Rutgers researchers develop AI-aided tool to help deter rail trespassers
8/31/2022
By Julie Sneider, Senior Associate Editor
Researchers at Rutgers University in New Jersey hope their efforts to create an artificial intelligence (AI)-aided tool will one day help prevent deadly situations like the derailment that occurred June 27 after an Amtrak train collided with a dump truck that had entered a grade crossing on a Missouri highway.
Asim Zaman, a Rutgers project engineer, and Xiang Liu, an associate professor in transportation engineering at Rutgers’ engineering school, created an AI-aided framework that automatically detects trespassers on rail property. The technology can differentiate types of trespassers and generate video clips of infractions.
The researchers define trespassers as unauthorized people or vehicles on rail or transit property not intended for public use, or vehicles that enter a signalized crossing after the signals have been activated.
Most research into rail trespassing incidents has focused on casualties. However, often overlooked in previous research is the “near miss” information that could help determine trespasser behavior that can then be used by railroads, transit and government agencies to discourage such behavior, the researchers say.
In the recent Amtrak case, the Amtrak Southwest Chief train was carrying 270 passengers and 12 crew members when it collided with a dump truck that was passing through a crossing near Mendon, Missouri. Three passengers and the truck driver died, and multiple passengers and crew members were injured.
Xiang Liu, an associate professor in transportation engineering at Rutgers University, worked with Rutgers project engineer and fellow researcher Asim Zaman to create an AI-aided framework that detects trespassers on rail property. Rutgers UniversityThe crossing was equipped with crossbucks and a stop sign. All locomotives and rail cars derailed with seven cars coming to a rest on their side. Amtrak and BNSF Railway Co., which owns the track, have estimated $4 million in property damage, according to the National Transportation Safety Board.
Trespassing along railroad rights-of-way is the leading cause of rail-related deaths in the United States, according to the Federal Railroad Administration. Nationally, more than 400 trespass fatalities occur each year. At the same time, incidents at grade crossings are the second-leading cause of rail-related fatalities. FRA-assisted programs have helped reduce those fatalities by 60% over the past two decades; however, federal, state and rail industry officials want those numbers reduced even more.
About 212,000 crossings exist on the 140,000 miles of track that make up the U.S. rail system, according to the FRA. Together, trespassing and crossing incidents account for 95% of all rail-related deaths and injuries, with nearly all of those incidents preventable, FRA officials say.
That’s where the Rutgers researchers believe their research into trespassing detection technology will help. Their newly developed system uses an algorithm to process video data into a single dataset.
In their study, Zaman and Liu used their AI and deep-learning tool to analyze more than 1,600 hours of video footage taken at a crossing in urban New Jersey. During 68 days of monitoring, the researchers learned that an average of 44 trespassing instances occurred per day; nearly 70% of trespassers were men; about a third trespassed before the train passed; and most violations occurred on Saturdays around 5 p.m.
The image shows two people detected by the AI on a railroad right-of-way in Thomasville, North Carolina. Image provided courtesy of Asim Zaman, Rutgers University“With this information we can answer numerous questions, like what time of day do people trespass the most, and do people go around the gates when they are coming down or going up,” says Zaman.
Although video cameras installed at the New Jersey location and at other crossings were installed after the 2015 Fixing America’s Surface Transportation Act, most video taken at those crossings today are either not reviewed or are reviewed manually, which is labor-intensive and expensive. By using the researchers’ system to process the video surveillance data into a single dataset, the granular data can be used by police departments to position officers near crossings during peak violation times or to inform railroads of more effective crossing solutions, such as grade-crossing separations or advanced gates and signals.
“Everyone loves data, and that’s what we’re providing,” Zaman says.
The study results from the New Jersey crossing were published in April in the journal Accident Analysis & Prevention. Additional studies are underway in Virginia and North Carolina, and the U.S. Department of Transportation recently awarded the researchers a $583,000 grant to expand their research to other states, including Connecticut, Massachusetts and Louisiana, according to Rutgers. Research collaborators include rail companies, transit agencies and state departments of transportation.
An Amtrak train derailed June 17 after it struck a dump truck that was traveling through a grade crossing near Mendon, Missouri. Three passengers on the train and the truck driver died as a result of their injuries. National Transportation Safety Board/BNSF Railway Co. For now, the technology is still in the research and development phase “to make sure the AI works,” Zaman says. It’s hoped the research will eventually lead to a commercially developed tool for railroad applications.
In June, Zaman discussed the research at the International Level Crossing Awareness Day (ILCAD) conference held in Denver. The conference is attended by rail industry representatives, road authorities, academics and others who raise awareness about the dangers surrounding grade crossings.
Operation Lifesaver Inc. (OLI), the 50-year-old U.S. rail-safety education organization, was a host of this year’s ILCAD conference. In July, OLI — citing FRA statistics — reported on its website that 2,147 crossing collisions occurred in the United States last year. About 68% occurred in the following states (ranked highest to lowest): Texas, California, Georgia, Illinois, Indiana, Florida, Alabama, Louisiana, Ohio, Michigan, Tennessee, Pennsylvania, Virginia, South Carolina and North Carolina.
In addition, 1,144 pedestrian rail-trespass casualties (fatalities and injuries) occurred last year, OLI reported. About 69% of those were in these states (ranked highest to lowest): California, Texas, Florida, New York, Washington, Illinois, Ohio, Pennsylvania, Georgia, New Jersey, Oregon, North Carolina, Louisiana, Indiana and Nevada.
