Skip to main content

Developing ‘intelligent’ wayside track detectors

December 2020

Jamie Williams, senior director mechanical operations and support, has led his team in collaborations with NS’ advanced analytics group to develop technology-driven rail car inspections.

NS’ use of AI to inspect rail cars enhances safety and reduces business costs

At around 6 a.m. on Saturday, Nov. 14, Mabby Amouie got out of bed and checked his overnight email. Scanning a series of messages, he learned that his team’s work very likely had helped prevent a train derailment earlier that morning on our busy Chicago Line, just west of Elkhart, Indiana.

“It was fantastic. I was really happy,” said Amouie, Norfolk Southern’s director of advanced analytics and chief data scientist.

Even while they’re sleeping, Amouie and his team of data scientists produce results. Call it the power of artificial intelligence.

In close collaboration with NS’ mechanical operations forces, Amouie’s analytics team has been developing deep learning algorithms that enable wayside detectors to intelligently inspect passing rail cars for defects that could derail a train. Having started with a focus on car coupler systems, the team now is making progress on algorithms to identify defects in air hoses, wheels, bearings, and trucks.

“With our deep-learning algorithms, we are embedding human intelligence into machines,” Amouie said. “Essentially, we’re training the machine to look for certain things, and the machine learns as it applies this knowledge to the world.” 

Mabby Amouie, director advanced analytics and chief data scientist, is leading the effort to develop intelligent wayside detectors to inspect passing rail cars.

Advancing toward technology-driven inspections

Combining cameras and artificial intelligence, these wayside machine vision detectors capture images and data, analyze them in real time, and send alerts when potential defects are identified. The email alerts are issued automatically, without human involvement, going to the mechanical wayside desk in Atlanta’s Network Operations Center – no matter the time of day or night.

As work advances, the goal in the years ahead is to develop a series of detectors capable of performing automated train inspections at track speed, with no need to slow or stop the train.

“With the work that Mabby and his team are doing, we’re setting the stage and pushing this dynamic train inspection in the right way, both for NS and the entire industry,” said Jamie Williams, senior director mechanical operations and support, whose team provides the business intelligence that our advanced analytics group uses to develop the algorithms.

Generating benefits

As the Nov. 14 success story proves, the efforts are producing significant benefits. Presently, intelligent machine vision detectors are installed at five strategic locations across the network, including at Vine Creek, Indiana, about 25 miles west of Elkhart, the site of the recent Saturday incident.

Shortly before 2 that morning, NS’ mechanical wayside desk received an email alert from a Vine Creek machine vision detector programmed to identify defects in car coupling systems, including missing cotter keys, retainer pins, and bolts. These are the components that keep the cars connected in the train. Looking at a photo image, the wayside desk quickly confirmed that a rail car on a mixed-freight train running over the Chicago Line was missing the cross key and retainer pin – a critical issue.

“That means the train is liable to separate, and you could have a derailment any minute,” said Mike Fabery, assistant manager wayside desk. He immediately contacted the Great Lakes Division dispatch desk. A dispatcher radioed the train crew to let them know they needed to bring the train to a slow stop. A car mechanical crew from Burns Harbor, Indiana, hit the road to meet the train and repair the car. By sometime after 7 a.m., the car had been repaired, and the train was on its way again.

Recognizing results

That morning, emails flowed from senior leaders recognizing the success – from the work of Amouie’s team to the full-court response by operations. In addition to enhancing safety, the effort avoided the costs associated with derailments.

“This was a great find and fix, likely avoiding a significant event. Thank you!” wrote Tom Schnautz, vice president advanced train control.

“Excellent work! We caught this one before it caught us,” Williams wrote. “This is a milestone that wouldn’t have been achieved without the continued deep dives into the processes by the team and the output of the algorithms from Mabby’s analytics. We will lead the industry in this effort and set the stage for the future of rail inspection. Keep doing the deep dives, keep working as a team, and never lose sight of the vision.”

Above are images of rail car coupling components taken by wayside machine vision detectors that use deep-learning algorithms developed by NS to inspect rail cars on passing trains. The Nov. 14 image – a quality photo showing a missing cotter key, retainer pin, and cross key – was taken shortly before 2 a.m. as a train passed a wayside detector at Vine Creek, Indiana, at more than 50 mph. The detector sent an automated email alert with the image to NS’ mechanical wayside desk in Atlanta, potentially diverting a train separation and derailment.