Deep Learning of Railway

Track Faults using GPUs

Swiss Federal Railways (SBB)

Swiss Center for Electronics and Microtechnology

(CSEM)

Nathalie Rauschmayr (CSEM)

Matthias Hoechemer (CSEM)

Marcel Zurkirchen (SBB)

Stefan Kenzelmann (SBB)

Maitre Gilles (SBB)

GTC 2018, Santa Clara

Defect Detection.

Fingerprinting.

Monitoring the conditions of the Swiss rail network.

Need for automation

▪ Manual inspection is infeasible

▪ Ever increasing traffic leads to faster attrition

▪ New train types (Cargo, High Speed and Tilting

Trains…)

cause the development of railway faults that have

not been observable 10 years ago

Multiple specially equipped trains «Diagnosis Trains»

▪ Travelling up to 100 mph

▪ Multiple high resolution cameras and other sensors

Big Data Deep Learning of Railway Track Faults using GPUs 2

Some Statistics

• 15k trains per day

• 1.2 million rides per day

• Size of rail network: 4000 miles

Monitoring the conditions of the Swiss rail network.

Deep Learning of Railway Track Faults using GPUs 3

Diagnosis train operates since 2007, BUT

▪ Software generates too many false positives/negatives

▪ Railway experts have to filter all of them by hand

System has not been of much use until now

Railcheck project.

Deep Learning of Railway Track Faults using GPUs 4

Our Project Goals: Use Deep Learning to:

reduce the time for onsite visual inspections to a minimum

minimize the time experts spend on false positives

increase network safety

Human Being / BusinessMachine LearningDiagnosis train

Supervised Learning

Unsupervised Learning

Coniditon monitoring

DFZ since 2006

IBN gDFZ 2018

Railcheck project.

Deep Learning of Railway Track Faults using GPUs 5

Joint project between Swiss Center for Electronics and Microtechnology (CSEM) and

Swiss Federal Railways (SBB)

CSEM: Swiss R&D Lab specialized in microtechnology, system engineering, robotics,

automation and computer vision mainly doing applied R&D for industry (www.csem.ch)

The Input Data.

Deep Learning of Railway Track Faults using GPUs 6

Challenges:

▪ Changing weather conditions: rain, snow, ice

▪ Artefacts: leaves, dirt

▪ Different forms and shapes

Small artefacts Rain TurnoutsIn the street

Snow

Preprocessing.

Deep Learning of Railway Track Faults using GPUs 7

Tensorflow Object Detection API

Transfer Learning on faster R-CNN inception resnet in order to segment railway

surfaces and clamps

Anomaly Detection.

Deep Learning of Railway Track Faults using GPUs 8

Next Step: Clustering of railway components

Anomaly: components that do not fit to any cluster

Clustering with Generative Adversarial Networks (GAN):

Generator DiscriminatorRandom

Noise

Fake Data

Training Data

Anomaly Detection.

Deep Learning of Railway Track Faults using GPUs 9

Generator and Discriminator learn the underlying structure of the data

Search for similar images (Clustering) – Example:

Input Output: most similar images

Anomaly Detection.

Deep Learning of Railway Track Faults using GPUs 10

Pre-Clustering of normal railway components

Train Convolutional Autoencoder per Cluster

Input Image Reconstructed Difference

Anomaly

Fault Classification.

Deep Learning of Railway Track Faults using GPUs 11

Major Problem:

▪ Very little training data < 100

▪ Tricky to define fault category: in principal 20 categories

Was simplified to 6 categories and later on to 4

▪ A lot of variation

Welding Joint Surface Defect

Fault Classification.

Deep Learning of Railway Track Faults using GPUs 12

Major Problem:

▪ Very little training data < 100

▪ Tricky to define fault category: in principal 20 categories

Was simplified to 6 categories and later on to 4

▪ A lot of variation

Wheel SlipSquatNot Defect

Fault Classification.

Deep Learning of Railway Track Faults using GPUs 13

Labelling faults: Tedious,

▪ railway surface does mostly not contain any fault (luckily)

▪ faults can be easily overlooked

Speed the labelling process up

Increased training data for faults by a factor 100 within a month

Transfer Learning & Inference

Inspect & Correct Output

Add to Training

Data

Fault Classification.

Deep Learning of Railway Track Faults using GPUs 14

Major Problem: bias, wrong categories, different expert opinions

The worst faults are sometimes the ones that are nearly invisible (e.g. squats)

Squat Surface Defect Surface Defect

Fault Classification.

Deep Learning of Railway Track Faults using GPUs 15

Solution: Crowd Decision Making

Experts can view output of neural network via a website and give feedback

Add to Training

Data

Transfer Learning & Inference

Inspect & Correct Output

StatisticalAnalysis

Railway Experts

Surface Defect

Fault Classification.

Deep Learning of Railway Track Faults using GPUs 16

Even with little training data network achieves astonishing results

▪ Detection rates:

Joint 99%, Welding 57%, Surface Defect 65%

No training data: Squat, Wheel Slip

Surface Defect Surface Defect Welding Tiny surface defect

Fingerprinting.

Deep Learning of Railway Track Faults using GPUs 17

«Diagnosis trains» go routes multiple times per year

▪ Which fault is new?

▪ Which fault is already recorded in DB?

Challenge:

▪ Faults change over time

▪ Environmental conditions can be different

▪ Faults << GPS accuracy

Fingerprinting.

Deep Learning of Railway Track Faults using GPUs 18

Input faulty image into

pretrained model

Reduce size of

feature vector

Locality Sensitive

Hashing

Check for images within GPS

Range and compares hashes

20162017

Are these surface

defects identical?

Fingerprinting.

Deep Learning of Railway Track Faults using GPUs 19

Recording faults over time helps to understand

▪ How do they develop

▪ How to prevent them

November

2016

February

2017

April

2017

May

2017

June

2017

August

2017

Summary.

Deep Learning of Railway Track Faults using GPUs 20

Summary.

Deep Learning of Railway Track Faults using GPUs 21

Highly parallel problem:

▪ Create more instances of Fault- and Anomaly-Detectors depending on available

GPUs

▪ Ideally: real-time processing (120-160 km/h) HPC

www.csem.chwww.sbb.ch