RPUG2019 - Pavement Distress Detection Using Advanced ... › PDFs › PE-2019 ›...
45
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data By Matthew Connelly-Taylor Andrea Annovi Fugro Roadware
Transcript of RPUG2019 - Pavement Distress Detection Using Advanced ... › PDFs › PE-2019 ›...
Pavement Distress Detection Using Advanced Machine Learning Methods
with Intensity and Depth Data By
Matthew Connelly-Taylor
Andrea Annovi
Fugro Roadware
2
Slide about who Fugro Roadware is - Coming from Allan
Fugro Roadware
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data 2
Founded in 1969
1st fully integrated road data collection vehicle (ARAN) in 1980
2019
56 ARANs operating in 18 countries
Over 10 Million miles of ARAN roadway data to date
Over 500 Thousand miles of ARAN roadway data each year
3
Overview
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
• Why develop crack detection algorithms?
• Automation increases value of pavement data:
• Less human intervention = less subjectivity = more dependable results
• Faster results = more time to use the data = better decisions
• Current automated algorithms aren’t good enough
• Why Machine Learning?
• Rapidly improving field
• Excellent at solving complex problems with unstructured data
• Why us?
• We have 50 years of experience in pavement condition analysis
• We have a lot of accessible pavement data = 3 PetaBytes = 2 Million Miles
• …and it is already annotated
4
Pave3DX Stereoscopic Imaging and Measurement
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
4.3m width 7296 pixels 0.6mm pixel
Image 2Image 1
5
1mm Lines Combined into Image Frames
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
100 km/h
6
2 Images from different angles
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Image 2 (Right)Image 1 (Left)
7
Combine Stereo Images
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
3D Model
Depth Image
3D Point Cloud
8
Pave3DX + WiseCrax Processing Pipeline
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Right Stereo
Image
Left Stereo
Image
3D
Reconstruction
Depth
Image
Colour
Image
Image
Processing
Computer
Vision
Machine
Learning
Crack Detection
Longitudinal Profile
Surface Texture
Transverse Profile
Rut Depth
Pothole Detection
…
Learning Feed
Forward
Learning Feed Back
3D
Model
3D
Point
Cloud
9
WiseCrax Detection Pipeline
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Noise removal (autoencoder) Image tiles
Supervised
Unsupervised
ML
model
Roadware Vision
DB
10
LeNet-5 – A Classic CNN Architecture
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
http://yann.lecun.com/exdb/publis/pdf/lecun-98.pdf
11
Convolution Operation
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Source: http://cs231n.github.io/convolutional-networks/
12
Instance Segmentation
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
13
U-Net: Convolutional Networks
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
https://arxiv.org/pdf/1505.04597.pdf
14
Data Augmentation
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
• Generate batches of image data with real-time data augmentation
• The data will be looped over (in batches)
• 250,000 images as Data Augmentation
• Transformations applied:
o Rotation
o Flip
o Translation
o Gaussian Noise
o Scale
o Mirroring
15
Measure similarity between two images:
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Modified Pixel-wise-based Method
1. Introduce buffer regions by applying erosion on the original crack map
2. Convert the 'thick crack line' to 1-pixel-wide crack line using Skeletonization
Accuracy = Skeleton of TP / Skeleton of Union
16
Detection Performance Definitions
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
17
Crack Detection Metrics - Precision
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Average : 0.909
18
Crack Detection Metrics - Recall
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Average : 0.999
19
Crack Detection Metrics – F1 Score
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Average: 0.948
20
Example 1 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
21
Example 1 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
22
Example 1 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
23
Example 2 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
24
Example 2 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
25
Example 2 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
26
Example 3 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
27
Example 3 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
28
Example 3 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
29
Example 4 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
30
Example 4 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
31
Example 4 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
32
Example 5 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
33
Example 5 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
34
Example 5 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
35
Example 6 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
36
Example 6 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
37
Example 6 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
38
Example 7 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
39
Example 7 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
40
Example 7 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
41
Example 8 – Real Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
42
Example 8 – Depth Image
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
43
Example 8 – Crack Detection
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
44
Conclusion
Pavement Distress Detection Using Advanced Machine Learning Methods with Intensity and Depth Data
Deep Learning - Well suited for pavement distress detection
WiseCrax with UNet – Very promising results
Precision = 90.9%
Recall = 99.9%
F1 Score = 94.8%
Continuing to learn rapidly
Large data sets improve results
Data Augmentation
Reduces burden of annotation
Increases speed of improvement
