Recognition of Traffic Lights in Live Video Streams on Mobile Devices Jan Roters Xiaoyi Jiang Kai...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
2
Transcript of Recognition of Traffic Lights in Live Video Streams on Mobile Devices Jan Roters Xiaoyi Jiang Kai...
Recognition of Traffic Lights in Live Video Streams on Mobile
Devices
Jan Roters
Xiaoyi Jiang
Kai Rothaus
2011 IEEE Transactions on CSVT
Outline
IntroductionProblemsSystem Architecture
IdentificationClassificationVideo AnalysisTime-Based Verification
Experiment ResultsEvaluationsConclusion
Introduction
People with visual disabilities are limited in mobility.
Orientate pedestrians with zebra crossings at intersections
Portable PC with a digital camera and a pair of auricular stereo
Present a system for mobile devices to help sightless people cross roads.
Problems
Program usage
Real world conditionsCamera resolutionDifferent appearances
Problems
The scale of traffic lightsMany traffic lightsOccludedIlluminationRotation
Pedestrian Lights in Germany
1) Installation
2) Shape
3) Color arrangement
4) Circuitry
5) Background
Mobile Device & Databases
Nokia N95330MHZ ARM processor18Mb RAM320240
2 publicly available databaseGround truth segmentation was made manually
System Architecture
1.
2.
3.
4.
1. Localization
Red and Green Color Filter(1/3)
1. Analyze the data
Red and Green Color Filter(2/3)
2. Design the filter rules (ex : red traffic light)
The Gaussian distribution of the red cluster is defined by its mean color = (0.48,0.06,0.07) and has three eigenvectors
A color c = (r, g, b) is a red traffic light color when
Red and Green Color Filter(3/3)
3. Optimize parameters different parameter settings for each color Use 300 images to train Measure the quality of each setting by TP, FP, FN
Recall = , Precision =
Size/Circuitry Filter
Assume the traffic light is 4 to 24 meters awayFixed camera focal length and possible aspect
ratios
1. Filter out regions that are too small or too large
2. Vertical neighbor should not have different color
Background Color Filter
Inspect the region under a red light candidate or above a green light candidate
If there are no dark pixels within search region, refuse this candidate
Search region
Search region
Validation of Localization
Validate the localization results with 201 images
Optimal Validation
recall precision recall precision
Red 76% 89.5% 71.8% 87%
green 85% 98.5% 83.3% 92.6%
Error = 33.7%
2. Classification
TLC is the broadestTLC has the smallest distance to the top of imageNo other traffic light has similar height with TLC
Performance of Classification
Red Green
Recall 86.3% 86.3%
Precision 97.4% 98.1%
3. Video Analysis(1/2)
Temporary OcclusionFalsified ColorsContradictory SceneRepeating Results
3. Video Analysis(2/2)
Find the motion vector between two framesUse KLT tracker to track feature pointsOnly search in a small area around crucial traffic light
candidate (30 pixels in each direction)Correlate the features by using SAD
Search region
Crucial traffic light
Candidate region
Feature point
𝑡𝑖 −1 𝑡𝑖
4. Time-Based Verification
Reduce the false positive detections by comparing 2 kinds of results
Use state queue with 4 scenarios1) Identification and video analysis are both successful
and the locations match with each other.
2) Identification and video analysis are successful but the locations are different.
3) Video analysis succeeds but identification fails.
4) Video analysis fails but identification succeeds.
Experiment Results
and Compute at least 5 frames per secondAt least consecutive correct detection with the
same color
Experiment Results
Evaluations
ReliabilityPrevent false positive green light detection
Evaluations
InteractivityTemporal analysis reduce the interactivityThe feedback is normally given within 2 seconds
Conclusion
The system can be helpful on driver assistance systems
Limited computational power on mobile devicesThe verification ideas can be improved