Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.
-
Upload
joseph-banks -
Category
Documents
-
view
239 -
download
6
Transcript of Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.
![Page 1: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/1.jpg)
Automated Drowsiness Detection For Improved
Driving Safety
Aytül ErçilNovember 13, 2008
![Page 2: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/2.jpg)
Outline
Problem Background and Description Technological Background Action Unit Detection Drowsiness Prediction
![Page 3: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/3.jpg)
Objectives/Overview•Statistical Inference of fatigue Using Machine Learning Techniques
![Page 4: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/4.jpg)
In over 500.000 accidents in 2005 (in Turkey): Injured: 123,985 people
Deceased: 3,215 people
Financial loss: 651,166,236 USD
Number of accidentsı
0
100.000
200.000
300.000
400.000
500.000
600.000
19
80
19
82
19
84
19
86
19
88
19
90
19
92
19
94
19
96
19
98
20
00
20
02
20
04
(*)
A. Kaza B. Ölü C. Yaralı D.Maddi Hasar Miktarı (ABD $) A
Driver error has been blamed as the primary cause for approximately 80% of these traffic accidents.
![Page 5: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/5.jpg)
The US National Highway Traffic Safety Administration estimates that in the US alone approximately 100,000 crashes each year are caused primarily by driver drowsiness or fatigue
![Page 6: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/6.jpg)
Growing Interest In Intelligent Vehicles US Department of Transportation Initiative European Transport Policy for 2010: set a
target to halve road fatalities by 2010.
Problem Background
![Page 7: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/7.jpg)
The Drivesafe ProjectThe Drivesafe Project
![Page 8: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/8.jpg)
Current Funding Status:
• Turkish Development Agency funding of Drive-Safe (August 2005-July. 2009)
• Japanese New Energy and Industrial Technology Development Organization (NEDO) (October 2005 -December 2008)
• FP6 SPICE Project at Sabancı University (May 2005- October 2008)
• FP6 AUTOCOM Project at ITU Mekar (May 2005- April 2008).
![Page 9: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/9.jpg)
Readiness-to-perform Mathematical models of alertness dynamics Vehicle-based performance technologies (Vehicle
Speed, Lateral Position, Pedal Movement) In-vehicle, on-line, operator status monitoring
technologies
Fatigue Detection and Prediction TechnologiesFatigue Detection and Prediction Technologies
![Page 10: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/10.jpg)
Physiological Signals (heart rate, pulse rate and Electroencephalography (EEG))
Computer Vision Systems (detect and recognize the facial motion and appearance changes occurring during drowsiness)
In-vehicle, on-line, operator status monitoring In-vehicle, on-line, operator status monitoring technologiestechnologies
![Page 11: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/11.jpg)
Computer Vision SystemComputer Vision Systemss
Visual BehaviorsVisual Behaviors• ExamplesExamples
Gaze DirectionGaze Direction Head MovementHead Movement YawningYawning
• No requirement for physical contactNo requirement for physical contact
![Page 12: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/12.jpg)
Facial Actions
Ekman & Friesen, 1978
![Page 13: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/13.jpg)
Background Information- Background Information- Action UnitsAction Units
![Page 14: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/14.jpg)
Proposed WorkProposed Work
Detection Of Driver Fatigue From A Recorded Video Using Facial Appearance Changes
The framework will be based on graphical models and machine learning approaches
![Page 15: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/15.jpg)
Proposed ArchitectureProposed Architecture
Sensing ChannelsEye Tracker
AU 61
Pupil Motion
AU 62
Gaze Tracker
Gaze
AU 51 AU 52
Eye Tracker
AU 61
Pupil Motion
AU 62
GazeTracker
Gaze
AU 51 AU 52
Features
Time n-1 Time n
Inattentive Falling Asleep
Fatigue
Inattentive Falling Asleep
Fatigue
Entire Face Behavior
Partial Face Behavior
Single AU
![Page 16: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/16.jpg)
Action Unit TrackingAction Unit Tracking
Previous techniques Previous techniques Do not employ a spatially and temporally Do not employ a spatially and temporally
dependent structure for Action Unit Trackingdependent structure for Action Unit Tracking Contextual information is not exploitedContextual information is not exploited Temporal information is not exploitedTemporal information is not exploited
![Page 17: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/17.jpg)
Classification- ChallengesClassification- Challenges
Which action units or combinations is a cue for fatigue?
![Page 18: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/18.jpg)
Learning from real examples
Posed Drowsiness
Actual Drowsiness
Different Neural pathways for posed/spontaneous expressions
![Page 19: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/19.jpg)
Initial Experimental Setup
Subjects played a driving video game on a windows machine using a steering wheel and an open source multi-platform video game. At random times, a wind effect was applied that dragged the car to the right or left, forcing the subject to correct the position of the car.
![Page 20: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/20.jpg)
Head movement measures
Head movement was measured using an accelerometer that has 3 degrees of freedom. This three dimensional accelerometer has three one dimensional accelerometers mounted at right angles measuring accelerations in the range of 5g to +5g
![Page 21: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/21.jpg)
The one minute preceding a sleep episode or a crash was identified as a non-alert state. There was a mean of 24 non-alert episodes with a minimum of 9 and a maximum of 35.
Fourteen alert segments for each subject were collected from the first 20 minutes of the driving task.
![Page 22: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/22.jpg)
Crash
Overcorrection
Seconds0 20
Steering
Distance from center
Eye openingEyes closed
![Page 23: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/23.jpg)
Histograms for Eye Closure and Eye Brow Up
Eye Closure: AU45 Brow Raise:AU2 Area under the ROC
![Page 24: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/24.jpg)
Pattern Recognition(Adaboost)
(SVM)
FeatureSelection
Machine Learning
Facial Action Unit Detection
AU1
AU2
AU4
….
….
AU46
++
![Page 25: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/25.jpg)
Drowsiness Prediction
The facial action outputs were passed to a classifier for predicting drowsiness based on the automatically detected facial behavior.
Two learning-based classifiers,
Adaboost and multinomial logistic regression are compared.
Within-subject prediction of drowsiness and across-subject (subject independent) prediction of drowsiness were both tested.
![Page 26: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/26.jpg)
Classification Task
Multinomial Logistic
Regression (MLR)
Frame
Alert
60 secBeforecrash
:
• 31 Facial Action Channels• Continuous output for each frame
AU1
AU2
AU4
AU31
![Page 27: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/27.jpg)
Testing: MLR Weighted Temporal Windows
![Page 28: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/28.jpg)
Within subject drowsiness prediction
For the within-subject prediction, 80% of the alert and non-alert episodes were used for training and the other 20% were reserved for testing.
This resulted in a mean of 19 non-alert and 11 alert episodes for training, and 5 non-alert and 3 alert episodes for testing per subject.
![Page 29: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/29.jpg)
Across Subject Drowsiness Prediction
Training : 31 actions -> MLR Classifier Framewise training
Cross validation: 3 subjects –> training 1 subject –> testing
Crash prediction:• choose 5 best features by sequential feature selection• Sum MLR weighted features over 12 second time interval• .98 across subjects (Area under the ROC)
![Page 30: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/30.jpg)
More when critically drowsy
Eye Closure Brow Raise Chin Raise Frown Nose Jaw Wrinkle Sideways
Predictive Performance of Individual Facial Actions
![Page 31: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/31.jpg)
Predictive Performance of Individual Facial Actions
Less when critically drowsy
Smile Squint Nostril Brow Lower Jaw Drop
CompressorA’ > .75
![Page 32: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/32.jpg)
We observed during this study that many subjects raised their eyebrows in an attempt to keep their eyes open, and the strong association of the AU 2 detector is consistent with that observation.
Also of note is that action 26, jaw drop, which occurs during yawning, actually occurred less often in the critical 60 seconds prior to a crash. This is consistent with the prediction that yawning does not tend to occur in the final moments before falling asleep.
![Page 33: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/33.jpg)
Drowsiness detection performance, using an MLR classifier with different feature combinations.
![Page 34: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/34.jpg)
Effect of Temporal Window Length
* 12 secondsA’
Seconds
![Page 35: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/35.jpg)
Coupling of Facial Movements
ALERT DROWSY
Eye Openness
Brow Raises
Brow RaisesBrow Raise
Eye ClosureBrow Raise
Eye Closure
r=0.87
0 Seconds 10 10Seconds0
![Page 36: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/36.jpg)
Coupling of Steering and Head Motion
ALERT DROWSY
r=0.27
r=0.65
Steering
Head Acceleration
Head Acceleration
SteeringSeconds 60 600 0 Seconds
![Page 37: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/37.jpg)
Coupling of Steering and Head Motion
![Page 38: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/38.jpg)
New associations between facial behavior and drowsiness
• Brow raise• Chin raise• More head roll• Possibly less yawning just before crash
• Coupling of behaviors– Head movement and steering– Brow raise and eye opening
![Page 39: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/39.jpg)
Future WorkFuture Work
Extend the graphical model so that it captures the Extend the graphical model so that it captures the temporal relationships using a discriminative temporal relationships using a discriminative approachapproach
![Page 40: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/40.jpg)
Future Work: More Data Collection in Future Work: More Data Collection in Simulator Environment Simulator Environment
Uykucu (Sleepy)
![Page 41: Automated Drowsiness Detection For Improved Driving Safety Aytül Erçil November 13, 2008.](https://reader036.fdocuments.net/reader036/viewer/2022081419/56649e2b5503460f94b195f0/html5/thumbnails/41.jpg)
Thank you