Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final...

Background Subtraction for Urban Traffic Monitoring

using Webcams

Master Graduation ProjectFinal Presentation

Supervisor: Rein van den Boomgaard

Mark Smids

December 12th 2006

Overview

• Introduction• Background Subtraction• Shadow Detection• Video Summarization • Demo’s

• Background Subtraction in action• Shadow Detector in action• Smart Surveillance using Video Summarization

• Evaluation • Conclusions

Introduction

• Introduction

• Background Subtraction

• Shadow Detection

• Video Summarization

• Demos

• Evaluation

• Conclusions

• Traditional ways of traffic monitoring

• using magnetic loops

Introduction

• Introduction




• Demos

• Evaluation

• Conclusions



• Limitations:

• These systems only count, very costly

Introduction

• Introduction




• Demos

• Evaluation

• Conclusions



• Limitations:


• For extended traffic monitoring we want to measure:

• road density, queue detection, vehicle speed, exact location of vehicles

Introduction

• Introduction




• Demos

• Evaluation

• Conclusions



• Limitations:


• For extended traffic monitoring we want to measure:

• road density, queue detection, vehicle speed, exact location of vehicles

• Solution: use cameras to monitor traffic automatically

Introduction

• Introduction




• Demos

• Evaluation

• Conclusions

• Why focus on an urban setting?

• Most research focused on a highway setting

• More challenging tasks

Introduction

• Introduction




• Demos

• Evaluation

• Conclusions

• Why focus on an urban setting?

• Most research focused on a highway setting

• More challenging tasks

• Components of a vision based traffic monitoring system:

• cameras, calibration, background subtraction, tracking, shadow detection, parameter extraction, video summarization, …

Background Subtraction

• Introduction




• Demos

• Evaluation

• Conclusions

• Deterministic approach

• Create an initial background model from the first

N frames

• For each new frame, subtract it from the background model to obtain a binary mask for all x,y: if I(x,y) – B(x,y) > T then M(x,y) = 1 else M(x,y) = 0

• Update the background model: for all x,y: if M(x,y) = 0 then B(x,y) = I(x,y)

• Introduction




• Demos

• Evaluation

• Conclusions

• Statistical approach

• Model each pixel in the background model by a mixture of Gaussians

),,(*)( ,,1

, titit

k

itit XXP


/

• Introduction




• Demos

• Evaluation

• Conclusions

• Statistical approach

• Model each pixel in the background model by a mixture of Gaussians

• How to determine those components that model

the background?• Observation: these Gaussians have the most supporting evidence and lowest variances • Order the K distributions in the mixture by the value of• The first B distributions are chosen as the background model, where:

TBb

kkb

1

minarg


),,(*)( ,,1

, titit

k

itit XXP

Shadow Detection

• Introduction




• Demos

• Evaluation

• Conclusions

• Shadows: cast and self shadows

Shadow Detection

• Introduction




• Demos

• Evaluation

• Conclusions

• Shadows: cast and self shadows

• Elimination of cast shadows can improve background subtraction results very much…

Shadow Detection

• Introduction




• Demos

• Evaluation

• Conclusions

• Consider the set of pixels classified as foreground pixels

Shadow Detection

• Introduction




• Demos

• Evaluation

• Conclusions


• A pixel is a candidate shadow pixel when the pixel value has a significant lower value

than it’s corresponding background value

Shadow Detection

• Introduction




• Demos

• Evaluation

• Conclusions




• Extend this idea: let c = (R,G,B) and

Rate of similarity:

),,( bgr

c

SRGB

Shadow Detection

• Introduction




• Demos

• Evaluation

• Conclusions




• Extend this idea: let c = (R,G,B) and

Rate of similarity:

• If tau < < 1 then pixel is a shadow pixel

),,( bgr

c

SRGB

RGBS

Video Summarization

• Introduction




• Demos

• Evaluation

• Conclusions

• Application: smart vision based surveillance system

Video Summarization

• Introduction




• Demos

• Evaluation

• Conclusions


• Record only frames which includes relevant foreground objects

Video Summarization

• Introduction




• Demos

• Evaluation

• Conclusions


• Record only frames which includes relevant foreground objects

How to guarantee that a full trajectory of a vehicle is recorded?

Demos

• Introduction




• Demos

• Evaluation

• Conclusions

1. Shadow Detector in action – 1 | 2

2. Background Subtraction in action det 1 | stat 1 - det 2 | stat 2

3. Smart Surveillance using Video Sum. - 1

Evaluation

• Introduction




• Demos

• Evaluation

• Conclusions

• Test videos: three different weather conditions (5 minutes each)

• Goal: test both background subtraction algorithms on these videos

• Limitation: no ground truth available!

Evaluation

• Introduction




• Demos

• Evaluation

• Conclusions

• Evaluation on another level: using the video summarization component.

• A frame level ground truth is created

• For each algorithm a score can be computed

Evaluation

• Introduction




• Demos

• Evaluation

• Conclusions

100)1( N

ES

• Introduction




• Demos

• Evaluation

• Conclusions

Score S

(deterministicapproach)

Score S

(statisticalapproach)

Totalnumber of Frames

Video A(wind/cloudy)

85.6% 88.3% 4581

Video B(sunny)

88.5% 94.6% 4163

Video C(rain)

83.4% 93.4% 3024

Evaluation

Conclusions

• For all weather conditions: the statistical approach outperforms the deterministic approach (5-10%)

• Introduction




• Demos

• Evaluation

• Conclusions

Conclusions


• Wind is the hardest problem from both algorithms

• Introduction




• Demos

• Evaluation

• Conclusions

Conclusions



• Statistical approach performs much better in the sunny settings

• Introduction




• Demos

• Evaluation

• Conclusions

Conclusions



• Statistical approach performs much better in the sunny settings

• Future work: create a pixel-level ground truth and evaluate both algorithms

• Introduction




• Demos

• Evaluation

• Conclusions

Questions?

• Introduction




• Demos

• Evaluation

• Conclusions

• Questions?

http://www.science.uva.nl/~msmids/afstuderen/master

MoG details

• Update Equations:)( k

tkkk E

kktkkk E )/(

22kk ))(/( 2

kkTkk

tkE

Z. Zivkovic, “Improved Adaptive Gaussian Mixture Model for Background Subtraction”

ktk X

• MoG: ),,(*)( ,,1

, titit

k

itit XXP

)()(2

1

2

12

1

)2(

1),,(

ttT

tt XX

nt eX

• Introduction




• Demos

• Evaluation

• Conclusions

Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final...

Documents

Transcript of Background Subtraction for Urban Traffic Monitoring using Webcams Master Graduation Project Final...