Effects of Post-processing on Background Subtraction Algorithms

Donovan Parks

Outline

What is background subtraction?

Project motivation

How is BGS performed and what makes it difficult?

Project goals and results

Concluding remarks

What is background subtraction?

Real-time method for identifying moving foreground objects within a video

Project motivation BGS is an important low-level step in

many computer vision applications: Video surveillance Traffic monitoring FG/BG segmentation

My interest is in using BGS to extract human silhouettes for pose estimation

How “good” are the obtained silhouettes in unconstrained environment?

Images from: Sminchisescu and Telea, “Human Pose Estimation from Silhouettes”, 2002.

How is BGS performed?

Static frame differencing BG model = first frame of video

BGDifferencing

Input Stream

BG Model

Output Masks

Threshold

What makes BGS difficult?

Moving background elements:

Adaptive, statistical BG models

BGDifferencing

Mean

Input Stream

BG Model

Output Masks

Threshold

Update BGModel

0 50 100 150 200 2500

0.005

0.01

0.015

0.02

0.025

0.03

Component Value

Pro

babi

lity

0 50 100 150 200 2500

0.005

0.01

0.015

0.02

0.025

0.03

Component Value

Pro

babi

lity

0 50 100 150 200 2500

0.005

0.01

0.015

0.02

0.025

0.03

Component Value

Pro

babi

lity

Variance

Gaussian Pixel Model

What makes BGS difficult?

Shadows:

Shadow removal

Shadows have little effect on chromaticity, but reduce luminosity

BGDifferencing

Mean

Input Stream

BG Model

Output Masks

Threshold

Update BGModel

Variance

ShadowRemoval

What makes BGS difficult?

Ghosting:

Ghost detection via optical flow

Low optical flow = ghost!

BGDifferencing

Mean

Input Stream

BG Model

Output Masks

Threshold

Update BGModel

Variance

ShadowRemoval

ConnectedComponents

OpticalFlow Test

What else makes BGS difficult?

FG/BG blending

Project goals

Evaluate a selection of state-of-the-art background subtraction algorithms Considering 10 algorithms in all

Analyze how post-processing influences the performance of these algorithms Shadow removal Optical flow testing Morphological “cleaning” Area thresholding

Initial resultsP

reci

sion

Recall

Example of shadow removal

Example of “cleaned” results

Conclusions

Many factors which make BGS difficult

Post-processing can significantly improve results

Results not as “clean” as more computationally expensive approaches

Questions?

Thank you.