Information Technology

Robust Background Subtraction Based on

Perceptual Mixture-of-Gaussians with

Dynamic Adaptation Speed

Mahfuzul Haque and Manzur Murshed

Agenda

Background Subtraction

Statistical Background Subtraction

Perception Inspired Background Subtraction

Dynamic Adaptation Speed

Experiments

Summary

Q&A

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 2

Background Subtraction

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 3

Input

Output

Background Subtraction: Challenges

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 4

Illumination variation

Local background motion

Camera displacement

Shadow and reflection

Challenges

Current frame

Background

Model

Foreground Blob

Dynamic Background Subtraction(e.g., MOG)

Basic Background Subtraction (e.g., BBS)

- =

Current frame Background Foreground Blob

Statistical Background Subtraction

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 5

Te

P(x)

x x

MOG: x = c1σ

μ

P(x)

x x

b

x = c2b

ω1

σ12

µ1

road

ω2

σ22

µ2

shadow

ω3

σ32

µ3

car

65% 20% 15%

BBS: x = c

Statistical Approaches Our Hypothesis (Perception Inspired)

Perception Inspired Background Subtraction

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 6

P(x)

x x

b

x = c2b Detection with

Low x

Current

Frame

Detection with

High x

Weber’s Law

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 7

Ernst Weber, an experimental psychologist in the 19th

century, observed that the just-noticeable increment ΔI

is linearly proportional to the background intensity I.

ΔI = c2I

How human visual system perceives noticeable intensity

deviation from the background?

Weber’s Law

Te

Ernst Weber, an experimental psychologist in the

19th century, observed that the just-noticeable

increment ΔI is linearly proportional to the

background intensity I.

P(x)

x x

b ? x b

ΔI = c2I

December 30, 2013 8 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed

x = c2b

Perceptual tolerance of HVS

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 9

Method 1

Method 2 Reference

Image

Distorted

Images

p dB

q dB |p – q| < 0.5 dB

Not perceivable

by human visual

system

What is the perceptual tolerance level in distinguishing

distorted intensity measures?

Our Problem: c2 = ?

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 10

Te

x = c2b

P(x)

x x

b

Weber’s Law

Perceptual Threshold, TP (0.5 dB)

1255

10log20255

10log20

xbxb2TP

x = c2b

Linear Relationship

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 11

x

b

Rod and Cone

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 12

Rods and Cones are two different types of

photoreceptor cells in the retina of human eye

Rods

– Operate in less intense light

– Responsible for scotopic vision (night vision)

Cones

– Operate in relatively bright light

– Responsible for photopic (color vision)

Error Sensitivity in Darker Background

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 13

Piece-wise Liner Relationship

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 14

Te

Scotopic Vision (R) Photopic Vision (C)

Dynamic Adaptation Speed

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 15

•Sleeping person problem

•Walking person problem

Dynamic Adaptation Speed

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 16

Dynamic Adaptation Speed

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 17

Dynamic Adaptation Speed

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 18

Experiments

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 19

Total 50 test sequences from 8 different sources

Scenario distribution

Indoor Outdoor Multimodal Shadow and Reflection Low background-foreground contrast

Test Sequences

Evaluation

Qualitative and quantitative comparison:

MOG (S&G) (TPAMI, 2000)

MOG (Lee) (TPAMI, 2005)

ViBe (TIP, 2011)

False Positive (FP)

False Negative (FN)

False Classification

Test Sequences

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 20

Te PETS (9) Wallflower (7) UCF (7) IBM (11) CAVIAR (7) VSSN06 (7) Other (2)

Experiments

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 21

Experiments

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 22

Experiments

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 23

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 24

First

Frame

Test

Frame

Ground

Truth

MOG

(S&G)

MOG

(Lee)

ViBe Proposed

Summary

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 25

Realistic background value prediction: high model agility

and superior detection quality at fast learning rate.

No context related information: high stability across

changing scenarios.

Perception based detection threshold: superior detection

quality in terms of shadow, noise, and reflection.

Perceptual model similarity: optimal number of models

throughout the system life cycle.

Parameter-less background subtraction: ideal for real-

time video analytics.

Q&A

December 30, 2013 Perceptual Mixture-of-Gaussians with Dynamic Adaptation Speed 26