Comparison of complex background subtraction algorithms using a fixed camera
description
Transcript of Comparison of complex background subtraction algorithms using a fixed camera
Comparison of complex background
subtraction algorithms using a fixed camera
Geoffrey Samuel
PhD ResearcherIntelligent Systems and Robotics Research Group (ISR)
Creative TechnologiesUniversity of Portsmouth
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Intro Background subtraction is a important
and vital step for computers to understand and interpreter a real-world scene
It allows a computer to ignore a background so to concentrate on a foreground object
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Hypothesis Each background subtraction algorithm
will have its advantages and disadvantages, and that looking and comparing these with a real-world situation, it would be possible to pick one algorithm or a method of combining algorithms to produce a algorithm capable of balancing speed with quality.
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
The Goal Test and evaluate the quality and speed
of existing background subtraction algorithms on a complex background with different everyday motions, and to compare the results with those of the extracted “Ground Truth”
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Complex BackgroundStatic Background:-Background does not contain any secondary “unwanted” motion. Controlled environment.
Complex Background:-Background contains secondary “unwanted” motion such as the winds effect on trees or blinds.Real-world data.
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Synthetic Test DataAdvantages:
• Automatically got the “Ground Truth”.• More control over each test clip.
Disadvantages:• Manual frame by frame “Ground Truth”
extraction.• Added artefacts from the Chroma keying
and compositing.
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
The Experiment To Create a set of synthetic data with
the “Ground Truth”
To test different motions with each background subtraction algorithm
To Compare the results of each algorithm with that of the “Ground Truth”
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
The Motions 7 everyday motions were chosen:
DrinkingJoggingPicking up walletScratching headSitting downStanding upWalking
Each motion started on the left of the screen and concluded on the right.
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed cameraCreating the test scenarios
Green Screen
Back Ground
Green Screen with actor
Final Composite “Ground Truth”
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Back Plate Difference
│framei – backplate│>Ts
The Algorithms
50
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Frame Difference
│framei – framei-1│>Ts
The Algorithms
50
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Approximate median
(x = ( framei - framei-1 – framei-2 . . .framei-n ) >
Ts )→ {background += 1}→ {background -= 1}
The Algorithms
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Mixture of Gaussians
frame(it = μ) = Σi=1 ωi,t .ț(μ,o)
The Algorithms
k
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Measuring the QualityCompare the Per-Pixel value ofeach frame with the “Ground Truth”
(0,0) (768,0)
(768,576)(0,576)
(0,0) (768,0)
(768,576)(0,576)
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Results - Quality
Test MotionsBackplate Difference Frame Difference Approximate Median Mixture of Gaussian% of image # of pixels % of image # of pixels % of image # of pixels % of image # of pixels
Drinking 90.78% 401577.3019 82.12% 363282.5031 89.52% 396024.7107 83.78% 370625.2327
Jogging 88.24% 390349.3529 88.88% 393194.9412 92.14% 407602.3824 88.20% 390146.7941
Picking up Wallet 91.26% 403717.114 88.22% 390256.9035 83.40% 368940.5088 90.19% 398979.9737
Scratch head 88.18% 390065.7255 84.87% 375422.2549 90.56% 400599.9216 86.15% 381117.049
Sitting down 88.51% 391528.6796 80.07% 354204.932 82.28% 363994.2039 81.68% 361327.3981
Standing up 89.40% 395491.6311 83.82% 370787.165 80.99% 358290.4563 83.78% 370631.6893
Walking 88.47% 391373.5094 89.81% 397309.3396 94.22% 416820.1321 90.01% 398195.3396
Most correct pixels Most incorrect pixels
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Results - Quality
Drinking Jogging Picking up Wallet
Scratch head Sitting down Standing up Walking70.00%
75.00%
80.00%
85.00%
90.00%
95.00%
100.00%
Percent of correctly identified pixels
Backplate DifferenceFrame DifferenceApproximate MedianMixture of Gaussian
Test Motions
Aver
age
Perc
ent o
f cor
rect
ly id
entif
ied
pixe
ls p
er fr
ame
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Results - Speed
Test MotionsBackplate Difference(Average of 100 times)
Frame Difference(Average of 100 times)
Approximate Median(Average of 100 times) Mixture of Gaussian
Drinking 0.0507 0.0004 0.3301 10.6954
Jogging 0.0507 0.0025 0.0691 10.8219
Picking up Wallet 0.0492 0.0819 0.0730 12.2895
Scratch head 0.0450 0.0850 0.0718 10.6132
Sitting down 0.0420 0.0692 0.0662 10.8503
Standing up 0.0416 0.0747 0.0529 12.7196
Walking 0.0319 0.0129 0.0541 10.5202
“Fastest” Algorithm “Slowest “Algorithm
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Results - Speed
Drinking Jogging Picking up Wallet
Scratch head Sitting down Standing up Walking0.0000
2.0000
4.0000
6.0000
8.0000
10.0000
12.0000
14.0000
Average time to process per frame
Backplate DifferenceFrame DifferenceApproximate MedianMixture of Gaussian
Test Motions
Aver
age
proc
essi
ng ti
me
per f
ram
e in
Sec
onds
(run
100
tim
es)
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Results - Speed
Drinking Jogging Picking up Wallet
Scratch head Sitting down Standing up Walking0.0000
0.0500
0.1000
0.1500
0.2000
0.2500
0.3000
0.3500
Average time to process per frame
Backplate DifferenceFrame DifferenceApproximate Median
Test Motions
Aver
age
proc
essi
ng ti
me
per f
ram
e in
Sec
onds
(run
100
tim
es)
...now ignoring the Mixture of Gaussian speed results
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Conclusion Backplate difference was the fastest and
produce the highest results in 4 out of 7 tests.
Frame difference was the ONLY algorithm to correctly remove the complex background, but couldn't correctly identify the foreground element.
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
ConclusionFrame Difference :-
Correctly Removed Complex BackgroundIncorrectly Removed inside of Subject
Backplate Difference :-Correctly Identified SubjectIncorrectly kept Complex Background
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Taking it furtherA new method that incorporated both thespeed of updating to remove thebackground and yet the knowledge of thebackground to properly extract the wantedforeground element.
Theory Framework idea:
Frame Difference Backplate Difference
ƒComplex background removed
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Where can this lead? Application of this technology could be
used in:
GamesSurveillanceMesh reconstruction and silhouette
extractionVarious computer vision tasks
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Any Questions?
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Acknowledgments
UK Engineering and Physical Science Research Council
Seth Benton for his Matlab code
Geoffrey Samuel www.GeoffSamuel.com
Comparison of complex background subtraction algorithms using a fixed camera
Thank you for your time
www.GeoffSamuel.com