Video Segmentation
-
Upload
smriti-jain -
Category
Engineering
-
view
210 -
download
1
Transcript of Video Segmentation
![Page 1: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/1.jpg)
Video SegmentationLabeling independently moving image regions
![Page 2: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/2.jpg)
2
Motion SegmentationForeground (object) and Background (noise)
Result could be a Binary image, containing foreground onlyProbability image, containing the likelihood of each
pixel being foreground
ApproachesMotion-based (optical flow)Color-basedTexture-based
![Page 3: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/3.jpg)
Motion (Change) Detection
Static camera : changed and unchanged regions.Moving camera : global and local motion regions.Limitations associated with motion estimationAperature problem : pixels in a flat image region
may appear stationary even if they are moving as a result of an aperture problem (hence the need for hierarchical methods)
Occlusion Problem : erroneous labels may be assigned to pixels in covered or uncovered image regions as a result of an occlusion problem.
![Page 4: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/4.jpg)
Motion Detection: using two frames : Image Differencing
![Page 5: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/5.jpg)
5
Subtract ImageCompute pixel-wiseSubtract previous image from input image:
Usually the absolute distance is applied
),(),(),( yxByxIyxF
),( yxF
1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise
![Page 6: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/6.jpg)
1
2
3
4
5
6
7
8
1
2
5
4
3
6
7
8
9
10
![Page 7: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/7.jpg)
7
ThresholdDecide, when a pixel is supposed to be
considered as a background pixel, or when it is to be considered as a foreground pixel:
Pixel is foreground pixel, if
Pixel is background pixel, if
Problem: What TH?!?
),( yx
THyxF ),(),( yx
THyxF ),(
1. Save image in last frame2. Capture camera image3. Subtract image4. Threshold5. Delete noise
![Page 8: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/8.jpg)
1
2
3
4
5
6
7
8
![Page 9: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/9.jpg)
125 4 3678
Frame no. 1 2 3 4 5 6 7 8 9
x 154 160 160 160 157 130 112 79 0
y 13 19 19 19 16 19 19 19 0
w s s w w w w w
![Page 10: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/10.jpg)
Dominant Motion : no optical flowSpatio temporal intensity gradient.Dominant motion segmentation
fitting a single parametric motion modelpartition the frame in two pixel groupsRepeat step 1 only to well represented pixels
group.
![Page 11: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/11.jpg)
ThresholdingNoise Removal Median Filter
Image subtraction
Bounding Box Centroid
![Page 12: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/12.jpg)
Multiple Motion SegmentationMultiple motion models compete against each
other at each decision site. They consist estimating:Motion within each region (motion model)spatial support of each regionnumber of regions.
The problem : associate each pixel to the right motion model, while simultaneously estimating motions and supports. Clustering (K-means, hough transform),Maximum Likelihood (ML) (pixel based) and Maximum A Posteriori probability (MAP).Region based label assignment
![Page 13: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/13.jpg)
Optical flow estimationmotion vectors at each frame
Thresholding Morphological closing on the motion
vectors
![Page 14: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/14.jpg)
Object tracking using kalman filters (prediction)
![Page 15: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/15.jpg)
15
Motion ModelPredicted position at time t:
Brownian Motion: According to a Gaussian model
0’th order:
1’th order:Similar for y
2’th order Similar for y
),( tt yx
),(),( 11 tttt yxyx
1 at time in velocity :1
11
t-xx
xtxx
t
ttt
1 at time in on accelerati :2
1
1
112
1
t-xx
xtxtxx
t
tttt
![Page 16: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/16.jpg)
16
Color Based segmentation : Background Subtraction
Use Neighborhood relation!! Compare pixel with its neighbors!! Weight them!!
Learn the background and its variations!!E.g. Gaussian models (mean,var) for each pixel!!!E.g. a Histogram for each Pixel The more images you train on the better!! Algorithm:
Consider each pixel (x,y) in the input image and check, how much it varies with respect to the mean and variance of the learned Gaussian models?
1. Calculate mean and variance for each pixel
2. Capture camera image3. Subtract image (= motion)4. Weight the distances (new)5. Threshold according to variance6. Delete noise
![Page 17: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/17.jpg)
Color Segmentation with Histograms
![Page 18: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/18.jpg)
18
Color Segmentation with Histogramsbrightness
![Page 19: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/19.jpg)
19
Color Segmentation with Gaussian Distribution
N(m, C)
![Page 20: Video Segmentation](https://reader030.fdocuments.net/reader030/viewer/2022032505/55c52dbfbb61eb7c638b466c/html5/thumbnails/20.jpg)
Thank you