CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren,...
-
date post
20-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren,...
![Page 1: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/1.jpg)
1 CVR05University of California Berkeley
Cue Integration in Figure/Ground LabelingCue Integration in Figure/Ground Labeling
Xiaofeng Ren, Charless Fowlkes, Jitendra MalikXiaofeng Ren, Charless Fowlkes, Jitendra Malik
![Page 2: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/2.jpg)
2 CVR05University of California Berkeley
IntroductionIntroduction
CRF
Conditional Random Fields on triangulated images, trained to integrate low/mid/high-level grouping cues
Approach:
![Page 3: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/3.jpg)
3 CVR05University of California Berkeley
Joint Contour/Region InferenceJoint Contour/Region Inference
Xe
Xe
Xe
Xe
Xe
Xe
Xe
Xe
Xe
XeXeXe
Xe
Xe
Xe
Xe
Xe
Xe
Yt
Yt
Yt
Yt
Yt
Yt
Yt
Yt
YtYt
Contour variables {Xe}
Region variables {Yt}
Object variables {Z}
Z
Integrating {Xe},{Yt} and{Z}: low/mid/high-level cues
![Page 4: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/4.jpg)
4 CVR05University of California Berkeley
Grouping CuesGrouping Cues• Low-level Cues
– Edge energy along edge e
– Brightness/texture similarity between two regions s and t
• Mid-level Cues– Edge collinearity and junction frequency at
vertex V
– Consistency between edge e and two adjoining regions s and t
• High-level Cues– Texture similarity of region t to exemplars
– Compatibility of region shape with pose
– Compatibility of local edge shape with pose
• Low-level Cues– Edge energy along edge e
– Brightness/texture similarity between two regions s and t
• Mid-level Cues– Edge collinearity and junction frequency at
vertex V
– Consistency between edge e and two adjoining regions s and t
• High-level Cues– Texture similarity of region t to exemplars
– Compatibility of region shape with pose
– Compatibility of local edge shape with pose
L1(Xe|I)L2(Ys,Yt|I)
M1(XV|I)
M2(Xe,Ys,Yt)
H1(Yt|I)H2(Yt,Z|I)H3(Xe,Z|I)
![Page 5: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/5.jpg)
5 CVR05University of California Berkeley
Conditional Random Fields for Cue IntegrationConditional Random Fields for Cue Integration
,,|,exp),(
1,,, IZYXE
IZIZYXP
ts
tse
e IYYLIXLE,
21 |,|
ts
etsV
V XYYMIXM,
21 ,,|
e
et
tt
t IZXHIZYHIYH |,|,| 321
Estimate the marginal posteriors of X, Y and Z
![Page 6: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/6.jpg)
6 CVR05University of California Berkeley
H3(Xe,Z|I): local shape and poseH3(Xe,Z|I): local shape and pose
shapeme i(horizontal line) distribution ON(x,y,i)
shapeme j(vertical pairs) distribution ON(x,y,j)
Let S(x,y) be the shapeme at image location (x,y); (xo,yo) be the object location in Z. Compute average log likelihood SON(e,Z) as:
eyx
oo yxSyyxxONe ,
),(,,log1
eOFFOFF
eONONe
XZeS
XZeSIZXH
),(
),(|,3
Then we have:
SOFF(e,Z) is defined similarly.
![Page 7: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/7.jpg)
7 CVR05University of California Berkeley
Training/TestingTraining/Testing• Trained on half (172) of the grayscale horse images from
the [Borenstein & Ullman 02] Horse Dataset.
• Use human-marked segmentations to construct groundtruth labels on both CDT edges and triangles.
• Uses loopy belief propagation for approximate inference; takes < 1 second to converge for a typical image.
• Parameter estimation with gradient descent for maximum likelihood; converges in 1000 iterations.
• Tested on the other half of the horse images in grayscale.
• Quantitative evaluation against groundtruth: precision-recall curves for both contours and regions.
• Trained on half (172) of the grayscale horse images from the [Borenstein & Ullman 02] Horse Dataset.
• Use human-marked segmentations to construct groundtruth labels on both CDT edges and triangles.
• Uses loopy belief propagation for approximate inference; takes < 1 second to converge for a typical image.
• Parameter estimation with gradient descent for maximum likelihood; converges in 1000 iterations.
• Tested on the other half of the horse images in grayscale.
• Quantitative evaluation against groundtruth: precision-recall curves for both contours and regions.
![Page 8: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/8.jpg)
8 CVR05University of California Berkeley
![Page 9: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/9.jpg)
9 CVR05University of California Berkeley
![Page 10: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/10.jpg)
10 CVR05University of California Berkeley
ResultsResults
Input Input Pb Output Contour Output Figure
![Page 11: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/11.jpg)
11 CVR05University of California Berkeley
Input Input Pb Output Contour Output Figure
![Page 12: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/12.jpg)
12 CVR05University of California Berkeley
Input Input Pb Output Contour Output Figure
![Page 13: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/13.jpg)
13 CVR05University of California Berkeley
ConclusionConclusion
![Page 14: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/14.jpg)
14 CVR05University of California Berkeley
Thank You
![Page 15: CVR05 University of California Berkeley 1 Cue Integration in Figure/Ground Labeling Xiaofeng Ren, Charless Fowlkes, Jitendra Malik.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d4d5503460f94a2cb3b/html5/thumbnails/15.jpg)
15 CVR05University of California Berkeley