Patch-Cuts: A Graph-Based Image Segmentation Method Using … · 2010-08-10 · Patch-Cuts: A...
1
Patch-Cuts: A Graph-Based Image Segmentation Method Using Patch Features and Spatial Relations Gerd Brunner 12 [email protected] Deepak R. Chittajallu 1 [email protected] Uday Kurkure 1 [email protected] Ioannis A. Kakadiaris 1 [email protected] 1 Computational Biomedicine Lab, Dept. of Computer Science, University of Houston, Houston, TX, USA 2 Section of Atherosclerosis and Vascular Medicine, Department of Medicine, Baylor College of Medicine, Houston, TX, USA 1 Background and Objective In this paper, we present a graph-based image segmentation method (patch- cuts) that incorporates features and spatial relations obtained from image patches. Without doubt, graph-based methods have advanced our under- standing of image segmentation [3]. The smoothness energy term in most graph-cuts methods is based on pixel intensities only. It is known that pixel intensities can be locally erroneous due to noise and other image acquisition problems. Thus, in these cases, noise may adversely affect the performance of graph-based segmentation methods. In this paper, we pro- pose patch-cuts, a graph-based segmentation method that: i) incorporates image patches in graph cuts; ii) introduces a tunable energy function that consists of intensity, shape, texture, and spatial terms; and iii) integrates a robust class of information theory energy terms. 2 Method: Patch-Cuts Patch Extraction: Patch-cuts incorporates aggregated pixel informa- tion, in the form of patches and their properties, into the formalism of graph-based segmentation using the following steps. An image Q(x; y) is split into K non-overlapping patches P = f p k j k = 1; 2; :::; Kg. The number of patches is determined by the number of quantized gray scale regions. Patch-cuts computes for each patch p k a set of Haralick [2] gray level co-occurrence matrix (GLCM) features: 1) correlation (ϒ); 2) en- ergy (Γ); 3) contrast (Δ); and 4) homogeneity (Ξ). Patch-Based Graph Representation: The proposed method builds a weighted undirected graph. The actual segmentation is obtained by par- titioning the graph [1, 3]. Specifically, U L (L k ) determines the energy to assign label L k to the k th patch. The energy function to be minimized consists of U L (L k ) and the sum of the four subsequent patch-based terms U P (L k ; L o ). E (L )= K ∑ k=1 U L (L k )+ K ∑ k=1 ∑ o2N k U P (L k ; L o ): The second term U P (L k ; L o ) computes the cost of assigning the labels L k ; L o to the neighboring patches k and o and N k is the neighborhood system for patch k. Patch-cuts incorporates the Kullback-Leibler and Jensen-Shannon divergence in the energy function. consists of four ad- ditive terms. Probability Distributions of Object and Background Regions: Patch- cuts uses the negative log-likelihood of the intensity probability distribu- tion as data term: U L (F )= ln p(H I k jF ) (foreground) and U L (B)= ln p(H I k jB) (background). Tunable Energy Function: Patch-cuts takes advantage of extracting re- gion properties which single pixels cannot provide, such as intensity, tex- ture, shape and spatial features. Incorporating region properties in the energy function positions patch-cuts as a highly adjustable graph-cut seg- mentation algorithm. The four energy terms incur costs based on indepen- dent characteristics of a region. We propose a tunable patch-based energy function U P (L k ; L o ) which incorporates intensity, texture, shape, and spatial distance information. We incorporate descriptors obtained from all patches in the energy function. The complete energy function E (L) is defined as E (L)= K ∑ k=1 U L (L k )+ w I K ∑ k=1 ∑ o2N k U I (L k ; L o )+ w T K ∑ k=1 ∑ o2N k U T (L k ; L o ) + w S K ∑ k=1 ∑ o2N k U S (L k ; L o ) 1 w D K ∑ k=1 ∑ o2N k U D (L k ; L o ) ; Figure 1: Sample patch-cuts segmentation results. First column: images with user-supplied fore- and background seed points; second column: images illustrat- ing the patches; third column: binary images showing the patch boundaries; and forth column: the segmentation results using patch-cuts with all weights=1. The number of patches, K is 1,709, 2,929, and 1,220 for the first, second, and third image, respectively. Best if viewed in color. Figure 2: Patch-cuts segmentation results for an axial non-contrast cardiac CT scan, near the aortic root. From the left: seed points, the patch image, the patch boundaries, and the segmentation result with all weights=1. K=11,333. Best if viewed in color. where U L (L k ) is the data term, U I (L k ; L o ) captures the intensity infor- mation, U S (L k ; L o ) denotes the shape energy, U T (L k ; L o ) represents the texture energy and U D (L k ; L o ) refers to the spatial energy. w I , w T , w S and w D denote the weights for the intensity, texture, shape and distance energy terms, respectively. Each weight is bounded by [0; 1]. 3 Experiments Figure 1 shows results of patch-cuts for different images. The weights w I , w T , w S , and w D were set to 1. Figure 2 shows the segmentation of an axial cardiac Computed Tomography (CT) scan acquired near the aortic root. In this case, all weights=1. Patch-cuts decomposed the scan into 11,333 patches. The fourth image shows the segmentation of the heart. 4 Conclusion We present patch-cuts, a novel graph-based image segmentation method. Patch-cuts introduces a tunable energy function that comprises intensity, shape, texture, and spatial terms. The initial experimental results are en- couraging and show that patch-cuts is robust with respect to noise and can be successfully applied to general images as well as to cardiac CT scans. [1] Y.Y. Boykov and M.P. Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In Proc. 8 th IEEE Interna- tional Conference on Computer Vision, volume 1, pages 105–112, Vancouver, Canada, Jul. 7-14 2001. [2] R.M. Haralick, K. Shanmugam, and I. Dinstein. Textural features for image classification. IEEE Transactions on Systems, Man and Cybernetics, 3(6): 610–621, 1973. [3] V. Kolmogorov and R. Zabin. What energy functions can be minimized via graph cuts? IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):147–159, 2004.
Transcript of Patch-Cuts: A Graph-Based Image Segmentation Method Using … · 2010-08-10 · Patch-Cuts: A...
Patch-Cuts: A Graph-Based Image Segmentation Method Using Patch Features and SpatialRelations
Gerd Brunner12
Deepak R. Chittajallu1
Uday Kurkure1
Ioannis A. Kakadiaris1
1 Computational Biomedicine Lab, Dept. of Computer Science,University of Houston, Houston, TX, USA
2 Section of Atherosclerosis and Vascular Medicine, Department ofMedicine, Baylor College of Medicine, Houston, TX, USA
1 Background and ObjectiveIn this paper, we present a graph-based image segmentation method (patch-cuts) that incorporates features and spatial relations obtained from imagepatches. Without doubt, graph-based methods have advanced our under-standing of image segmentation [3]. The smoothness energy term in mostgraph-cuts methods is based on pixel intensities only. It is known thatpixel intensities can be locally erroneous due to noise and other imageacquisition problems. Thus, in these cases, noise may adversely affect theperformance of graph-based segmentation methods. In this paper, we pro-pose patch-cuts, a graph-based segmentation method that: i) incorporatesimage patches in graph cuts; ii) introduces a tunable energy function thatconsists of intensity, shape, texture, and spatial terms; and iii) integrates arobust class of information theory energy terms.
2 Method: Patch-CutsPatch Extraction: Patch-cuts incorporates aggregated pixel informa-tion, in the form of patches and their properties, into the formalism ofgraph-based segmentation using the following steps. An image Q(x,y)is split into K non-overlapping patches P = {pk ∣ k = 1,2, ...,K}. Thenumber of patches is determined by the number of quantized gray scaleregions. Patch-cuts computes for each patch pk a set of Haralick [2] graylevel co-occurrence matrix (GLCM) features: 1) correlation (ϒ); 2) en-ergy (Γ); 3) contrast (∆); and 4) homogeneity (Ξ).
Patch-Based Graph Representation: The proposed method builds aweighted undirected graph. The actual segmentation is obtained by par-titioning the graph [1, 3]. Specifically, UL(Lk) determines the energy toassign label Lk to the kth patch. The energy function to be minimizedconsists of UL(Lk) and the sum of the four subsequent patch-based termsUP(Lk,Lo).
E(L ) =K
∑k=1
UL(Lk)+K
∑k=1
∑o∈Nk
UP(Lk,Lo).
The second term UP(Lk,Lo) computes the cost of assigning the labelsLk,Lo to the neighboring patches k and o and Nk is the neighborhoodsystem for patch k. Patch-cuts incorporates the Kullback-Leibler andJensen-Shannon divergence in the energy function. consists of four ad-ditive terms.
Probability Distributions of Object and Background Regions: Patch-cuts uses the negative log-likelihood of the intensity probability distribu-tion as data term: UL(F ) = −ln p(HI
k ∣F ) (foreground) and UL(B) =−ln p(HI
k ∣B) (background).
Tunable Energy Function: Patch-cuts takes advantage of extracting re-gion properties which single pixels cannot provide, such as intensity, tex-ture, shape and spatial features. Incorporating region properties in theenergy function positions patch-cuts as a highly adjustable graph-cut seg-mentation algorithm. The four energy terms incur costs based on indepen-dent characteristics of a region. We propose a tunable patch-based energyfunction UP(Lk,Lo) which incorporates intensity, texture, shape, andspatial distance information. We incorporate descriptors obtained fromall patches in the energy function. The complete energy function E(L) isdefined as
E(L) =K
∑k=1
UL(Lk)+
(wI
K
∑k=1
∑o∈Nk
UI(Lk,Lo)+wT
K
∑k=1
∑o∈Nk
UT (Lk,Lo)
+ wS
K
∑k=1
∑o∈Nk
US(Lk,Lo)
)∗(
1−wD
K
∑k=1
∑o∈Nk
UD(Lk,Lo)
),
Figure 1: Sample patch-cuts segmentation results. First column: images withuser-supplied fore- and background seed points; second column: images illustrat-ing the patches; third column: binary images showing the patch boundaries; andforth column: the segmentation results using patch-cuts with all weights=1. Thenumber of patches, K is 1,709, 2,929, and 1,220 for the first, second, and thirdimage, respectively. Best if viewed in color.
Figure 2: Patch-cuts segmentation results for an axial non-contrast cardiac CTscan, near the aortic root. From the left: seed points, the patch image, the patchboundaries, and the segmentation result with all weights=1. K=11,333. Best ifviewed in color.
where UL(Lk) is the data term, UI(Lk,Lo) captures the intensity infor-mation, US(Lk,Lo) denotes the shape energy, UT (Lk,Lo) represents thetexture energy and UD(Lk,Lo) refers to the spatial energy. wI , wT , wSand wD denote the weights for the intensity, texture, shape and distanceenergy terms, respectively. Each weight is bounded by [0,1].
3 Experiments
Figure 1 shows results of patch-cuts for different images. The weights wI ,wT , wS, and wD were set to 1. Figure 2 shows the segmentation of an axialcardiac Computed Tomography (CT) scan acquired near the aortic root.In this case, all weights=1. Patch-cuts decomposed the scan into 11,333patches. The fourth image shows the segmentation of the heart.
4 ConclusionWe present patch-cuts, a novel graph-based image segmentation method.Patch-cuts introduces a tunable energy function that comprises intensity,shape, texture, and spatial terms. The initial experimental results are en-couraging and show that patch-cuts is robust with respect to noise and canbe successfully applied to general images as well as to cardiac CT scans.
[1] Y.Y. Boykov and M.P. Jolly. Interactive graph cuts for optimal boundary andregion segmentation of objects in N-D images. In Proc. 8th IEEE Interna-tional Conference on Computer Vision, volume 1, pages 105–112, Vancouver,Canada, Jul. 7-14 2001.
[2] R.M. Haralick, K. Shanmugam, and I. Dinstein. Textural features for imageclassification. IEEE Transactions on Systems, Man and Cybernetics, 3(6):610–621, 1973.
[3] V. Kolmogorov and R. Zabin. What energy functions can be minimized viagraph cuts? IEEE Transactions on Pattern Analysis and Machine Intelligence,26(2):147–159, 2004.
