Maximally Stable Extremal Regions and Extensions
description
Transcript of Maximally Stable Extremal Regions and Extensions
Maximally Stable Extremal Regions and Extensions
Loris Bazzani, PhD Student
Department of Computer Science, University of Verona, Italy,
VIPS Lab.
Medical Image Processing Course
Supervisor: Prof. Vittorio Murino
Introduction
I. Maximally Stable Extremal Region
II. Maximally Stable Volume: 3D Extensiono Segmentation of volumes
III. Maximally Stable Colour Region: RGB Extensiono Objects of interest modeling
IV. Conclusions
Outline
I. Maximally Stable Extremal Region
II. Maximally Stable Volume
III. Maximally Stable Colour Region
IV. Conclusions
Maximally Stable Extremal Region(MSER) [Matas2002]
• Set of all thresholdings of to a binary img:
• MSER = connected region in with little size change across several thresholdings
• Margin = the number of thresholds for which the region is stable
MSER (1)
1
2
3
4
5
[Images from Matas’ presentation]
Math. Details
Outline
I. Maximally Stable Extremal Region
II. Maximally Stable Volume
III. Maximally Stable Colour Region
IV. Conclusions
Maximally Stable Volumes (MSV) [Donoser2006]
•Find the level sets of a connected, weighted graph
• Node: pixel
• Edge: connection relationship (e.g. 4-neghborhood)
• Weight: pixel intensity
• contains a set of nodes that have a weight above a given threshold
• Build a component tree from a connected, weighted graph
• Nodes: the connected components of
• Edges: inclusion relationship between and
New interpretation/formulation of MSER (2D):
MSV (1)
Extension to the third dimension: spatial or temporal
•Find the level sets of a connected, weighted graph
• Node: voxel
• Edge: 3D connection relationship (e.g. 6-neghborhood)
• Weight: voxel intensity
• contains a set of nodes that have a weight above a given threshold
• Build a component tree from a connected, weighted graph
• Nodes: the connected volumes of
• Edges: inclusion relationship between and
MSV (2)
• A connected volume fulfills:
is the set of all boundary voxels of a volume
•A connected volume is son of iff
i.e., an inclusion relationship between connected volumes
MSV (3)
• MSVs are identified as the connected volumes with high stability:
• Local minimum along the path to the root of the tree
• Computation of the tree:– number of edges + nodes– inverse Ackermann function
3D segmentation (1)
• Applied to simulated brain MR images• Size: , with different noise
MSV detection result of brain segmentation.
Images from [Donoser2006]
3D segmentation (2)3D visualization of human brain,which was detected as a single MSV
Images from [Donoser2006]
3D segmentation (3)• Applied to paper fiber network images
• Sequences of cross-sectional images with max resolution of
Images from [Donoser2006]
3D segmentation (4)Segmented fiber detected as MSV
Images from [Donoser2006]
Outline
I. Maximally Stable Extremal Region
II. Maximally Stable Volume
III. Maximally Stable Colour Region
IV. Conclusions
Maximally Stable Colour Region (MSCR) [Forssen2007]
Original set of images MSCR representation
• Novel colour-based affine covariant region detector
• Extension of the MSER to colour
•Look at successive time-steps of an aggloramerative clustering of image pixel, based on proximity and similarity on colour
•Modelling of the distribution of edge magnitudes
•Novel edge significance measure based on a Poisson image noise model
• Perform better than MSER and other state-of-the-art blob detectors
• Applications: 3D object recognition and view matching
MSCR (1)• Evolution process over the image
that successively clusters neighbouring pixels with similar colours
• For each time step , the evolution is a map of labels
• Any two positions are connected by a path of distances which are smaller than
MSCR (2)
• is all zeroes• is constructed from by assigning
new regions to all pair of pixel with• If one pixel of the pair already belongs to a
region, the non-assigned pixel is appended to the region
• If both pixels belong to regions the corresponding regions are merged
Evolution Process with agglomerative clustering
MSCR (3)• How the distance is defined:
• Sensors count the number of photons• Noise follows the discrete Poisson distribution • For high , good approximation is a Gaussian:
• Measure of edge significance: probability that a pixel has a larger mean than its neighbour:
Chi-squared distance
MSCR (4)• Dynamically adapt the threshold :
• Linearly increasing: very fast image evolution in the beginning and very slow at the end of the evolution
• Change according to the inverse Cumulative Distribution Function (CDF)
• Observation: edge significance measure follows a Chi-squared distribution:
• Evolution thresholds:
MSCR (5)
• Detecting stable regions:– For each region in the label image, we store the area
and the distance threshold – When the area increases more than a threshold , and are re-initialized– The slope of the area and distance function is used
for the detection
if is the best (smallest), the region is stored
MSCR (6)
• Descriptor for the MSCRs:– Region area – Centroid– Inertia Matrix– Average colour
• These measures define an approximating ellipse for the detected region as:
Tracking-by-detection (1)• Tracking: spatial and temporal localization of a mobile
object in an environment monitored by sensor(s)• Multi-target (MTT): keeping the identity of
different targets• Reliable: insensible to noise and occlusions
• Detection: identify all the objects of interest into the image
• Tracking-by-detection: • targets are detected for every frame• IDs are associated from frame (t-1) to frame (t), with
a data association process
Tracking-by-detection (2)• Tracking-by-detection using the MSCR
descriptor• Our method extracts the MSCR from the foreground of
the detected objects• We define a distance measurement in order to compare
the objects at time (t-1) with the objects at time (t)• For each pair of blobs, we have:
• Color distance: • y distance:
• Distance between the objects :
Euclidean distance
Qualitative Results (1)
Image in the database
Probe ImageMSCR MSCR
Qualitative Results (2)
Quantitative ResultsTagging error
Rate for each t
Total Tagging Success Rate
Tagging error Rate for each N of ped
Person Re-identification (1)• Multi-camera scenario with (non-)overlapping
fields of View (FoV)– Objective: recognize an object, when it is being seen in different
FoV– Challenging problem with non-overlapping FoV
• Idea: – Keep a database of all the history of the seen objects– Once a new object enters in the scene, the method retrieves the
IDs of the object from the database (if it is being seen before)– If the object is not in the database, a new ID is given to it and it is
added to the database
Person Re-identification (2)• The method is the same used for tracking-by-
detection problem• Compute the distance• Extraction of part-based HSV histogram
– Divide the image in three parts: legs, torso, head– Compare the hist. of each part using the
Bhattacharyya distance
• MSCR and HSV hist. distance are combined:
Quantitative Results (1)• Evaluation in term of:
– Cumulative Matching Characteristic (CMC): represents the expectation of finding the correct match in the top n matches
– Synthetic Recognition Rate (SRR): represents the probability that any of the m best matches is correct
• Using challenging publicly available datasets: VIPeR and iLIDS Dataset– pose variation and shape deformation– illumination changes, camera movement, and occlusions– noise and blurring
Quantitative Results (2)
CMC SRR
VIPeR dataset
Thank to M. Farenzena and C. Cristani
Quantitative Results (2)
CMC
iLIDS datasetMatching
Thank to M. Farenzena and C. Cristani
Conclusions
• Two extensions of the MSER feature had been discussed– MSV that deals with 3D segmentation and modeling
of medical images – MSCR that deals with hard problems in very different
applications: tracking-by-detection, and person re-identification
• MSER and extensions seem to be good features for representing and segmenting of object of interest in different kind of application
Thanks!Questions?
References
[Matas2002] J. Matas, O. Chum, M. Urban and T. Pajdla, Robust Wide Baseline Stereo from Maximally Stable Extremal Regions, In BMVC, 2002.
[Donoser2006] M. Donoser and H. Bischrof, 3D Segmentation by Maximally Stable Volumes (MSVs), In ICPR, 2006.
[Forssen2007] P. Forssen, Maximally Stable Colour Regions for Recognition and Matching, In CVPR, 2007.