Allan Hanbury Mathematical Morphology: from Erosion to Watersheds PRIP, Vienna University of...

Allan HanburyAllan Hanbury Allan HanburyAllan Hanbury

Mathematical Morphology: from Erosion to Watersheds

Mathematical Morphology: from Erosion to Watersheds

PRIP, Vienna University of TechnologyFavoritenstraße 9/1832A-1040 Vienna, Austria

[email protected]://www.prip.tuwien.ac.at/~hanbury

PRIP, Vienna University of TechnologyFavoritenstraße 9/1832A-1040 Vienna, Austria

[email protected]://www.prip.tuwien.ac.at/~hanbury

mailto:[email protected]












ContentsContents

Introduction Erosion and Dilation Opening and Closing Binary Geodesic Operators Numerical Geodesic Operators SKIZ and Watershed Applications


Image Processing (I)Image Processing (I)

=> Image processing may be divided into three classes of questions, namely Codification, Extraction of characteristics and Segmentation.

1) Codification:Codification concerns the representation of the images. It comprises:

Acquisition: Transformation of analogue images into numerical ones.

Compression: Modification of image representation.

Synthesis: Generation of an image from a more symbolic representation.

Analog ImageNumerical Image

Acquisition

NumericalImage n° 2

Compression

NumericalImage n° 1

Synthesis

Numerical Image Numbers

Image Processing (II)Image Processing (II)

2) Extraction of Characteristics:

The aim here is to improve image quality or to exhibit some of its features. This includes in particular measurements, noise reduction, and filtering.

3) Segmentation:

Segmentation consists in partitioning the images into zones which are homogeneous according to a given criterion.

Image2Image1

ImageTranformations

Numbers

Pixels RegionsSegmentation

Definitions of Mathematical MorphologyDefinitions of Mathematical Morphology

Mathematics Physics

Engineering

Lattice theory for objects or operators in continuous or discrete spaces;Topological and stochastic models.

Signal analysis techniques based on set theory aiming at the study of relations between physical and structural properties.

Algorithms and software / hardware tools for developing signal processing applications.

Signal Processing

Nonlinear signal processing technique based on minimum and maximum operations.

Extensivity and Anti-extensivity: A transformation is extensive if its output is always greater than its input. It is anti-extensive when the output is always smaller than the input.

Extensivity : X (X) Anti-extensivity : X (X) X

Set (extensivity) Function (extensivity)

Input Output Comparison (I)Input Output Comparison (I)

X

(X)

f(f)

Idempotence: A transformation is idempotent if its output is invariant with respect to the transformation itself:

Idempotence: [ (X) ] = (X)

Increasingness: A transformation is increasing if it preserves the ordering relation between images:

Increasing : f , g, f g ( f ) ( g )

Input Output Comparison (II)Input Output Comparison (II)

ContentsContents



• A structuring element (SE) is a small set used to probe the image under study.

• For the Erosion of a set, we ask the question “Where does the structuring element fit the set?”

• The erosion of a set X by a structuring element B is denoted by

B(X) and is

defined as the locus of points x such that B is included in X when its origin is placed at x:

B(X) = XB = { x: Bx X }

• A structuring element (SE) is a small set used to probe the image under study.

• For the Erosion of a set, we ask the question “Where does the structuring element fit the set?”

• The erosion of a set X by a structuring element B is denoted by

B(X) and is

defined as the locus of points x such that B is included in X when its origin is placed at x:

B(X) = XB = { x: Bx X }

ErosionErosion

Initial set X

Erosion of X

Image size 256 256 pixelsSE size 15 15 pixels

• Dilation is the dual operator of erosion.

• It is based on the question “Where does the structuring hit the set?”

• The dilation of a set X by a structuring element B is denoted by

B(X) and is

defined as the locus of points x such that B hits X when its origin coincides with x:

B(X) = XB = { x: Bx X }

• Dilation is the dual operator of erosion.

• It is based on the question “Where does the structuring hit the set?”

• The dilation of a set X by a structuring element B is denoted by

B(X) and is

defined as the locus of points x such that B hits X when its origin coincides with x:

B(X) = XB = { x: Bx X }

DilationDilation

Initial set X

Dilation of X


Equivalence between Sets and Functions

Equivalence between Sets and Functions

A function can be viewed as a stack of decreasing sets. Each set is the intersection between the umbra of the function and a horizontal plane.

X (f) = { xE , f(x) } f(x) = sup {xX (f) }

Function

Function

Sets

Function => Sets Sets => Function

Stack of sets

Dilation and Erosion by a flat structuring Element

Dilation and Erosion by a flat structuring Element

Definition: The dilation (erosion) of a function by a flat structuring element B is introduced as the dilation (erosion) of each set Xf

by B. They are said to be planar.

This definition leads to the following formulae :

( fB) (x) = sup{ f(xy), yB }

( fB) (x) = inf { f(xy), yB }

• Erosion shrinks positive peaks. Peaks thinner that the structuring element disappear. It also expands the valleys and the sinks.

• Dilation produces the dual effects.

8060402000

10

20

30

40

50

60

Space

grey levels

Original

Dilate

Eroded

Structuring element

Greyscale Dilation and Erosion examples

Greyscale Dilation and Erosion examples

Initial imageDilate

Erode


Properties of Dilation and Erosion (I)

Properties of Dilation and Erosion (I)

• Dilation and Erosion are dual transformations with respect to complementation.

• This means that any erosion of an image is equivalent to a complementation of the dilation of the complemented image with the same structuring element (and vice-versa).

• In terms of symbols:B = B

• Dilation and Erosion are dual transformations with respect to complementation.

• This means that any erosion of an image is equivalent to a complementation of the dilation of the complemented image with the same structuring element (and vice-versa).

• In terms of symbols:B = B

B

B

Properties of Dilation and Erosion (II)

Properties of Dilation and Erosion (II)

• Dilation and Erosion are increasing transformations.

• Dilation and Erosion are increasing transformations.

gf

gfgf

Residues of TransformationsResidues of Transformations

Definition

• The residue between two transformations et is their difference

Set case:

Functions case: (X) = X) (X)

Residue

Residue

,(X) = (X) \ (X)

Morphological GradientsMorphological Gradients

Gradient by erosion:

• It is the residue between the identity and an erosion , i.e.:

for sets g (X) = X / (XB)

for functions g (f) = f (fB)

The goal of gradient transformations is to highlight contours. In digital morphology, three Beucher’s gradients based on the unit disc are defined:

Gradient by dilation:

• It is the residue between a dilation and the identity, i.e.:

for sets g+ (X) = (XB) / X

for functions g+ (f ) = (fB) f

Original

Erosion

Gradient

Original

Dilation

Gradient

Morphological Gradients (II) and Laplacian

Morphological Gradients (II) and Laplacian

Symmetrical gradient:

• It is the residue between a dilation and an erosion

for sets g (X) = (XB) / (XB)

for functions g(f) = (fB) (fB)

Laplacian:

• It is the residue between the gradients by dilation and erosion, for functions

L(f) = g+ (f ) g (f)

Note: These notions correspond the “classical” notions of gradient and Laplacian (if they exist), in the limit when the radius of the disc tends towards zero.

Original

Dilation

Gradient

ErosionOriginal

Dilation

Laplacian

Erosion

Morphological Gradient examples

Morphological Gradient examples

Initial image

Gradientby erosion


SymmetricalGradientGradient

by dilation

ContentsContents



The mapping is called opening, and is denoted by

XoB = [(X B)

By commuting the factors and we obtain the closing

XX B]

Opening and Closing Opening and Closing

The problem of an inverse operator

Several different sets may admit a same erosion, or a same dilate. But among all possible inverses, there always exists a smallest one (a largest one). It is obtained by composing erosion with the corresponding dilation (or vice versa) .

Ero

sion

structuringelement

?

Effects of OpeningEffects of Opening

Geometrical interpretation

• The opened set (X) is

the union of the structuring elements Bx which are included in set X.

When B is a disc, the opening reduces the capes, removes the small islands and opens isthmuses.

Structuring element

Opening

Effects of ClosingEffects of Closing

Geometrical interpretation

• The closing (X) is the complement of the domain swept by B as it misses set X.

• All background structures that cannot contain the structuring element are filled by the closing.

When B is a disc, the closing closes the channels, fills completely the small lakes, and partly the gulfs.

Structuring element

Closing

Effects on FunctionsEffects on Functions

• The opening and closing create a simpler function than the original. They smooth in a nonlinear way.

• The opening (closing) removes positive (negative) peaks that are thinner than the structuring element.

• The opening (closing) remains below (above) the original function.

8060402000

10

20

30

40

50

60

Sample

Level

Original

Closing

Opening

Structuring element

Greyscale Opening and Closing examples

Greyscale Opening and Closing examples

Initial imageClosing

Opening


Properties of Opening and ClosingProperties of Opening and Closing

• Opening and Closing are dual transformations with respect to complementation.

• Openings are anti-extensive transformations (some pixels are removed) and closings are extensive transformations (some pixels are added).

• Opening and closing are both increasing transformations.

• Opening and closing are both idempotent transformations:

and

• Opening and Closing are dual transformations with respect to complementation.

• Openings are anti-extensive transformations (some pixels are removed) and closings are extensive transformations (some pixels are added).

• Opening and closing are both increasing transformations.

• Opening and closing are both idempotent transformations:

and

gf

gfgf

Suprema of OpeningsSuprema of Openings

Theorem:• Any supremum of openings

is still an opening.• Any infimum of closings is

still a closing.

Theorem:• Any supremum of openings

is still an opening.• Any infimum of closings is

still a closing.

Application:

For creating openings with specific selection properties, one can use structuring elements with various shapes and take their supremum.

Application:

For creating openings with specific selection properties, one can use structuring elements with various shapes and take their supremum.

OriginalOpening by

Opening by

Opening by

sup.

Goal • The “top-hat” transformation, due to F. Meyer, aims to suppress

slow trends, therefore to enhance the contrasts of some features in images, according to size or shape criteria. This operator is used mainly for numerical functions.

Definition• The “top-hat” transformation is the residue between the identity

and an opening:

(f) = f (f) ( functions) ; (X) = X \ (X) (sets)

• A dual top-hat can be defined: the residue between a closing and the identity:

*(f) = (f) f ( functions) ; (X) = (X) \ X (sets)

“Top-hat” Transformation“Top-hat” Transformation

Use of the Top-hatUse of the Top-hat

Sets• The top-hat extracts the objects that

have not been eliminated by the opening. That is, it removes objects larger than the structuring element.

Functions• The top-hat is used to extract

contrasted components with respect to the background. The basic top-hat extracts positive components and the dual top hat the negative ones.

• Typically, top-hats remove the slow trends, and thus performs a contrast enhancement.

8060402000

20

40

60

80

100

Sample

Level

Opening:

Original

Top hat

Example of a Top-hatExample of a Top-hat

Negative image of the retina.

Top hat by ahexagonal opening of size 10.

Top hat by the sup of three openings by linear segments of size 10.

Comment: The goal is the extraction of the aneurisms (the small white spots). Top hat c), better than b) is far from being perfect. Here opening by reconstruction yields a correct solution.

ContentsContents



Geodesic DistanceGeodesic Distance

Definition

The set geodesic distance dX: nn , with respect to reference set X is defined by

dX(x,y) = Infimum of the lengths of the paths going from x to y and included X;

dX(x,y) = + if no such path exists.

Properties

1) dX is a generalized distance, sincedX(x,y) = dX(y,x)dX(x,y) = 0 x = ydX(x,z) dX(x,y) + dX(y,z)

2) The geodesic distance is always larger than the Euclidean one;

3) A geodesic segment may not be unique.

X

x

z

y

y'

Examples of geodesics in 2 :

N.B. the portions of geodesics included in the interior of X are line segments

Geodesic DiscsGeodesic Discs

• The notion of geodesic path is seldom used. By contrast, the notion of geodesic discs appears very often:

BX, (z) = {y, dX(z,y) }

• When the radius r increases, the discs progress as a wave front emitted from z inside the medium X.

• For a given radius , the discs BX, can be viewed as a set of structuring elements which vary from place to place.

z

X

BX, (z)

• The geodesic dilation of size of Y inside X is written as follows:

XYBX,

(y) ,yY

• The geodesic dilation of size of Y inside X is written as follows:

XYBX,

(y) ,yY

Geodesic DilationGeodesic Dilation

X

Y

geodesic dilation

Geodesic ErosionGeodesic Erosion

• The geodesic erosion is the dual transformation of the geodesic dilation with respect to set complementation.

• But the complement is taken inside the mask X (i.e. YX \ Y = X YC), which results in the erosion :

X(Y) = X \ X (X \ Y)i.e.

X(Y) = ( Y Xc ) X

where stands for an isotropic erosion of size 1.

• Note the difference between and X(Y) and (Y)X.

X

Y

(Y) X

Y(Y)

X

Geodesic erosion

• In the digital metrics on n , and when (x) stands for the unit ball centered at point x, then the unit geodesic dilation is defined by the relation :

XY

Y X

• The dilation of size n is then obtained by iteration :

X,n(Y) = X(n)Y, with

X(n)Y Y X X ... X

• X is called the mask, and Y the marker.

(Binary) Digital Geodesic Dilation(Binary) Digital Geodesic Dilation

X

Y

X(Y)

X(2)(Y)

Reconstruction by dilationReconstruction by dilation

• The reconstruction by dilation of a mask image X from a marker image Y is defined as the geodesic dilation of Y with respect to X until stability.

• This can be seen as an infinite geodesic dilation:

X(Y) = X,(Y) , or as an iteration of unit geodesic dilations until there is no further change:

RX(Y) = X(i)Y

where i is such that X(i)YX

(i+1)Y• This an opening as it is

• Increasing [ Y1 Y2 RX(Y1) RX(Y2) ]

• Anti-extensive [ RX(Y) X ]

• Idempotent [ RR (Y)(Y) = RX(Y) ]

• The reconstruction by dilation of a mask image X from a marker image Y is defined as the geodesic dilation of Y with respect to X until stability.

• This can be seen as an infinite geodesic dilation:

X(Y) = X,(Y) , or as an iteration of unit geodesic dilations until there is no further change:

RX(Y) = X(i)Y

where i is such that X(i)YX

(i+1)Y• This an opening as it is

• Increasing [ Y1 Y2 RX(Y1) RX(Y2) ]

• Anti-extensive [ RX(Y) X ]

• Idempotent [ RR (Y)(Y) = RX(Y) ]

X

Y

X

Application: Filtering by Erosion-ReconstructionApplication: Filtering by Erosion-Reconstruction

• Firstly, the erosion XBsuppresses the connected components of X that cannot contain a disc of radius This is then used as the marker Y.

• Then the opening RX(Y) of marker Y = XB“re-builds” all the others.

• Firstly, the erosion XBsuppresses the connected components of X that cannot contain a disc of radius This is then used as the marker Y.

• Then the opening RX(Y) of marker Y = XB“re-builds” all the others.

a) Initial image b) Eroded of a) by a disc

c) Reconstruction of b) inside a)

Application: Removal of the edge grainsApplication: Removal of the edge grains

• Let Z be the set of the edges, and X be the grains under study;

• Set Y is the reconstruction of Z€X inside set X ;

• the set difference between X and Y extracts the internal particles.

• Let Z be the set of the edges, and X be the grains under study;

• Set Y is the reconstruction of Z€X inside set X ;

• the set difference between X and Y extracts the internal particles.

X

Z

a) Initial image b) Particles thathit the edges

c) Residue a) - b)

Application: Hole FillingApplication: Hole Filling

initial image X

A = part of the edge that hits XC

reconstruction of A inside XC

Comment: efficient algorithm, except for the particles that hit the edges of the field.

Individual Analysis of ParticlesIndividual Analysis of Particles

Algorithm (J.C. Klein)

While set X is not empty do {

~ p := first point of the video scan;

~ Y := connected component of X reconstructed from p;

~ Processing of Y (and various measurements) ;

~ X := X \ Y

}

Algorithm (J.C. Klein)

While set X is not empty do {

~ p := first point of the video scan;

~ Y := connected component of X reconstructed from p;

~ Processing of Y (and various measurements) ;

~ X := X \ Y

}

X

Y new X

Individual extraction of particles

ContentsContents



Numerical Geodesic Dilations (I)Numerical Geodesic Dilations (I)

• Let f and g be two numerical functions from d into , with g f.

• The binary geodesic dilation of size of each cross section of g inside that of f at the same level induces on g a dilation f,(g) (S. Beucher).

• Function g is the marker, and f is the mask.

• Let f and g be two numerical functions from d into , with g f.

• The binary geodesic dilation of size of each cross section of g inside that of f at the same level induces on g a dilation f,(g) (S. Beucher).

• Function g is the marker, and f is the mask.

Numerical geodesic dilation of g with respect to f

f

g

f,(g)

Numerical geodesic Dilations (II)Numerical geodesic Dilations (II)

• The digital version starts from the unit geodesic dilation:

f (g) = inf (g B , f )

which is iterated n times to give a geodesic dilation of size n f ,n(g) = f

(n)(g) = f (f .... (f (g))).

• The Euclidean and digital erosions derive from the corresponding dilations by the following duality

f(g) = m mf ) (m g) ,

which is different to the binary duality.

• m is the maximum possible value in the image (i.e. 255 for 8-bit images).

• The digital version starts from the unit geodesic dilation:

f (g) = inf (g B , f )

which is iterated n times to give a geodesic dilation of size n f ,n(g) = f

(n)(g) = f (f .... (f (g))).

• The Euclidean and digital erosions derive from the corresponding dilations by the following duality

f(g) = m mf ) (m g) ,

which is different to the binary duality.

• m is the maximum possible value in the image (i.e. 255 for 8-bit images).

Numerical geodesic erosion of f with respect to g

f

g

f,(g)

Numerical ReconstructionNumerical Reconstruction

• The reconstruction opening of f from g is the supremum of the geodesic dilations of g inside f, this sup being considered as a function of f:

Rf(g) = ƒf,(g) ,

or, as before:

Rf(g) = f(i)g

where i is such that f(i)gf

(i+1)g• The corresponding reconstruction by

erosion is:Rf*(g) = f

(i)g


(i+1)g

• The reconstruction opening of f from g is the supremum of the geodesic dilations of g inside f, this sup being considered as a function of f:

Rf(g) = ƒf,(g) ,

or, as before:

Rf(g) = f(i)g


(i+1)g• The corresponding reconstruction by

erosion is:Rf*(g) = f

(i)g


(i+1)g

Numerical Reconstruction of g inside f :

g

rec(f ; g)

f

Applications of ReconstructionApplications of Reconstruction

• The three major applications are : • swamping, or reconstruction of a function by imposing markers for

the maxima;

• reconstruction from an erosion

• contrast opening, which extracts and filters the maxima.

• The three major applications are : • swamping, or reconstruction of a function by imposing markers for

the maxima;

• reconstruction from an erosion

• contrast opening, which extracts and filters the maxima.

Reconstruction Opening by ErosionReconstruction Opening by Erosion

Goal: contour preservation

Whereas the standard opening modifies contours, this transform is aimed at efficiently and precisely reconstructing the contours of the objects which have not been totally removed by the filtering process.

Algorithm

- the mask is the original signal,

- the marker is an eroded of the mask.

Structuring element

Ero

sion

Dil

atio

nR

econ

stru

ctio

n

Original

opening by a disc

Opening by reconstruction

:B

Rf [ B(f) ] = { f

(n)[B(f )] , n > 0 }

Application to Retina ExaminationApplication to Retina Examination

a) Initial image c) difference a) minus b) followed by a threshold

b) closing by dilation-reconstructionfollowed by opening byerosion-reconstruction

Comment: The aim is to extract and to localise aneurisms. Reconstruction operators ensure that one can remove exclusively the small and isolated peaks(case study due to F. Zana and J. C. Klein).

Reconstruction of a Function from MarkersReconstruction of a Function from Markers

Goal To remove the useless maxima (or

minima) of a function.

Algorithm• The “marker” is a bi-valued (0,m)

function identifying the peaks of interest.

• The reconstruction process result is the largest function f and admitting maxima at the marked points only. It is called the swamping of f by opening (S. Beucher, F. Meyer),

Swamping of f by markers m ( by opening )

marker

An Example of Swamping: Contrast OpeningAn Example of Swamping: Contrast Opening

Algorithm

- Shift down the initial function f by constant c;

- Rebuild f from function f - c, i.e.

Rf(fc) = { f

(n)(fc) , n > 0 }

GoalBoth morphological and reconstruction openings reduce the functions according to size criteria which work on their cross sections. In opening by dynamics, the criterion holds on grey tone contrast (M.Grimaud).

C

Original

Résultat

Application to Maxima DetectionApplication to Maxima Detection

• The maxima of a numerical function on a space E are the connected components of E where f is constant and surrounded by lower values.

• Therefore they are given by the residues of contrast opening of shift c = 1

• More generally, the residuals associated with a shift c extract the maxima surrounded by a descending zone deeper than c. They are called Extended Maxima (S. Beucher).

• The maxima of a numerical function on a space E are the connected components of E where f is constant and surrounded by lower values.

• Therefore they are given by the residues of contrast opening of shift c = 1

• More generally, the residuals associated with a shift c extract the maxima surrounded by a descending zone deeper than c. They are called Extended Maxima (S. Beucher).

ContentsContents



SKIZ, or Skeleton by Influence Zones SKIZ, or Skeleton by Influence Zones

Definition (C. Lantuejoul) Consider a compact set X in 2 . • The zone of influence of a component Xi of X is the set of points of

the plane that are closer to Xi than to any other component.• The SKIZ is then defined as the boundary of all zones of influence.

Algorithm• In the digital case, the SKIZ is constructed in two steps:

1) Thinning of the background (with L in the hexagonal case)2) Pruning of the thinned transform (with E in the hexagonal case)

Skiz

Influence zone

Let Y = { Yi , ibe a set of the Euclidean plane made of I compact connected components, and included in a compact set X.

• The geodesic zone of influence of a component Yi in X, is formed by all points of X whose geodesic distance to Yi is smaller than to any other component of Y

zi (Yi \ X) = {aX , ki, dX(a, Yj) dX(a,Yk)}

where the geodesic distance from point a to set Y is the inf of the geodesic distances from a to all points of Y.

• The geodesic SKIZ is then the boundaries of all geodesic zones of influence.

Geodesic SKIZGeodesic SKIZ

An example of geodesic SKIZ

SKIZ (Y)

Y3

Y2Y 1

X

ziX (Y1)

Distance function (I)Distance function (I)

Definition :The distance function is an intermediate step between sets and functions.

• When a distance has been defined on E, it is possible to associate, with each set X, the subset X composed of all those points of X whose distance to the boundary is larger than .

• As increases, the subsets X are included within each other (and parallel in the Euclidean case). They can be considered as the horizontal thresholds of a function whose grey level is at point x if x is at distance to the boundary. This function is called Distance Function.

1 23 4

X

Distance Function (II): an Example

Distance Function (II): an Example

Set X Corresponding Distance Functions

Comment : Figure b shows the supremum of both distance functions of X and Xc

xy

x

y

Topographic Image Representation

Topographic Image Representation

• The height in the topographic representation corresponds to the greylevel in the initial image.

• The height in the topographic representation corresponds to the greylevel in the initial image.

Watershed Lines for Numerical FunctionsWatershed Lines for Numerical Functions

surface ofthe function

Minima

CatchmentBasinsWatershed

Lines

topographical patterns of a numerical image

N.B. Beucher-Lantuejoul’s algorithm is presented below for the sake of pedagogy. But it is not the unique one, it has been improved by P. Soille and L. Vincent. Another implementation, based on hierarchical queues, due to F. Meyer is more efficient.

• If a drop of water falls on such a topographic surface, it will obey the law of gravitation and flow along the steepest slope path until it reaches a minimum.

• The whole set of points of the surface whose steepest slope paths reach a given minimum constitutes the catchment basin associated with this minimum.

• The watersheds are the zones dividing adjacent catchment basins.

• If a drop of water falls on such a topographic surface, it will obey the law of gravitation and flow along the steepest slope path until it reaches a minimum.

• The whole set of points of the surface whose steepest slope paths reach a given minimum constitutes the catchment basin associated with this minimum.

• The watersheds are the zones dividing adjacent catchment basins.

• Suppose that holes are made in each local minimum and that the surface is flooded from these holes. Progressively, the water level will increase.

• In order to prevent the merging of water coming from two different holes, a dam is progressively built at each contact point.

• At the end, the union of all complete dams constitute the watersheds.

• Suppose that holes are made in each local minimum and that the surface is flooded from these holes. Progressively, the water level will increase.

• In order to prevent the merging of water coming from two different holes, a dam is progressively built at each contact point.

• At the end, the union of all complete dams constitute the watersheds.

Construction of the Watersheds by Flooding (I)

Construction of the Watersheds by Flooding (I)

Minima

Dams in construction

Water level

Construction of the Watersheds by Flooding – Example

Construction of the Watersheds by Flooding – Example

Construction of the Watersheds by Flooding (II)

Construction of the Watersheds by Flooding (II)

Flooding algorithm (S. Beucher, Ch. Lantuejoul) • Let m be the minimum of function f. Put:

X0 = { x: f(x) = m}, Xk = { x: f(x) m+k } with 1 k max f

• Denote by Y1 the geodesic zones of influence of X0 inside X1. Distinguish three types of connected components of X1

– those, X1,1 that do not contain points of X0: then they do not belong to Y1

– those, X1,2 that contain a unique c.c. of X0: then they fully belong to Y1

– those, X1,3 that contain several c.c. of X0: Y1 recovers then X1,3 minus the branches of its geodesic SKIZ.

Flooding algorithm (S. Beucher, Ch. Lantuejoul) • Let m be the minimum of function f. Put:

X0 = { x: f(x) = m}, Xk = { x: f(x) m+k } with 1 k max f

• Denote by Y1 the geodesic zones of influence of X0 inside X1. Distinguish three types of connected components of X1

– those, X1,1 that do not contain points of X0: then they do not belong to Y1

– those, X1,2 that contain a unique c.c. of X0: then they fully belong to Y1

– those, X1,3 that contain several c.c. of X0: Y1 recovers then X1,3 minus the branches of its geodesic SKIZ.

X1.1

c.c. of X0

X1.3

X1.2

c.c. of X0

skiz ( X0 \ X1 )

Evolution of the flooding

Construction of the Watersheds by Flooding (III)

Construction of the Watersheds by Flooding (III)

• Since the X1,1 's are minima which appear at level 1, we have to incorporate them into the flooding process. Thus we replace

X1 by Y1X1,1

• ...and we iterate. The geodesic zones of influence

– Y2 of Y1X1,1 inside X2 are calculated;

– They provide markers Y2X2,1 ; etc...

• The process ends when level k = max (f) is reached. Then one has:

Ymax f = union of the basins;

[ Ymax f ]c = Watershed lines.

• Since the X1,1 's are minima which appear at level 1, we have to incorporate them into the flooding process. Thus we replace

X1 by Y1X1,1

• ...and we iterate. The geodesic zones of influence

– Y2 of Y1X1,1 inside X2 are calculated;

– They provide markers Y2X2,1 ; etc...

• The process ends when level k = max (f) is reached. Then one has:

Ymax f = union of the basins;

[ Ymax f ]c = Watershed lines.

An example of Watershed by Flooding (I)

An example of Watershed by Flooding (I)

Initial image. Minima (1), and next level (2).

2

1

Geodesic skiz of (1) into (2)(in white lines).

An example of Watershed by Flooding (II)

An example of Watershed by Flooding (II)

Level 2, minus the first skiz , and level 3.

Second skiz(note that it extends the first one).

Final watershed(The result is significant in spite of the small number of gray levels).

ContentsContents



Two ExamplesTwo Examples

Electrophoresis: ( S. Beucher)

1- Delineate the spots,2- Determine the neighbors.

Highway: ( S. Beucher)

Delineate the lanes.

Morphological Segmentation Paradigm( S. Beucher, F. Meyer )

Morphological Segmentation Paradigm( S. Beucher, F. Meyer )

One, or more, images

Pre-processingMarkers M

Function f

synthetic function f' ( f, M )

Watershed of f'

Possible Hierarchy

Intelligent Phase

Automatic Phase

Input VariablesInput Variables

• The watersheds extend the minima of an image as far as its topography allows it. One can act on these minima

- either by means of filtering which removes some minima,

- or by swamping, which impose their markers as new minima.

• One can also force the segmentation to pass along some portions of contours which are known a priori (the dotted lines on the road in example 2). Then the watersheds are conditioned by means of an input image where these portions are maxima.

• Finally, a first watershed can generate a new image, if we fill its divides with convenient values, and create a mosaic image. This new image is no longer based on pixels, but on the components of a planar graph. One can apply new watersheds or swampings, leading to a second mosaic image, etc.

• The watersheds extend the minima of an image as far as its topography allows it. One can act on these minima

- either by means of filtering which removes some minima,

- or by swamping, which impose their markers as new minima.

• One can also force the segmentation to pass along some portions of contours which are known a priori (the dotted lines on the road in example 2). Then the watersheds are conditioned by means of an input image where these portions are maxima.

• Finally, a first watershed can generate a new image, if we fill its divides with convenient values, and create a mosaic image. This new image is no longer based on pixels, but on the components of a planar graph. One can apply new watersheds or swampings, leading to a second mosaic image, etc.

Minima selection by FilteringMinima selection by Filtering

As a general rule, images have too many minima, and a careless computation of their watersheds often leads to a disastrous over-segmentation.

• In order to obtain significant minima, one can begin with filtering the images:

– either “horizontally” by plane alternating filters, with or without reconstruction ;

– or “vertically” by closings Rf*(fh) of dynamic h. In particular, for h=1 all the minima are extracted.

• If dealing with maxima, one takes Rf (f h).

As a general rule, images have too many minima, and a careless computation of their watersheds often leads to a disastrous over-segmentation.

• In order to obtain significant minima, one can begin with filtering the images:

– either “horizontally” by plane alternating filters, with or without reconstruction ;

– or “vertically” by closings Rf*(fh) of dynamic h. In particular, for h=1 all the minima are extracted.

• If dealing with maxima, one takes Rf (f h).

“horizontal” filtering

“vertical” filtering

Changing the Minima by SwampingChanging the Minima by Swamping

• The markers may not coincide with the minima of f. In that case, they act on f via the swamping transformation.

• The marker image is calculated as

• Two steps:– The point-wise minimum between the

input image and the marker image is computed:

(f + 1) gRemark: f +1 is used to avoid the case where two minima to impose belong to the same minimum in f.

– A reconstruction by erosion of (f + 1) gfrom g is performed.

R*[(f + 1) g](g)

• The markers may not coincide with the minima of f. In that case, they act on f via the swamping transformation.

• The marker image is calculated as

• Two steps:– The point-wise minimum between the

input image and the marker image is computed:

(f + 1) gRemark: f +1 is used to avoid the case where two minima to impose belong to the same minimum in f.

– A reconstruction by erosion of (f + 1) gfrom g is performed.

R*[(f + 1) g](g)

otherwise

marker a tobelongs if0

maxg

xxg

Changing the Minima by Swamping – ExampleChanging the Minima by Swamping – Example

1st Example: Electrophoresis Gels1st Example: Electrophoresis Gels

(a): Initial image of electrophoresis (b): Watershed of the initial image

(

Problem: to segment the spots, and to characterize the adjacency relations.The solution for this famous case study is due to S. Beucher and F. Meyer.

Electrophoresis: MinimaElectrophoresis: Minima

(c): Inital image minima (d): Minima of the filtered image (e)

Criticism: the over-segmentation comes from the excess of minima; so we will filter initial image by a (opening followed by closing) of size 1 before computing the watershed.

Electrophoresis: zones of influenceElectrophoresis: zones of influence

(e): Hexagonal alternating filter of (a) (f): Watershed of the filtered image (e)

Criticism: the segmentation, now correct, provides the zones of influence ofthe spots, but not their contours. The latter should be derived from the watershedof thegradient

Electrophoresis: Watershed of the Gradient (I)Electrophoresis: Watershed of the Gradient (I)

(g): Modulus of the gradient of the filtered image

(h): Watershed of the gradient (g)

Criticism: a few contours are revealed, blurred in a maze of over-segmentation. Since we know the significant minima of (e), we can introduce them, by swamping, as markers in the gradient image (g)

Electrophoresis: Watershed of the Gradient (II)Electrophoresis: Watershed of the Gradient (II)

(j): Watershed of (i) (i): Swamping of gradient (g) by the minima (d)

Criticism: We forgot to mark that the gradient is zero not only at the centresof the spots, but also in the background. We must swamp (g) with the watershed (f).

Electrophoresis: Contours Electrophoresis: Contours

(l): Watershed of (k), superimposed on the initial image

(k): (g) Swamped by the union of the minima (d) and of the watershed(f)

Criticism: BRAVO !

Electrophoresis: Edge EffectsElectrophoresis: Edge Effects

Comment: We must make an assumption about the outside of the field . Up to now, we implicitly assumed it was white, i.e. lighter than all the pixels in the image. If, on the contrary, we take a black outside, then the graph no longer crosses the boundary of the field.

Lessons Drawn from the ExampleLessons Drawn from the Example

• In a first analysis, over-segmentation may be corrected by filtering, but this approach can only suppress minima. If we want to add some new ones, or to move some of them, we have to apply swamping.

• An object is individualized when it has a unique minimum. The watershed lines of individualized objects delineate their zones of influence. Therefore they are exclusively located in the background, for which they can provide an ideal marker.

• For obtaining the contours of individualized objects, we must deal with the watershed of their gradients.

• The outside of the image has always to be chosen either as being a minimum or a maximum.

....Finally, a number of situations are far from pertaining to that of the previous example, as we will see now.

• In a first analysis, over-segmentation may be corrected by filtering, but this approach can only suppress minima. If we want to add some new ones, or to move some of them, we have to apply swamping.

• An object is individualized when it has a unique minimum. The watershed lines of individualized objects delineate their zones of influence. Therefore they are exclusively located in the background, for which they can provide an ideal marker.

• For obtaining the contours of individualized objects, we must deal with the watershed of their gradients.

• The outside of the image has always to be chosen either as being a minimum or a maximum.

....Finally, a number of situations are far from pertaining to that of the previous example, as we will see now.

Segmentation of Road Lanes (I)Segmentation of Road Lanes (I)

Problem: Delineate the traffic lanes of a road, from a sequence of 450 views taken from a fixed camera with a high angle shot.

Differences: Unlike the previous case:

- we now start from a sequence of images. How to synthesize information?

- The segmentation holds on the road only, without any distinction between foreground and background. Then, how do we build up markers?

- The watershed must pass through the white dotted lines. How can we introduce this constraint?

Problem: Delineate the traffic lanes of a road, from a sequence of 450 views taken from a fixed camera with a high angle shot.

Differences: Unlike the previous case:

- we now start from a sequence of images. How to synthesize information?

- The segmentation holds on the road only, without any distinction between foreground and background. Then, how do we build up markers?

- The watershed must pass through the white dotted lines. How can we introduce this constraint?

(a): Image extracted from the initial sequence.

Road (II) : Synthetic ImagesRoad (II) : Synthetic Images

(a): Sum ( fi , 1 i 450 ) 450 (b): Sum (fi+1 fi , 1 i 449 ) 449

Comment: We summarized the whole useful information in two images (instead of 450 ). One of them emphasises the static portions, and the otherthe moving ones.

Road (III) : MarkersRoad (III) : Markers

(c): thresholded version of (b)

(d): dilation of (c) by a horizontal segment whose length increases as it moves down theimage.

(e): difference between(c) and (d). In white,the four zones to thin.

Comment: image (b), under threshold, provides a first set of markers (c);it is completed by the complement (d) of the road, which marks the outside (e).

Road (IV) : ConditioningRoad (IV) : Conditioning

Comment: the condition of passing through the dotted lines demands we build upa function in which these dotted lines are maxima. It results in variants (g) and (h).

(f): top hat of image (a).

(g): 3 level function, derived from (e) and from the filtered restriction of (f) to the centralstripes of (c)

(h): (inversed) geodesic distance function of the higher level of (g) inside its median level. The two stripes without dotted white lines have been added.

Road (V) : ResultsRoad (V) : Results

(i): Watershed of the three level function (g)

Comment: OK for (j). The three level function is also not such a bad startingpoint. In both cases, the digitalization of the watershed creates slight variationsfrom the dotted lines.

(j): Watershed of the geodesic distance function

Lessons Drawn from the ExampleLessons Drawn from the Example

• For imposing some points or lines on a watershed, one must introduce them as maxima in the source image. This goal can be achieved by taking their geodesic distance function with respect to a significant zone of the image. Such an image synthesis may viewed as complementary to the swamping procedure, which holds on the minima.

• In the limit, it is possible to concentrate information in a three level synthetic image based upon the two following conditioning sets

- low level: markers of the minima,- high level: arcs which are imposed to the watershed .

The median level is a constant value given to the rest of the input function.Finally, if the upper level is missing, the watershed comes back to a SKIZ of the minima.

• In sequences taken by a fixed camera without zoom, the first two abstracts of the information are the means of the images and of their time gradients.

• For imposing some points or lines on a watershed, one must introduce them as maxima in the source image. This goal can be achieved by taking their geodesic distance function with respect to a significant zone of the image. Such an image synthesis may viewed as complementary to the swamping procedure, which holds on the minima.

• In the limit, it is possible to concentrate information in a three level synthetic image based upon the two following conditioning sets

- low level: markers of the minima,- high level: arcs which are imposed to the watershed .

The median level is a constant value given to the rest of the input function.Finally, if the upper level is missing, the watershed comes back to a SKIZ of the minima.

• In sequences taken by a fixed camera without zoom, the first two abstracts of the information are the means of the images and of their time gradients.

SummarySummary

• You have seen– Some of the basic operations of mathematical morphology.

– Some examples of where they are useful.

• Many more exist, which haven’t been covered: Leveling, Granulometry, Morphology for graph representations, etc.

• Applying these operations to colour images is more complicated, as will be discussed in my next talk.

• You have seen– Some of the basic operations of mathematical morphology.

– Some examples of where they are useful.

• Many more exist, which haven’t been covered: Leveling, Granulometry, Morphology for graph representations, etc.

• Applying these operations to colour images is more complicated, as will be discussed in my next talk.

AcknowledgementsAcknowledgements

• This lecture is based on:– Jean Serra’s one week course on Mathematical Morphology held at the

Paris School of Mines (it’s part of the GEI-Athens European courses, so it’s most probably possible to get money from the EU to attend it).

– Pierre Soille’s book.

• This lecture is based on:– Jean Serra’s one week course on Mathematical Morphology held at the

Paris School of Mines (it’s part of the GEI-Athens European courses, so it’s most probably possible to get money from the EU to attend it).

– Pierre Soille’s book.

Allan Hanbury Mathematical Morphology: from Erosion to Watersheds PRIP, Vienna University of...

Documents

Transcript of Allan Hanbury Mathematical Morphology: from Erosion to Watersheds PRIP, Vienna University of...