Design of Morphological Operators by Learningjb/lectures/morphological operator design/i… ·...

Design of Morphological Design of Morphological

Operators by LearningOperators by Learning

Junior BarreraJunior Barrera

[email protected]@ime.usp.br

IMEIME--USPUSP

22

LayoutLayout

�� IntroductionIntroduction

�� Binary operator design: WBinary operator design: W--operators operators

�� Binary operator design: constraint WBinary operator design: constraint W--

operatorsoperators

�� GrayGray--scale operator design: aperturesscale operator design: apertures

�� GrayGray--scale operator design: stack filtersscale operator design: stack filters

�� Conclusion Conclusion

IntroductionIntroduction

44

dA eA

⁄Ÿ

n

i Ÿ neA

Morphological Machine (MMach)

∏

55

PropertiesProperties

�� Any Any finite latticefinite lattice operator can be operator can be

implemented as a implemented as a programprogram of a of a MMachMMach

�� Finite lattices of practical importance are the Finite lattices of practical importance are the

lattice of lattice of binarybinary and and graygray--scale imagesscale images

66

Morphological ToolboxMorphological Toolbox

�� Library of Library of hierarchical functionshierarchical functions::

�� -- primitivesprimitives: elementary operators and : elementary operators and

operations;operations;

�� -- high order operatorshigh order operators: primitives and high : primitives and high

order operators order operators

77

Heuristic DesignHeuristic Design

�� DivideDivide the problem in the problem in subproblemssubproblems

�� Each Each subproblemsubproblem is solved by a is solved by a toolbox toolbox

functionfunction

�� IntegrateIntegrate the the subproblemssubproblems solutionsolution

88

Automatic DesignAutomatic Design

�� Operator learning in a standard Operator learning in a standard

representationrepresentation

�� Finding an equivalent and more efficient Finding an equivalent and more efficient

representationrepresentation

99

Operator Design Operator Design

1010

ApplicationApplication

1111

Change of RepresentationChange of Representation

dAeA

e ⁄ e ⁄ Ä Ä Ä ⁄ eA1 A2 An

y

Phrases

Operators

Operator DesignOperator Design

1313

The problemThe problem

Find an image operator that transforms the observed

image to the respective ideal (or “close to the ideal”)

image.

observed ideal

1414

Binary image operatorsBinary image operators

Binary image :

Binary images can be understood as sets :

is a complete Boolean lattice

Binary image operators = set operators :

1515

Translation invarianceTranslation invariance

translation-invariant iff

Translation of S by z :

is

1616

Local definitionLocal definition

An image operator is locally defined within iff

Window :

1717

WW--operatorsoperators

Translation invariance

+

local definition within

=

-operators

z

W-operators are characterized by Boolean functions.

1818

1

11

1

RepresentationRepresentation

Window

X11 1X0 11X

1919

Statistical HypothesisStatistical Hypothesis

X and Y are jointly stationary

),( YWSP z∩

is the same for any z in E

2020

Stationary ProcessStationary Process

2121

Join Stationary ProcessJoin Stationary Process

2222

Error measureError measure

Loss function

Risk (expected loss) of a function :

Design goal is to find a function

with minimum risk.

X is a random set

Y is a binary random variable

2323

Optimal MAE function

Example : MAE loss function

≤

>

=

),0(),1( 0

),0(),1( 1

)(

XpXp

XpXp

Xψ

MAE exampleMAE example

2424

Design procedureDesign procedure

2525

PAC learningPAC learning

L is Probably Approximately Correct (PAC)

For examples),( δεmm >

δεψψ −><− 1) |)()(Pr(| optRR

)1 ,0(, ∈δε

2626

Edge detectionEdge detection

Test images

Training images

2727

Noise filteringNoise filtering

Training images

Test images

2828

Texture extraction Texture extraction (1)(1)

Training images

2929

Texture extraction Texture extraction (2)(2)

Test images

3030

Fracture DetectionFracture Detection

Training images Test images

3131

18,10 %

3,13 %2,68 %

2,21 %2,01 %

1,82 %1,69 % 1,60 %

Noise 1 image 2 images 3 images 8 imagess 13 images 25 images 32 images

Last Image

Amount of data availableAmount of data available

3232

addition: 2%

subtraction: 1%

distict patterns : 140.060

in 1.548.384

addition: 3%

subtraction: 3%

distinct patterns : 266.743

em 1.548.384

addition: 6%

subtraction: 6%

distinct patterns: 487.494

in 1.548.384

window 5x5, 6 training images

3333

Size of the windowSize of the window

Error rate in function of window size

0.00%

0.10%

0.20%

0.30%

0.40%

0.50%

0.60%

0.70%

3x3 4x4 5x5 6x6 7x7

1

2

3

4

5

6

7

8

Constraint DesignConstraint Design

3535

optψ

Ψ

ConstraintsConstraints

3636


Err

or

N0N1 N2

Sample size

εdes

εdes-con

εopt

εopt-con

3737


Structural Constraints

- impose maximum number of elements in the basis

- use alternative structural representations(e.g., sequential)

3838


- consider a constraint by an envelope of operators

- consider a constraint by a multi-resolution criteria

- consider a constraint by a multi-resolution envelope criteria

Algebraic constraints

- consider a class of operators satisfying a given algebraic property (e.g., increasingness , idempotence, auto-dualism, etc)

Structural ConstraintStructural Constraint

Minimum number of intervals Minimum number of intervals

in the basisin the basis

Iterative designIterative design

4545

is a -operator

Motivation : composition of operators over small windows produces an operator over a larger window

4646

Application exampleApplication example

test image

iteration 1

iteration 2

4747

Application exampleApplication example

test image iteration 1

iteration 4 iteration 5 iteration 6

iteration 2 iteration 3

Algebraic ConstraintAlgebraic Constraint

4949

Increasing W-operators

increasing non-increasing

5050

increasing

increasingnon-increasing

Design of increasing W-operators

MultiMulti--resolution constraintresolution constraint

5353

00

01

10

11

00

01

10

11

0

1

0 1

2 variables: 22=4 4 variables: 24=16

0000000100100011010001010110011110001001101010111100110111101111

0010

0011

0000

0001

0110

0111

0100

0101

1010

1011

1000

1001

1110

1111

1100

1101

8 variables: 28=256

x1 x2 x3 x4 x5 x6 x7 x8

z1 z2 z3 z4

w1 w2 2 variables

4 variables

8 variables

5454

W0

W1

D1 = P(W1)

D0 = P(W0)

zi = pi(xi1, ... , xi9) , z = p(x) , p=(p1, .. p9)

Let φ:D1→{0,1} , it defines the operator Ψφ on Do by

Ψφ (x) = φ(p(x))

The operador Ψφ is constrained by resolution to D1

Equivalence classes defined

by

p(x) = p(y)

D0

5555

Properties

Operators over D0

Operators over D0 constrained by resolution on D1

Operators over D1

Q φΨφ

5656

W2

W1

W0

D2 = P(W2), D1 = P(W1), D0 = P(W0)

x ∈D0, z ∈D1, v ∈D2,

zi = pi(xi,1, ... , xi,9) , z = p(x) ,

p=(p1, .. p81)

vi = wi(xi,1, ... , xi,81) , v = w(x) ,

w=(w1, .. w9)

The equivalence classes defined byp may be different by the onesdefined by w.

p

w

5757

ψ

ψ

ψ ρ

ψ ρ ρ

ψ ρ ρ

( )

( ) ( )

( ) ( ) , ( ( ))

( ) ( ) ,..., ( ( )) , ( ( ))

( ) ( ) ,..., ( ( )) , ( ( ))

,

,

,

,

x

x x

x x x

x x x x

x x x x

=

>

= >

= = >

= = >

− − −

−

0

1 1

1 2 1

1

0

0 0

0 0 0

0 0 0

N

N

m N m m

m N m m

if N

if N N

if N N N

if N N N

M

6161

image noise image + noise

MultiresolutionMultiresolution NoiseNoise

6262

3x3 window

Pyramid

Restauration Persisting noise

6464

Example

W11

W10

W9

W8

W7

W6

W5

W4

W3

W2

W1

W0

Hybrid Multiresolution Filter: Experiment I

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

Ψ11 Ψ10 Ψ9 Ψ8 Ψ7 Ψ6 Ψ5 Ψ4 Ψ3 Ψ2 Ψ1 Ψ0

Window sequence

MA

E

Single operator Pyramid operator

Envelope constraintEnvelope constraint

6666

Independent ConstraintsIndependent Constraints


Restriction of Restriction of

the operators spacethe operators space

K(K(ψψψψψψψψoptopt) ) ∈∈∈∈∈∈∈∈ Q Q ⊆⊆⊆⊆⊆⊆⊆⊆ PP((PP(W))(W))

P(P(W))

Q

K(ψψψψopt)

Independent Constraint Independent Constraint

Let be ALet be A,,BB ⊆⊆⊆⊆⊆⊆⊆⊆ PP(W) with (W) with AA⊆⊆⊆⊆⊆⊆⊆⊆BB::

hhψψψψψψψψ(x)=1 (x)=1 ∀∀∀∀∀∀∀∀ xx∈∈∈∈∈∈∈∈AA & h& hψψψψψψψψ(x)=0(x)=0∀∀∀∀∀∀∀∀xx∉∉∉∉∉∉∉∉BB, ,

∀ψ∀ψ∀ψ∀ψ∀ψ∀ψ∀ψ∀ψ :K(:K(ψψψψψψψψ) ) ∈∈∈∈∈∈∈∈QQ

A

B

P(W)

{1}

{0,1}

{0}

P(P(W))

A

B

Q

6767

Independent ConstraintsIndependent Constraints

PropositionProposition: if Q is an independent restriction then exist a par of : if Q is an independent restriction then exist a par of

operatorsoperators

((αααααααα,,ββββββββ) such that, for any ) such that, for any ψ∈Ψψ∈Ψψ∈Ψψ∈Ψψ∈Ψψ∈Ψψ∈Ψψ∈ΨWW

K(K(ψψψψψψψψ) ) ∈∈∈∈∈∈∈∈Q Q ⇔⇔⇔⇔⇔⇔⇔⇔ α≤α≤α≤α≤α≤α≤α≤α≤ ψψψψψψψψ ≤β≤β≤β≤β≤β≤β≤β≤β

where K(where K(αααααααα) = ) = AA and K(and K(ββββββββ) = ) = BB

•• All independent constraint is characterized by two operators All independent constraint is characterized by two operators αααααααα and and

ββββββββ

•• The pair (The pair (αααααααα,,ββββββββ) is called ) is called ““EnvelopeEnvelope””

A

B

P(W)

{1}

{0}

{0}

A

B

P(W)

{1}

{1}

{0}

hαααα hββββ

6868

Definition

Desing of two heristic filters α and β such that when we

know that α ≤ ψopt ≤ β, the restriction is defined by:

Q = {ψ : α ≤ ψ ≤ β}

and any filter ψ can be projected into the restriction by

ψ ψ α βenv = ∨ ∧( )

6969

Definition

α

β

Operators over P(W)

constrained by envelope

Operators over

P(W)

ψ

ψenv

Q

7070

Properties

♦ (ψopt ∨ α ) ∧ β is optimal in Q.

♦If α ≤ ψopt ≤ β then Error[ψenv] ≤ Error[ψ]

♦If α ≤ ψopt ≤ β is not true, then

LimN→∞ Error[ψenv,N] > LimN→∞ Error[ψN]

7171

Example

7272

Noise

Addition

Ground

Image

Noisy Image Filtered

Image

Edge

DetectedRestorationEdge

Detection

Edge Detected

Direct Edge Detection

Noise Edge DetectionNoise Edge Detection

7373

Machine design of

the restoration

Human-Machine

design of the

restoration

0.28 % 0.13 %

0

0,05

0,1

0,15

0,2

0,25

0,3

Error (%)

Machine design

of the restoration

Human design of

the restoration

Restoration a) Machine design of the restoration

b) Human-machine design of the restoration

ψψψψpac designed by examples

ψψψψcon = (ψψψψpac ∩β∩β∩β∩β) ∪∪∪∪ αααα

αααα = δδδδB⊕⊕⊕⊕BεεεεB⊕⊕⊕⊕BδδδδBεεεεB and ββββ = εεεε B⊕⊕⊕⊕BδδδδB⊕⊕⊕⊕B εεεε BδδδδB

αααα and ββββ are alternating sequential filters with

P[ αααα(S) ≤≤≤≤ I ≤≤≤≤ ββββ(S) ] ≈≈≈≈ 1

B is the 3x3 square

7474

Edge

Detection

a) Machine design over noisy images

b) Human design after restoration

ζζζζ pac designed by examples from noisy images

ζζζζ = Id - εεεεB

B is the 3x3 square

c) Machine design after restoration

ζζζζ pac designed by examples from restored images


Machine

design over

noisy images

Human

design after

restoration

Machine

design after

restoration

0.65 % 0.27 % 0.24 %

0

0,1

0,2

0,3

0,4

0,5

0,6

0,7

Error (%)

Machine design over

noisy images

Human design after

restoration

Machine design after

restoration

7575

Machine design over

noisy images

Error = 0.65%

Human design after

restoration

Error = 0.27%

Machine design after

restoration

Error = 0.24%


Envelope multiEnvelope multi--resolution resolution

constraintconstraint

7777

Definition

♦ W1 ⊂ W0 , ρ:D0 → D1 is a resolution mapping

♦ α, β: D1 → {0,1} with α ≤ β

♦ ψ : D0 → {0.1}

♦ ψρ = (ψ ∧ β’) ∨ α’ , α’(x) = α(r(x)) and β’(x) = β(r(x))

=

=

=

otherwhise

xif

xif

)(

0))((0

1))((1

)(

x

x

ψ

ρβ

ρα

ψ ρ

7878

ψρ

QD0

φenv

β

α

D1

φ

Definition

Ψφ

β’

α’

7979

Teorema

ψ

ψρ

Q

D0

D1

φenv

β

α

φ

β’

α’

8080

Piramidal Design:

Let ψi,env,N = ( ψi,N ∧ β’) ∨ α’, be the projection of the resolution

constrained filter inside the envelope (α’, β’)

ψ

ψ

ψ ρ

ψ ρ ρ

ψ ρ ρ

env mres

N

env N

m env N m m

m env N m m

if N

if N N

if N N N

if N N N

−

− − −

−

=

>

= >

= = >

= = >

( )

( ) ( )

( ) ( ) , ( ( ))

( ) ( ) ,..., ( ( )) , ( ( ))

( ) ( ) ,..., ( ( )) , ( ( ))

,

, ,

, ,

, ,

x

x x

x x x

x x x x

x x x x

0

1 1

1 2 1

1

0

0 0

0 0 0

0 0 0

M

8181

Properties:

♦ ψenv-mres is a consistent estimator of ψopt

♦ If the envelope is well defined on D1, then the ρ-envelope of a

resolution constrained filter is advantageous

8282

α = εB(γEφEγE)

β = δB(φEγEφE)

EB

GrayGray--scale operator design: scale operator design:

apertureaperture

8585

Spatial Translation Invariance

8686

Gray-scale Translation Invariance

8787( ) ( ) )(/)( xWfxf xΨ=Ψ

Locally defined in W

8888

}},)(,{{))(/( kyzukzKu y −−∨∧=

Locally defined in W and K

8989

0000

5555

10101010

15151515

20202020

25252525

30303030

inputinputinputinput

outputoutputoutputoutput

-2 1 2 2 2

-2 1 2 2 2

-2 1 2 2 2

-2 1 1 1 1

-2 -2 -2 -2 -2

-2 -1 0 1 2

2

1

0

-1

-2

ψβ

Aperture Operator

W { }2,1,0,1,2 −−=K

9090

-2 1 2 2 2

-2 1 2 2 2

-2 1 2 2 2

-2 1 1 1 1

-2 -2 -2 -2 -2

-2 -1 0 1 2

2

1

0

-1

-2

ψβ

ψ

12 13 14 15 16

12 13 14 15 15

12 13 14 14 12

12 13 13 11 12

12 12 10 11 12

10 11 12 13 14

14

13

12

10

11

=

10 11 12 13 14

10 11 12 13 14

10 11 12 13 14

10 11 12 13 14

10 11 12 13 14

10 11 12 13 14

14

13

12

10

11

+

)(ou ψβ

2 2 2 2 2

2 2 2 2 1

2 2 2 1 -2

2 2 1 -2 -2

2 1 -2 -2 -2

10 11 12 13 14

14

13

12

10

11

Aperture Operator

9191

�� Let Let a, ba, b ∈∈ Fun[Fun[W,LW,L]], , aa ≤≤ bb iffiff aa((xx) ) ≤≤ bb((xx), ), x x ∈∈ WW

�� Interval Interval [[aa,,bb] = {] = {uu ∈∈ Fun[Fun[W,LW,L]:]: aa ≤≤ uu ≤≤ bb}}

a

b|W| = 2

9292

�� SupSup--generating operator:generating operator:( ) ],[1, bauuba ∈⇔=λ

0 0 0 0 0

0 0 1 1 1

0 0 1 1 1

0 0 1 1 1

0 0 0 0 0

[a,b]

ba ,λ

9393

Kernel of ψ at y: K(ψ)(y) = {u ∈ Fun[W,L]: y ≤ ψ(u)}

0 1 2 2 2

0 1 2 2 2

-1 1 2 2 2

-1 1 1 1 1

-2 -1 -1 -1 -1

K(ψ)(2)K(ψ)(-1) K(ψ)(0) K(ψ)(1)K(ψ)(-2)

-2 -1 0 1 2

2

1

0

-1

-2

9494

Basis of ψ at y: B(ψ) is the set of maximal intervals contained in K(ψ)

K(ψ)(2)K(ψ)(-1) K(ψ)(0) K(ψ)(1)K(ψ)(-2)

B(ψ)(2)B(ψ)(-1) B(ψ)(0) B(ψ)(1)B(ψ)(-2)

9595

SupSup--representationrepresentation

( ) { }{ }1))((],[:)(: , =∈∈= yBbauMyu ba ψλψ UU

K(ψ)(2)K(ψ)(-1) K(ψ)(0) K(ψ)(1)K(ψ)(-2)

ψ(-1,-1) = 1

9696

These are part of the observed and ideal images (512x512)

Observed Ideal

9797

MAE x Number of ExamplesMAE x Number of Examples

9898

DebluringDebluring -- Aperture x Optimal Aperture x Optimal

linearlinear

Aperture 17p x 5 x 5 Optimal linear 7x7

9999

Resolution Enhancement

100100

Resolution Enhancement

(0,0) (0,1)

0Ψ

3Ψ

2Ψ

3Ψ

1Ψ

101101

Original Aperture: 3x3x21x51

Linear Bilinear

Resolution Enhancement - Results

102102

Original Aperture: 3x3x21x51

Linear Bilinear

Zoom

Resolution Enhancement - Results

GrayGray--scale operator design:scale operator design:

stack filtersstack filters

104104

A stack filter is a gray-scale operator characterized

by a positive (i.e., increasing) Boolean function

}1])[(:max{)( =∈= fTKtf tψψ

where

})(:{][ txfWxfTt ≥∈=

105105

Impulse noise removal Impulse noise removal (1)(1)

training images

106106



107107



108108

Robustness Robustness (1)(1)


109109

Robustness Robustness (2)(2)


110110

Conclusion Conclusion

�� Design of Morphological Operators:Design of Morphological Operators: a a

discrete nature problemdiscrete nature problem

�� Fundamentals: Fundamentals: Algebra, Statistics, Algebra, Statistics,

CombinatoryCombinatory

�� Real problemsReal problems solutionsolution

�� Design techniques Design techniques adequate to introduce adequate to introduce

prior knowledgeprior knowledge

�� Identification of Lattice Dynamical Systems Identification of Lattice Dynamical Systems

Design of Morphological Operators by Learningjb/lectures/morphological operator design/i… ·...

Documents

Transcript of Design of Morphological Operators by Learningjb/lectures/morphological operator design/i… ·...