iAstro Workshop Granada 21-22 February 2002

Samia Bouchafa

Bertrand Zavidovique

THE COST-TIST 283 

Symmetry operators and their application in computer vision

IEF University of Orsay France

Vito Di Gesù

Cesare Valenti

DMA University of Palermo Italy

iAstro Workshop Granada 21-22 February 2002

Symmetry and perceptionSymmetry is a property that characterizes the invariance of a given system.

o It is one of the most prominent spatial relations perceived by human beings.

o Psychologists of perception, assign a relevant role to symmetry in attentive mechanism in both visual and auditory systems

o Image segmentation

o Object-parts representation and description

o Detection of points of interest

iAstro Workshop Granada 21-22 February 2002

iAstro Workshop Granada 21-22 February 2002

Computing Symmetry

Computation paradigm

edge gray levels hybrid

global symmetry

Computer vision tasks

local symmetry

iAstro Workshop Granada 21-22 February 2002

Edge Based Computation

Symmetry Axial Transform (SAT) (Blum, Nagel, 1978) Smoothed Local Symmetry (SLS) (Brady, Asada, 1984) Affine transformations and symmetry(Mukhergee, Zisserman, Brady, Chan, Cipolla, 1995)

Partial occlusion (Sato, Cipolla, 1997)

String oriented approach(Atallah, 1985), (Bruckstein and Shaked, 1995)

iAstro Workshop Granada 21-22 February 2002

Gray Levels Approaches

Texture analysis and symmetry measures (Cheterikov and Haralick, 1995)

Measures based on the Radom’s transform (Kiryati and Gofman, 1996)

Context free attentional operators(Reisfeld, Wolfson and Yeshurun 1995) 

iAstro Workshop Granada 21-22 February 2002

Symmetry TransformDi Gesù, Valenti, 1994

O

pdprprOT ,,

nk

gn

ks

n

krT

h

hr

h

hs

sjrikji

,,1

cossin ,

2

,

Circular Symmetry

n

k

n

k

kji

kjii,j T

nT

n1

2

1

,2

,11

1T

iAstro Workshop Granada 21-22 February 2002

Discrete Symmetry Transform

DEDTDDST

1,,,

,,,

kk CsrCmh

srmhji ggE

iAstro Workshop Granada 21-22 February 2002

Points of interest

iAstro Workshop Granada 21-22 February 2002

Pyramid-DST(Di Gesù,Valenti 1996)

Discrete Fourier Transform of D0 and

yx)( , max0 then:

04

1

td

0log

d

t

iAstro Workshop Granada 21-22 February 2002

iAstro Workshop Granada 21-22 February 2002

iAstro Workshop Granada 21-22 February 2002

Tracking problems

iAstro Workshop Granada 21-22 February 2002

Face analysisApplications:security systems, criminology. physical access control, man-machine interactions

iAstro Workshop Granada 21-22 February 2002

Expression analysisNeutral, Sadness, Disgust, Happiness, Fear, Anger, Surprise

Neutral SmilingNeutral 92% 8%Smiling 20% 80%

iAstro Workshop Granada 21-22 February 2002

Object recognition systems Chella, Di Gesu’, Infantino, Intravaia, Valenti 1997

o Object Recognition Using Multiple Viewso 3D shape reconstruction from image sequences

O

O

s

pdphG

pdphpOpO

OA1

2

2/1/1

e

ph

iAstro Workshop Granada 21-22 February 2002

Iterated Object TransformDi Gesù, Zavidovique, 2002

The IOT computes the symmetry transform, T, on steadily intensity reduced versions of the input image

11,

1,

mforDD

DD

mm

TETIOT

IOT

E operator erosion

iAstro Workshop Granada 21-22 February 2002

Contrast change definition

Non-decreasing funtion g

Level set :

Contrast change impact

• some level sets disappearance

• no geometric deformation

Motion impact (+ noise)

• some new level sets appearance

• Geometric deformation level lines crossing

Contrast change and level lines

)(SgI

SI

)p(/p II

iAstro Workshop Granada 21-22 February 2002

How can we reconstruct the scene S ?

Possibilities for each line :1. The line is present

no detection2. The line is not present

Doubt :Is the reference complete ?Is the background uniform ?

3. The line crosses another one detection

Week Detection

Strong Detection

Detection criteria

SI

i

NIII

S

...10

iAstro Workshop Granada 21-22 February 2002

Motion detection algorithm

Level line Extraction

Characterization

Updating

Comparison

Detection

Image Sequence

Reference

Local Orientations

tracking of level lines

- Appearance of new lines

- Crossings between lines

iAstro Workshop Granada 21-22 February 2002

• Two possibilities :

Global characterization surface, other moments of

inertia, etc.

Local characterization local orientation

100 Associated level line

Level line characterization

Our choice : local characterization- Point detection

- No level lines occlusions management

iAstro Workshop Granada 21-22 February 2002

The original sequence presents some contrast changes due the automatic gain control of the camera and to natural scene illumination changes.

In the sequence, only points affected by motion are displayed

The result of the detection algorithm that is insensible towards contrast changes.

iAstro Workshop Granada 21-22 February 2002

iAstro Workshop Granada 21-22 February 2002

Fast Marching Methods and Level Set Methods are numerical techniques which can follow the evolution of interfaces. These interfaces can develop sharp corners, break apart, and merge together. The techniques have a wide range of applications, including problems in fluid mechanics, combustion, manufacturing of computer chips, computer animation, image processing, structure of snowflakes, and the shape of soap bubbles.

These are two fundamentally different approaches to the problem of tracking moving interfaces, yet they share a common theory and numerical methodology.

iAstro Workshop Granada 21-22 February 2002

Edge Based Computation

Symmetry Axial Transform (SAT) (Blum, Nagel, 1978)

iAstro Workshop Granada 21-22 February 2002

Smoothed Local Symmetry (SLS)

(Brady, Asada, 1984)

iAstro Workshop Granada 21-22 February 2002

DST

Edge based operatorYeshurun

Input

iAstro Workshop Granada 21-22 February 2002

Face analysis and algorithmsCardaci, Di Gesu’, Intravaia, 1998

• The algorithm is based on an attentive architecture.

• local and global symmetry operators Reisfeld, Wolfson,Yeshurun (1995) Di Gesù, Valenti, Strinati, (1997)

• graph theoretical algorithms Zhan (1972)

• facial anatomy (model driven) Russel, (1994)

iAstro Workshop Granada 21-22 February 2002

iAstro Workshop Granada 21-22 February 2002

Gelstat clustering (GC)

Structural information are represented by a simple Internal Model (IM) based on psycho-visual correlation between components of face Chen, Yachida (1996)

A relational graph (FG) is then built from the retrieved FC

Neutral SmilingNeutral 92% 8%Smiling 20% 80%

iAstro Workshop Granada 21-22 February 2002

Séquence initiale

ResultsA sequence with global contrast changes

iAstro Workshop Granada 21-22 February 2002

Results

The same crossing junction but different lighting conditions

iAstro Workshop Granada 21-22 February 2002

ApplicationsRoad environment

Vehicle/pedestrian detection and counting

Subway environment

Stationnary objects/human detection

iAstro Workshop Granada 21-22 February 2002

Comparisons

Reference sequence Six months before Level lines

Grey levels Laplacian Gradients orientation

iAstro Workshop Granada 21-22 February 2002

Comparisons

Gradient orientations

Problems with stability and thresholding !