Matthias Wimmer, Bernd Radig, Michael Beetz Chair for Image Understanding Computer Science TU...

Matthias Wimmer, Bernd Radig, Michael Beetz

Chair for Image UnderstandingComputer Science

TU München, Germany

[email protected]

A Person and Context Specific Approach for

Skin Colour Classification

2005-12-18 2/9Technische Universität MünchenMatthias Wimmer

high-level vision module

low-level vision module

Output Parameters

Input Parameters

Calculation Rules

Motivation

Face LocatorSkin Colour Classifier

motivation our approach results outlook

see: A.E. Broadhurst, S. Baker: Setting Low-Level Vision Parameters, CMU-RI-TR-04-20, Robotics Institute, Carnegie Mellon University, 2004.

Our scenario: Adaptive Skin Colour Classification Classifier adapts to person and context

low-level vision module

Input Parameters

simple way:

promote parameters: high-level to low-level vision module mathematically transform parameters


Motivationmotivation our approach results outlook


Observations Skin colour depends on image conditions:

illumination: light source, light colour, shadow, shading,… camera: type, settings,… visible person: ethnic group, tan,…

Skin colour occupies a large area within colour space Skin colour varies greatly between images. Skin colour varies slightly within an image.


gree

n

red

skin colour pixels (red) and other pixels (blue), static skin colour clusters (white), adaptive skin colour clusters (yellow)

gree

n

red

image 1 image 2


Our ApproachOffline step: learn a mask that extracts skin colour pixels

specific for the face detector


Online steps: Step 1: detect the image specific skin colour

using the face detector using the skin colour mask

Step 2: calculate the input parameters Step 3: adapt the skin colour classifier

Face Locator(Step 1)

Skin Colour Classifier(Step 3)

Output Parameters

Input ParametersCalculation Rules

(Step 2)


Learn the Calculation Rules

Gather many training images Manually annotate images with ground truth Learn calculation rules via machine learning techniques

e.g. linear regression, neural networks, model trees, …


Face LocatorSkin Colour Classifier

Output Parameters

Input Parameters

Calculation Rules

specify these (ground truth)

specify these (ground truth)

learn those


Results good robustness for

coloured persons exact shape outline detection of facial parts:

eyes, lips, brows,…

correctly detected pixels: fixed parameters: 90.4% 74.8%

40.2% adaptive parameters: 97.5% 87.5%

97.0% improvement: 0.08 0.17 1.41

motivation our approach results outlooka

da

ptive

pa

ram

ete

rs

fixed

pa

ram

ete

rs

orig

ina

l

ima

ge


Outlook We will create further

adaptive colour classifiers lip teeth eyes brows, hair …

Preliminary results for lip colour classifier:


original fixed adaptive


Thank you!


Motivation

Skin colour detection supports… face model fitting

mimic recognition person identification gaze estimation fatigue detection (e.g. vehicle)

hand tracking gesture recognition action recognition supervising work

challenge our approach results outlook


Challenge Skin colour depends on image conditions:

illumination: light source, light colour, shadow, shading,… camera: type, settings,… visible person: ethnic group, tan,…

Skin colour occupies a large area within colour space



Challenge (2): non-skin colour pixels Skin colour pixels have to be separated from non-

skin colour pixels. Areas of skin colour and

non-skin colour overlap. Colour can not make a

distinctive separation.



Our approach

Offline step: learn the skin colour mask

specific for the face detector

Online steps: Step 1: detect the image specific skin colour model

using the face detector using the skin colour mask

Step 2: adapt a skin colour classifier Step 3: calculate the skin colour image



Offline: Learn the skin colour mask face image database with labeled skin colour pixels skin colour mask: array with 24 x 24 cells

Computational steps:

1. detect the face in every image

2. every cell is assigned the relative number of labeled skin colour pixels at its position

3. apply threshold

1. 2. 3.



Step 1: Detect the image specific skin colour model

detect the face extract the skin colour pixels normalized RGB colour space:

base = R + G + B

r = R / base

g = G / base

skin colour model: mean values: μr, μg, μbase

standard deviations: σr, σg, σbase



Step 2: Adapt a skin colour classifier non-adaptive skin colour classifier:

skin := 0.35 ≤ r ≤ 0.5 0.2 ≤ g ≤ 0.7 200 ≤ base ≤ 740

adaptive skin colour classifier:skin := lowr ≤ r ≤ highr

lowg ≤ g ≤ highg lowbase ≤ base ≤ highbase

learn the bounds via the skin colour model mean value and standard deviationlowr := μr – 2σr

highr := μr + 2σr . . . . . . . . .

linear function:lowr := aμr + bμg + cμbase + dσr + eσg + fσbase + g

. . .



Related work Feedback of information from

high level vision components to low level vision components



Conclusion Challenge: much variation within skin colour

illumination, camera, visible person skin colour occupies a large area within colour space

We propose a way to reduce those variations exploit an image specific skin colour model adapt a skin colour classifier to that skin colour model

We proved our approach using a simple but real-time capable skin colour classifier comparison: non-adaptive ↔ adaptive



Ongoing research Learn skin colour mask for other face detectors Specialize more powerful skin colour classifiers Recognize other feature images/colour images

lip colour image tooth colour image eye colour image hair colour image eye brow colour image

example: lip colour detection



Adaptive skin colour classifier non adaptive skin colour classifier:

skin := 0.35 ≤ r ≤ 0.5 0.2 ≤ g ≤ 0.7 200 ≤ base ≤ 740

adaptive skin colour classifier:skin := lowr ≤ r ≤ highr lowg ≤ g ≤ highg

lowbase ≤ base ≤ highbase

learn the bounds out of the skin colour model

Matthias Wimmer, Bernd Radig, Michael Beetz Chair for Image Understanding Computer Science TU...

Documents

Transcript of Matthias Wimmer, Bernd Radig, Michael Beetz Chair for Image Understanding Computer Science TU...