Eye Feature Detection

Rui Liu

PhD student M.E.

Final Project for Computer Vision

CONTENT

‣ 1 Introduction

‣ 1.1 Motivation

‣ 1.2 Goals

‣ 1.3 Experiment Devices and Environment

‣ 2 Methods & Algorithms

‣ 3 Experiment Results

‣ 4 Result Analysis

‣ 4.1 Assessment

‣ 4.2 Small Problems and Its Analysis

‣ 4.3 Moving Forward

1 Introduction

1.1 Motivation for Eye feature detection

(1) What could eye feature indicate?

reflecting psychological state

He sees the truth. It’s written all over our faces.

Indicating human intention

Maybe want to leave Maybe want to drink

(2) Motivation: detecting eye features for intuitive human-robot interaction

1.2 Goals

Pupil detection

pupil localization, pupil diameter/ mean area and its standard deviation in a certain time.

Blink detection

blinking status detection, blinking times and blinking rate in a certain time.

1.3 Experiment Devices and Environment

Head Mount

HD Camera

Pic Captured

(1) Devices

Common lab (400 lux)

(2) Environment

2 Methods & Algorithms

1.1 pupil detection

(1)Method:

a. Circle(d≈44)

b. Dark(Area≈1500)

c. Concentric

d. Region

Pupil

(2) Algorithm:

a. Circle detection: Two-stage Circular Hough Transform (‘imfindcircle’ )

b. Dark region detection: ‘regionprops’

d= 42Concentric

‣ (3) Parameter Calculation

pupil center (x, y)

Pupil Diameter d circle detection

Mean Area A = 𝜋𝑑2 𝑡

4

𝑡2𝑡1

/(𝑛𝑡2 − 𝑛𝑡1)

Standard Deviation of Pupil Diameter 𝜎 = (𝑑 𝑛𝑡 −𝑑 )2𝑡2𝑡1

𝑛𝑡2−𝑛𝑡1

1.2 blink detection

(1) Method:

Blinking pupil disappearing time >𝒕𝟎

OpenBlinking

‣ (2) Parameter Calculation

Blinking Rate Rate=N/(𝑡2-𝑡1)

Blinking Times N

3 Experiment Results

3.1 results in different situations

a. Beginning b. Blinking

c. Open d. Near Blinking

3.2 Result Video

3.3 Statistics Results

a. Eye status b. Blinking Rate (n/sec)

0 5 10 15 20-0.5

0

0.5

1

1.5

Time(s)

Eye S

tatu

s

Blinking Status

1 -- Blinking

0 -- Open

0 5 10 15 20-0.5

0

0.5

1

1.5

Time(s)

blin

k r

ate

(Tim

es/s

ec)

blink rate

d. Mean Pupil Area

0 5 10 15 20

1300

1400

1500

1600

Time(s)

Mean P

upil

Are

a(p

ixel2

)

Mean Pupil Area

0 5 10 15 200

20

40

60

Time(s)

Pupil

Dia

mete

r(pix

el)

Pupil Diameter

c. Pupil Diameter

0 5 10 15 200

1

2

3

Time(s)

Sta

ndard

Devia

tion o

f P

upil

Siz

e

Standard Deviation of Pupil Size

e. Standard Deviation of Pupil Size

4. Result Analysis

4.1 Assessments

Successful Rate for Pupil detection 99%

Successful Rate for Blink detection 100%

Successful Rate for Pupil Contour detection 90%

4.2 small problems and its analysis

(a) A few pupil contours could not be detected accurately when eye is

near blinking/open

Because

Open Num= 1

Rate= 0.469

—A(d)=1439.44

(d)= 2.314

Actual contour shape is ellipse which

is detected as 2 circles

(b) Pupil in some frames could not be detected near blinking/open.

Light intensity is too low near blinking/open.

Dark region detection : Failed

Because

4.3 Moving Forward

In the future, this project could be optimized in next aspects:

(1) The circle detection could be revised as circle/ellipse detection

(2) The sensitivities and thresholds of these algorithms could be

adjusted adaptively

Q&A

Thank You !

Rui Liu