Lecture 7

The Eye and Neuromorphic Vision

1

Outline

• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing

2

• Eyeball Cross Section and Retina

3

Retina Cells

4

Schematized Retinal Cells

5

Outline

• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing

6

Frame-based Sensors Vs Biological Vision

Artificial Vision Biological Systems

Frame-based Continuous Detection

Global AGC Local AGC

Redundancy Low redundancy

High power consumption Low power consumption

7

Notion of Frame

Tframe

TintegrationTime

x

y

Tintegration<Tframe

8

Dynamic Range

Rüedi et al, JSSC 2003 ”A 128x128 Pixel 120dB Dynamic-Range Vision-Sensor Chip for Image Contrast and Orientation Extraction”

Bad choice of the integration time

The human eye has a dynamic range higher than 10 decades

9

Outline

• The eye and the retina• Artificial Vs biological vision systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing

10

Depletion Region and Phototransduction

11

Sili

con

Electromagnetic Spectrum

12

Some Possible Photodetectors in CMOS Technology

13

A. Moini. ‘Vision Chips’. Kluwer Academic Publishers, 2000.

Photogates

14

Logarithmic Photoreceptors

(a) (b)

15

Photoreceptors with Negative Feedback

(a)

(b)16

Classic Active Pixel Sensor (APS)

17

Outline

• The eye and the retina• Artificial vision Vs biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing

18

Read-out Strategies:

Addressing/Scanning Charge Coupled Devices (CCD) Address Event Representation (AER)

19

Addressing/Scanning

Token System20

Charge Coupled Devices (CCD)

21

Address Event Representation (AER) (I)

22

Address Event Representation (AER) (II)

23

Outline

• The eye and the retina• Artificial Vs vision biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing

24

Delbrück’s Adaptative Photo Cell (I)

25

Delbrück’s Adaptative Photo Cell (II)

Non-linear element26

Mahowald’s Pixel

27

Retina Pixel Boahen

28

Dynamic Vision Sensor (DVS)

P. Lichtsteiner, C. Posch, and T. Delbrück, “A 128 128 120 dB 15 µs latency asynchronous temporal

contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008.

29

Common Source Amplifier

30

Can used as comparator

Examples of Biomorphic Vision Sensors

Spatio-Temporal Contrast Detector:http://folk.uio.no/juanle/Projects/Spatial_contrast_retina.html

Dynamic Vision Sensor:http://folk.uio.no/juanle/Projects/DVS.html

31

Outline

• The eye and the retina• Artificial vision Vs biological systems• Fundamentals of photo receptors • Read-out strategies• Examples of neuromorphic vision systems• Further processing

32

Further Image Processing

• Motion Detection• Features extraction

33

Token Based Motion Processing: Reichardt Detector

34

Intensity Based

35

Features Extraction

36

’Difference of Gaussians’ Kernel (I)

37

’Difference of Gaussians’ Kernel (II)

38

Result of Edge Detection

39

45 Degrees Edge Kernel (I)

40

45 Degrees Edge Kernel (II)

41