A Human Eye Retinal Cone Synthesizer

Michael F. Deering

Implementation Sketch For The SIGGRAPH 2005 Paper:Implementation Sketch For The SIGGRAPH 2005 Paper:

A Photon Accurate Model of the Human Eye

Michael F. Deering

Use Graphics Theory To Simulate VisionUse Graphics Theory To Simulate Vision

GoalGoal

• Build a computer program to properly simulate the complex sampling pattern of the human eye retinal cone mosaic.

• Use this in a photon by photon simulation of display devices onto the human eye.

Why Eye Sampling Pattern MattersWhy Eye Sampling Pattern Matters

OverviewOverview

• Background about human retinal cones

• Growth algorithm overview

• Cone force equation

• Re-forming cone cell borders

• Touch-up

• Results

Eye ModelEye Model

What Does A What Does A ConeCone Look Like? Look Like?What Does A What Does A ConeCone Look Like? Look Like?

What Do Cone Retinal Arrays Look Like?What Do Cone Retinal Arrays Look Like?

• For years all we had were photo micrographs of sliced and diced dead eyeballs.

• Now we can obtain images of living retinas.

Roorda And Williams ImageRoorda And Williams Image

Retinal Cone DistributionRetinal Cone Distribution

• Most data is from Curcio et. al. ’90

• Large variation in maximum density

• More recent data: Williams, Millar, Roorda

• Cone density varies primarily biased on eccentricity, but also by retinal meridian

Terminology: Cell BordersTerminology: Cell Borders

Plants have cell walls Animals don’t havecell walls; they have

cell borders(or cell membranes)

High Resolution Foveas Are A Relatively Recent AdditionHigh Resolution Foveas Are A Relatively Recent Addition

-2 months birth +6 years

Synthetic Retina GenerationSynthetic Retina Generation

• Use rectangular lattice.

• Use triangular lattice.

• Use perturbed triangular lattice.

• Take real retinal images as representative patches then flip and repeat.

I want all 5 million cones:

A new computer model to generate parameterized retinas (not synthesizing rods yet).

Possible Retina Generation AlgorithmsPossible Retina Generation Algorithms

• Add one new cone at a time, placing each into its final position.

– Too simplistic to work

• Simulate the interactions of all 5 million cones simultaneously.

– Too computationally complex to work

Retina Generation AlgorithmRetina Generation Algorithm

• Add new cones in concentric rings, varying target cell density by Curcio data

• Merge new cones into existing mosaic

• Grow on curved spherical surface

• Keep only changing cones in memory

Two Phase Cone Growth AlgorithmTwo Phase Cone Growth Algorithm

• Phase I: update the center location of all still active cone cells using the cone force equation.

• Phase II: re-form all cone cell borders from updated cone centers using pattern matching algorithm.

Run paired phases for 21-41 cycles per ring of new cones added.

DefinitionsDefinitions

• Normalized distance between cones p and n:

• Two cones p and n are neighbors if:

p-nD[p,n]

(p.r+n.r)

N[p,n] = D[p,n] 1.5

The Cone Force EquationThe Cone Force Equation

N[p,n]

n

p = p

+ K1 pn

+ K2 r

n-p - K3 spline[D[p,n]]

n-p

��������������

The Cone Force PictureThe Cone Force Picture

p

To center of fovea

p

p’

Definition Of Spline[ ] FunctionDefinition Of Spline[ ] Function

0 1

1

x

Spline[x]

Re-form Cone Cell Borders From Updated Cone CentersRe-form Cone Cell Borders From Updated Cone Centers

Why Vornoi Cell Construction Is InappropriateWhy Vornoi Cell Construction Is Inappropriate

No way to enforcecell size or shapeconstraints

Why Vornoi Cell Construction Is InappropriateWhy Vornoi Cell Construction Is Inappropriate

Always looking atthree vertices ata time.

Correct answerhere is just asingle newborder vertexfor all 4 cones.

My Cell Border Construction AlgorithmMy Cell Border Construction Algorithm

• Sequentially visit each cell.

• Using spatially indexed data structure, find all the neighbors of the cell and sort them into clockwise order.

• Apply cell border construction pattern rules to successive sequences of neighbors.

• Result is new set of border edges for that cell.

Sort Neighbors Into Clockwise OrderSort Neighbors Into Clockwise Order

p

n0

n1

n2

njnmax

ii N[p, n ]

Try Pattern Rules From Most Complex To Least ComplexTry Pattern Rules From Most Complex To Least Complex

• Only try a simpler pattern rule after all the more complex ones have failed.

• (The following slides will present the rules in the opposite order.)

Three Cone Centers Share Edge VertexThree Cone Centers Share Edge Vertex

p

nini+1

ej

N[ni, ni+1]

Three Cone Centers Don’t Share Edge VertexThree Cone Centers Don’t Share Edge Vertex

p

ni ni+1

ejej+1

i+1iN[n , n ]

Four Cone Centers Share Edge VertexFour Cone Centers Share Edge Vertex

p

ni ni+2ej

ni+1N[ni, ni+1]

N[ni+1, ni+2]

N[ni, ni+2]

D[p, ni] < D[p, ni+1] orD[p, ni+2] < D[p, ni+1]

Complex 5 Vertex CaseComplex 5 Vertex Case

p

ni

ni+2

ej

ni+1

q

N[p, q]

New Completed Cell BorderNew Completed Cell Border

pe0

e1 e2

e3

e4e5

Touch-upsTouch-ups

• Check re-formed cell borders for voids as large or larger than the local cone size; if they persist seed them with new cones.

• Check re-formed cell borders for cones too much smaller than their birth target size; if they persist delete them.

Extreme Cone Density Change Test CaseExtreme Cone Density Change Test Case

• Change the density control knob by a factor of 8 in area within a small distance.

Growth Sequence MovieGrowth Sequence Movie

Growth Movie ZoomGrowth Movie Zoom

Retinal Zoom Out MovieRetinal Zoom Out Movie

3D Fly By Movie3D Fly By Movie

Larger View Of My Synthetic RetinaLarger View Of My Synthetic Retina

Roorda Blood VesselRoorda Blood Vessel

Roorda vs. SyntheticRoorda vs. Synthetic

30x30 Pixel Face Input30x30 Pixel Face Input

Retinal Image ResultsRetinal Image Results

30x30 Pixel Movie30x30 Pixel Movie

Result MovieResult Movie

AcknowledgementsAcknowledgements

• Michael Wahrman for the RenderMan™ rendering of the cone data.

• Julian Gómez and the anonymous SIGGRAPH reviewers for their comments on the paper.