Early Vision
description
Transcript of Early Vision
![Page 1: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/1.jpg)
Early Vision
Bruce DraperDepartment of Computer ScienceColorado State University
![Page 2: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/2.jpg)
Overview
“The Biomimetic Vision Trilogy”1. Selective Attention
– Understanding the problem– Last week
2. Early Vision– Understanding the literature– Today
3. Ventral vision – Understanding object recognition– March 9 (next week)
![Page 3: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/3.jpg)
General Theme
Vision evolved to serve the needs of animals– Vision is action oriented (it guides behavior)
Actions may be immediate (e.g. grasp, navigate) Actions may be delayed (“perception”)
– Vision is not one system As animals became more complex, more and more visual
capabilities evolved in separate systems
Note: this is not a new idea. See The Visual Brain in Action by Milner & Goodale 1996; The Metaphorical Brain by Arbib 1972; or even Cybernetics by Weiner 1948.
![Page 4: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/4.jpg)
Input: The Eye(s)
Start at the beginning: Lens focuses light Iris serves as aperture Retina contains receptors Optic nerve transmits to brain
Lens, iris are controlled by muscles under the control of the brain
S. Palmer. Vision Science. P. 27
![Page 5: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/5.jpg)
Retina as Processor
• Five cell types:• receptor (rod/cones)
Species dependent• horizontal• bipolar cells• amacrine cells• ganglion cells
• Its inside out!• Blind spot where optic nerve passes through retina
S. Palmer. Vision Science. P. 30
![Page 6: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/6.jpg)
Retina (cont.)
Ganglion Cells– The first cells to produce spike discharges
Other retinal cells use graded potentials Spikes are needed for long distance communication
– On-center off-surround– Off-center on-surround
![Page 7: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/7.jpg)
Two Types of Ganglion Cells
P ganglion cells in primates (like Y cells in cats):– Large receptive fields (low frequency?)– Transient response– Fast transmission– Receive inputs from all colors and from rods
P ganglion cells (like X cells in cats):– Small receptive fields (high frequency?)– Sustained response– Medium transmission– Color opponent channels
![Page 8: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/8.jpg)
Fields of View & Stereo
Right hemisphere receives the left visual field from both eyes
– And vice-versa– Splitting the field of view
supports disparity computations
High resolution in fovea, lower elsewhere
– Fovea is ±2° (thumbnail at arms length)
![Page 9: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/9.jpg)
Visual Projections
The eye + optic nerve is a shared device There are eleven projections (“endpoints”) of the
optic nerve:– Retinogeniculate Projection
Onto LGNd (Lateral Geniculate Nucleus, dorsal) and then to V1 (a.k.a. primary visual cortex, striate cortex)…
This path is dominant in people; barely evident in non-mammals
– Retinotectal Projection Onto Superior Colliculus, then the Pulvinar Nucleus, then LGN,
V1, MT, higher level centers… This path is dominant in non-mammals; evolutionarily older Involved in eye movements, motion, tracking
![Page 10: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/10.jpg)
Projections (LGN & S.C.)
S. Palmer. Vision Science. P. 25.
![Page 11: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/11.jpg)
Projections (II)
At least 8 more (minor) projections!– Retina Suprachiasmatic Nucleus
Circadian rhythms SN is part of the hypothalamus (like LGNd) SN receives multi-modal projections
– Retina Nucleus of the Optic Tract (NOT) Optokinetic nystagmus
– Retina Accessory Optic Nuclei Visual control of posture, locomotion
– Retina Pretectum Pupillary Light Reflex
![Page 12: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/12.jpg)
Two LGNd Channels P cells in the retina project to the two magnocellular
(“large cell”) layers in the LGN.– Livingstone & Hubel: color-blind, fast, high contrast sensitivity, low
spatial resolution P cells in the retina project to the four parvocellular
(“small cell”) layers in the LGN.– L&H: color selective, slow, low contrast sensitivity, high spatial
resolution LGNd also has interlaminar layers with unknown
role/properties– Receives projections from optic nerve, S.C.
Right at the levels of cells; wrong at the level of populations
![Page 13: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/13.jpg)
Primary Visual Cortex (V1)
First cortical visual area– Columnar (like all cortex)
Retinotopically mapped Ocular dominance columns Edges (Gabor filters), color,
disparity & motion maps Connects to other
retinotopic areas (V2, V3, MT)
http://webvision.med.utah.edu/imageswv/capas-cortex.jpg
![Page 14: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/14.jpg)
Proof of Retinotopic Mapping
Pattern flashed (like a strobe) in front of monkey
injected with sugar dye
Left primary visual cortex of the same
monkey
Tootell, et al. 1982.
![Page 15: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/15.jpg)
Single Cell Recordings in V1
![Page 16: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/16.jpg)
Two Channels in V1
P Magnocellular LGN Layers layer 4C of V1.– Projects further to layer 4B
Motion direction selective Orientation selectivity & binocular No color
P Parvocellular LGN 4C of V1– Projects to layers 2 & 3.
Layers 2 & 3 subdivide into “blobs” and “interblobs”:– Blobs are color selective, simple receptive fields– No orientation/movement direction selectivity, binocularity.– Prefer low frequencies, have small Magnocellular input – Interblobs have reduced (non-zero) color selectivity– Binocular, high-frequency, orientation selective
![Page 17: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/17.jpg)
Simple Cells
The first orientation selective cells found in V1 were labeled “simple cells” – Well approximated by Gabor functions with fixed
orientations, scales and phases
Jones & Palmer 1987
![Page 18: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/18.jpg)
Complex Cells
The second set of orientation selective cells were called complex cells– Well modeled as combining 90° out-of-phase
Gabor responses (quadrature pairs)– Captures energy at a particular orientation &
scaleevenfilter
oddfilter
+
![Page 19: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/19.jpg)
Organization of cells in V1
Hubel & Weisel proposed the following organization for cells in V1
![Page 20: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/20.jpg)
Well, a little more complex…
Ocular dominancecolumns
Color-coded orientationSensitivity columns
![Page 21: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/21.jpg)
More on V1
Only about 27% of V1 cells are orientation selective About 70% of orientation selective cells are complex
cells Orientation selective cells also include end-stopped
cells (a.k.a. hypercomplex cells), and grating cells. Non-orientation selective cells include cells that
respond to:– Color (Hue/Sat maps?)– Disparity– Motion
![Page 22: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/22.jpg)
V1 Connections
V1 is the starting point of cortical visual processing.
Dorsal projections lead to somatosensory and motor control areas
Ventral projections lead toward associative memories
From Van Essen 1992. Image can be found athttp://webvision.med.utah.edu/imageswv/Visual-Cortex1.jpg
![Page 23: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/23.jpg)
Anatomical Maps of Visual Cortex
1983 Version 1990 Version
![Page 24: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/24.jpg)
Two Visual Subsystems
In 1982, Ungeleider & Mishkin propose that there are two primary visual pathways in humans and primates:– The dorsal (or “what”) pathway
Ends in the posterior parietal cortex– The ventral (or “where”) pathway
Ends in the inferotemporal cortex
![Page 25: Early Vision](https://reader035.fdocuments.net/reader035/viewer/2022062811/56815f6d550346895dce747b/html5/thumbnails/25.jpg)
Visualizing Two Subsystems
D. Milner & M. Goodale, The Visual Brain in Action, p. 22