Basic Visual ProcessesBasic Visual Processes

Introduction and brief review of Introduction and brief review of theories in visual perceptiontheories in visual perception

Outline of the course as a wholeOutline of the course as a whole

Week 1 – outline of the course as a whole, Week 1 – outline of the course as a whole, theories of visual perceptiontheories of visual perception

Week 2 – ‘Review’ of visual anatomy and Week 2 – ‘Review’ of visual anatomy and physiologyphysiology

Succeeding weeks – coverage of colour, Succeeding weeks – coverage of colour, contour, depth and motion with about two contour, depth and motion with about two weeks devoted to each topicweeks devoted to each topic

Format of the courseFormat of the course

First two weeks will be mostly lectureFirst two weeks will be mostly lectureSucceeding weeks will be seminar format Succeeding weeks will be seminar format – I’ll give some introductory material and – I’ll give some introductory material and you’ll present summaries of papers (about you’ll present summaries of papers (about 3/week, so one presentation every 2 3/week, so one presentation every 2 weeks or so)weeks or so)Paper due at the end of the termPaper due at the end of the termGrade will be some generous combination Grade will be some generous combination of work on paper and on presentationsof work on paper and on presentations

Brief History of Theories in Visual Brief History of Theories in Visual PerceptionPerception

Early psychophysicsEarly psychophysics

The Gestalt movementThe Gestalt movement

Brunswik’s probabilistic functionalismBrunswik’s probabilistic functionalism

ConstructivismConstructivism

Direct PerceptionDirect Perception

ComputationalismComputationalism

Bayesian modelsBayesian models

The birth of psychophysicsThe birth of psychophysics

Ernst WeberErnst Weber (1795-1878)(1795-1878)

Weber Fraction (1846):Weber Fraction (1846):

S/S=KS/S=K

Fechner’s revelationFechner’s revelation

Gustav Fechner Gustav Fechner (1801-(1801-1887)1887)

S=C*log(I) (1860)S=C*log(I) (1860) This represented more This represented more

than an algebraic re-than an algebraic re-arrangementarrangement

Fechner imagined that Fechner imagined that the JND’s represented the JND’s represented in the equation above in the equation above and also in the Weber and also in the Weber fraction must be the fraction must be the units of sensationunits of sensation

The philosophical contextThe philosophical context

Kant argued that any Kant argued that any science of the mind science of the mind was impossible was impossible because the nature of because the nature of the external world the external world (and hence its (and hence its relationship to the relationship to the mental world) was mental world) was inherently inherently unknowable.unknowable. Immanuel Kant (1724-1804)

Structuralism and the early Structuralism and the early empiricistsempiricists

““The ‘taste’ of lemonade is made up of a The ‘taste’ of lemonade is made up of a sweet taste, an acid taste, a scent (the sweet taste, an acid taste, a scent (the fragrance of lemon), a sensation of fragrance of lemon), a sensation of temperature and a pricking (cutaneous) temperature and a pricking (cutaneous) sensation. The ‘taste’ of limewater is sensation. The ‘taste’ of limewater is made up of a weakly sweet taste, a made up of a weakly sweet taste, a sensation of nausea (organic sensation), a sensation of nausea (organic sensation), a sensation of temperature and a biting sensation of temperature and a biting (cutaneous) sensation.”(cutaneous) sensation.”

-Titchener (1901)--Titchener (1901)-

The Gestalt movementThe Gestalt movement

Max Wertheimer and Max Wertheimer and the phi-phenomenonthe phi-phenomenon Two alternating Two alternating

flashing lights can flashing lights can produce a sensation of produce a sensation of pure movement that pure movement that inheres in neither light inheres in neither light alone.alone.

Max Wertheimer (1880-1943)

“The basic thesis of gestalt theory might be formulated thus: there are contexts in which what is happening in the whole cannot be deduced from the characteristics of the separate pieces, but conversely; what happens to a part of the whole is, in clearcut cases, determined by the laws of the inner structure of its whole.” (1924)

Wertheimer’s laws of groupingWertheimer’s laws of grouping

Key principles of the Gestalt Key principles of the Gestalt approachapproach

Reformulation of the problem to be solved Reformulation of the problem to be solved (looking out vs. looking in):(looking out vs. looking in):

There seems to be a single starting point for psychology, exactly as for There seems to be a single starting point for psychology, exactly as for all the other sciences: The world as we find it, naively and all the other sciences: The world as we find it, naively and uncriticallyuncritically

-Wolfgang Kohler (1947)-Wolfgang Kohler (1947)

Key principles of the Gestalt Key principles of the Gestalt approachapproach

PragnanzPragnanz Perception moves towards simplicity, Perception moves towards simplicity,

symmetry and wholeness, as in Wertheimer’s symmetry and wholeness, as in Wertheimer’s rulesrules

An interesting and ironic twist here is that the An interesting and ironic twist here is that the Gestaltists took inspiration from Kant – they Gestaltists took inspiration from Kant – they believed that they were discovering the believed that they were discovering the physical basis of his metaphysicsphysical basis of his metaphysics

Key principles of the Gestalt Key principles of the Gestalt approachapproach

Kohler’s troubled Kohler’s troubled formal theory of formal theory of isomorphismisomorphism

Wolfgang Kohler (1887-1967)

Brunswik’s probabilistic Brunswik’s probabilistic functionalismfunctionalism

““The environment The environment scatters its effects”scatters its effects” There is only a There is only a

probabilistic probabilistic relationship between relationship between the proximal stimulus the proximal stimulus and the external worldand the external world

The perceiver as a The perceiver as a boxer (“smallness of boxer (“smallness of error at the expense of error at the expense of highest frequency of highest frequency of precision”)precision”)

Egon Brunswik (1903-1955)

Brunswik’s lasting influenceBrunswik’s lasting influence

One of the first theorists to recognize the One of the first theorists to recognize the complexity of the proximal stimulus complexity of the proximal stimulus (influenced Gibson).(influenced Gibson).

Probabilistic approach to perception was Probabilistic approach to perception was very modernvery modern Eg. Brunswik and Kamiya (1953) did scene Eg. Brunswik and Kamiya (1953) did scene

analysis showing why Wertheimer’s grouping analysis showing why Wertheimer’s grouping laws made senselaws made sense

Why Brunswik isn’t better known:Why Brunswik isn’t better known:

Here’s a quote:Here’s a quote:““Perception, then emerges as that relatively primitive Perception, then emerges as that relatively primitive

partly autonomous, institutionalized, ratiomorphic partly autonomous, institutionalized, ratiomorphic subsystem of cognition which achieves prompt and subsystem of cognition which achieves prompt and richly detailed orientation habitually concerning the richly detailed orientation habitually concerning the vitally relevant, most distal aspects of the environment vitally relevant, most distal aspects of the environment on the basis of mutually vicarious, relatively restricted on the basis of mutually vicarious, relatively restricted and stereotyped, insufficient evidence in uncertainty-and stereotyped, insufficient evidence in uncertainty-geared interaction and compromise, seemingly geared interaction and compromise, seemingly following the highest probability and smallness of following the highest probability and smallness of error at the expense of the highest frequency of error at the expense of the highest frequency of precision.” (1956)precision.” (1956)

The physiological revolutionThe physiological revolution

Can facts about the nervous system Can facts about the nervous system explain facts about perception?explain facts about perception?

Helmholtz and colour visionHelmholtz and colour vision

Hermann Helmholtz (1821-1894)

Early trichromaticity theory based on clever inferences and good guesses

Warren McCulloch, brains and Warren McCulloch, brains and cyberneticscybernetics

““The assumption has always The assumption has always been that the eye mainly been that the eye mainly senses light, whose local senses light, whose local distribution is transmitted to distribution is transmitted to the brain in a kind of copy by a the brain in a kind of copy by a mosaic of impulses. Suppose mosaic of impulses. Suppose we held otherwise, that the we held otherwise, that the nervous apparatus in the eye nervous apparatus in the eye is itself devoted to detecting is itself devoted to detecting certain patterns of light and certain patterns of light and their changes, corresponding their changes, corresponding to particular relations in the to particular relations in the visible world.”visible world.” Warren McCulloch (1898-1972)

What the frog’s eye tells the frog’s What the frog’s eye tells the frog’s brainbrain

Humberto Maturana (1928-)Jerome Lettvin (1912- Walter Pitts (1923-1969)

McCulloch

Feature detection in the amphibian Feature detection in the amphibian nervous systemnervous system

‘‘Convexity detectors’ looked like they Convexity detectors’ looked like they would be good for catching bugswould be good for catching bugs

Again, the ghost of Kant is invoked as Again, the ghost of Kant is invoked as Lettvin et al claim they have found the Lettvin et al claim they have found the physical basis of Kant’s transcendental physical basis of Kant’s transcendental aesthetic.aesthetic.

Hubel and Wiesel’s physiological Hubel and Wiesel’s physiological studies of cortexstudies of cortex

David Hubel (1926-) Torsten Wiesel (1924-)

Single units and visual perceptionSingle units and visual perception

Horace Barlow

“A description of that activity of a single nerve cell which is transmitted to and influences other nerve cells and of a nerve cells’ response to such influences from other cells, is a complete enough description for functional understanding of the nervous system. There is nothing else ‘looking at’ or controlling this activity, which must therefore provide a basis for understanding how the brain controls behaviour.”

Modern constructivismModern constructivism

Irvin Rock (1922-1995)Richard Gregory (1923-)

Perception as inferencePerception as inference1. Fuse two images to produce

two planes in depth

2. While keeping images fused, move head back and forth.

3. You will see that the front plane follows motion of your head

4. Rock’s explanation is that there is no accretion/deletion of texture elements as one would see if the front plane was stationary, so the visual system constructs a model of the world that fits the facts.

Another example for the stereo-Another example for the stereo-blindblind

The Necker cube gives impression of depth and we can choose more or less at will which face of the cube is at the front

While viewing a real 3-dimensional version of the Necker cube, see what happens when you move your head

Charlie Chaplin IllusionCharlie Chaplin Illusion

Gibson’s direct perceptionGibson’s direct perception

James Gibson (1904-1979)

“Perceiving is an act, not a response, an act of attention, not a triggered impression, an achievement not a reflex”

Key features of direct perceptionKey features of direct perception

Perception does not usually proceed in Perception does not usually proceed in laboratory conditions (note nod to laboratory conditions (note nod to Brunswik)Brunswik)

The stimulus for vision is a very The stimulus for vision is a very complicated pattern of light that is complicated pattern of light that is transformed in useful and interesting ways transformed in useful and interesting ways when we move.when we move.

The ambient optic arrayThe ambient optic array

Static observation is relatively rare. Instead, we move about in space and obtain vast amounts of information from these movements

Perceptual invariantsPerceptual invariants

Everything we need to know is contained Everything we need to know is contained directly in the optic array in the form of directly in the optic array in the form of invariants. These invariants do not require invariants. These invariants do not require inference or computation.inference or computation.

One example of an invariant – The One example of an invariant – The focus of expansionfocus of expansion

Another example – size constancyAnother example – size constancy

The ratio of a/b (a’/b’) remains constant with object distance

David Marr and the computational David Marr and the computational revolutionrevolution

David Marr (1945-1980)

“The best way of finding out the difficulties of doing something is to try to do it.”(1982)

“Trying to understand perception by studying only neurons is like trying to understand bird flight by studying only feathers: It just cannot be done.” (1982)

The levels at which information The levels at which information processing must be understoodprocessing must be understood

Computational theoryComputational theory What is the goal and why is it appropriate?What is the goal and why is it appropriate?

Representation and algorithmRepresentation and algorithm What is the representation for the input and What is the representation for the input and

output and what is the algorithm for the output and what is the algorithm for the transformation?transformation?

Hardware implementationHardware implementation How can the representation and algorithm be How can the representation and algorithm be

realized physically?realized physically?

Bayesian models of perceptionBayesian models of perception

P(S|I)=P(S)*P(I|S)/P(I)P(S|I)=P(S)*P(I|S)/P(I)

EXAMPLE:EXAMPLE:

-trying to figure out whether that dark spot in your -trying to figure out whether that dark spot in your periphery is a spider climbing the wall:periphery is a spider climbing the wall:

-should depend on P(S), the likelihood that a -should depend on P(S), the likelihood that a spider would be climbing the wallspider would be climbing the wall

-should depend on the likelihood that a spider -should depend on the likelihood that a spider climbing the wall would produce such a dark spot, climbing the wall would produce such a dark spot, normalized by the total probability of getting Inormalized by the total probability of getting I

Bayesian perceptionBayesian perception

Should recognize traces of constructivism Should recognize traces of constructivism (constructing a world based on evidence of (constructing a world based on evidence of senses)senses)

Should also see a clear (and acknowledged) link Should also see a clear (and acknowledged) link to Brunswikto Brunswik

Some argue that these models are compatible Some argue that these models are compatible with direct perception but I think Gibson would with direct perception but I think Gibson would have disagreed.(argument seems to be that P(I|have disagreed.(argument seems to be that P(I|S)/P(I) is something like a formalization of an S)/P(I) is something like a formalization of an invariant.)invariant.)

SummarySummary

The modern consensus takes much from each of the The modern consensus takes much from each of the approaches that I’ve describedapproaches that I’ve describedMost agree with Marr that the information processing Most agree with Marr that the information processing required for even very simple visual tasks is enormously required for even very simple visual tasks is enormously complex and unlikely to be directcomplex and unlikely to be directNew Bayesian models of perception emphasize how New Bayesian models of perception emphasize how much of perception must be probabilistic – much of much of perception must be probabilistic – much of these ideas are compatible with constructivism and with these ideas are compatible with constructivism and with Marr’s approachMarr’s approachSome new models (eg. Joel Norman) are rooted in Some new models (eg. Joel Norman) are rooted in anatomy and suggest that more than one of the anatomy and suggest that more than one of the approaches I’ve described find life in our brains.approaches I’ve described find life in our brains.