PERCEPTION TOPIC NOTES: Introduction to the Senses

1. Classification

There are five major groups of senses:

Sense Stimulus Receptor Sensory Structure

Vision Electromagnetic energy (light) photoreceptors Eye

Hearing Air pressure waves mechanoreceptors Ear

Touch Tissue distortion mechanoreceptors Skin, muscles, etc.thermoreceptors

Balance Gravity, acceleration mechanoreceptors vestibular organs

Taste/smell Chemical composition chemoreceptors nose, mouth

Certain basic physical and perceptual principles apply to all the senses.

2. General physical principles

1) TransductionAll senses need to convert ('transduce') environmental energy into electrical energy.

Receptors in each sense are specialised for transducing particular forms of energy into electricalsignals for transmission to the brain.

2) Sensory pathwaysElectrical signals from all the senses are transmitted ultimately to the cerebral cortex via

relays in the thalamus. All senses involve both cortical and sub-cortical processes, but differ inthe area of cortex involved, and the involvement of sub-cortical structures. Cortical analysis isthought to be linked with conscious experience, and sub-cortical analysis is thought to beinvolved in unconscious events eg. reflexes.

3) SelectivityIndividual neurons in each sensory system are very selective in their response to a

sensory stimulus; each has a RECEPTIVE FIELD, a restricted area of the sensory space withinwhich the stimulus must fall for the cell to respond. Different cells have receptive fields indifferent parts of the space (eg. spatial position on the body, position in the visual or auditoryfield, colour, sound frequency).

4) Sensory mapsCells with similar receptive fields tend to be found near to each other in the sense organ

or cortex, so that a map of the sensory surface is formed on the surface of the organ or cortex.These maps tend to exhibit distortions favouring certain areas of the visual field, or parts of thebody, or certain sound frequencies. Sensory coding requires interactions between cells withsimilar properties, so proximity simplifies wiring.

5) Specific nerve energyAll sense organs generate the same kind of electrical signals. How can they evoke

different sensations? Differences between senses are not reflected in the sensory signals, but intheir destination in the brain. During neurosurgery, small electrical signals are often applieddirectly to the cortex of an awake (anaesthetised) patient. This stimulation evokes sensationsassociated with the particular sense organ connected to that part of the cortex.

3. General perceptual principles

1) ThresholdsAll sense organs require a certain minimum amount of stimulation ('absolute threshold')

before they evoke a perceptual sensation. Note that detection threshold is not the same asthreshold for discrimination or identification. We can also measure the 'difference threshold' -minimum difference between two stimuli that can be detected (look up Weber's Law).

2) Sensory magnitudeAbove absolute threshold, there is a range of stimulus intensity levels which generates

corresponding variations in sensory magnitude, providing valuable information (eg. proximity,force). The variation in sensory magnitude with stimulus intensity is not linear (look up Stevens'Power Law).

3) AdaptationSensitivity must be matched to the physical stimulation available. Extreme sensitivity

would be undesirable (eg. itching, deafening). Lack of sensitivity is obviously also undesirable.All modalities have the ability to shift their operating range ie. shift the range of physicalintensity levels which generate a response. This allows us to adapt to prevailing environmentalconditions and remain sensitive to change (light and noise level, skin stimulation). Withinmodalities, many subtle adaptation effects can also be generated, which tell us a lot about howthe system operates.

4. Coverage of the senses in this course Vision will receive the most coverage, followed by hearing. The other senses will be

covered in much less detail. Why? Because we know most about vision, and because it is themost important sense in several respects:

1) It has the greatest range of operation.

2) Cortical areas devoted entirely to vision cover about 60% of the total cortical surface (inprimates).

3) Experiments demonstrate that when visual information is set into conflict with other senses,vision dominates eg. McGurk effect, ventriloquism, Lee and Aronson (1974), Rock and Harris(1967).

PERCEPTION TOPIC NOTES: Touch and balance

1. Touch

1) PhysiologyOur sense of touch is based on a set of receptors which respond to deformation of tissue.

There are four basic types:- Meissner corpuscles (most superficial, punctate)- Merkel discs- Ruffini endings- Pacinian corpuscles (deepest, diffuse)

The type of the receptor affects the adaptive properties of its nerve fibre (corpuscles tend to givetransient responses), and its location affects the fibre's spatial properties (punctate vs. diffuse).In addition, there is a network of free nerve endings (with no end receptor) below the skinsurface and wrapped around the base of hair follicles, which respond to slight bending of thehair.

The afferent fibres from receptors enter the spinal cord. Some connect with motorneurons which travel back to muscles in the body region where the afferent fibres originated.This in-out circuit mediates reflex reactions. Signals from other fibres are relayed to the brain(lemniscal pathway), where they arrive eventually in the somatosensory cortex. The corticalsurface is arranged in an orderly way, with some body areas covering more cortical surface thanothers (an property known as cortical magnification).

2) PerceptionQuantitative aspects Our sense of touch differs markedly in its sensitivity and in its

acuity in different body regions. Sensitivity is highest at the lips and fingertips, and lowest in theback and stomach. Similarly, two-point acuity is only 2mm on the finger, 30mm on the forearm,and 70mm on the back. This variation in response corresponds with the variation in cortical areadevoted to different body parts.

Qualitative aspects Our sense of touch varies qualitatively - we can perceive roughness,pressure, warmth, and pain. Early theories proposed that different receptors were associatedwith different sensory qualities, but experiments have not supported them (some investigatorsmapped sensory spots on their skin and then dissected it to find what receptors wereunderneath). A single event will generate responses in several receptor types simultaneously.Our perception of the event arises from a combination of all responses.

2. Balance The vestibular sense provides information about the head's attitude relative to

gravitational vertical, which is used to maintain posture, stabilise eye position, and compensatefor image motion introduced by head movement.

Unlike other senses, the information generated by the vestibular sense does not usuallyimpinge on consciousness unless unusual or violent body movements are involved (eg. abrupthalt after spinning). Individuals who suffer loss of vestibular function (due to toxic drugs ordisease) are not aware of an obvious deficit, but they have difficulty in maintaining balance onuneven or compliant surfaces, and experience visual disturbance (movement of the visual field)during irregular head movement (eg. travelling over uneven surfaces).

1) PhysiologyThe vestibular apparatus (about the size of a pea) contains three fluid-filled curved tubes

(semicircular ducts) at right-angles to each other. Each duct contains a sheet of sensory hair cells(cilia) stretched across it to form a diaphragm. The cilia are connected to sensory cells whichrespond to deflection of the cilia. The three ducts are arranged so that angular acceleration of thehead in any plane will deflect at least one set of cilia, and so generate a sensory response.

The vestibular afferents terminate primarily in the vestibular nucleus, which also receivesinformation from joint receptors and from the eyes. Complex interconnections are made with

other sub-cortical brain structures, such as the cerebellum, and a small area of the cortex receivesprojections from the vestibular nucleus (via the thalamus as usual). The complex sub-corticalprocessing and relatively small cortical projection probably explains why vestibular signals havelittle conscious representation. However, we do have some conscious awareness of bodymovement, and direct brain stimulation in the appropriate area evokes sensations of bodyrotation and displacement.

2) PerceptionIt is very difficult to study the vestibular sense in isolation from other senses, because

there are many non-vestibular cues to orientation which must be excluded. Apart from vision,pressure receptors in supporting tissue, joint position receptors, and muscle feedback all giveinformation about forces acting on the body. Elaborate precautions are needed, such as totalimmersion in water, so instead most studies examine the interactions between vestibular andnon-vestibular sensations. The clearest example of this interaction is motion sickness,characterised by nausea and vomiting, which is thought to arise from a mismatch of sensorycues from vestibular and non-vestibular senses. For example, the movement of ships or planescan generate vestibular signals which are not accompanied by visual information about themovement. The problem is worsened by the tendency for the changes in direction and magnitudeof acceleration to be abnormally slow. Another example of vestibular sensations intruding inconsciousness is the nausea and vertigo caused by alcohol consumption. There is evidence thatalcohol alters the relative density of the fluid in the semicircular ducts and the sheet of hair cellsstretched across them, so that abnormal signals are generated which vary with head position.Some have speculated that nausea associated with abnormal vestibular signals evolved as anadaptive response to ingested poisons, which may interfere with sensory signals and motor co-ordination.

PERCEPTION TOPIC NOTES: Smell and Taste

Smell

In humans smell is considered a minor sense compared to vision and hearing, though itis much more important for many animals. Even in humans it can be surprisingly important.Smell is a major component of 'taste', accounting for the relative unpalatability of food when wehave a cold. Potato and apple are indistinguishable if eaten holding the nose, and the flavour ofmeats, fruits, butter, and coffee is also largely removed. Experiments have demonstrated that wecan distinguish gender on the basis of hand or breath smell.

1) PhysiologyReceptors Olfactory receptors are found in the roof of the nasal cavity, on a patch of

tissue called the olfactory epithelium. In humans this tissue area covers 2-4 sq.cm. and contains10 million receptors (in dogs it covers 18 sq.cm. and contains 200 million receptors). The actualreceptor sites are on hairs (cilia) which project from each receptor cell. Chemicals given off byvolatile substances trigger responses in individual cells, and different cells appear to respond todifferent chemical components of odours. The exact mechanism of transduction is still disputed.The olfactory epithelium also contains free nerve endings which have no cilia or receptor sites.Although they cannot mediate odour perception, it is thought that free nerve endings mediate the'chemical sense' described below.

Pathways Signals from receptors terminate in the olfactory bulb, which protrudes fromthe front of the brain, and are then relayed up the olfactory tract. They travel to a small area ofcortex (via the thalamus), and also to the limbic system (thought to be involved in emotion).

2) PerceptionDimensions In 1916 Henning devised the 'smell prism' based on subjects'

classifications of over 400 odours. The six corners of the prism contain categories of smell(putrid, flowery, fruity, burnt, spicy, resinous), and Henning argued that each smell weperceive can be located on the surface of the prism according to the combination of qualities itevokes. Since then others have disputed whether it is possible to arrive at neat verbal labels forall smells. A different technique simply asks subjects to judge the similarities between smells,searching for clustering of qualities. Smells were found to vary along only two dimensions; onecorresponds to pleasant/unpleasant, the other has no simple verbal label.

Detectability varies hugely with the odour. Some chemicals are detectable atconcentrations 10 million times weaker than others. Sensitivity also varies with gender -womenare more sensitive than men.

Identification is poor near detection threshold, but once above threshold we candistinguish many more odours than we can attach verbal labels to. Women are, again, better thanmen. Practice has a major influence on our ability to identify odours.

Adaptation The perceived intensity of a strong smell drops by 30% after continuousexposure, and weak smells may become undetectable. This explains why we cannot smell ourown body odour. Cross-adaptation experiments show that adaptation is selective to similarsmells Eg. adaptation to lemon does not affect the smell of peanut butter, but does affect thesmell of limes (implications for food order in multicourse meals).

Disorders Odour blindness or anosmia can be temporary, following a blow or inhalationof caustic agents, or can be permanent and specific. Eg. 3% of people are insensitive to sweatsmell, 12% to musk, and 47% to urine.

Common Chemical Sense Many smells produce a 'feeling' as well as an odour(coolness, tingle, burning), which is thought to arise from stimulation of free nerve endings byhigh concentrations of chemicals.

Taste

In today's developed society, taste is used mainly to define our preferences for specificfoods, but historically it played a major role in distinguishing between palatable and poisonoussubstances, and it can still have powerful effects (see Conditioned Taste Aversion below). Many

plants and animals have evolved to taste bitter, fending off predators eg. the skin of toads iscovered in glandular cells that secrete a bitter-tasting substance.

1) Physiology

Receptors are carried in the taste buds, found on the tongue, in the lining of the mouth,and in the throat (some insects have them on their feet). There are about 10,000 in the mouth,though numbers vary with age, peaking at about 40. The exact mechanism of transduction is stillnot known. Pathways involve destinations in the parietal cortex as well as the limbic system.

2) Perception

Dimensions There are thought to be four categories (sweet, sour, salty, bitter) which canbe picked out as components of a taste.

Sensitivity depends on the taste category and the location on the tongue. Sweet & saltyreceptors are at the front, sour along the sides, and bitter at the back. Sensitivity also depends ontemperature, with bitter/salty sensitivity dropping at higher temperatures, and sweet sensitivityimproving (season food at serving temperature).

Adaptation occurs for taste just as it does for smell, and is also specific for taste categoryeg. adaptation to salty taste does not affect perception of sour.

Bitterness sensitivity varies between individuals, and has a genetic component. It may beinvolved in preferences or dislikes for coffee (do you take sugar and milk?), bitter beer, andsaccharin.

Conditioned taste aversion If a food induces illness, then many animals show anaversion to consuming that food again. The effect is very powerful, and requires only a singlelearning episode. There have been many experimental demonstrations of the effect in animalsgiven food laced with nausea-inducing chemicals (eg. in rats, mice, cats, monkeys, ferrets,birds, fish, reptiles, humans after too much beer?). The aversion apparently requires noconscious awareness, and may even develop when a particular food is consumed just before about of illness unconnected with consumption of the food.

PERCEPTION TOPIC NOTES: Light and the eye

Early Greeks thought that the eyes emitted a kind of 'fire' that touched and revealed objects.Today we know that the eye receives rather than emits energy.

1. Why have a light sense?

Our senses are specialised for gathering information about the environment. Without thisinformation we would obviously not survive very long. There are several reasons why the lightsense reigns supreme as a source of information-

(1) It is a distance sense, not requiring proximity to the source.(2) Light travels fast (186,000 miles/sec in a vacuum).(3) Light travels in straight lines, so an image can be formed which reflects the geometricproperties of the object emitting or reflecting it.(4) There is a lot of it about.

2. What exactly is light?

Light is a form of electromagnetic radiation, which behaves in two ways, both importantfor perception-

(1) It travels in waves, which can be specified in terms of wavelength or frequency. Thewave nature of light underlies our sensation of colour.(2) It is emitted as many small packets or quanta. The quantal nature of light underliesour sensation of intensity.

The easiest way to reconcile these apparently differing descriptions is to think of radiation interms of small wave packets, each packet is a short train of waves extending over a distanceroughly corresponding to the size of a quantum.

The wavelength of electromagnetic radiation varies from a few trillionths of a metre to manykilometres. The range of wavelengths visible to animals as light occupies only a very small part(1/70th) of the spectrum, from about 360 nm to 760 nm (1 nm is a billionth of a metre). Why?

(1) Atmospheric ozone filters solar radiation so that 4/5ths of it lies between 300 and1100 nm.(2) Water filters out the extremes of the remaining wavelengths: a few metres willremove the short ultraviolet wavelengths, and about a metre will remove infrared,leaving only a range between 380 and 760 nm.(3) Imperfections in the lens (chromatic aberration) would make it impossible to form asharply focussed image if more extreme wavelengths were detectable by thephotoreceptors.

'White' light actually contains many different wavelengths that combine to give an impression ofwhite light. Different 'white' light sources often contain slightly different distributions ofwavelengths.

3. Intensity, brightness, and lightness

These terms are often used interchangeably in everyday life, but they have very specificmeanings in vision science. Physical measures of light intensity must not be confused withperceptual measures of brightness and lightness. Light intensity is measured with devicescalled photometers, usually in units of candelas per square metre (cd/sq.m.). Brightness is thesubjective experience of the intensity of a light. Lightness is the impression of a surface’sreflectance (what proportion of the incident light it reflects). As with other perceptual quantities,brightness and lightness are often measured using matching or discrimination techniques.

The human visual system is sensitive to an enormous range of light intensities, varying by afactor of 1016 (a change by a factor of 106 is a million-fold increase or decrease). Pupilvariation cannot account for this sensitivity range, since pupil size varies between 2 and 8 mm (asixteen-fold variation). Pupil size indicates emotional state.

4. Optical properties of light.

When light strikes an interface between two surfaces with different optical properties,one of three things can happen:

(1) It may be absorbed and converted into another form of energy,usually heat.(2) It may be reflected back. Smooth surfaces (surface inhomogeneities small comparedto wavelength, eg. mirrors) preserve the spatial layout of the light waves (specularreflectors). Rough surfaces relect light diffusely in all directions.(3) It may be transmitted through the medium. Light slows down when passing throughany medium other than a vacuum (eg. air, water, glass) and different media slow up lightby differing amounts. When light passes across the interface between different media itmay change direction (refraction).

Lenses exploit refraction, so that rays which diverge from a point on a source are refracted onpassage through the lens so that they converge on a second point, creating an image of then firstpoint.

5. How do eyes gather information about the distribution of light?

Animals have evolved a variety of image forming devices, from pinhole eyes, to compoundeyes, to the single lens eyes of vertebrates. The goal of any eye is to form an image whichpreserves the spatial ordering of the points in space from which the light rays originated. Thevertebrate eye contains a convex lens. For a sharp image of a distant object, the image plane(retina) must be positioned at exactly the right distance given the lenses optical power (ability torefract light) and the distance of the object. One can consider the formation of an image of a verysmall point of light, since all emitting surfaces can be considered as collections of points. Eventhe best lenses have imperfections, so the image of a point is usually a blurred circle of light(point spread function). In the human eye, most (2/3) of its optical power is supplied by thecornea, with the rest from the lens. The optical power of the lens is variable, to cope withfocussing objects at differing distances in front of the eye (accommodation).

The formation of a sharp image on the retina is only the starting point of visual processing. Thepattern of light and dark on the retina results from the interaction of -

(1) The light source; its position, intensity, and spectral qualities. (2) The layout or spatial arrangement of visible surfaces. (3) The reflectances of the visible surfaces. (4) The viewing position of the observer.

All these factors interact to determine the pattern projected onto the retina. It is the job of thevisual system to disentangle them. A simple light-dark edge, for example, could be a shadow ora change in reflectance. The complex interaction of the different factors thus introduces manyambiguities which have to be sorted out.

6. Eye movements

The main purpose of eye movements is to image the point in space that is of most interest on thecentral retinal region of each eye (where acuity is highest). There are various types of eyemovement.

1. Voluntary

(a) Saccades: rapid shifts of gaze. Reaction time about 200 msec, duration of movement about45 msec = 400 deg/sec (therefore can make about 3-4 per sec as in reading). When resting wetake in about three words per fixation, thus limiting reading speed to a maximum of 12 wordsper second = 720 per minute. Moves are ballistic, i.e. once started movement runs off with nofeedback. There is some suppression of vision in the course of a saccade.

(b) Smooth pursuit: eyes can lock on to a moving target and follow it up to a rate of 30 deg/sec.Impossible to make smooth following movements without a moving target. Control systems forsaccades and smooth following movements are different. Smooth following movements areeliminated by barbiturates.

(c) Vergence: very slow smooth movement in which the two eyescome together (converge) or move apart (diverge) in order toadjust for fixation on a point at a different distance to the onepreceding.

2. Involuntary

(a) Physiological nystagmus: involves tiny drifts, flicks, and tremor in eye position, probablyto help to preserve vision against adaptation. When an image is stabilised on the eye, it rapidlydisappears. Since cones are 20" of arc tremor would appear to move the image roughly acrossone cone.

(b) Optokinetic nystagmus: a series of saccades and smooth following movements caused bymoving the scene past the observer (e.g. as in a train).

(c) Vestibular nystagmus: like optokinetic nystagmus, but caused by the effects of accelerationand deceleration on the vestibular system, eg head rotation.

PERCEPTION TOPIC NOTES: Physiology

1. The eye

1) Transduction50% of the light reaching the eye is lost by absorption and scattering. The rest hits retinal

photoreceptors, which contain light sensitive molecules. These molecules release transmittersubstances which pass on electrical activity to collector cells. This process takes less than 1millisecond.

2) Dynamic rangeAmbient light levels on earth can vary hugely. White paper seen in sunlight reflects

100,000,000 times more light than the same paper seen by starlight. The pupil can vary indiameter from 2-8 mm, only a 16-fold variation in light admitted, clearly inadequate to cope withlight variations It is physically impossible to build a single photoreceptor system which canrespond to such a wide range of light levels, so the retina contains two sets of receptors, one forhigh light levels and the other for low light levels, which complement each other:

Rods Cones

120 million 6 millionHigh sensitivity Low sensitivityPeripheral retina Central retinaSingle pigment type Three pigment types

The shift from cone vision to rod vision takes about half an hour, well matched totwilight, but not to modern cinema-goers.

3) BottleneckThere are 126 million photoreceptors, but the output from the retina (optic nerve)

contains only 1 million fibres, so there has to be a massive convergence of many photoreceptorsonto few ganglion cells (whose fibres form the optic nerve). How?

a) Retinal receptive fields - each ganglion cell receives inputs from a large number ofphotoreceptors, defining its receptive field. Signals from receptors in the centre have theopposite effect on the cell to signals from receptors in the surrounding region of the receptivefield (antagonistic centre-surround organisation), so the cell responds only to differences in lightlevel between centre and surround.

b) Retinal inhomogeneity - Convergence limits acuity, so to allow high acuity despiteconvergence the receptive fields of ganglion cells are not all the same size (ie. homogeneous) butsome cells have small fields (high acuity) and others have large fields (low acuity). Fields incentral vision tend to be small, and those in peripheral vision tend to be large.

2. Visual pathway In humans, the majority of optic nerve fibres terminate in the thalamus, in two bodies

called the lateral geniculate nuclei (LGN). Fibres from the LGN then travel up to the visualcortex. Our monocular visual fields overlap considerably, so a single object will generate signalsin both eyes. The two monocular signals must be brought together in the cortex for analysis, sooptic nerve fibres cross-over before reaching each LGN:

- Fibres from the right visual field of each eye converge on the left LGN and cortex- Fibres from the left visual field of each eye converge on the right LGN and cortex.

Optic nerve fibres terminate in a highly ordered arrangement in each LGN, with inputs fromeach eye arriving in different layers, and different cell types grouped together.

3. Visual cortex 1) Primary Visual Cortex (V1)

All LGN fibres terminate in the Primary Visual Cortex. As mentioned before, corticalcell receptive fields are mapped topographically. The cortex has a layered stripey appearance,and contains a variety of cell types, distinguished in several ways. Here we will concentrate on

distinctions according to preferred stimuli.- Simple cells prefer oriented lines or edges, correctly positioned.- Complex cells prefer particular movements of lines or edges, position less critical.- Hypercomplex cells are sensitive to terminations, corners, etc.

Early theorists considered these cells to be ‘feature detectors’, and used them as the basis fortheories of object recognition. This view of cortical cells is far too simplistic (see objectrecognition notes).

The cortex is highly ordered, with cells preferring similar stimuli near to each other eg.orientation and ocular dominance columns.

2) Higher cortical areas: parallel streams of analysisA number of other areas of cortex contain cells which respond exclusively to visual

stimuli. They are labelled V2, V3, V4, V5, VP, PO, etc. There are complex interconnectionsbetween the areas, and consistent differences in the response properties of cells in differentareas. This had led to the theory that the visual system contains 'parallel streams' of analysis.with whole areas devoted to analysis of particular visual attributes:

- Preliminary analysis in V1- Contour in V2 and V3- Colour in V4- Motion in V5

On a broader scale, two routes to perception have been identified:

- One passes from V1 to V2, V3, V4 and then the inferior temporal cortex, and isthought to be involved in recognition ie. 'what is being viewed?'.- A second passes from V1 to V2, V5 and then the parietal cortex, and is thought to beinvolved in perception of motion and location ie. 'where is it?'.

PERCEPTION TOPIC NOTES: fundamental visual functions & phenomena

1. Photopic and scotopic vision

The eye contains two photoreceptor systems, specialised for different levels ofillumination. Vision at high light levels, mediated by cones, is called photopic vision. Vision atlow light levels, mediated by rods, is called scotopic vision.

1) Dark adaptation - the shift from photopic to scotopic vision.We can measure a subject's sensitivity to the presence of a small, well-defined light

stimulus, while the subject is adapted to bright conditions. The subject is then placed in a darkenvironment, and sensitivity re-measured at regular intervals. Over a period of about 25minutes, the subject can detect progressively dimmer lights as they adapt to the dark conditions.This improvement in performance corresponds with a shift from cone to rod vision.

2) Spectral sensitivity - the Purkinje shiftRods and cones respond slightly differently to light of different wavelengths (ie.

colours). So in photopic vision we are most sensitive to wavelengths in the yellow-red part ofthe spectrum (560nm), while in scotopic vision we are most sensitive in the green part of thespectrum (500nm). Consequently, as we dark adapt, different colours change their relativebrightnesses. If two red and green colours appear equally bright in high illumination, the redwill appear darker than the green in dim illumination (because rods are more sensitive to greenthat to red). This shift in apparent brightness is called the Purkinje shift.

The differing spectral sensitivities of rods and cones can be exploited. If a person'sactivities require them to shift from bright illumination into darkness, and perhaps back again,the process of dark adaptation can be accelerated (or preserved) by using red light. Rods arerelatively insensitive to red, so they begin adapting (or remain dark adapted) as soon as red lightis used.

2. Spatial and temporal sensitivity

1) Visual angleBefore discussing acuity, we must introduce a fundamental unit of measurement in

vision. The visual effect of a spatial pattern or object depends on the information it offers in theretinal image. It is clearly inappropriate to specify the spatial dimensions of an object in terms oflinear units (cms, ins), because its retinal image will vary with viewing distance. We need ameasure of the size of the retinal image. The standard measure used is the angular extent of theobject's image on the retina, in degrees, minutes (1/60ths of degree), and seconds (1/60ths ofminute). Eg. the diameter of the moon (and, coincidentally, the sun) is 0.5 degrees or 30 min;the width a thumb at arm's length subtends about 2 degrees.

2) The blind spotAll nerve fibres and blood vessels serving the retinal leave the eye at one place - the optic

disk. Clearly there can be no receptors at the optic disk, so we have no visual sensitivity in thatregion of the retina. It is a blind spot. We are not usually aware of this blind spot, partly becausethe spots are at different places in the visual field in the two eyes, and partly because the visualsystem seems to 'fill in' the blind area on the basis of stimulation nearby.

3) The foveaAnother landmark on the retina is the macula lutea (=yellow spot in Latin), a small

yellow-tinged area at the centre of each retina . At the centre of the macula is a small depressioncalled the fovea (=pit in latin), covering about 0.5 arc deg. Retinal blood vessels avoid this area,which contains very high concentrations of cone photoreceptors. This anatomical feature hasimportant consequences for acuity.

4) Spatial acuity

Visual receptive fields detect spatial changes in image intensity. These edges are importantbecause they mark the borders of shapes and objects in the world. It is important to establish thelimits of our ability to detect these spatial variations in image intensity. A simple measure is thenarrowest bars in a grating required that we can just detect (or the highest frequency). Typically,the highest frequency we can see is 60cpd. Why? We already know that, due to imperfectoptics, the retinal image is slightly blurred. In particular, a very thin line is blurred across severalminutes of angle, as shown in line spread function. So, many narrow bars in a row will beresolved provided that they are far apart, but will blur together when they are too close.

5) Temporal acuity

We need to detect variations over time as well as variations over space. Spatial variation relatesto object shape. Temporal variation relates to object motion. We can estimate our ability to detecttemporal variations using a flickering spot that repeatedly turns on and off. The frequency of theflicker is specified in cycles per second or Hertz. What is the highest flicker frequency that wecan detect (known as the critical flicker frequency)? In high illumination and central vision theCFF is about 25-30Hz. It rises slightly in the near periphery (2-4 deg. eccentricity), but fallswith image intensity. Why? The explanation is similar to the explanation for spatial acuity limits.Retinal response to a flickering light is not instantaneous, but rises and falls over manymilliseconds (thousandths of a second). The flicker can be resolved only if the frequency is notso high that successive responses merge together.

6) The spatiotemporal window of visibility

For spatial and temporal frequencies below the acuity limits, we are not equally sensitive to allfrequencies. If we measure the minimum spatial (or temporal) luminance change needed to justdetect the presence of spatial (or temporal) modulation in the image, then the optimal spatialfrequency lies at about 3 cycles per degree, and the optimum temporal frequency lies at about 10Hz. Above and below these frequencies, sensitivity tails off. So we can define a spatiotemporal'window' within which a stimulus must fall if we are to detect it effectively. This information isuseful for optimum design of visual equipment.Eg. TV pictures are made up of numerous smalldots 'painted' by a spot that moves across the screen. If the dots are small enough, and speedhigh enough, human observers are not aware of them, and a complete image is seen. What sizeof dot, and speed of movement are needed? TV designers use the window of visibility to selectvalues that fall outside the range of visibility. In UK, the spot traverses the screen 50 times asecond, above CFF. Light bulbs and strip lights also flicker. There is some evidence that thisflicker is uncomfortable even when it is not visible. The window of visibility can also bemeasured for various animals. Cats, for example, cannot see high spatial frequencies as well ashumans, but can see low spatial frequencies better. Small flying insects have much higher CFFsthan humans (eg. 300 Hz for bees).

7) Visual latencyThe visual response to light takes a short time to generate (about 30 ms), known as visuallatency. Latency depends on light intensity - dimmer lights have longer latencies. The perceptualconsequences of this latency variation are vividly illustrated in the Pulfrich Pendulum.

PERCEPTION TOPIC NOTES: Spatial Vision

This part of the course will examine how the visual system represents spatial propertiesof the image:

- how it extracts information about local contours- how it structures contour information into more extensive shapes and textures- how it maintains constant representations of size and shape

1. Representing contours at multiple scales: spatial channels

Images contain detail at many different scales, from fine scale detail (eg. surface texture,brick patterns, leaf shape), to coarse scale detail (eg. building and tree shape). The visual systemneeds to represent these multiple scales of detail simultaneously, because they are useful indifferent ways. For example, coarse detail may allow us to identify the object uniquely (eg. aparticular building, or species of animal, a person's face), whereas fine detail may conveyspecific information about object properties (eg. what material an object is made of, the type offur an animal is covered in, the expression on a face). In order to represent information atdifferent scales simultaneously the system requires visual mechanisms which can select onlydetail at a particular scale, and ignore other scales. It can then inspect responses in particularmechanisms to isolate particular image scales.

Centre-surround receptive fields respond to contours, but can only respond to contoursat a level of detail which is suited to the size of the receptive field. So each size of field 'filtersout' all detail except that at a particular scale in the image. Different sized fields conveyinformation at different scales. Scale-specific receptive fields have been found in animals, fromcell recordings. Indirect psychophysical evidence shows that the human system also has scale-specific receptive fields or 'spatially-tuned channels'.

1) Evidence for spatial channelsWe saw earlier that a convenient simple stimulus for probing the visual system is a

grating consisting of repetitive bright & dark bars. Conventionally, experimenters use gratingswith a smoothly varying or sinusoidal luminance profile (the significance of which will becomeapparent later). We can only detect gratings with bar widths within a certain broad range - toowide (low frequency), or too narrow (high frequency) and human observers cannot detect thepresence of the bars. Our sensitivity to different frequencies can be represented graphically in aContrast Sensitivity Function. In 1969, Blakemore and Campbell found that even frequenciesthat we can normally detect become less visible of we adapt to those frequencies for a while. Theunderlying neurons become fatigued. They found that adaptation only affects the visibility ofgratings similar in frequency to the adapting grating - evidence for channels which respond onlyto particular frequencies (ie. scales). Since 1969, hundreds of experiments have been conductedto probe the properties of these spatially-tuned channels, usually called spatial-frequencyselective channels.

2) Fourier AnalysisWhy do experimenters use sinusoidal gratings? The 18th century mathematician J.

Fourier discovered that any complex pattern can be decomposed into a unique set of sine waves,which must have specific frequencies, amplitudes, and relative positions to define that patternand no other. So, we can predict the visual system's response to any image if we know how itresponds to individual sine waves - we break the image down into its component sine waves (ie.perform a Fourier Analysis on the image), remove or attenuate some of the components becausethe visual system cannot detect them, and then reconstitute the image (eg. visibility of humanfaces to infants).

So far we have seen that images contain detail at many different spatial scales, and the visualsystem extracts local scale-specific information using tuned receptive fields or 'spatial channels'.The next step in spatial analysis requires that larger regions of the image are segmented intoshapes that eventually form the basis for object recognition.

2. Grouping contours into shapes

1) Gestalt psychologyIn the early part of this century, the Gestalt Psychologists formulated a set of rules or

'laws' of perceptual organisation, which demonstrate how the visual system seems to have in-built preferences for grouping parts of the image together on the basis of certain properties.Examples include:

- proximity- similarity- common fate- good continuation- pragnanzRecent computer models of visual segmentation exploit Gestalt principles.

2) Visual search and texture segmentationRecent research has systematically explored the local image properties which allow the

visual system to segment areas of the image on the basis of textural differences. The basic idea isto superimpose on a texture pattern an area of different texture, and give subjects the task offinding the different area (or just reporting whether one is present or not). Some texturaldifferences 'pop out' very easily, indicating that the system has built-in descriptors which cancode the difference in texture (eg. line orientation, curvature, colour, motion direction, contrast).Other texture differences are difficult to detect, requiring serial search of the elements (eg.elements containing conjunctions of properties).

3. Perceptual constancies

The spatial appearance (ie. size and shape) of objects in the image is not dictated purelyby the size and shape of their retinal image. If it were, objects would appear to change in sizeand shape as viewing position changed. Instead, the visual system has in-built ways of makingallowances for viewing conditions when estimating object size and shape, so that theseproperties tend to remain relatively constant despite variation in viewing conditions eg. peopleappear to stay the same size as they walk away from us, despite the fact that their retinal imageshrinks dramatically. A number of cues have been identified, which the system uses to achieveconstancy:

- known size, only if other cues are absent- relative size, again a weak cue used if others are absent- distance cues, most important. Demonstrated by Holway & Boring.

Thus apparent size (s) is determined jointly by retinal image size (r) and apparent distance (d),and remains fixed if the product (r.d) remains constant (ie. as the retinal image, r, gets smaller,the apparent distance, d, gets larger); size-distance scaling. A number of illusions show howsize scaling breaks down when distance cues are disturbed. Shape constancy can be viewed asan extension of size constancy.

PERCEPTION TOPIC NOTES: Motion Perception

1. The importance of motion

Information about motion in the visual image is of paramount importance. Even thesimplest sighted animals have a motion sense. Among the advantages of a motion sense are:

1) Figure/ground segregation - shapes which are invisible when static can be detected as soon asthey move using a motion sense. Motion therefore acts a a cue for shape segmentation.

2) Precise information about 3-D object shape is available in the pattern of movements createdwhen the object moves.

3) As we move about in the world, its image `flows about' on our retina, and a great deal ofinformation can be extracted from the pattern of movements eg. the point of impact of anaeroplane as it descends toward a runway.

A dramatic illustration of the importance of motion comes from the case of a woman who losther motion sense after brain damage. Apparently simple tasks (pouring tea, talking to others)became very difficult.

2. Detecting movement

Information on motion is not available directly in the image, but requires specialised visualprocesses to extract it. Individual receptive fields at each point in the image can only signalincreases and decreases in intensity over time. To detect motion, the visual system must compareimage intensity over time as well as over space. It is easy to build 'delay-and-compare' motiondetectors using elementary retinal receptive fields. Both humans and other animals certainlypossess hard-wired detectors that operate according to this principle (eg. cell recording data, themotion after-effect).

3. Object motion and eye motion

It is not sufficient to possess motion detectors. Motion on the retina, signalled by suchdetectors, could result either from motion of an object in the world, or from movement of theeyes and/or body. We correctly attribute image motion to these two potential sources (the worlddoes not usually swing around when we move our eyes; object do not appear stationary whenwe track them). How? The visual system appears to use two strategies. During rapid shifts ineye position (saccades) vision is suppressed. For slower eye movements, the visual systemseems to compare a record of the commands sent to the ocular muscles to initiate eye movementsagainst signals coming in from retinal motion detectors. If these two signals are equal andopposite (eg. retinal motion left, eye motion right) then they cancel out and no motion isperceived. When there is an imbalance between them then object motion is perceived.

4. Real movement vs. apparent movement

When the image of an object shifts across the retina continuously (without pauses or gaps) theobject really is moving, and we may perceive this REAL MOVEMENT eg. an actor walkingacross the stage during a theatre performance.

When the image of an object shifts position on the retina discontinuously ie. suddenlydisappearing from one static position and re-appearing at a nearby position a short time later(stroboscopic motion), it is not really moving but it may appear to be moving (APPARENTMOVEMENT). For example when we see an actor in a cimema film walk across the screen, themotion is an illusion, because the cimema film consists of a succession of static images of the

actor in slightly different positions. When the series of static frames is presented rapidly (24frames every second), it creates an impression of movement.

Why are we susceptible to illusions of apparent motion? A common fallacy is that the illusionresults from 'persistence of vision' - each static image persists in our vision for a short time, andsomehow merges with the next static image. However, visual persistence and apparent motionare demonstrably separate visual effects. A clue to the correct explanation comes in theobservation that good apparent movement is indistinguishable from real movement.Stroboscopic motion is effective because it excites neural motion detectors in the same way asdoes real motion. Support for this explanation comes from experiments which record activity inneural motion detectors. These cells repond to apparent motion stimuli as well as to real motionstimuli. While at least partially correct, this theory cannot easily explain all instances of apparentmotion.

5. High-level motion analysis

1) High-level detectors?Neural motion detectors respond to the movement of local intensity edges in the image.

However, apparent motion can be perceived in motion sequences that contain no simplecorrespondences between intensity edges. For example, if a rectangle made from vertical stripesin one frame is suddenly replaced by a nearby rectangle made up of horizontal stripes in the nextframe, then it can create an impression of motion. This kind of motion is now called ‘second-order’ motion. It cannot stimulate the simple intensity-based detectors described earlier. Howcan we explain the perception of second-order motion? There are two general types ofexplanation -

a) The apparent motion is detected by motion detectors of the same general kind asdescribed earlier, but the input receptive fields to the detectors respond not just to simpleintensity edges but to texture-defined shapes. There is some physiological evidence forthe existence of such detectors, and growing perceptual evidence for them.

b) The perceived apparent motion is not mediated by hard-wired detectors, but is drivenby a shift in the focus of our visual attention. Certain motion stimuli are inherentlyambiguous, because they contain at least two equally strong signals in differentdirections. Observers can switch between them by shifting the focus of their attention.This explanation borders on cognition. There is also evidence to support thisexplanation.

2) Motion integration

A number of perceptual effects demonstrate that following low-level detection of motion,sophisticated analysis takes place at a more global scale. For example:

a) The Kinetic Depth Effect (KDE). Wallach and O'Connell (1953) were the first to describe this effect, in which the 3-Dstructure of an object can be perceived on the basis of the moving two-dimensionalimage it creates on the retina, in the absence of other cues as to the object's shape.

b) Biological motion. Johansson attached points of light at the joints of a person, and then filmed him walkingin total darkness. When subjects were later shown the film (containing disconnectedmovements of about a dozen points) they immediately recognised the pattern as awalking person. The pendular movements of the individual points were not apparent,only the organised shape of a walking figure.

The exact nature of the high-level analyses underlying these percepts is not known. Onetechnique for studying them is to develop computer programs which can correctly infer 3-Dshape, or the movement of a figure, given the same information as a human observer.Successful attempts at simulating perceptual interpretations rely on in-built assumptions aboutthe nature of the visual world, such as rigidity: natural objects tend to be rigid or articulated(made from rigid parts which are connected at flexible joints).

PERCEPTION TOPIC NOTES :Depth perception

A depth cue is defined as anything that is used by the visual system to evaluate the depthof a point in space or to perceive depth in a 3-D shape. Depth cues can be divided into threemajor groups. 5-10% of the general population cannot use the stereoscopic cue, and must relyon the other cues. In one case of brain damage, a patient was found to have intact stereo visionbut an inability to use monocular depth cues.

1. Oculomotor cues

1) Accommodation.

Based on information from muscles controlling lens shape. It is potentially useful onlyfor nearby objects (muscles are relaxed for distances over a few metres). Heinemann, Tulving &Nachmias (1959) and Kunappas (1968) found that subjects could not use accommodationreliably when other cues held constant (the size of the target was kept at a constant visual angleusing luminous discs and no background).

2) Convergence.

Based on information from extraocular muscles controlling eye vergence. Again thepotential cue diminishes with distance. Heinemann, Tulving & Nachmias (1959) found thatjudgements based on convergence alone were very inaccurate although when convergenceincreased they did see the object as smaller. The problem could be caused by conflictinginformation from other cues.

2. Monocular cues

1). Linear perspective.Lines: Parallel lines receding from the observer produce images on the retina that become

closer together as they recede. Any set of parallel lines have the same vanishing point as oneanother.

Texture: Where texture is formed by elements of the same size (e.g. bricks) the retinalsize decreases with distance and hence produces a cue to slope. Gibson pointed out that evenwhen the elements of texture were not of the same size but had an average size (e.g. grass in afield), average size on the retina decreases with distance and hence provides a cue to depth.

2) Unlearned size.Where two objects of the same shape are presented with no other cues to depth, they are

usually seen as of the same size but at different distances.

3) Height in visual field.For any surface below the observer (e.g. the ground) the further away an object resting

on that surface is, the higher is its base in the visual field.

4) Interposition.When one object is partly obscured by another, T junctions are formed in the retinal

image. The object to which the stroke of the 'T' belongs is in front of that to which the tailbelongs.

5) Shadows.The direction of shadows can yield a cue to the shape of the object causing the shadow,

as can the luminance of different parts of the surface. In looking at pictures people assume thatlight is falling from above. The cue appears to be innate in chickens since even if they are raisedwith light coming form below, they peck at small objects casting a shadow below not above(Herschberger, 1970).

6) Aerial perspective.

On misty days or over great distances the further away an object the more blurred itlooks. Blue light is more scattered than red by particles in the air and hence blue light producedby scattered sunlight is added to the visual image of distant objects. For similar reasons the skylooks blue and sunsets may be red.

7) BlurCameras, and eyes, have limited depth-of-focus, so depth variation generally leads to

variation in blur across the image. This variation provides a cue to depth.

8) Learned size.If the size of an object is known to the visual system and all other cues are removed, its

retinal size determines its apparent distance (playing card illusion). Note that consciousknowledge of the real size does not abolish this effect.

9) Motion parallax.If the observer moves, points at different distances move at different velocities over the

retina. If he moves his head from side to side while fixating a point in the middle distance,points beyond the fixation point are swept in the direction of the head movement and pointsnearer than the fixation point are swept in the opposite direction. Rogers, using random dotdisplays, has successfully established that this is a very powerful cue to distance. If the observermoves forwards, nearby objects expand rapidly on the retina, and distant objects less rapidly.

10) Kinetic depth.This is similar to motion parallax except that the observer is stationary and the object is

moving. Provided the object is rigid and the observer assumes that it is rigid, the relative depthsof its parts can be determined from the rate at which they move if the object is rotated. Motioncues can produce powerful effects of depth in the cinema.

3. Binocular cues: stereopsis

1) Binocular disparityA very important cue. As a result of their different positions in the head, each eye

receives a slightly different view of the world. Consequently, objects further away from (ornearer to) the observer than the object being fixated will stimulate slightly different positions onthe retinas of the two eyes. This displacement in relative position is known as disparity - the signor direction of the disparity depends on whether the non-fixated object is nearer or further away,and the magnitude of the disparity depends on how much nearer or further away is the non-fixated object. Far objects are displaced to the left in the left eye's view and to the right in theright eye's view (far or uncrossed disparity). Near objects are displaced to the right in the lefteye's view and to the left in the right eye's view (near or crossed disparity). You can verifythese statements by viewing two fingers placed at different distances from your eyes.

2) The horopterIf we maintain fixation at a particular point, and position other points in the visual field

so that they appear to be located at the same distance away as the fixated point, we can build upan imaginary surface called the horopter, representing all points in the visual field that appear tobe the same distance away. Curiously, this perceptually-determined surface does not agreeexactly with the geometrically correct surface calculated from measurements of eye dimensions(the Vieth-Muller circle).

3) StereogramsStereo images can be generated in a number of ways, in addition to the natural way of

looking at a three-dimensional world. All work by delivering a slightly different image to eacheye, mimicking the images one would receive in a 3-D scene.Early stereoscopes used prisms ormirrors to display stereo-pairs of photographs or drawings to each eye More recent techniquesinvolve selecting each eye's image using coloured filters or polarising lenses. A populartechnique is to interleave each eye's view in a single picture, visible by slightly mis-convergingthe eyes (autostereograms).

4) Random Dot StereogramsStereoscopes were invented in the mid-1800's, and until the 1960's it was thought that

the underlying visual mechanism operated as follows. First, objects in each monocular viewwere identified, then the visual system matched or fused the pair of images corresponding toeach object to extract its depth. However Julesz created stereograms made up of random dots, inwhich the only cue to depth arose from slight changes in dot position in one eye's image relativeto the other eye's image. Random dot stereograms generate powerful impressions of depth,demonstrating that disparity alone (without monocular shape cues) is sufficient. Binocular cellstuned to disparity have been discovered in the visual system of cats and of monkeys. Depthperception, in the form of disparity tuned cells, is therefore hard-wired at the earliest level ofvisual analysis in the cortex.

Random dot stereograms highlight a difficult problem that the visual system apparentlysolves with ease - the correspondence problem. Each eye's image contains many thousands ofdots. How does the system work out which dot in one image matches a corresponding dot in theother image? For each correct match there are thousands of incorrect 'ghost' matches. Theinformation in the image is not sufficient to arrive at a unique solution. The visual system seemsto make assumptions based on the properties of real-world images to rule out many potentialmatches, simplifying the problem. These assumptions are expressed in terms of interactionsbetween disparity tuned cells that weed out incorrect matches and preserve correct matches.

4. Cue combination in depth perception.

Until recently, research on depth perception was dominated by random dot stereograms.But it is important to remember that depth perception involves a multitude of cues. Wheneverone cue is ‘isolated’ experimentally, conflicting information from the other cues is still present,and may have some effect on results. Recent studies have attempted to examine how the visualsystem resolves these conflicts and integrates the information provided by different cues. Forexample, in a stereogram, if the right and left eyes' views are swapped, then all disparity valuesreverse (ie. far disparities become near and vice-versa). In a random dot stereogram, perceiveddepth also reverses, but in a stereogram depicting a realistic scene perceived depth does notreverse - other cues overcome the disparity reversal.

PERCEPTION TOPIC NOTES: Colour Vision

Colour vision aids the detection of objects against their background (eg.fruit on a tree),and the discrimination of one object from another (eg. oranges and lemons). Colour also hasemotional power, as illustrated by the case of a man who lost his colour vision (eg. food becameunpalatable, people were unattractively "rat-coloured").

2. Physics of colour

Subjective experiences of colour are derived from the wave properties of light.Wavelengths toward the short end of the visible spectrum appear blue, and wavelengths towardthe long end appear red. Light from natural sources and reflecting surfaces contains manywavelengths, and their appearance depends on the actions of those wavelengths on thephotoreceptors. As Newton discovered in 1704, sunlight contains a whole spectrum ofwavelengths, but appears white. Artificial light also contains many wavelength components,differing in relative energy from sunlight so, for example, light from tungsten lamps appearsmore yellow than sunlight.

Of course, as Newton observed, light at a particular wavelength is not itself coloured."Colour" is a purely subjective or psychological concept. For example, energy from the non-visible part of the spectrum differs from visible light only in terms of wavelength, but we wouldnot describe radio waves as 'coloured'.

2. Subjective properties of colour

1) Dimensions.Humans can discriminate over a thousand different colours presented side-by-side, but

can recognise only about a dozen presented one at a time. Colour has three subjectivedimensions.

(a) Brightness depends primarily on the intensity of the light received, though certaincolours look brighter than others at the same physical intensity. For example, for thesame physical intensity, yellow would appear considerably brighter than red or blue.

(b) Hue is related primarily to the wavelength of the light. Most hues are spectral hues,i.e. they can be produced by monochromatic light at different wavelengths. Some hues,however, are extra-spectral. These consist of mixtures of red and blue light (e.g. purple)reinforcing the distinction between the physical and perceptual properties of light .

(c) Saturation refers to how pure colour seems. The more a hue appears to be mixed withwhite light, the less saturated it is. Thus, red is a saturated colour, whereas pink is thesame hue as red, but is much less saturated. Unsaturated hues look pale.

A further demonstration of the subjective nature of colour can be found in colour constancy. Theset of wavelengths reflected by a surface depend not just on the reflectance properties of thesurface but also on the wavelength composition of the light source illuminating the surface. Forexample, tungsten lamps emit more energy in the yellow region of the spectrum than daylightdoes, so a white surface, for example, will contain more 'yellow' under tungsten lamps. Yet thecolour of the surface remains white. Photographic films have no mechanism for achievingconstancy, so photographers need to use different films (or compensating filters) for differentlighting conditions to maintain colour balance.

2) Colour circleNewton experimented with mixtures of light from different parts of the spectrum,

observing the resultant colour. He was able to summarise his observations in the colour circle,which allows us to predict the appearance of various mixtures by joining the two colours with aline and locating the sector in which the line's centre-of-gravity falls. Refinements to the colourcircle have been made since Newton's original idea,re-arranging the spacing of colours so that

complementary colours (mixtures appear white) fall opposite each other in the circle eg. red vs.green, blue vs. yellow.

3) Colour mixtureNewton employed 'additive' mixtures of colour, in which light at different wavelengths

is simply added together (as in TV displays). However, painters have been mixing pigments ofdifferent colours for centuries, and get different results. For example, mixtures of blue andyellow pigment make green, not white. Why? Painters employ 'subtractive' mixtures. In thisexample, one pigment illuminated by broad-spectrum light appears blue because it absorbs mostwavelength components, reflecting only those near the blue/green-ish region. The other pigmentabsorbs all except wavelengths in the yellow/green-ish region A mixture of the two will spareonly a relatively small set of wavelengths that neither pigment absorbs very well, in the greenpart of the spectrum So subtractive mixtures ' take out' more wavelengths from the resultinglight, but additive mixtures, as in Newton, 'add in' more wavelengths. The two techniquesfrequently produce different results.

4) Metameric matchingAs the colour circle indicates, the same colour sensation can be evoked by lights having

different wavelength composition, provided that the relative intensities of the components areadjusted appropriately Eg. spectral orange = (red+yellow); white = (red + green) = (blue +yellow). These subjectively identical, but physically different, sets of lights are called metamers.A subjective match with any colour can be made by appropriate mixture of no more than threepure wavelengths. This led Young (1802) to propose that human colour vision was trichromatic('three-coloured'), an idea later taken up by Helmholtz (1852). Rushton (1961), using atechnique called retinal densitometry, provided precise physical measurements of two differentpigment types in human eyes, partially verifying the theory.

5) Colour after-images and contrastAfter staring at a red coloured card for a while, white card appears to be green (and vice

versa). Adaptation to a blue card causes a white card to appear yellow (and vice versa). A whiteregion surrounded by red appears to be green, and a white area surrounded by blue appearsyellow. These observations led Hering (1878) to propose that red and green were pairedtogether perceptually, as were blue and yellow - the opponent-process theory of colour vision.Hurvich and Jameson (1957) provided precise psychophysical data in support of opponentprocessing.

3. Physiology of colour vision

By the middle of this century, there was no obvious way to reconcile the trichromaticand opponent-process theories of colour perception. A resolution was finally achieved as a resultof physiological discoveries.

1) Retinal receptorsMacNichol (1964) and Dartnall, Bowmaker and Mollon (1983) used a technique called

microspectrophotometry, which involves shining monochromatic light on an individual receptorin a piece of excised retina mounted on a microscope slide, and measuring the amount of lightpassing out the other side (ie. not absorbed). They found three different types of cone receptor,with peak absorbances at 430, 530 and 560nm . Note only 1 in 100 cones is tuned to blue. Thehuman retina is therefore trichromatic.

2) Retinal ganglion cells and LGN cells.About half are spatially opponent but spectrally non-opponent ie. respond to intensity

differences between centre and surround regardless of colour. The rest are both spatially andspectrally opponent, eg. with green exciting, red inhibiting, (or vice versa). Initial work wasdone by De Valois and De Valois (1975), and the table below gives estimates of the type andrelative frequency of different spectrally opponent cells (from Zrenner 1983).

Positive Input Negative Input Frequency

Red cones Green cones 30%Green cones Red cones 16%Blue cones Red/Green cones 6%Red/Green cones Blue cones 1%

Note that cells with blue cone inputs are very rare, so their properties are less certain. Cells inthe visual pathway are therefore opponent-process.

3) Cortical colour cells

a) Cell types - most are 'double-opponent', responding in opposite ways to colour dependingon whether the centre or surround is stimulated (e.g. R+ G- in centre, R- G+ in surround);somehave the same properties as LGN cells; and some are non-opponent but excited by a narrowrange of wavelengths.

b) Organisation - cells responsive to colour are grouped into blobs in V1, and into stripes in V2.There are projections from V2 to area V4, which appears to specialise for colour. Zeki (1983)has claimed that, whereas cell responses in earlier cortical areas are wavelength dependent, cellsin V4 are colour dependent ie. accomplish colour constancy. This claim has still to be verified.Many believe that this route from V1 through V2 to V4 constitutes a specialised processing‘stream’ for colour information, that has a very limited ability to process information aboutmotion and depth.

4. Congenital colour deficiencies

These take many forms - the eye has the right number of cones, but appears to lack oneof the three types of cone pigment ('dichromacy') or one cone class has an abnormal response('anomalous trichromacy'). Only in very rare cases are two or all three cone types are missing.An absence or anomaly of the green cones is most common (6% of males), followed by red(2% of males). Blue deficiencies are very rare. Colour deficiencies are linked to the X (male)chromosome, so are much less common in females (less that 0.2% in total).

PERCEPTION TOPIC NOTES: Object Recognition

1. Background

So far, we have covered the extraction of simple local image attributes (contour, motion,depth, colour), and touched on issues of shape segmentation. How do we progress from thisintermediate level of visual analysis to the recognition of objects? No-one has found a completeanswer to this question yet. Instead, theories have restricted themselves to particular classes ofobject and/or specific viewing conditions. We will briefly review some of these theories, andhopefully gain some appreciation of the difficulties encountered when trying to construct acomprehensive theory of object recognition.

The central problem is stimulus equivalence - many images can equal one recognisedobject, eg. different views of the same object, different styles of the same object (different chairsare all recognised as a chair). How does the visual system achieve this?

Early work by computer scientists concentrated on recognising alphanumeric patterns,skirting around the problem of achieving stimulus equivalence by restricting themselves to a 1-to-1 mapping of each character onto its stored representation. One and only one possible imagecorresponded to each character in the set.

2. Template matching

A pattern is represented in terms of the image it projects onto the retina (its template).recognition proceeds by fitting each stored template onto the image until a match is found. Thesystem works in the restricted world of alphanumerics, but

1) false alarms can occur to partial matches eg. P vs. R.

2) the system cannot cope with shapes which may be transformed in the image by size,rotation, intensity modulation, perspective projection, etc. In other words, itcannot solvethe stimulus equivalence problem.

To overcome some of these drawbacks, more complex versions have been devisedinvolving flexible templates. Each template is made from `rubber', or linked plates, allowing adegree of deformation. Even these systems have difficulty in dealing with complex scenescontaining several objects, some partially occluding others.

3. Feature Analysis

A development of template matching, in which shapes are broken down into componentfeatures, each detected by a small template. The idea was inspired partly by Hubel and Wiesel'sdiscovery of `feature detectors' in the cortex. An example is the pandemonium model. Featureanalysis has its own limitations -

1) Neural feature detectors are ambiguous in their responses.

2) The system ignores the structural relations between features.

3) The system ignores the differences between patterns belonging to the same class,which remain detectable. Eg. different fonts can all be read, as well as be recognised asdifferent.

4. Structural descriptions

A structural description of an object consists of a list of propositions which specifies the

parts of the object, their properties, and the relations between them. The description isintrinsically 'object-centred'. Two current theories are:

1) Marr and Nishihara's, based on generalised cylinders which extend along an axis within thevolume occupied by each part of an object. The representation is 3-D, derived from acombination of 2-D information and information about depth from stereo, and information frommotion. Parts are linked in a hierarchy.

2) Biederman's geons ('geometric ions'). Object parts are specified in terms of a restricted set ofgeometric shapes, such as wedges and cylinders. These are, in turn, constructed from basicprimitives such as parallel edges and curves. The representation is 2-D.

One problem with structural descriptions is that many objects have no obvious decompositioninto parts, yet we can still recognise them.

5. Pictorial representations

This approach emphasises the role played by pictorial shape information. It has beenshown that many objects can be economically represented in terms of a small number ofcharacteristic views, removing the need for complex structural analysis.

6. Evidence

There is relatively little evidence bearing on theories of object recognition. Templates andfeature analysis have been rejected for human vision because of their inherent limitations.Structural descriptions have proved too difficult to implement fully in computer vision systems.Available evidence indicates that:

1) Orientation is sometimes important, supporting the view that the system uses view-specificrepresentations at some stage during recognition:

- Mental rotation effects- The Thatcher effect- Perrett has evidence for view-specific cells in higher visual areas in monkey cortex.- Evidence for preferred viewing angles during shape learning.

2) 'Top-down' effects can be important, but are not accommodated in most theories ofrecognition.

PERCEPTION TOPIC NOTES: Perceptual Development

1. HearingInfants below 6 months have higher absolute thresholds for sound than adults, but even newborns

show head-turning to the direction of a sound source, and this ability seems to be related to intensitydifferences rather than time differences (Clifton, Morongiello, and Dowd, 1984).

2. Touch, pain, taste, smell.Touch sensitivity is present right from birth (eg. the rooting response). A taste preference for sweet

tastes is also present at birth, but below about 4 months there is no sensitivity to salty, sour, or bitter tastes.Studies measuring heart rate and general bodily activity show that very young infants react to strong smells,especially unpleasant ones. Two-week old infants will turn towards an object carrying their mother's scent.

3. Infant VisionAlthough the size of the body increases by 20 times during development, the eye only doubles in volume. Itslength increases from 16-24mm. Despite the relatively small change in the eye, visual capabilities changevery markedly. For example, newborns have very poor acuity (20/800), and very limited accommodation(their eye is virtually fixed-focus at about 20cm, the distance to the mother's eye).

1) Methodology

Infants present particular difficulties for researchers studying the development of perception. Theyare non-verbal and have a very short attention span. A range of techniques have been developed formeasuring infants perceptual capabilities.

a) Preferential looking. Infants prefer to look at more complex visual patterns, and this can beexploited when assessing, for example, visual acuity. The infant is given a choice between a blankscreen and a low contrast grating.b) Habituation. Infants prefer to look at more novel patterns, and this can be exploited to determinewhether a child can detect a change in the stimulus. If he/she can, then dishabituation should occurwhen a previously habituated pattern is replaced by a new one.c) Reaching. An infant ( > 4 mths) will spontaneously reach for the nearer of two objects, useful formeasuring depth perception.d) Visual evoked potentials (VEPs). Electrodes placed on the scalp can be used to measure smallchanges in electrical activity evoked by visual stimuli. A direct physical measure of the effectiveness ofstimuli.

2) Developmental changes in visual capabilities.

Most major visual capabilities therefore seem to develop to adult levels during the first 3 months of life(though see below for the development of binocularity). For example -

Colour vision. 1-5 day old infants can discriminate reds and greens, but not blues. By 2-3 months,infant colour vision is close to that of adults. Younger infants were studied using preferential looking,between chequerboard stimuli made up of grey and coloured squares (matched for brightness) and auniform grey stimulus.Visual acuity and contrast sensitivity. Acuity for high contrast stimuli is very poor at birth (about20/500 at one month, legally blind), but gradually improves to adult levels by 6 months. Sensitivity tolow contrast gratings is also very poor near birth - sensitivity is low overall, and peak sensitivity fallsat much lower SFs (.5-1.0 cpd) than in adulthood (2-3 cpd). Measures include both preferentiallooking and VEPs.Depth perception. Reliable convergence cannot be demonstrated until infants are about 3 months old.Using preferential looking at depth gratings vs. zero-depth gratings, and visual tracking using RDKs,it was found that stereopsis emerges at about 3.5 months, and improves thereafter. Use of pictorialcues (interposition and familiar size) can be demonstrated at 7 months (infants reach for the nearerobject on the basis of the cue).

The gradual improvement in visual function could represent a slow physiological maturation of the visualsystem, or it could represent learning through experience. What is the physical state of development at birth,and does it follow a pre-determined path?

3) Genetic programming vs. experience

Physical signs of development can be found: newborns have poorly developed foveas, with a muchlower cone density (1/3rd) than in adults. The visual cortex is also poorly developed at birth, with fewer cellbodies and sparser connections than in adults. However, one cannot reach firm conclusions concerninggenetics vs. experience from either the physical or the perceptual data. It is not possible to avoid some formof visual experience before testing even the youngest infants, and this experience may contribute todevelopment. Nor can the delayed appearance of a visual capability be taken as evidence for the importanceof experience, since physiological maturation may have been responsible.

It is likely that development results from a complex interaction between genetics and experience:visual experience acts as a guide, steering the path of development along a route that ensures that the infant iswell matched with his/her visual environment. Evidence comes from two sources: experimental studies ofdevelopment in animals, and clinical studies of developmental deficiencies in humans.

4) Animal studies of development

a) Newborns. 8-16 day old kittens with no visual experience have adult-like cortical receptive fields.Although the cells are binocular, accurate alignment only emerges at about 5 weeks, so there is no wayof testing for stereopsis at birth.b) Deprivation studies. Deprivation causes a loss of acuity which worsens as the period of deprivationlengthens. The loss is greater with monocular deprivation than with binocular deprivation. Monoculardeprivation results in a loss of binocularity. Hubel and Weisel found that monocular deprivation inkittens has the most drastic effects on binocularity during a sensitive period between the 2nd and 6thmonths of life. Loss of binocularity can also be induced by experimentally misaligning the animal'seyes. The cortex is obviously changeable for a long period after birth. Why? Growth causes the eyesto move apart. The visual system must be able to compensate for this change in visual input byadjusting the response properties of its neurones.c) Selective rearing. Blakemore and Cooper exposed kittens only to patterns of vertical (or horizontalstripes) stripes for the first 5 months. Behviourally, the cats seemed blind to stripes at right angles tothose they has been exposed to. Electrophysiological recordings found that cortical cells preferred lineorientations near those used for rearing, not in the orthogonal orientation. Equivalent effects ofselective rearing have been found for motion detection.

These studies demonstrate that the visual system `tunes' itself to the environment in which its owner lives.Though basic visual capabilities are present at birth and the brain imposes limits on the influence of theenvironment on visual functions.

5) Human studies.

About 5% of adults suffer from`amblyopia', or reduced visual acuity in one eye which cannot beattributed to refractive (focusing) errors. The defect varies in severity. There are two causes:

a) Early strabismus. Some children are born with a defect in the ocular muscles which preventsbinocular co-ordination - the eyes point in different directions. If the defect is not corrected until thechild is 4-5 years old, then he/she will lack stereoscopic vision (ie. binocular neurones). Binocularitycan be assessed using the inter-ocular transfer of the tilt after-effect. Studies have shown that thesensitive period in humans peaks during the second year.b) Early differences in refractive error eg. astigmatism. If the two eyes differ greatly in terms offocussing power, then this difference is likely to become fixed in neural responses. For example,corneal distortion can cause an inability to focus lines at different orientations simultaneously. Thisastigmatism results in meridional amblyopia later in life - the astigmatism remains even whencorrecting lenses are worn. A bias in early development results in a bias in neural functioning.

Thus studies of early defects in human vision support the experimental findings using animals: visualdevelopment reflects an interaction between innate visual capabilties and environmental influences. Plasticityallows the visual system to adjust for growth and to fine-tune itself to the individual's visual environment.The cost is a tendency for early defects to cause irreversible changes in visual functioning later in life.

4. Cases of restored sight in humans

In rare cases when sight is restored to an individual who has been blind since birth, or shortly after, there israpid progress in certain visual abilities over the first few months eg. reading, face recognition, objectrecognition. However, other abilities remain poorly developed, particularly the use of pictorial and otherdepth cues, and the use of perceptual constancies.

5. Downhill from here...

Of course, vision does not remain at its peak during later life but gradually declines in a number of ways, forexample:

1) Reduced accommodation by the mid-forties, eventually requiring reading glasses.

2) Reduced sensitivity due to a smaller pupil size and opacities in the cornea and lens.

3) Age-related loss in visual cortical neurons - 20-yr olds have 46 million cells per gram of cortical tissue,but 80-yr olds have only 24 million.

There are also losses in other senses - a marked drop in high-frequency hearing, and losses in smell andtaste sensitivity.

PERCEPTION TOPIC NOTES: Individual Differences in Vision

So far we have concentrated on the many aspects of perception that we are assumed to shareuniversally. In this lecture we turn to aspects of perception that are subject to individualvariation. Since perception relies upon physical function, most differences in the former can betraced to differences in the latter. I will concentrate on two sources of variation – Clinicalconditions, and gender differences.

1. Clinical Conditions

To look for physical conditions that lead to individual differences we can start right at the firststage of vision – the cornea and lens in the eye.

1) Defects of accommodationIn order to project a sharp image on the retina, the optical power of the lens must be adjusted

appropriately to refract incoming rays of light by just the right amount. Near objects requiregreater optical power than far objects. In a correctly functioning eye, accommodation (variationin the shape of the lens) maintains a sharp image on the retina regardless of distance (except forvery near distances) and image shape. Three defects can lead to blurred vision:a) Myopia or shortsightedness

An inability to focus far objects. The problem is that parallel light rays from a far objectare refracted too much by the lens even in its relaxed state. Accommodation only serves torefract the rays even more. Near objects can be seen clearly with relatively little accommodation.Corrective lenses diverge the rays from far objects so that they fall into focus on the retina.b) Hypermetropia or longsightedness

An inability to focus near objects. Light rays from far objects are not refracted enough,so accommodation must be used to bring them into focus. Near objects are outside the range ofaccommodation. A corrective lens can be prescribed to converge light rays from far objects,removing the need to accommodate.c) Astigmatism.

The surface of a normal cornea has spherical curvature (like a ball), and myopia orhypermetropia can be compensated using a `spherical' correction. An astigmatic cornea haselliptical curvature (like a spoon). The variation in curvature means that the refractive power ofthe cornea differs for different orientations, and lines of different orientation cannot be broughtinto focus simultaneously, resulting in strain. The error is not usually severe enough to requirecorrection by itself, but if there are problems with accommodation as well, then a `cylindrical'correction can be included in the prescription.

2) Optic nerve damageGlaucoma is increased pressure inside the eye, due to a blockage in the out-flow of

ocular fluid (aqueous humour), reduces blood supply to the retina and presses on the head of theoptic nerve. The result can be degeneration of optic nerve fibres and consequent blindness. Themost common type of glaucoma responds readily to drug treatment to reduce production ofaqueous and free the blockage.

3) Central visual disordersThese all result from brain damage, and depend on the site and extent of damage.

a) AgnosiasLack of knowledge about certain aspects of the visual world, even though basic visual

functions appear intact. For example, associative visual agnosia involves an inability to nameobjects presented visually, depsite correct naming with auditory or tactile stimuli.

b) NeglectIn this curious disorder, the patient selectively ignores or neglects one half of the visual

world. The effect is not simply tied in with one side of the visual field. The individual deniesone side of the world, shaving and dressing on one side, eating from one side of a plate,drawing one side of a picture, imagining one side of a remembered scene.

2. Gender differences

There are a number of ways in which men and women have been found to be slightly butconsistently different. This table shows examples of gender variation:

Females Superior Males Superior

Smell sensitivity Photopic acuityTaste sensitivity Visual reaction timeTouch sensitivity Time discriminationHearing sensitivity (high freq.) Time-to-contact judgementScotopic acuity Visuo-spatial ability

There is evidence that at least some of these differences can be traced to hormonal differences.For example, the female superiority in smell and taste (and, incidentally, their preference forsweet tastes) varies with the menstrual cycle. However, some consistent gender differences arelikely to reflect systematic differences in learning and experience between males and females.For example, boys tend to be given toys that involve a spatial component (Lego sets, videogames) and therefore tend to have more experience at tasks requiring visuo-spatial skills.

Summary Lecture on the Visual System

Organisation of visual analysis

According to our subjective impressions, there is a direct link between the retinal image andperception of object and scene properties. However, you should now be convinced that inbetween there a largely hidden but very sophisticated system at work, extracting informationfrom the image and deriving abstract knowledge from it.

We have seen that this ‘black box’ system of analysis can actually be broken down into severalsequential stages. Each of these stages can itself be broken down into component operations.

How can we be sure that the visual system is organised in this way?

Evidence

RetinalImage

VisualAnalysis

Object & Scene

Properties

The Visual System

RetinalImage

RetinalProcessing Modular Analysis

Shape & SurfaceRepresentation

Object & Scene

Properties

ReceptorSampling

ON cells

OFF cells

Motion

Depth

Colour

Contour

Grouping Sketches

RetinalImage

Object & Scene

Properties

1.Retina We have direct and detailed knowledge of retinal processing from anatomy andphysiology - the retina is covered by a matrix of receptor cells, and these feed a network of cellsthat respond to variations of intensity and colour in the retinal image. Some perceptual illusionscan be attributed to retinal processing in a straightforward manner.

2. Modular analysis We have covered physiological evidence that individual cells in the cortexrespond to only a restricted range of stimuli. Eg. some respond to colour and not much else,others respond to motion but are indifferent to colour, etc. There also appear to be completevisual areas that specialise in responses to particular visual attributes. There is a great deal ofperceptual evidence to support the idea of specialised modular analysis, and we have looked atsome of this evidence in detail. Eg. adaptation to motion does not affect perception of colour,and vice-versa. Stereo cues alone are sufficient to yield an impression of depth.

3. Shape analysis Many old perceptual phenomena reveal that the visual system attempts toorganise the image in terms of larger scale regions and shapes, based on elementary properties(eg. common motion, colour). These shapes are precursors for objects.

4. Many people regard this system as fairly self-contained and distinction from memory andcognition (involved in object recognition). One justification for this belief can be found in theproblems of individuals with agnosia. They appear to have an intact and functioning visualsystem, but the link to object recognition has been severed, so they can no longer relate visualinformation to knowledge.

Outstanding Issues

1. This organisation begs a major question. How can we reconcile the modularity of visualprocessing with our apparently unified and seamless visual experience? All of the visualattributes (colour, motion, depth, etc.) are glued together in a single experience, despite the factthat this information is being transmitted through entirely different cells and cortical areas.Where does the glue come from? This is a very difficult question, bordering on morephilosphical questions on the nature of consciousness, and no-one has a good answer.

2. Historically, research in perception has been driven by a desire to explain particular visualphenomena, particularly visual illusions. Eg. motion after-effect, colour contrast. This is notnecessarily the best way to motivate research, because it may not be easy to see how the illusionand its explanation relate to larger perceptual issues. Nevertheless, many insights have beengained in this way (motion detection, opponent-processing). This approach starts with aphenomenon, and tries to generate a theoretical model. A complementary approach is to startfrom a model and generate predictions for phenomena. Where does the model come from? Wecan ask a question about how to acquire the information needed to solve a particular visualproblem, eg. estimate the slant of a surface.This is more popular today, with the use ofcomputers to develop precise and explicit models.

Future advances in our knowledge of perception are likely to involve both research strategies.