Processing Facial Emotion

Darwin (1872) argued that facial expression evolved from expressions in other animals and are therefore innate.

Universality of Facial Emotion

Ekman & Oster (1979) review evidence for cross-cultural similarities in emotion processing.

Disgust Anger

Fear

Happiness

Surprise

Sadness

Literate Cultures Show Distinctive Facial Expression For:

HappinessSadnessDisgustAngerFear ) Preliterate cultures show Surprise ) confusion over fear and surprise

But where might emotion be processed in the brain?

Are there separable brain regions for processing each of these different emotions?

Laterality and Emotion

Carmon and Nachshon (1973): Left ear advantage for discriminating emotion eliciting sounds (crying, laughing etc.).

Damage to RH more likely to result in impaired emotion recognition (Bowers et al., 1985) and loss of prosody in voice(emotional expression).

LH generally has positive affect - RH negative affect. Damage to LH leave patients feeling anxious or angry, damage to RH leaves them with flattened affect or indifference toward failure. Emotional processing different across the hemispheres, but perhaps emotion more right than left.

Laterality and Emotion

Original Face

Left Half Composite

Right HalfComposite

Sackheim & Gur (1978) showed subjects chimeric faces and asked which ones showed the strongest emotion. Left halves more expressive than right halves (Left half perceived by RH).

The Limbic System

System for Reward, Motivation, Arousal, Emotion etc.

ElectrodeImplanted inSeptum Area

Rat presses lever to stimulate the brain - Yeah baby!

Rat falls asleepexhausted but happy!

Rat wakes and immediately starts topress the lever again. Rats will even neglect babies.

The Amygdala

There are two routes to the Amygdala:

1: Subcortical Route

Via the superior colliculus and pulvinal thalamus.

2. Cortical Route

Via the visual cortex.

Subliminally presented fearful faces activate the amygdala in normal participants. In fact unattended presentation gives highest level of activation.

Blindsight patients and those with hemispatial neglect can discriminate emotional expression.

The Amygdala

Lesion studies find impaired recognition of emotional faces when bilateral damage has occurred.

This is mostly associated with fear (Calder et al., 1996, Morris et al., 1996), but may also be found in other “negative” emotions such as angeranger, sadnesssadness and disgustdisgust (Adolphs et al., 1991; Adolphs, 2002 Cur. Biol).

Some have argued that the amygdala is particularly important in processing emotion related to perceived threat.

Individuals with bi-lateral lesions also appear to be indiscriminately friendly and have problems judging trust in others. Elderly people also show reduced amygdala activation - they also get conned more by people?

fMRI Evidence

But so does comparing happy faces with neutral faces!(Somerville et al., 2004)

A. Comparing Fearful Faces with Neutral Faces shows bi-lateral Amygdala activation.

B. Comparing Angry Faces with Neutral Faces also shows Amygdala activation.

Whalen et al. (2001)

But Why?

What is it doing?

Heberlein et al. (1998) Impaired attribution of socialmeanings to abstract dynamic visual patterns following damage to the amygdala.

The Amygdala as an Ambiguity Detector

Whalen et al. (2001) argue that the pattern of activation found for the amygdala may not reflect the processing of negative emotion/threat per se, but more about detecting ambiguity in the face and how that relates to predictibilitypredictibility of the persons

actions.

Both convey perceived threat but ANGRY face conveys information about the source of the threat.

Emotion and Gaze Information

Are you looking at me?Are you looking at me?

Unambiguous

He’s angry at me!

Ambiguous

Who’s he angry at?

Emotion and Gaze Information

Are you looking at me?Are you looking at me?

Unambiguous

Something threatening to

my left!

Ambiguous

Where is the threat?

Emotion and Gaze Information

Are you looking at me?Are you looking at me?

Ambiguous

Summary of Amygdala Function in Emotion Recognition

Part of rapid system for processing highly salient stimuli.Responds most to autonomic/unconscious stimuli.More responsive to fearful than angry faces.Also active for ascribing social meaning to dynamic representations.May actually be important in prediction of threat in ambiguous situations (Whalen et al., 2001; Adams et al., 2003).

The Orbito-Frontal Cortex (OFC) and Negative Emotion

OFC unlike amygdala responds more to explicitly encoded aversive stimuli perhaps with a view to generating a response to them.

Responds more to fear than happiness (Kawasaki et al., 2001) and to anger over sadness (Blair et al., 1999)

Response latency in OFC 120-150ms. Thus OFC like amygdala can make rapid responses to emotionally salient stimuli. Can modulate early aspects of perception

Insula

The anterior insula is connected to the ventro–posterior–medial thalamic nucleus, and has been identified in primates as gustatory cortex (Rolls, 1994), containing neurons that respond to pleasant and unpleasant tastes (Yaxley et al., 1988).

The Insular Cortex & Basal Ganglia and Disgust

Insula

Calder et al. (2000) report a patient with damage to left insula and left basal ganglia who was impaired at recognizing disgust in facial expression and in the ability to experience disgust himself.

Adolphs et al. (2002) report a patient with bi-lateral insula damage who was severely impaired at experiencingexperiencing and recognizing disgust from all types of stimuli.

BasalGanglia

The Insular Cortex & Basal Ganglia and Disgust

Phillips et al. (1997)

75%

150%

150% > 75%activates rightanterior insula

The Insular Cortex & Basal Ganglia and Disgust

The Insular Cortex & Basal Ganglia and Disgust

Diseases that specifically attack the basal ganglia such as Huntingdon’s Disease lead to disproportionate impairment in disgust processing (Sprengelmeyer et al., 1996)

The Insular Cortex & Basal Ganglia and Disgust

Summary

Recognition of disgust appears to occur in a region associated with experiencing the feeling of disgust.

Somatosensory Cortex

Lundqvist & Dimberg (1995)

Recorded facial muscle movements while participants viewed emotional faces. They noted that the muscles used to produce facial expressions were activated simply by viewing them.

Thus recognition may occur from simulation of the emotion by the perceiver.

Somatosensory Cortex

Do we therefore recognize facial emotions in others by simulating them ourselves?

Somatosensory Cortex

The Simulation Hypothesis

Support from other literatures….

Rizzolatti and colleagues (Gallese et al., 1996; Rizzolatti et al., 1996) report a “mirror system” in premotor cortex.

When observing the actions of others, brain activity is also observed in the motor cortex of the perceiver.

Thus we simulate the movements of others.

Referred to as the “Mirror System” or “Mirror Neurons.

In emotion there is also evidence of a causal link between the production of emotional expressions and changes in emotional state (Cacioppo et al., 1992).

Thinking happy thoughts can also enable you to fly! (JM Barrie)

Viewing the facial expressions of others can lead to changes in the emotional experience of the perceiver (Wild et al., 2001).

So do we simulate to recognize emotion in others?

Möbius Syndrome

Rare congenital disorder. Infant has problems feeding and exhibits a mask-like expression.

Diagnosis reserved for palsies of the VIth and VIIth cranial nerves and skeletal defects (missing fingers, toes, club foot).

Also problems with eye movements in horizontal plane.

Möbius Syndrome

Möbius Syndrome

So can Möbius individuals perceive emotions in others in the absence of simulation themselves?

Giannini et al. (1984). MS individual asked to watch video of face of someone playing a slot machine and to guess what value “jackpot” they were playing for. Found that MS was unable to perform this task. Also informed experimenters that she could not interpret facial expression.

Möbius Syndrome

Calder et al. (2000) tested 3 MS individuals and found no impairment in recognizing basic types of facial affect (anger, fear, sad, happy, disgust). Did show mild impairment in face recognition - may be due to problems with eye movements.

Suggests that simulation not necessary for facial emotion recognition.

Damasio (1994, 1999) argues that one can bypass the generation stage of producing the expression and utilize the direct modulation of somatic (feeling) structures to simulate.

The Time Course and Network for Emotion Processing

Core system engaged in person perception (IOG, STS, FG). Processing in areas like FG can take ~170ms (N170).

But…

Discrimination of emotion can occur at ~80 to 110ms (Pizzagali et al., 1999; Halgren et al., 2000).

Amygdala responds to emotion at ~120ms (Halgren et al., 1994), but activation is reduced when subjects asked to report the emotion exhibited by the face. Suggestion that activation in amygdala inhibited by the frontal cortices here.

OFC: Damage affects emotion processing. OFC activation occurs ~150ms (Marinkovic et al., (2000).

OFC activation also found ~120ms (Kawasaki et al., 2001) so perhaps OFC can respond rapidly to salient stimuli also.

Early processing of salient stimuli. Subcortical route feed-back to OC.Bottom-Up Process

Detailed perception through Core system and emotional reaction (OFC).

Interaction between visual and somatosensory areas in recognition of facial emotion, and possible simulation.

Top-Down ProcessAdolphs (2002)

Different Neural Systems At Different Time Points: Amygdala & OFC

Summary

Multiple distinct brain areas involved in the perception and recognition of emotion and in the resultant observer expression of emotion. Different systems reflect different requirement for information extraction at different times, and bottom-up vs. top-down processes.

Perception-Reaction-Conceptual Processes

Recommended Reading

Adams R (2003) Science, 300.Adolphs R (2002) Current Opinion in Neurobiology, 12.Adolphs R (2002) Beh. & Cog. Neuro. Revi., 1.Adolphs R (2003) Nat. Reviews. Neuro., 4.Calder et al. (2000) Cognitive Neuropsychology, 17.Kawasaki et al. (2001) Nature Neuro., 4.Phillips et al. (1997) Nature, 389.Whalen PJ et al (2001) Emotion, 1.Whalen et al. (1998) Curr. Dir. Psych. Science, 7.