Story Reaction Structures to Emotion Detection

Nelson Zagalo, Vasco Branco, Anthony Barker

University of Aveiro, Portugal

Story Reaction Structures to Story Reaction Structures to Emotion DetectionEmotion Detection

ACM MULTIMEDIA WORKSHOP SRMC, 2004

IntroductionIntroduction

We’ll present story structures that can be elicited by emotion recognition in storytelling, in order to maximize user emotions and therefore the signification experience.

We feel emotional when we understand story events as important to us.

Emotions are responsible for our cognition capacity.

1/20

Emotion in StorytellingEmotion in StorytellingOral and Cinematic process

Affective Storytelling Process 2/20

Style and RepresentationStyle and Representation

Story

StyleRepresentation

Nuclei

Catalysers

ORAL

Tone of VoiceFacial Expression

PostureGesture

Eye Direction

CINEMATIC

FramingEditingMusic Sound

Lighting

ORAL / CINEMATIC

3/20

Cinematic and OralCinematic and Oral

Representation

Style

Cinematic

Oral

Static

Dynamic

Storyteller reinforces or subtracts according to the story-listeners

expression of emotions, using new elements from Total Story

Representation

Style

Static and Dynamic

Viewer Emotions

4/20

Viewer EmotionsViewer Emotions

Two axes dimensions, one for intensity (arousal) as the “tension-sleep” axis and the other for quality (valence) as the “pleasant - unpleasant” axis.

The points of convergence between Intensity and Quality axes, give us pre-defined labels that identify the emotional-state in a categorical point-of-view.

Four dramatic clusters. General emotional areas that influence the story viewer - anxiety, happiness, sadness and relax

Intensity

Quality

Coding Emotions

Story Viewer Emotions

5/20

Dynamic CinemaDynamic Cinema

Applying the model of the story viewer emotions to cinematic elements in search for a dynamic development.

Emotional Film Structures

6/20

Emotion FlowEmotion Flow

Director’s Version

over-stimulation

under-stimulation

(A) Overall Story Emotion(B) Intermediate Tension sequences

Viewer’s Will

traditional dramatic arcs

Director and Viewer Control

In the Viewer’s Will, the unbalancing of the tension sequences, results in an (A) overall equal or below to any (B) intermediate sequence.

7/20

We’ll analyse here the small unit intermediate tension arc of the Director’s version emotional flow.

Intermediate Tension Arc

Intermediate tension arc unitEmotion FlowEmotion Flow

8/20

Reaction StructuresReaction Structures

The application of the four dramatic emotion clusters to the intermediate tension arc, give us the arc of emotions, the reaction structures guider.

viewer arc of emotions

Arc of emotions

Story Viewer Emotions Intermediate tension arc

=+

9/20


Framing Editing Sound Lighting Music Catalysers

Intensity(variation)

Strong Close-ups

Reduce frame

duration

Fast tempos, crescendo

Darker Explicit

Quality(adding)

Hardly distinguishing.

Shade near night light.

Dissonant chords. Very

high/very low pitches

Anxiety cluster: anger, fear and

disgust representations

Having detected an emotional state in the viewer with low levels of arousal, in a sequence of rising tension, the story would react to get the intended emotional effect from the user

ANXIETY

10/20



Medium frames,

some close ups

Expand frame

duration

Reduce tempo,

decrescendoGreyer Implicit

Quality(adding)

Sufficiently distinguishing.

Cloudy day light.

Consonant chords. Low

pitches

Anxiety cluster: sadness, misery and depression representations


SADNESS

Sadness is a negative low tension emotion, so rhythm needs to decelerate, become greyer and treat emotion catalysers implicitly.

11/20



Medium frames,

some Close-ups

Reduce frame

durationFast tempos Brighter Explicit

Quality(adding)

Very Clear distinguishing.

Sunny day light.

Melodic. High and medium pitches

Anxiety cluster: excitement, delight or euphoria

representations


HAPPINESS

Happiness is a positive high tension emotion, rhythm accelerates, becomes brighter, crystal clear picture definition, treat emotion catalysers explicitly.

12/20



Medium and general frames

Extended frame

duration. Very Linear

Slow tempos, decrescendo

Mellower Implicit

Quality(adding)

Sufficiently distinguishing.

Sunny end afternoon

Melodic. Low and medium pitches

Anxiety cluster: calmness or

repletion representations


RELAX

Relax is an optimal state formed by low tension and positive emotion, it tends to balance everything.

13/20

Case StudyCase StudySilence of the Lambs (1991)

We’ll present a theoretical usage of how reaction structures could operate, analysing the second, 10 minutes period of the movie.

Sequence: First visit Clarisse pays to Hannibal Lecter in prison

The sequence wil be presented divided in four phases: Entrance, Confrontation, Retreat, Flashback

We can’t show the 10 minutes, so we’ll analyse only some frames 14/20

Silence of the LambsSilence of the Lambs(A) Entrance

Entrance of Clarisse into the high-security chamber

Night light, music starts slowly in a crescendo

Anticipation: Medium disgust catalyser

System could analyse emotional effects on the viewer to this disgust catalyser, and learn what levels the viewer is prepared to support

Editing accelerates, framing gives Clarisse regular close-ups

Sounds dissonant with higher pitches

The lighting is dark, difficulty in identifying prisoners’ faces.

“I can smell your cunt”night lightentrance

Tension

15/20

Silence of the LambsSilence of the Lambs(B) Confrontation – part 1

strong close up medium close upconfrontation

Confrontation starts with viewer arousal already activated

Confrontation is pure anxiety, rises the whole story arc.

Anxiety is triumphantly carried off by very strong actors’ performances

Framing is anxiety effective with extreme close-ups

Lighting could become darker

Editing rhythm could be accelerated

Music could be added to help reinforcing anxiety.

Anxiety

16/20

Silence of the LambsSilence of the Lambs(B) Confrontation – part 2

implicit explicit

Strong emotional disgust catalyser

Prisoner Miggs masturbates and then throws his semen at Clarisse’s face

To Intensify – system could show explicit masturbation

To smooth – system could use suggestion instead of explicitly show semen dripping or even could eliminate the scene from the representation.

Anxiety (false ending iniciates sadness cluster)

17/20

Silence of the LambsSilence of the Lambs(C) Retreat

Retreat sequence is the declared arc peak and it’s conducted through sadness cluster.

Clarisse is presented crying, the day light is grey and the melodic low pitched score helps the rising note of sadness.

System, could zoom to tears, add some rain or expand sequence duration with music

sadness grey light

Sadness

18/20

Silence of the LambsSilence of the Lambs(D) Flashback

Flashback constitutes the relaxation structure.

Flashback changes from a grey day in Foster’s present to a sunny end of day in her past.

Action rendered in slow motion and music is melodic.

Calmness and familiarity enters the viewer’s body and mind, relaxing and restoring composure

mellower light familiarity

Relax

19/20

We tried to develop a theoretical story computer system based in oral and cinematic storytelling able to react to emotion detection.

We developed a kind of a dynamic cinema, supporting undirected interactivity in multilinear scenarios.

The analysis we have presented here must be seen as a draft for the next stage of research in the field

Next stage – reactions to emotion detection within VR storytelling.

For that we’re starting to work, inside an european project, in developing an interactive storytelling taxonomy of emotions, to support the development of story reactions within virtual environments.

We are convinced that story reactions to emotions detected will become a field of great research in interactive storytelling

Conclusions & PerspectivesConclusions & Perspectives

20/20

Story Reaction Structures to Emotion DetectionStory Reaction Structures to Emotion Detection Nelson Zagalo1, Anthony Barker3, Vasco Branco2

1,2 Dept. of Communication and Art, University of Aveiro3810-193 Aveiro, Portugal

[email protected], [email protected]

3 Dept. of Languages and Cultures, University of Aveiro3810-193 Aveiro, Portugal

[email protected]

ACM WORKSHOP SRMC, 2004

Story Reaction Structures to Emotion Detection

Business

Transcript of Story Reaction Structures to Emotion Detection