SceneMaker:Automatic Visualisation of Screenplays

School of Computing & Intelligent SystemsFaculty of Computing & EngineeringUniversity of Ulster, Magee, Northern Ireland

hanser-e@email.ulster.ac.uk, {p.mckevitt, tf.lunney, j.condell}@ulster.ac.uk

Eva Hanser Prof. Paul Mc Kevitt Dr. Tom Lunney Dr. Joan Condell

SCENEMAKER OVERVIEW

• Automatically generate animated scenes from screenplays

AIMS

• Realistic visualisation of emotional aspects

• Well-designed representation through 3D animation, speech, audio and cinematography

• Enhance believability of virtual actors and scene presentation

: AIMS & OBJECTIVES

Input:Screen-play SceneMaker

SystemOutput: Animation

OBJECTIVES

• Inferencing emotions and semantic information within story context

• Common sense, affective and cinematic knowledge reflecting human cognitive reasoning

• Automatic genre recognition from text

• Design, implementation and evaluation of SceneMaker

: AIMS & OBJECTIVES

• Standardized format and language of screenplays

• Automatic access to information on location, timing, props, actors, events, manners, dialogue and camera direction (Jhala 2008)

SEMANTIC TEXT PROCESSING: RELATED PROJECTS

INT. M.I.T. HALLWAY -- NIGHT Lambeau and Tom come around a corner. His P.O.V. reveals a figure in silhouette blazing through the proof on the chalkboard. There is a mop and a bucket beside him. As Lambeau draws closer, reveal that the figure is Will, in his janitor's uniform. There is a look of intense concentration in his eyes. 

LAMBEAUExcuse me!

 WILL

Oh, I'm sorry. 

LAMBEAUWhat're you doing?

 WILL

(walking away)I'm sorry.

Screenplay Extract from ‘Good Will Hunting (1997)’

• Reflecting emotions in automatic physical transformation of 3D models

• Manner influences intensity, scale, force, fluency and timing of an action

MODELLING EMOTIONAL BEHAVIOUR

Personality & Emotion Engine(Su et al. 2007)

Greta (Pelachaud 2005)

: RELATED PROJECTS

• WordsEye – Scene composition(Coyne & Sproat 2001)

• ScriptViz – Screenplay visualisation(Liu & Leung 2006)

• CONFUCIUS – Action & speech animation(Ma 2006)

• CAMEO – Cinematic and genre visualisation(Shim & Kang 2008)

VISUALISING 3D SCENES

WordsEye CONFUCIUSScriptViz CAMEO

: RELATED PROJECTS

Animation Player

Animation Player

Script EditorScript Editor

Screen-play

Text & LanguageProcessing

Text & LanguageProcessing

ContextInterpretation

ContextInterpretation

MultimediaGenerationMultimediaGeneration}

Genre

Emotion

Action

}

ARCHITECTURE OF SCENEMAKER: DESIGN AND IMPLEMENTATION

GENRE IDENTIFICATION: DESIGN AND IMPLEMENTATION

GateTokenizer

Film Script

.txt Part-Of-Speech

Gazetteer

Sentence Splitter

WordNet

WordNet-Affect

TaggedFilm Script.xml

Script ElementOntology

Training corporaLocation key-phrases

Training corporaLight key-phrases

ControlledIndexing Free

Indexing

KEA Key-phrase Extractor

Genre LocationOntology

Genre Relevant Locations

Genre LightOntology

Genre Relevant Lighting

Theme

ConceptNet

Emotion Module

Most Dominant Emotions

GenreDirectorNotation(Film Ontology)

Dialogue Length

MotionContent

Film Pace

EVALUATION OF SCENEMAKER

Aspect EvaluationCorrectness of screenplay interpretation

Hand-animating scenes

Effectiveness of output scenes

Existing feature film scenes

Suitability for genre type Unknown scripts categorised by readers

Functionality of interface Testing with drama students and directors

: DESIGN AND IMPLEMENTATION

SUMMARY

SceneMaker

• Heightens expressiveness, naturalness and artistic quality of computer generated animations

• Assists crew on set – directors, actors, camera men, ... – drama students, script writers and advertising agencies

• Mobile user interface for immediate use on set or anywhere

• Animations manually editable

Thank you.

QUESTIONS OR COMMENTS?