Mapping Gestures Across Art Forms

8/13/2019 Mapping Gestures Across Art Forms

1/99

Mapping gesture across art

formsPhD by Project

Roger Alsop


2/99


3/99

Presentation structure and projectsummary Overview of the project considering the:

purpose theoretical base conceptual factors developmental factors Resolutions

Chapter outlines Demonstration of works


4/99

Project purpose

To develop a set of approaches totranslating motion, text, and visualgestures into temporal, ephemeral,perceiver focused audio-visual (screen)works through a perceiver interactiveprocesses.


5/99

Purpose

To create audio visual works that use: Movement gesture Speech gesture Visual gesture

to form the works at their most fundamental,structural level


6/99

Purpose

It is essential that the interaction betweenthe audio and visual aspects of the worksbe attributable, and not based onpersonal, aesthetic interpretations of theworks considered in creating the newworks.

In doing, so demonstrate a direct translationbetween gesture from one form ofcommunication to another.


7/99

Gestures

Three types of physical gesture ofdecreasing size are considered: The movement gesture: is the conscious

physical movement of the body. The speech gesture: is the intuitive motion of

the mouth as, notated through the phoneme

The visual gesture is saccadic motion, theunconscious motion of the oculomotormuscles


8/99

Theoretical base

Internal theories (what I am bringing to theproject)

The author expresses Allotts impulse

through assembling elements (data) in acertain order That this data can be manipulated via

multidimensional, interdisciplinary processingto create new works

That the perceiver of any art work has equal& similar agency in the interpretation of the

impulse to the author


9/99

Theoretical base

External theories There are two strands to the external

theoretical bases behind this project;

specific discussions regarding communication,interaction, and process development specific art works that exemplify concepts

influential in the project

This project considers an artwork per se asbeing a theoretical discussion


10/99

Theoretical base (artworks)

Specific artworks that inform this work Onos Bo x o f sm i le

Cages Theatre Piece No 1, Radio Music,and 433 Nonos Fragmente Stille, An Diotima

Yuskavages True Blondes Pollocks Number 11


11/99

Theoretical base (discussions)

Specific discussions inform this work Lyotards Differend

Jakobsons Tripartite schemata of verbalcommunication Kansei

Notation Algorithmic processes


12/99

Conceptual factors The gesture as the universal communicative tool

Some gestures are deliberate and conscious, some areintuitive, and some are automatic


13/99


14/99

Conceptual factors The gesture as the universal communicative tool

Some gestures are deliberate and conscious, some areintuitive, and some are automatic

The majority of communication is aural and/or visual

Aural and visual communications are broadcastthrough subtle changes in a medium (airmolecules or photons/ electromagnetic waves)


15/99


16/99

Conceptual factors These changes in a medium can be represented as

numeric data (which can be manipulated) That the data produced by the creator is an

interpretation of their intentions

This data received is interpreted by the perceiver, andis separate from the creator


17/99

Conceptual factorsFundamental tenets of the project

All art is essentially ephemeral, existing only in itsperception.

All art can be interpreted as data. The data-fication of art allows its reinterpretation and

translation (bridging Derridas horizon of aconsensual sense, of a translatability)


18/99

Developmental factors in creatingthe works

Developing interface design processes fortranslating aspects of verbal gesture into an audio visual result motion gesture into an audio visual result ocular gesture into an audio visual result

Exploring, developing and articulatingconceptual frames that have intuitivelyinformed my processes


19/99

Developmental factors in mypractice

This project has developed my practice to include visual aspects

to the formation of concrete relationshipsbetween visual and aural aspects to emphasize the ephemeral, perceiver

centered nature of art works


20/99

Resolution

The creation of three interactive worksbased on perceiver interaction

These works are created from perceiver movement perceiver data input perceiver interpretation

and are essentially ephemeral, designed toexist only when being acted with by theperceiver


21/99

Project summary/description

This project explores ways that the results of different artsmaking processes can create novel results in a single artmaking process.

Premises on which the work is based: that the points of intersection between creative

processes are fundamentally greater than the points ofdivergence;

that while an artworks experiential results may bedissimilar, their intellectual and instinctual causes havefundamental similarities.

that the perceivers role in the making an art work issimilar to the authors


22/99

Project summary/description

The project is the creation of systems that can be used bythe artist in translating the results of one art form intoresults in another.

These systems are similar to other systems used totranslate concept and idea into a perceivable result (egthe piano or paintbrush).

When creating the systems interaction with the perceiverwas an essential part of the design


23/99

Chapter outline


24/99

Chapter 1

Introduction and document outline: the conceptual/theoretical frames the research question and approach the approach to the literature background to the works and processes discussion of each of the three processes


25/99

Chapter 2

Conceptual frames Role of the maker/perceiver Role of the process

Questions What is a gesture How can gestures be translated

Interpreting gesture as data Computer as pivot


26/99

Chapter 3

Literature - Artworks Detailed discussion on

Onos Box of smile Cages Theatre Piece No 1, Radio Music, and

433 Nonos Fragmente Stille, An Diotima Yuskavages True Blondes Pollocks Blue Poles


27/99

Chapter 4

Literature - Concepts Lyotards Differend

Jakobsons Tripartite schemata of verbalcommunication


28/99


29/99

Background to works

Overall goals: to create works that allow interaction that is novel

but predictable Interface design

that is simple but requires developing an intuitiveunderstanding

Processes that develop similar interfaces across the three

different translators Anticipated outcomes

processes that is engaging due to its outcomes


30/99


31/99

Chapter 7

Discussing TEXT Goals

To use the speech motion, as defined through

phonemic notation to create a computerizedviewing of an image

Process To develop interfaces that reflect the user through

structures based on speech gesture

Outcomes


32/99


33/99


34/99

Discoveries

How stylized motion can create motion inanother art form Motion/interaction to AV outcome

How sonic gestural structure can betranslated from one temporal art form toanother

Text/interaction to AV outcome How automatic motion creates a gesture

Ocular motion/interaction to AV outcome


35/99

Implications

Forming an open ended approach to artmaking

Forming a new set of tools for artmaking/art perception

POTENTIAL DEVELOPMENT OF NEWPARADIGMS FOR ART MAKING The artist creates Field of play for audience


36/99

Outcomes

Creation of an open-ended making processdeveloped from any input

Development of an improvisational, perceiverfocused approach to creating abstract audiovisual

works Developing relationships between visual and sonicelements in art making in the viewer

Developing processes for crossing boundariesbetween art forms

Creating fulcrum or pivot around which art formsmay orbit


37/99

Conclusions

Creation of real time ephemeral works Direct transformation between

performative processes and concreteprocesses


38/99

(To be further explored here)

Tools are not benign Direct transformation between

performative processes and concreteprocesses


39/99

(Value of art)

To expand the imagination beyond what canbe imagined

Direct transformation between performative

processes and concrete processes My role is similar to that of a playwright, I

create a set of possibilities that can beinterpreted by the user

A play is interpreted by a director/actor/etcwithin the paradigms of the play just as mywork creates a paradigm for interpretation.the


40/99


41/99


42/99

Developing the discussion


43/99


44/99

Issues 1

An artwork exists in its perception andinterpretation

The constantly shifting relationship

between maker and perceiver Perceiver is an active agent in the art

making process

Maker and perceiver are simultaneouslyINTER/INTRA DEPENDANT in thecreation of a work


45/99

Issues 2

All communication can be interpreted asdata

Most data is perceived extereoceptively,forming interoceptive responses

Maker and perceiver are simultaneouslyINTER/INTRA DEPENDANT in thecreation of a work


46/99

Issues 3

My work is Multidisciplinary & Multi ModalThe focus is the relationship between the sender, object

and perceiver.The perceiver is as active in the creation of a work as the

creatorThe perceiver is the active receiver of the dataVery slippery concept as it is very subjective

Responses are subject to too many thingsPerceiver Art forms are transcending boundaries


47/99

The Maker Perceiver relationship

The maker creates an art gesture to which they are tangiblyconnected. The gesture affects the perceiver creating a virtualconnection between the maker and the perceiver. The perceiver alsoconnects to the gesture, remaking it with each viewing orrecollection.

Maker Gesture Perceiver

Virtual relationship between Perceiver andMaker

Virtual relationship between Maker and Perceiver


48/99

Questions

What is a gesture How can a gesture be used in transformational

mapping

Role of the perceiver In perceiving the artistic gesture Perception is a creative act

Role of the author In creation of the artistic gesture


49/99

Questions: What is a gesture

A gesture is a movement or change ofstate perceived over time .

Whatever the manifestation of the gesture itbegins in an internalized, mental state.

The gesture is externalized and made materialby the author, broadcast through a medium,perceived, and then finally returned to amental state in the perceiver


50/99


51/99


52/99

4 areas of art

intersection

sound

image

text

movement

ART


53/99

Questions: Role of the perceiver

To paraphrase Barthes, from Death of the Author:The perceiver is the anonymous space in which all

gestures are inscribed, the unifying, impersonaldestination, holding all gestures.

The perceiver creates the work through theirown interpretation of the inscribedgestures

All perception is unique, individual, andimmediate to the perceiver in the act ofperceiving

Questions: Role of the perceiver in my


54/99

Questions: Role of the perceiver in mywork

The perceiver is creates the work, their input,regarding: choice of interpretation processes (TEXT) choice of materials and interpretation processes

(IMAGE) choice of interpretation processes and their

motion (MOTION)creates the work that they experience through

applying gestures to the tools andsubsequently interpreting the resultant AVgesture.


55/99

Questions: Role of the author

The author is a pointer to the intentions ofan art work.

The author externalizes their intentions

through translating them into sonic orvisual data to be perceived by an external,anonymous perceiver.

The author creates potential interpretations,of the authors intentions, in the perceiverthrough providing visual or sonic data

l f h h


56/99

Questions: Role of the author in mywork

Here the author builds on their role as apointer to the intention of the art work.

By creating an environment in which theperceiver uses the paradigms of thatenvironment created by the author tocause a re-viewing of visual and sonicinput.


57/99

Methodology

To create pivots around which art works canbe translated from one mode to another

Explore and discuss approaches to artmaking and perception throughconsidering specific artworks and specifictheories


58/99

Methodology: Creating Pivots

PROCESSESTechnologies

Previouswork

Inspirations Histories

Conceptualframes

OUTCOMES


59/99

Methodology: Creating Pivots

The process creates the pivot on whichthe transformation can manifest

The pivot is the liminal, interstitial point atwhich a mapping can be made


60/99

Methodology: Discussion

Elevate the process to the resultProcesses exits in art making

An artists model is a process, creating a template

for a paintingSerial composition, fugue, counterpoint areexamples of templates in music

Make the process the result


61/99

Creative outcomes

Three interactive audiovisual works/processes:1.TEXT2.IMAGE

3.MOTIONThese use different input from external sources to

create AV works based on processes created

here.The user/perceiver can create a work based on theparadigms built into the processes.


62/99

Previous work

Master of arts Papers


63/99

Previous work

Master of arts


64/99

Previous work

Master of arts Papers


65/99

Location: Conceptual frames

Communication/making sense Everything is data Lyotard Jakobson Cage

C t l f Everything is


66/99

Conceptual frames: Everything isdata

C t l f Lyotard & the


67/99

Conceptual frames: Lyotard & theDifferendthe very point of The Differend is the very horizon of a

consensual sense, of a translatability, of a possible totranslate. (Derrida, 2000)

C t l f Jakobson &


68/99

Conceptual frames: Jakobson &Communication Insert image of communication

environment here

C t l f Cage & Theatre


69/99

Conceptual frames: Cage & TheatrePiece No 1


70/99


71/99

Conceptual frames: Making Sense


72/99

Location: Influences

Artists Non- Electronic arts Electronic arts

Systems Approaches to art making

Creativity Generation Recognition


73/99

Location : Non- Electronic Artists

Sound Nono, Webern, Cage

Vision Ono, Yuskavage, Pollock

Text Joyce, Bukowski, Gomringer


74/99

Location : Electronic Artists

Sound Ikeda, Lansky, Ashley, Lucier, Cage

Vision (Time based) Polli, Marclay, Czarnecki, Barbara Kruger,

Wearing, Viola there are also VJ artists as a

point of contention, visual music


75/99

Location : Systems

Cage Kansei Computers

Notation


76/99

Location: Systems: Cage

iChingIndeterminacyNegation of ego


77/99

Location: Systems: Kansei


78/99

Location: Systems: Notation

Note-fication is data-ficationNotation is an abstracted representation of

tangible events, allowing those events tobe considered algebraically

This allows the artist to develop andinterpret works through mathematicalprocesses

And to adjust events without reference totheir materiality


79/99

Location: Notation : Sonic Notation

Musical notation: Pitch and duration representation


80/99


Sonic notation: Frequency, amplitude and duration

representation


81/99


Graphic notation: Representing form to be interpreted by the

performer


82/99

Location: Notation : Motion Notation

Laban notation


83/99

Location: Notation : Motion Notation

visual representation


84/99

Location: Notation : Image Notation

Paint by numbers


85/99


Paint by numbers

Completed image


86/99


The computer usually stores or generatesan image as 4 numbers located at aspecific point (pixel) within a screen, the

first three numbers relate to the red,green, blue values of the pixel, and thefourth to the opacity of the resultant colour.


87/99

Location: Notation : Image Notation Adjusting numbers in computer stored

images.

RGB to RGB (Red Green Blue toRed Green Blue)

h l l


88/99

Location: Notation : Morphological Notation

Denis Smalley described a way ofconsidering the morphological changesthat occur over time in electro-acoustic

music, using text as his communicationtoolKevin Patton developed these ideas using

visual images to represent morphologicalchange in sound.

This representation can be used torepresent any kind of morphological

change

h l l


89/99

Location: Notation : Morphological Notation

C


90/99

Location: Systems: Computers

C i i (i i i )


91/99

Location : Creativity (imagination)

Generation Recognition


92/99

P


93/99

Processes

These systems are creative works that haveresonances with:Playwriting, in which the playwright sets a text that may

be reinterpretedJazz improvisation, in which a sequence of harmoniesare interpreted by a musician

MOTION


94/99

Processes : MOTION

Improvised creation of a AV work Ephemeral input Reactive/Collaborative

machine human human machine

Based on the kinetic/silent gesture The perceiver becomes the art work


95/99

TEXT


96/99

Processes : TEXT

Static/set input Concrete User interactive based on the sonic/temporal gesture

COMMENT


97/99

Processes: Text : COMMENT

Static/set input Concrete User interactive based on the sonic/temporal gesture

IMAGE


98/99

Processes : IMAGE

Static/set input Concrete User interactive Based on silent/non-temporal

(IMMEDIATE) gesture

P I COMMENT


99/99

Processes: Image : COMMENT

Static/set input Concrete User interactive Based on silent/non-temporal

(IMMEDIATE) gesture

Mapping Gestures Across Art Forms

Documents

Transcript of Mapping Gestures Across Art Forms