Self Compiled Plot - Step to Understanding - Ana Rink - Ronny Reichmann

8/14/2019 Self Compiled Plot - Step to Understanding - Ana Rink - Ronny Reichmann

http://slidepdf.com/reader/full/self-compiled-plot-step-to-understanding-ana-rink-ronny-reichmann 1/70

gesture based interactions in museums – prof. p. heidkamp – ws 07/08 – kisd

ana rinkronny reichmannself compiled plot step to understanding



Table of Contents

Introduction ............................................................................................................. 1

Research phase I .................................................................................................... 5

First proposals ..................................................................................................... 17

Research phase II ................................................................................................. 19

Research phase III ................................................................................................ 21

Second proposals ................................................................................................. 33

Final concept ........................................................................................................ 49

Testing facility ...................................................................................................... 57

Conclusion ............................................................................................................ 59

References etc. ..................................................................................................... 61



i n t r o d u c t i o n r e s e a rc h p h a s e o n e f i r s t p r o p o s a l s r e s e a r c h p h a s e t w o

“A chain can only be as strong as it’s weakest link.”

1 / 6 6

“Museums have recently developed a strong interest in technology, as they are

more than ever before in the orbit of leisure industries. They are faced with the

challenge of designing appealing exhibitions, handling large volumes of visitors,

and conserving precious artwork.

They look at technology as a possible partner which can help archive a balance

between leisure and learning as well as help them be more effective in convey-

ing story and meaning. Technology can help construct a coherent narrative of

an exhibit for the visitor by creating experiences in which the objects on display

narrate their own story in context. Using interactive techniques embedded in the

physical space museums can present a larger variety and more connected ma-

terial in an engaging manner within the limited space available. The interaction

systems enhance the memory of the visit and help build a constructivist-style

learning experience for the public.” [1].



research phase three second proposals f inal concept test ing facil i ty conclusion

The concept of the spanish agency GPD is fresh and something new in the land-

scape of museums around the world. The contents architecture and the meta-

phorical connection to the architecture of the building, are the strong aspects in

it´s exhibitions.

By testing the systems, made available to us by Alterface and Realtime Visions,

we encountered usability and ergonomic difficulties that cannot be dismissed.

Using both systems is not intuitive at all. It feels unresponsive and slow. One has

to consciously and perpetually adjust its gestures to “force” the system to react

in a proper way. Reducing the extent of the target group to technical enthusi-

asts is not the right way to follow. After discussing this issue we realized that

we didn’t want to contaminate GPD’s strong concept with such a weak link and

therefore decided to NOT call this situation a problem, but a chance to create a

less error-prone and by design more ergonomic and easier to implement inter-

face.

2 / 6 6




By one of our periodic state-analysis one more difficulty was encountered: The

amount of unconscious and therefore usable gestures is small, but a by-pass-

ing visitor has in every case to be taught how to use such an interface anyway.

Especially by keeping in mind the average time a visitor should be allowed to use

the interface, teaching gestures, that a visitor is not used to, seemed the w rong

way for us. We didn’t want to fiddle with more or less graphically / superficial

quick- and dirty-solutions. There had to be more in order to stay with the high

level of GPD’s concept.

3 / 6 6




4 / 6 6




Research phase I

“Research is the basis of knowledge”

In the first “Gesture based interactions in the museums” project meeting some

important questions about the use of interactive interfaces in the museums

were made. The project group split the questions into the following categories:

technology, information structure, target group, gestures, attention,

group mode and added value. Our group summarized the questions

about gestures. Then the search for detailed information about dif-

ferent gesture types and existing gesture-based computer systems

followed. Here we provide the questions about gestures:

The first and very general question: Why using gestures?

We found different answers to that question:

Interacting with gestures is a natural and powerful way of communi-

cation, it´s the first thing that babies learn [2].

“Brainstorming” in the project room

5 / 6 6




Using gestures adds a value by: personal dislinking of keyboards, mice and other

transducing devices expand the range of possibilities of computer usage start

being physical again (natural interaction) [3].

Feedback – How can we make the user believe and understand that his

gestures are causing an action in the interface (the user has to get a

feedback about his actions)?

The representation of the user on the screen. Physical feedback about

his actions.

Example 1: Hiroshi Ishii in Tangible User Interfaces (“image: music

bottles”) uses physical output to user´s actions, like changing of col-

ors, sound and even form of the interactive device [4];

Example 2: While analysing the Wii interface (Nintendo Wii) we en-

countered some problems like the lack of calibration between user´s

movements and the movements of the cursor which left the visible

Hiroshi Ishii, tangible interface

“Music bottles” (1999)

Nintendo Wii interface

6 / 6 6




projection space by every hand movement; there is a positive aspect in the Wii

interface - the vibration of the control device when the virtual hand is touching

the surface of the buttons. This kind of feedback is very important for instance

in the “Virtual flights” learning systems where the tactile connection between the

pilot and the machine gets lost [5];

Conventions – What human gestures are universal and culture independent?

a) Gestures don’t have single meanings/are not monosemantic unambiguous se-

mantic meaning.

b) Gestures for elementary needs are same in different cultures (like threatening

with a finger).

c) Gestures are a culturally formed working process: the humans need for hapti-

cal movements – grabbing working tools. The hand as the most developed

working tool is involved in the creation process of virtual objects (for instance

the gesture of forming a hole in the sand) [5].

d) Gestures have cultural background (outgoing gestures in italian culture, ingo-

ing getures in japanese c ulture).

Some simple hand gestures

7 /66




e) Gestures developed by groups (ex.: football coach gestures).

f) Gestures can not only convey the content but be involved in moderat-

ing the process of interaction [6]

How to avoid errors of different meanings?

a) Create a clear and simple Norm-language

b) Accommodate the user´s inexperience and errors: using daily-life

gestures for the interaction with the interface can cause misunder-

standing (when the interaction gesture duplicates the normal-life

gesture) [6].

Which gesture fits to which navigation convention?

Use of the hand gesture for appropriate tasks: it is important to choose carefully

the tasks that gesture input is going to be used for. While gesture input is natural

for some navigation and direct manipulation tasks, it is inappropriate for tasks

that require precise interaction or manipulation. These tasks typically require

some form of physical contact. [6].

Using the Alterface interface “Salto” in the project-room at KISD

8 / 6 6




Example: using the interface of Alterface “Salto”, the finger is pointed to the

screen as a mouse pointer for moving the cursor, but when you have to select

something you tend to tense your arm and make short movements similar to

clicking the mouse and if you don’t get the feedback it becomes a negative expe-

rience.

Input types – are other input types than gestures (voice command, acustic sig-

nals, movements) possible?

a) Body movements. In the game “Collabola” the body movements are sent as

command to the computer [7]. A simple body movement can be interpreted as

a gesture too. We found an example, where a natural body movement can de-

termine the displaying of the information on the computer system. This is an

experimental work “Draussen” by Patrick Juchli (HyperWerk/fachhochschule

beider Basel) - video installation with sensoring the breathing of the user.

A frame from film “Draussen” by P.Juchli

“Collabola” game, Interaction

Design Institute Ivrea

9 / 6 6




“Draussen” installation is a reflective interaction between analog and digital

time. The author is building different narrative sequences in accordance to

human body movements (breathing) [8]

b) Distribution of weight: the system detects users declination – when his body

tends to one side or to the other

c) Simply approaching the system

d) Gaze detection. “Even rudimentary gaze recognition adds power to the inter-

face. As an example of this Bolt used the Media Room to design an interface

called “Gaze-Orchestrated Dynamic Windows”. This involved showing up to

thirty different moving images at once on the wall display, complete with a ca-

cophony of their soundtracks all combined into one. Gaze detection was then

used to change the relative size of each of the moving images. If the users

gaze was fixed on a particular channel all the soundtracks to the other chan-

nels were turned off. If they continued watching, the image would zoom to fill

the wall display. The net effect is that the computer filters out all but the in-

formation of immediate interest to the user, just as humans do when engaged

in conversation” [9].

Media Room by Richard Bolt, 1984

1 0 / 6 6




Intuitive and ergonomic – How to make the user feel confortable

by controlling the interface?

a) Using phidgets (physical widgets or phidgets are to physical user interfaces

what widgets are to graphical user interfaces. Similar to widgets, phidgets

abstract and package input and output devices [10]). Phidgets make the user

believe that his natural body movements influence the physical action in the

system (like pedals in the c ars controlled by electronic system)

b) Using simple actions based on common experience: “If something has a

handle, we grab it, if something has a base bigger than it’s apex – we should

keep the base downward [4].

c) Using natural and intuitive gestures. Using an interface is a subjective experi-

ence, a harmonic gesture, adapted to the psychology and physique of the user,

is accepted better than a forced innatural movement [11].

Example 1: Roller Controller - an input device for young children. Children

are having trouble looking at the screen while touching the control device.

The rotation movements on a big bal l are accepted better by children than

using a mouse-similar device [12].

Phidget RFID prototype

Roller controller – an input

device for children

1 1 / 6 6




Example 2 : Nintendo Wii

The cursor isn’t calibrated well with the screen-size and the gesture of the

user – it moves too fast in comparison to the input device movements. A

small movement of the hand holding the device and the cursor goes out of

the visible area of the screen.

Example 3 : QuarkXpress programme

By the movement of the mouse towards and backwards the

image is zoomed out and zoomed in – intuitive like move-

ment in the space [4].

Example 4: Video Wall and interactive carpet by F.Sparacino:

it´s large scale installation which uses two projection sur-

faces: one vertical and one horizontal. The horizontal sur-

face is a large map projected on the floor. People physically

walk onto different locations of the floor map and trigger

Movement of the cursor on the

Nintendo Wii game projection

1 2 / 6 6

QuarkXpress application-window




consequently the front surface projection to show the correspondent vi-

sual and auditory information. One can see the floor map projection like a

mouse pad, the person walking onto the map like the mouse driving the sys-

tem, and the vertical large projection screen as the computer display [13].

This system is intuitive because the user influences the image on the front

screen by walking like moving through the real-life space.

d) Not too complicated system. The aimed immersion of the user into the system

occur when the psycological connection with the medium is understood [14].

e) Avoiding tiredness/repetition. To many mouseclicks or repetetive gestures can

tire the user. On the other Hand, if the gesture is repeated to rarely, the user

can forget how to use the gesture.

f) Avoiding errors by setting the right duration. In critical situations the most

of the people tend to use familiar gesture patterns. Sometimes the slowing

down/deceleration of the gesture can be the scope of the project, because it

permits the user to remember better the command and its consequence. For

instance a quick mouse clicking can lead to “overclicking” important informa-

tion and warning messages.

F.Sparacino Video Wall and

Interactive Carpet, 1999

1 3 / 6 6




g) A meaningful connection of the gestures and the action. In one of the first

gesture based interface languages the gesture “Loslassen” is intuitive. The

other gestures require a learning process and are not intuitive and connected

to natural actions [15].

Is there a sense for calibration and training before starting using the interface?

If the interface attracts the user and its use is a longer period of time and in

many places of the exhibition it’s worth to make the training. But if the system

will be used only once and it’s usage is too complicated, the user will probably

leave. On the other hand the trained users will feel challenged by the complicated

systems and will feel satisfaction after the usage.

The Introduction can scare the visitor if he is not con-

vinced (“what will it bring to me”) – it´s the entrance/

door effect (A.Horbelt, Triad Berlin).

1 4 / 6 6

One of the first gesture-

based-languages for

communicating with

a computer

Andreas Horbelt (Triad, Berlin) talking about interactions in the museums




The list of analysed interactive interfaces:

• Hirosh Ishii Music Bottles (1999) [29]

• Video Wall and interactive carpet: MetaSpace by F. Sparacino (1999)

• Charade by Baudel&Beaudouin-Lafon (1993)

• Alterface “Salto” interface

• Realtime Visions interface

• Alive magic mirror by Pattie Maes (1995) - the first

interface with user projected inside the picture

• Media Room by Negroponte´ s Architectu re

Machine Group (1984)

• Collabola game (Ivrea Interaction Institute, 2004)

• Nintendo Wii (2006)

1 5 / 6 6

Exploring the interface provided by Realtime

Visions, project room at KISD




1 6 / 6 6




First proposals

Grabbing cubes (Ana)

The proposal “Grabbing the cubes” is based on the re-

search about natural human gestures. The user is sup-

posed to point at the “flying” menu points and then grab

with both hands the points he would like

to know more about and drag them into the

main area of the screen. Then the cube would

become larger and in one of the cube faces

would start a movie clip.

“Grabbing” the cube with both hands

Selecting a cube from the rotating menu

1 7 / 6 6




Rotating exhibition pictures (Ronny)

This proposal is about how the idle-mode of the interac-

tive system might look like and how the system could catch

the attention of the bypassing visitor. The process is di-

vided into three parts: idle mode, passive mode and active

mode. The system has to differenciate between 2 distances

a visitor might pass by. The distance of the proper track-

ing and recognition of user´s gestures is a short distance.

If a bigger distance separates the user from the interac-

tion device, the system is able to realize at least that there

is something moving from one position to another. The

long-distance-recognition is used for the passive-mode,

the short-distance for the (inter)active-mode. The action

of coming-closer, the in-between long- and short-distance

has also to get recognized.

IDLE MODE

some elements are circling around

invisible centre

1 PASSIVE MODE

visitors passing by are affecting the

foremost elements

2.1elements aim to face the visitor

2.2

elements revert to IDLE MODE after

visitor did pass by

3 PASSIVE MODE

same behaviour if visitor is passing

by from other direction ...

4 ... and revert to IDLE MODE5

ACTIVE MODE

orbit of elements gets minimized

orbit speed increases

6.1visitor in process of getting nearer

6.2visitor in optimum p osition:

elements get ordered

6.3

Storyboard “Rotating exhibition pictures”

1 8 / 6 6




Research phase II

In addition, a research about the classification of existing interactive systems

was made and the classification of interaction experiences by italian researcher

Flavia Sparacino:

Interactive experiences:

a) Scripted Systems (rigid interaction)

b) Responsive systems (a certain amount of options)

c) Behavioral systems (the response is not always the same)

d) Learning systems (the system can learn new b ehavior)

e) Perceptive systems (allow to simulate emotion) [16].

One of the first responsive systems,

ALIVE interface, 1995

1 9 / 6 6




By analising this taxonomy we realised that interaction systems like the one pro-

vided by Alterface could be described as Responsive systems (it gives a certain

amount of options to the visitor and the public after trying all the possible op-tions is often not attracted back to the piece). So the critical point in such a sys-

tem is to be able to engage the visitor to the experience. Therefore to engage the

visitor we have to concentrate on elaborating an original and unexpected content

presentation style and conveying the visitor through the system by using drama-

turgy, drawing his attention away from the technological limitations.

Behavioral interactive system “Rea”

by Cassell, 1999

20 / 66




Research phase III: Dramaturgy

2 1 / 6 6

Gustav Freytags analysis

After the presentation of available technologies (Alterface and Realtime visions)

and trying them out some usability p roblems were encountered. The user is con-

centrated too much on “struggling” with the interface and could loose the con-

nection to the value of the exhibition. The focus switched to the construction of

dramaturgy which would involve the user and make the

interaction more natural.

The dramatist and novelist Gustav Freytag [17] is well

known for his analysis of Greek and Shakespearean

drama. He abstracted Aristotle’s drama-structure

further and adapted it to some more modern aspects

of storytelling. Where Aristotle named the vertices,

in the so called Freytag’s Pyramid there are also two

additional names for the meta-levels of drama – the

raising and the falling action – alongside a more meaningful nomenclature of the

vertices. Another approach that is opposite to Aristoteles structure is Bertolt

Brecht s Epic Theater concept. “It was this conception that Brecht abhorred,

Exposition

Climax

Rising Action Falling Action

Denouement

Gustav Freytags Pyramide




and that he called, knowing that Goethe and Schiller had based their theory on

Aristotle’s Poetics, the Aristotelian concept of drama, the drama of catharsis by

terror and pity, the drama of spectator identification with the actors, the dramaof illusion, which tries to create magical effects by conjuring up events which

are represented as totally present, while palpably they are not. Such a theater

therefore was a fraud. Brecht, the rationalist demanded a theater of critical

thoughtfulness, an epic theater” [18].

The research about different dramaturgies revealed, that the epic theater by

Bertolt Brecht was closest to the spanish classic theater tradition in it´s didactic

character. The so called “World-theater” - typical for the spanish theater school

has didactic character in the representation of a thrustful relation between God

and mankind. “Do it right, God is above you” - in this phrase from the theater we

see the non-acceptance of misbehaviour and the ability to change things: “God

gave the man the possibility of making his life better” [19]. The mentioned state-

ment is very close to the Brecht´s theater concept of

non-acceptance of the reality.

2 2 / 6 6




Recording the speech of B. Micka (GPD), where he told that the aim of the Exhi-

bition “Memory of Andalusia” is the didactic effect on the visitors, the decision

of working on dramaturgy was made. The dramaturgy for this exhibition shouldleave out the pure technology-demostrating immersive effects and concentrate

on the way of introducing the content. Nowadays the most of the exhibitions

follow the Gesamtkunstwerk concept (typical to the american theater and cin-

ema school too) where the actors create the illusion of immersion and make the

visitors identify with the characters represented by ac tors. The visitors are im-

mersed in this created world, they experience trance, magic, narcotic and hyp-

notic effects and they accept what is happ ening on the stage - “life is like that”.

But they are not forced to think. Whereas in the classic theater and the epic

theater the alienation is created by wearing masks, keeping distance between the

actor and his character and therefore between the actor and the audience. T he

audience doesn´t accept, what is happening on the stage, it´s vision is irritated

by contradictory actions of the ac tors, sudden alterations of the scenes, involv-

ing of unexpected characters, this way the barrier to empathy is created. “Life

doesn´t have to be like that, there are options”. This theatrical approach has a

2 3 / 6 6




Gestures in the theater

The research about gestures in the theater science followed. We analysed the

french theater scientist Francois Delsarte work about gestures [20]. One of the

most characteristic Delsarte s statements is, that the gesture of the actor can

convey personality and emotions. T his opinion is opposed to Stanislavski´s meth-

od [21] used in the Soviet Union and American theater schools, which teaches to

imagine the characters emotions and feelings that will automatically convey to

the right gestures. According to Delsarte´s, there are three types of gestures:

a) individual gestures

b) cultural gestures

c) emotional reactions

Delsarte also describes the attitudes of the hand by talking about an imaginary

cube: he divides the hand gestures into 7 types. For instance, one of them is the

“support” gesture when the palm of the hand is on the bottom face of the invis-

ible cube [22]. This classification of the gestures could be really useful on small-

2 5 / 6 6




scale-interfaces, but not suitable to the “Andalusia Memory” exhibition interface

because of the technological limits. The tracking of the hand movement, position

and rotation requires a very sophisticated technology and the imprecise inter-pretation of hand movements can cause misunderstanding and delusion. Al-

though a very precise hand-movements interpretation is possible with so-called

“cyber-gloves”, but in the environment of a museum a distribution of such expen-

sive devices to every visitor is unconvenient.

Gestus

The next step in d eveloping our concept was finding out more about the tech-

niques of Bertolt Brecht´s Epic Theater [18]. The gesture or “Gestus” has a very

important role in it. T his term expresses all basic human at titudes, not merely

gesture, but all signs of social relations: department, intonation, facial expres-

sion. The gestic language of Brecht is characterised by pause, parallelism, coun-

terpointing. There is a basic attitude in all scenes, for instance despair, defiance,

submission.

2 6 / 6 6




The Gestus is a central basic action that repeats in all scenes. All scenes are

rounded and complete themselves, but there is the main attitude that connects

them.

Analysis of an example of Epic Theater technique:

“Baal” (B. Brecht)

The analysis of

the film “Baal” by

B.Brecht [23] rein-

forces the knowledge

of his cinematic tech-

niques that could be

applied to the exhibition environment too. For sure an exhibition is not a cinema

nor a theater piece, it differs by it´s inconstant character like the movement of

the visitors through the space, no boundaries like tickets or start-time. Therefore

the timing of the piece in the interactive system should consider this absence of

27 / 6 6




Construction of the concept on the base of research

“Combination of understanding people and telling stories rather than understand-

ing technology and building devices” (David Kelley)

On the base of our research we realised that the movement of the body could

be interpreted as a gesture if it transmits information to the system and can be

interpreted by the system.

The gesture influences the state of mind of the actor according to F.Delsarte

[20], so it´s very important to choose the right gesture to convey the user to the

right psycological reaction. The reduction of the usage of gestures to a minimum

is motivated by the current technology p ossibilities, that don´t allow a perfect

interpretation of every human gesture like in the interpersonal communication.

Therefore the following decision was made: to concentrate on the development

of a dramaturgy, which will involve the visitor of the exhibition, instead of using

complex interaction methods.

2 9 / 6 6




The chosen dramaturgy is based on the Epic Theater concept by B.Brecht be-

cause of it´s high didactic effect and effective narrative approach.

We will adapt the following methods from the Epic Theater concept:

a) create distance between visitor and the system;

b) alienation effect experienced by user through clear separation present-past

time;

c) connection to the content by storyfying it and choosing common and known

topics, focus on social relations, human value through time;

d) creation of contradiction by unexpected past-present scenes alteration (non-

linear narrative structure) - the user has to be slightly irritated and forced to

think;

e) “boil down” to one essence the interaction, not making people playing too

much with the system.

30 / 6 6




It is possible to transfer this cinematic and theatric techniques to an exhibition

concept, by keeping the duration in mind. Cinematique techniques used in anexhibition environment have to be much shorter in order to not conflict with the

expectations of the visitor. At the end, this techniques are nothing more or less

than tools to formulate the content, to create a narrative structure.

3 1 / 6 6




3 2 / 6 6




In order to combine the two quite contradictory functions: to alienate the user

from the system and to connect him to the content, we decided to use in additionto the front screen a bottom screen which is distanced from the front screen.

The initial idea was to separate these two information layers - the one shown

on the front screen (is referred to the past) and the one shown on the bottom

screen (which is the connection between the past and the p resent). The visitor

cannot affect directly the reality of the front screen - he´s not allowed

to, because it is impossible to change the past. Therefore he can find

the connection to the past events through metaphors - objects ap pear-

ing on the bottom screen, which symbolise the left-overs from the past,

the memories which he can affect by stepping on them and they will

guide the spectator through time.

Second proposal: front screen + bottom screen

3 3 / 6 6

First front- and bottom-screen proposal




Constructing a story

As mentioned above the exhibition environment is not the same like cinema or

theater environment. But the presentaton of the information in these differentenvironments is based on the construction of a narrative structure, and ap-

plying of a dramaturgy. “People experience their lives as a narrative. Amongst

cognitive psychologists, Jerome Bruner stressed the importance of story, as

the means which structures our perception and communication. He reminded

us that thinking cannot be reduced to mere information processing and sorting

into categories and that narrative is our main instrument of making-meaning,

the embodiment of culture, communication, and education. The history of archi-

tecture offers innumerable examples of places which embed and narrate a story

through their spatial layout and décor. By looking at the sequence of floor plans

of historical buildings through the centuries, from the Greek temple, to the Ro-

man church, the medieval dome through today, we understand how a rectangle,

a circle, a cross, or other more complex figures, transmit a message through the

3 4 / 6 6




centuries. This message is a story about how people through times relate to life,

nature, and spirituality.” [24].

The concept “Step to understanding” is based on a dramaturgic approach, there-

fore the content is the main actor and a constructed example story can help to

test the visitors reaction.

The first attempt was the creation of a moodboard based on our general knowl-

edge about Andalusian history and creating a sketch of a story which would be

based on human relationships and values seen through time. In this sketch our

intention was to show how the alteration of past and present events connected

by one topic would work.

Step to understanding

The basic gesture is a step on a screen that is embedded into the floor. The

screen has to be able to recognize feet-movements on it’s surface. Therefore

just an easy change in position means interaction. As mentioned above the at-

tempt to “boil down” the possible gestures to one simple and strong gesture or

“Gestus” allows the user to concentrate maximally on the narration. The ges-

3 5 / 6 6




ture of stepping is one of the most simple and intuitive gestures, that does not

require a big concentration. According to the example mentioned in the chapter

“Research phase I” - Video wall and interactive carpet by F.Sparacino, the sys-

tem created by this scientist encountered positive results in terms of ergonomy

and intuitive exploration of the narrative space. Of course the switching of the

attention from the bottom to the front screen and viceversa has to be taken into

concideration. This swoping of attention seems to be natural, because in the

everyday life by walking we gaze to the ground time by time. Nonetheless, to

be sure of the functionality of this system a series of experiments and testing

should be made in the future.

On the second stage we designed a proposal based on 2 screens approach, these

are the main points describing it:

3 6 / 6 6




Organisation of the space

2 screens in every corner of the exhibition hall. The 4 screen-areas in the exhi-

bition hall are referred to 4 themes or filters: Fire, Water, Sand and Air. The fire

area represents the more dramatic point of view – crucial points in the A ndalu-

sian History like wars, revolutions, catastrophes, emigrations, political changes

etc. Water area represents changes in the social structure. Air refers to religion

questions. Sand is about agriculture, industry, economy issues.

Interaction

• The visitor is not selecting options, he is made believe that he can decide, but

he is guided by the dramaturgy

• The story is transferring him into a mood of “i-wish-to-understand”, by: alien-

ation, confusion, objection

3 7 / 6 6




• In the beginning the visitor is tought by the system which movements he can

use. He sees the footprints appear in front of him - that gives him the idea to

step on them.

• By stepping onto another screen-area, the visitor performs the “step to un-

derstanding”

• The user is set in the mental situation of letting him “touch” the past by his

mind through understanding

• If more than one person is standing on the bottom screen, the first step per-

formed is recognised.

• So if one user leaves the screen, another can continue without interrupting

the story

3 8 / 6 6




Information structure

• Information has to be “storyfied”

• Story has to be split up into scenes with a past-present-alteration

• Unidimensional structure where interaction is just confirmation on one hand

and a non-linear story on the other hand.

Creating alienation by contradiction

• past-present-transition

• sound- & talk-contradiction (sound and projection is used for time-transitions

or time-shifts, to create feelings of alienation, confusion or objection)

• bottom-front-screen-contradiction

3 9 / 6 6




Story-example

1) A bunch of sand on the bottom screen

2) Step onto it: see the front screen change to display bare feet walking on

sand

3) Hear high heels walking on concrete

4) Suddenly the scene is changing: showing a woman on high heels meeting her

friends on the street

5) People are talking to each other – getting more and more amused

6) Suddenly someone begins to talk about a special topic; the group is entering

a restaurant: camera shows them passing the doors

40 / 66




7) After a while they are all transferred into the past, talking about the same

topic, that also fits into that time

8) The group is sitting down to a table

9) The scene suddenly switches ones more to the present, but the sounds and

talks stay the same

10) The topic of talking is changing to religion, camera rotates to show the floor

between the tables

11) A muslim in an archaic vesture is going along, kneeling down from time to

time

12) As the camera completes its rotation the people are no longer sitting on the

table, but on the floor, they are religion-students in an old musque

4 1 / 6 6




13) The camera is once more moving across. A pylon is visible. After passing

the pylon we are back with the people sitting around the table

14) They begin to eat and one can hear the sound of spoons and knifes on por-

celain

15) The camera is moving from showing the whole scene to only showing some

part of the table, while sounds go on

16) Suddenly the surface of the wooden table changes being aged and looking

used now. We hear the same sounds like before

17) The camera is moving up and shows woman that is washing the dishes in a

tub: she is making those sounds

4 2 / 6 6




Andalusian history

After the presentation of this concept the research about real facts from Anda-lusian history was made in order to concentrate on the content itself. Taking into

consideration the main p eriods of Andalusian history, the most influent on the

formation of the Andalusian culture and religion is the Arabian invasion or Emir-

ate Al-Andalus and later the Caliphate of Córdoba. That was the time that affect-

ed very much the development of andalusian society, art, culture, religion. The

geographic position of Spain was of a big importance in the historical changes,

so as it plays an important role nowadays, when the immigration rates increase.

Some guidelines that connect that period with the modern spanish society were

created with a particular focus on one of the topics proposed by GPD - the An-

dalusian Art and Culture. In following the extract from Andalusian History de-

scription in short is provided:

“The independent Emirate Al-Andalus and later the Caliphate of Córdoba (chal-

lenging Bagdad) mark the blooming time of the Omayyade d ynasty and thus the

Arab culture in Andalusia. Córdoba becomes the center and melting pot of dif-

4 3 / 6 6




ferent cultures and religions. Trade, science, handicraft and arts experience a

long succeedful period. Starting from 1031 the Caliphate disintegrates into small

Islamic reigns. Almoravids and Almohads (Berber) alternate their rulership until

the 13th Century. After the fall of Córdoba (1236) and Seville (1248) the Nasrid

Kings maintain their kingdom in Granada for two and a half centuries more. The

last Moor king Boabdil hands over the city of Granada to the Catholic Monarchs

Isabel and Fernando in January 1492 and moves to the

Alpujarra mountains.” [25].

Many aspects of Andalusian society life were affected during the Arabian period:

the scripture (even an mixed Arabian-spanish scripture was invented by Mude-

jares (Mauren); All the wood-carving works were made by Arabian woodcarvers.

In the architecture the Arabian architectural elements influenced the Spanish

architects. Folk songs and dressing manners changed too [26], [27].

4 4 / 6 6




The story in the concept “Step to understanding” is based upon the following

topics: crossing the borders, transferring to another country as a metaphor of

big changes in Andalusian History, religion and culture. The story of transfer-

ring, of migration is talled, the integration in a strange culture of a symbolic

person – a man from Morocco, who could b e a symbol of conquest process in

medieval times and migration nowadays. The visitor is invited to think how these

simple situations influenced the everyday life of the w hole country. The specta-

tor is connected to the story, because the situation seems to be familiar to him

– he knows about the migration problems from the media. But the comparison of

this modern process to the conquest process in Arabian times seems unexpect-

ed, new and contradictory. This comparison makes him ask questions and think

why we confront these scenes from different times. Moving forward through the

story more and more elements show our intention – the metaphoric objects, that

show connections between Arabian and Spanish culture appear, like a box found

in an Andalusian church made by and Arabian woodcarver, or some Arabian

scriptures on a house wall in a Spanish village.

4 5 / 6 6




serves to understand better the history and to find a connection to it. In this

case the visitor of the exhibition is like a system that is preserved of elab orat-

ing the information from scratch because it is provided with some facts linked

to something already existing in his memory. This linkage to his own experience

(like the facts about imigration in contemporary Spain) helps to

involve the visitor.

4 7 / 6 6




Finalisation of the concept

The concept “Step to understanding” gives the visitor few possibilities to interact

and aimes to guide him through this process by dramaturgic narration. The group

mode interaction has to be concidered too (when the other visitors approach in

the middle of the process they don´t know what is the story about). The doubts

about visitors attention during the whole storytelling p rocess leaded to the idea

of “Self Compiled Plot”.

4 8 / 6 6




Connection

The concept “Self compiled plot” is tailored to the structure of the museum, soevery quarter has its own meaning based upon the blueprints of GPD. The kind

of information our concept is about is best described as “hu man value”. This

means the state of mind of a time. Immaterial values like emotions and faith pl ay

a decisive role. The awakening of the collective memory is the aim of the con-

cept.

BenefitsThe etimologic meaning of a gesture is a movement which could be interpreted

as meaningful to the receiver of the information (the system) and the surround-

ing persons. Therefore every movement of a human b ody can be interpreted as a

gesture (like the breathing gesture in the project “D raussen”[8]). The non-solv-

able ergonomic- and simplicity-problems experienced with the provided systems

would reduce the extent of the target group to technical enthusiasts. Therefore

the decision of designing an extended interface with more than one front screen

came to daylight. Stepping is a much more natural gesture. Everyone is used to

Self compiled plot – Step to understanding The final concept

49 / 6 6




it, because everyone moves through space. Additionally an electronic system can

recognize this gesture much more reliably than mentioned systems. According

to F. Delsarte the gesture can bring the actor into a certain emotional state [20].

Therefore the gesture of stepping is a self-explaining metaphor for stepping to

the past.

The main aspect in the concept “Self compiled plot” is storytelling. Especially

Brecht s Epic theater on one hand and the likewise didactical approaches of the

historical Spanish theater on the other hand. To draw visitors attention with a

meaningful story, you need dramaturgy, you need to force the attention of thevisitor. The perception has to be leaded to the intrinsic value: the content. It

shouldn´t even accidentally be threaded by some more or less usefull possibil-

ity to choose. By analysing different ways of storytelling, we experienced, that

especially the controversial and contradictory nature of the Epic theater cre-

ates the best setting to open the perception of the visitor, draw his attention and

make him curious.

50 / 66




Idle mode

When no visitor is using the system, it will display a rotating wheel of stills. Ran-domly one of the stills will be zoomed to fullscreen, playing a half or one second

and zoom back to their position. When a visitor appears in front of the screen

this wheel scales down to

the lower right corner and

changes functionality to

that of a progress-indicator

that displays the current

position in the “framework

plot”.

5 1 / 6 6




Moments of choice

With it’s non-linear narrative structure the concept spreads possibilities tochoose across the timeline. By interacting through stepping on objects on the

bottom screen the visitor can add content plots to the framework plot and there-

fore compiles it’s own content. Multiple users can take part in this interaction.

Everyone can

participate and

help to enrich the

framework plot

with informative

content.

5 3 / 6 6




Content Plot

In difference to the framework-plot that is designed to create curiosity andinterest, the content plots are of didactical nature. it also differs from the frame-

work-plot in it’s stylistic appearence. while the pictures of the framework-plot

are blurred, black and white and

metaphoric, the content plots are

colored, clear and informative.

the more content plots are added

to the framework plot the clearer

the framework plot gets. the blur-

rines is removed step by step and

the user gets more conscious

about the meaning of the story

5 4 / 6 6




Contet plot – storyboard

5 5 / 6 6

frame 1metaphoric object, thatrepresents andalusianculture

frame 2symbolic scene stands for important populationtransitional moment

frame 3imigration scene from thepresent

frame 4farewell with the house

frame 5metaphor for integrationin a new country




5 6 / 6 6

frame 6arabian conquistors goingto andalusia

frame 7newcomers stay on the way, problems to face

frame 8remembering religion to hold on in difficult mo-ments

frame 9a content plot about merg-ing of religions is added

frame 10metaphor of giving thepower to the conquistores




The interface „self compiled plot – step to understanding“ should be tested in

the future in order to understand how it works in an environment with exhibi-

tion visitors and how it affects the visitor. Therefore two projectors connected

to two computers are required. One of them is projecting on the wall in front of

the visitor, and the other one is projecting on the floor. A technology similar to

Alterface interface “Salto” should be ap plied in this case to track visitor´s move-

ments on the floor and to enable the interaction with the objects represented on

the bottom projection.

Testing facility

5 7 / 6 6



research phase three second proposals f inal concept test ing faci l i ty conclusion

5 8 / 6 6

Outline-drawing

of the testing

facility




Looking back to the workflow process – initial questions, research about ges-

tures, dramaturgy, Andalusian history, available technology, working on the

concept, on the content of the exhibition – there is an impression of a really

deep and accurate approach to the topic. We tried to approach the designing

of the interaction system from the side of the content, dramaturgy, giving less

importance to the development of the gestures because of the limitations of the

provided technology. We found the solution of a more natural and common used

gesture – walking, stepping on the ground. And the division of the visual area

into 2 levels – front and bottom screen – was aimed to divide clearly the inter-action and the information area, so the visitor has a clear understanding that he

can interact on the bottom screen and the result of his interaction is visible on

the front screen.

Conclusion

59 / 6 6



research phase three second proposals f inal concept test ing faci l i ty conclusion

On one hand we are all used to the visualisation of the interaction on the same

screen where we see the information (like navigation on the website), although

the input activity is performed on the keyboard and mouse. On the other hand,

affecting directly the area of vision (like touchscreen) is something we are

getting used too. The question is, in which case the direct manipulation of the

objects on the screen is better than the manipulation through additional devices?

Maybe in some cases the clear division between the information area and the

input area helps to switch the attention and keep pureness and clearness of the

information on the main projection? At least that is what we believe and that isthe motivation why we devided the information between the main screen and the

additional screen, so the symbolic meaning of the pure and untouchable, distant

and mysterious image of the past can be kept.

6 0 / 6 6



Movies used for constructing the example framework plot

1) Iberia Dance by Carlos Saura

2) Lawrence of Arabia by David Lean

3) Nowhere in Africa by Caroline Link

4) Return to Kandahar by Paul Jay and Nelofer Pazira

5) The Sheltering Sky by Bernardo Bertolucci

6) Casablanca by Alfred Hitchcock

Soundtrack from: “Iberia Dance”

6 1 / 6 6



References:

[1] Flavia Sparacino, Narrative Spaces http://xenia.media.mit.edu/~flavia/Papers/

NarrativeSpaces.pdf, p.10

[2] Recognizing gestures: Interface design beyond point and click http://www.

w3c.org/TR/1999/REC-html401-19991224/loose.dtd

[3] M. Billinghurst, Gesture Based Interaction, www.billbuxton.com/input14.

Gesture.pdf

[4] B. Moggridge, Designing Interactions

[5] H. Sturm, Geste und Gewissen im Design, Dumont,1998, p.81

[6] M. Billinghurst, Gesture Based Interaction, p.23

[7] Interaction Design Institute Ivrea, Collabola project, http://www.interaction-

ivrea.it/en/gallery/collabolla/index.asp

[8] P. Juchli, “Draussen”, http:/ /www.fhnw.ch/hgk/ ihw/projekte/de/projekte/

thema/metaworx/diplom-03-metaworx

[9] M. Billinghurst, Gesture Based Interaction, p.30

6 2 / 6 6



[10] Phidgets: Easy Development of Physical Interfaces through Physical Widgets,

Saul Greenberg and Chester Fitchett,Department of Computer Science

University of Calgary, http://grouplab.cpsc.ucalgary.ca/papers/2001/01-

Phidgets.UIST/phidgets-uist-2001.pdf

[11] H. Sturm, Geste und Gewissen im Design, p.79

[12] B. Moggridge, Designing Interactions, p.293



[14] H. Sturm, Geste und Gewissen im Design, p.82

[15] H. Sturm, Geste und Gewissen im Design, p.80[16] Flavia Sparacino, Media Actors, http://alumni.media.mit.edu/~flavia/

publications.html, p.3

[17] http://www.uni-essen.de/literaturwissenschaft-aktiv/Vorlesungen/dramatik/

freytagdra.htm

[18] Bertolt Brecht and Epic Theater: http://faculty-staff.ou.edu/L/A-Robert.

R.Lauer-1/Brecht.html

6 3 / 6 6



[19] Heinz Kindermann, Theatergeschichte Europas I Band, Spanisches Theater,

Otto Müller Verlag, Salzburg, 1959, p.339-351

[20] The Delsarte Method: 3 Frontiers of Actor Training, E. T. Kirby, http://links.

jstor.org/sici

[21] Konstantin Stanislavski and method acting, http://www.bbc.co.uk/dna/h2g2/

A5133151

[22] Stacy S. Marcella , An Exploration of Delsarte´s structural acting system,

http://www.isi.edu/~marsella/Delsarte_CameraReady.pdf

[23] http://www.deutsches-filmhaus.de/filme_einzeln/sch_einzeln/schloendorff/

baal.htm



[25] History of Andalusia: http://www.andalusia-web.com/index.htm

[26] K. Jörg Ruhl, Spanische und portugiesische Berichte zum Nachschlagen 1986

Verlag Plötz Freiburg p.44

6 4 / 6 6



[27] Wilhelm Giese, die Kultur Spaniens, Portugals und Iberoamericas, Frankfurt

am Main, 1972, p.62

[28] Total Interaction, edited by G. M. Buurman, Birkhäuser, p.303

[29] Hiroshi Ishii Music Bottles http://tangible.media.mit.edu/projects/

musicbottles/

6 5 / 6 6



s e l f c o m p i l e d p l o t – s t e p t o u n d e r s t a n di n g

g e s t u r e b a s e d i n t e r a c t i o n s i n m u s e u m s

a n a r i n k – r o n n y r e i c h m a n n

p r o f . p . h e i d k a m p

w s 0 7 / 0 8

k i s d

Self Compiled Plot - Step to Understanding - Ana Rink - Ronny Reichmann

Documents

Transcript of Self Compiled Plot - Step to Understanding - Ana Rink - Ronny Reichmann