Interaction

2

Contents

Understanding & Conceptualizing Interaction

Interaction models Interaction styles Interaction paradigms

3

Understanding & Conceptualizing Interaction It is tempting to design an interface

by going directly to the nuts & bolts Whether to use touch screen GUI details Whether speech should be used

While these decisions are necessary, they should be done after understanding the problem space

4

Understanding the Problem Space Before designing a solution, you need to

understand the problem What is the problem with the current

interface? What do the users find difficult? What is the functionality which the users

want? Do all users have the same requirements? Are all users willing to put time into

understanding the problem?

5

Example: Understanding Browser Deficiencies A major web browser has been

losing market share to competitors The development team meets to

discuss the problem After asking users, it is found that

many have never used bookmarks They zero in on the bookmarks as

a potential problem area

6

Example: Understanding Browser Deficiencies There are various ideas to explain the

problem Organizing bookmarks requires dragging URLs

to folders which is tedious and error-prone Another member agrees with this assessment

based on feedback A software engineer, who scrolls through

history rather than using bookmarks, assumes others do the same. He recommends that the history facility be removed due to it being overly complex and unused.

7

Example: Understanding Browser Deficiencies The result of this discussion is:

They have various theories as to what the problem is

They still do not understand the real problem

They must go back, ask more users for feedback, and think about the problem for a longer time

You cannot create a solution until you understand the problem !

8

Conceptualizing the Design Space

“A conceptual model is a high-level description of how a system is organized and operates”

Johnson & Henderson

The conceptual model is how the user will see and understand the system

9

Conceptualizing the Design Space

“The most important thing to design is the user’s conceptual model. Everything else should be subordinated to making the model clear, obvious, and substantial. This is almost exactly the opposite of how most software is designed”

Liddle

10

The Conceptual Model

Outlines what people can do with a product

Presents the concepts necessary to interact with the product

The metaphors and analogies used to convey an understanding of the product to the user

11

The Conceptual Model

The model Gives the user an understanding of

how to use the product Forms a shared vision between all of

the designers Should be designed after the problem

space is understood but before the rest of the solution is designed

12

Contents

Understanding & Conceptualizing Interaction

Interaction models Interaction styles Interaction paradigms

13

Interaction Models

Interaction models can help us understand the interaction between the user and the system

Terminology Domain

Important concepts about the problem space

In graphic design these would be shapes, drawing tools and a drawing surface

14

Interaction Models Tasks

Operations to manipulate the concepts in the domain

Goals The desired result of a task

Intention A specific action designed to achieve a goal

Task analysis The identification of the problem space in terms

of domain, tasks, goals and intentions

15

Interaction Models Core language

A language used to describe computational aspects of the state of the system

Task language Describes psychological aspects of the

domain relevant to the state of the user

16

The Execution-evaluation Cycle This is also called Norman’s model

of interaction It is probably the most influential

model It is broken into two major phases

Execution Evaluation

These are further broken into a total of 7 stages

17

The Execution-evaluation Cycle

1. Establishing the goal2. Forming the intention3. Specifying the action sequence4. Executing the action5. Perceiving the system state6. Interpreting the system state7. Evaluating the system state with

respect to the goals and intentions

18

The Execution-evaluation Cycle Each of these stages is performed

by the user The user determines a goal, figures

out how to achieve it, performs the action, and then looks at the system to see if it worked as expected

Norman uses this model to explain why users have problems with interfaces

19

The Gulf of Execution Remember that the system operations are

defined in terms of the core language and the user describes actions in terms of the task language

Since these languages are not the same, it can result in misunderstandings

This means that there is a gulf between the users actions to achieve a goal and the systems interpretation of those actions

20

The Gulf of Evaluation The gulf of evaluation is the distance

between the actual state of the system and the state expected by the user

The more difficulty the user has interpreting the system state, the less effective the interaction

Norman’s model provides a simple framework for the understanding of interaction

21

The Interaction Framework Norman’s model is for

understanding the interaction It does nothing to consider the

system’s side of the communication This is addressed by an extension

to Norman’s model by Abowd and Beale

This extension is the interaction framework

22

The Interaction Framework This breaks the system into 4 major

components The system The user The input The output

It uses the core language and the task language as well as languages for both input and output

23

The Interaction Framework There are 4 steps in the

interaction cycle1. The user formulates a goal

and a task to achieve it. This is expressed in the input language.

2. Input languages is translated into core language

3. System transforms itself into a new state

4. The state of the system is then rendered in the output language and sent to the user

S

O

I

Ucore

output

input

task

24

The Interaction Framework There are 4 translations

involved in the interaction1. The users task must be

articulated in the input language

2. The input language is then translated into stimuli for the system

3. Once the system state is updated, evaluation begins with the presentation of results in the output language

4. Finally, observation translates the output language into understanding by the user

S

O

I

Ucore

output

input

task

presentation observation

performancearticulation

25

Articulation Poor articulation

A user wants to turn on lights at one end of a room

The user is confronted with a bank of unlabeled switches

The user has no idea of which switch controls the lights at the end of the room

The problem is that the user cannot articulate the goal (turn on lights at one end) in the language provided (unlabeled switches)

26

Articulation Good articulation

Virtual reality systems use data gloves This allows every motion of the user’s

hand to be translated directly into operations to be performed by the system

This is an example of a direct manipulation interface

It provides an excellent translation from the task language to the core language

27

Output Consider writing a paper with a word

processor You must be able to see the effects of

your actions Often, you can see only the immediate

paragraph without the surrounding text where there might be changes as well

This means that all of the state changes cannot be be displayed in the output language

28

Using the Interaction Framework

The interaction framework can be used to judge the usability of the interface

By concentrating on translations, it allows us to determine if concepts are being communicated correctly

This might be used as the basis for choosing one tool over another

29

Contents

Understanding & Conceptualizing Interaction

Interaction models Interaction styles Interaction paradigms

30

Interaction Styles The choice of interaction style has a profound

impact on the nature of the interaction We will look at

Command line interface Menus Natural language Question/answer dialog Forms WIMP Point and click Three-dimensional interfaces

31

Command Line Interfaces

The earliest interfaces were command line

Despite the availability of more modern interfaces, they continue to be used

Command line interfaces Provide abbreviations and function

keys to speed use

32

Command Line Interfaces Provide direct access to commands

without having to navigate menus Allow options to be used to modify

command behaviour Are very powerful Have a steep learning curve before

they can be used effectively This can be lessened by using

meaningful command names

33

Menus These display the available choices The user can select one by mouse,

numeric entry, or arrow keys Larger menus are organized

hierarchically so not all of the menu is displayed at once

This makes navigation more difficult since the user cannot see where they want to go

34

Menus

The advantage of menus is that they lessen the load on the user’s memory since commands no longer have to be remembered

This still means that the commands must be meaningful and easy to understand

35

Natural Language On the surface it is attractive to allow

instructions in a natural language like English

Problems are soon encountered due to the ambiguity of natural language The boy hit the dog with the stick

Did the boy use a stick to hit a dog or did the boy hit a dog carrying a stick?

36

Natural Language

There is also ambiguity in the meaning of individual words as well as the sentences

Although research continues into natural language understanding, it remains problematic for the near term

37

Question/answer Dialogs The user is asked a series of questions

and provided with a series of answers from which to choose

These are easy for novices to use They restrict queries to those supported

by the system General purpose query languages

provide more flexibility but are more difficult to learn and use

38

Forms This is used primarily in data entry and

occasionally in data retrieval The user is presented with a form that

looks like a familiar paper form Advanced forms allow the user to leave

fields blank and to fill in the fields in different orders

A spreadsheet is a special type of form in which the results of changes can be seen immediately

39

WIMP Interface This is the familiar windowing system

consisting of Windows, Icons, Menus and Pointers

This has become a standard interface supported by many computer systems including Microsoft Windows X Windows Apple Macintosh

40

WIMP Interface

Windows Areas of the screen where text and or

graphics can be rendered Icons

Small pictures representing concepts or actions

Used for buttons, labels, and representing windows in collapsed form

41

WIMP Interface Pointers

Pointing devices such as mice, touchpads, trackballs, tablets, etc.

These are essential to WIMP interfaces The pointer is represented on screen by

a cursor which can take on a variety of shapes

Often the shape of the cursor is used to indicate state information about the system

42

WIMP Interface Menus

These allow the user to issue commands by selecting from menus

There are pull-down menus from a menu-bar as well as pop-up menus

They all reduce the load on the user’s memory

There are also menus which are circular as well as ones which pick up gestures by the user with the pointing device

43

WIMP Interface Buttons

Buttons are small windows which can be clicked to perform an action

Buttons can be labeled with text, graphics or both

Buttons can change appearance to indicate their state

Radio and checkboxes are buttons with a slightly different behaviour and appearance

44

WIMP Interface Toolbars

This is an area of the screen where buttons and icons are collected

It acts similarly to a menu by allowing the user to select from many commands

It often makes selections more obvious by showing them all at once

Often, the user can customize their display by showing a selection of multiple toolbars which are available

45

WIMP Interface Palettes

A palette is a collection of controls which can alter the state of the system

They usually select values for various options

They provide immediate feedback on the state of these options

Graphics systems often use palettes to se4lect the drawing color, line thickness, etc.

46

WIMP Interface Dialog boxes

A dialog box is a window which conducts an interaction with the user to

Provide information Notify the user of an error Solicit input from the user

Often, these dialogs are modal so that the user must interact with them before continuing

System modal locks the entire windowing system and are usually a bad idea as the user cannot interact with other programs

47

Contents

Understanding & Conceptualizing Interaction

Interaction models Interaction styles Interaction paradigms

48

Paradigms for Interaction

This section will look at the key technologies which have contributed to interactivity

This will be partially historical in nature so that you can see how it evolved over time

49

Time Sharing Initially, computers were batch systems

used by a single user In the 1960s advances in computing

power made time sharing possible This allowed a single computer to be

used by several individuals They were able to interact with it via

teletypewriters Many people at the time were excited

by the possibilities

50

Video Display Units The teletypewriter gave way to the

video display unit Ivan Sutherland invented Sketchpad, a

program which allowed you to draw on the screen and have it stored in memory

This formed the basis for all modern windowing systems and established VDUs and the main display technology

51

Programming Toolkits Douglas Engelbart had the idea that the

computer should be used to augment human capabilities, not just for data processing

He created systems which were forerunners of modern word processors

He advocated the creation of software toolkits which could be used to easily build ever more complex software

52

Personal Computing The 1970s saw computing power

offered to the masses The LOGO system was designed to allow

children to draw simple graphics using a turtle which draw with its tail

Alan Kay at Xerox Parc had a vision of the Dynabook, a handheld personal computer which is more advanced than what we have today

The tablet PC is the closest thing to a Dynabook

53

Windowing Systems The success of personal computing with

the introduction of the PC created a need for richer interaction

Kay developed the WIMP interface at Xerox which produced the Xerox Star in 1981

One of the programmers from the project was hired by Apple…and the Mac was born

Soon WIMP came to predominate

54

The Metaphor Metaphors are a good way to introduce

users to new technology They allow people to understand

something new based on something they already understand

LOGO used the metaphor of a turtle dragging its tail on the ground

Many systems are based on the metaphor of the office desktop

55

The Metaphor Although the metaphor is useful to get

users used to a new system it can be harmful if carried too far the word processor looks and seems to act

just like a typewriter Few people know that they should format

paragraphs rather than putting an extra space between them

Few people realize that space is a character, since it is just empty space on a typewriter

56

The Metaphor

The most extreme example of the metaphor is the virtual reality system

This places the user in the metaphor, creating an alternate world

These interpret the users normal actions as instructions to manipulate virtual objects in the virtual world

57

Direct Manipulation

Direct manipulation interfaces are characterized by rapid feedback on every action performed by the user

This makes it appear that the user is manipulating an object in real time

This is a very satisfying experience that mirrors what happens in the real world

58

WYSIWYG What You See Is What You Get These are interfaces that show you

what the end result will look like in real time

They are ubiquitous in word processing The trouble is that a lot of processing

power is spent on it and sometimes it is less convenient to include diagrams this way than to use other techniques

59

Language vs. Action While direct manipulation is good for

many tasks, it can be less efficient when applied to repetitive tasks

Languages are better for this sort of task The user can describe the action to be

performed and the system will interpret and perform the action as many times as desired

This is why command languages are still used for certain types of tasks

60

Hypertext Vannevar Bush proposed the idea of

interlinked text in 1945 Since then, it has been used for

Linking of text in help systems Linking of documents in the Web

Among the problems of hypertext is the tendency for users to get lost and not be able to navigate back to a specific point or to forget their location

61

Multi-modality Normally humans communicate

with computers using two modes Typing, a haptic sense Reading, a visual sense

Multi-modality uses the other senses including haptic and auditory

This can allow more information to be conveyed at once

62

Computer Supported Cooperative Work The development of the network led to

the desire for people to work on shared documents remotely

This created Email Chat systems Shared whiteboards Systems for the delivery of lectures

remotely This area is also called groupware

63

The World Wide Web The introduction of the web in 1989 is

the most important change in computing It moved the internet out of the research

lab and made it available to all It formed a union of computing and

communications It made everyone aware of computers,

not just the few who used them on a daily basis

64

Agent-based Interfaces An agent is something which performs

some action on your behalf An email filter is an agent which

identifies and flags junk email Help agents watch the user’s actions

and make suggestions or provide help Some agents search the internet to find

specific items Others can be dispatched to sites on the

net to perform actions on your behalf

65

Ubiquitous Computing Ubiquitous or pervasive computing

moves it out of beige boxes so that computing permeates our environment

We have refrigerators which can surf the net for menus

We have appliances which can be programmed from a PC anywhere on the internet

We are developing RFID tags where items can report their location to a central computer

66

Sensor-based Interaction We are used to entering data into a

computer What if the computer had sensors which

gathered information from the environment?

This happens with lights which detect someone entering a room

Computers can have sensors which let them find out about the real world and respond appropriately

67

Context-aware Interaction

Once the computer knows about the state of the environment, it can make decisions as to what is appropriate for the environment

This can use artificial intelligence to take actions which are appropriate to what the user is currently doing