POINTING, SELECTING &

DIRECT MANIPULATION

Old models applied to new technologies…

http://www.io.com/~hcexres/power_tools/teams/glove_small.gif

http://www.freewebs.com/sonypsfree/nintendo_ds_lite.jpg

http://scr3.golem.de/screenshots/0807/iPhone_3G/xx_iphone3g_pair.jpg

Presentation for the class: HCI Desktop to Mobile

Lecturer: Dennis Krannich

Place & Time: Uni Bremen, 14. Jan. 2009

Presenters: Mohammad Shahin & Jan Smeddinck

DIRECT MANIPULATION – REDUCE COMPLEXITY1

4 J

an

ua

ry 2

00

9

2

DOMAIN - TERRY WINOGRAD

From direct manipulation to “being there”:

We interact with the world around us in three main

ways; manipulation, locomotion, and conversation.

manipulation: move things around with hands

locomotion: move oneself from place to place

conversation: say something and another person

says something back

14

Ja

nu

ary

20

09

3

http://www.stanford.edu/group/dschool/images/left/people_terry_winograd_top.jpg

ABOUT FACE 3.0: THE ESSENTIALS OF

INTERACTION DESIGN

by Alan Cooper, Robert Reimann, Dave Cronin

A “how-to” book

How to design the user experience based on first

identifying user goals and then building and utilizing

personas to build user-centered interaction

Not all that scientific, but useful

First edition (1995), now: 3.0 (2007)

Today: Chapter 19

www.cooper.com

14

Ja

nu

ary

20

09

4

DIRECT MANIPULATION

Definition:

A human-computer interaction style

which involves continuous

representation of objects of interest,

and rapid, reversible, incremental

actions and feedback

14

Ja

nu

ary

20

09

5

EXAMPLE

Showing direct manipulation in Windows Explorer

14

Ja

nu

ary

20

09

6

COINED BY SHNEIDERMAN, 1974

1. Visual representation of manipulated objects

2. Physical actions instead of text entry

3. Immediately visible impact of the operation

14

Ja

nu

ary

20

09

7

7 VARIETIES OF DIRECT MANIPULATION

1. Pointing

2. Selection

3. Drag and drop

4. Control manipulation

5. Palette tools

6. Object manipulation

7. Object connection

Program manipulation vs. content manipulation

14

Ja

nu

ary

20

09

8

http://www.allhatnocattle.net/swiss-knife.jpg

POINTING DEVICES

The mouse

Alternatives:

Light Pens / CRTs

Trackball

Tablets

Touchpads

Touchscreens

14

Ja

nu

ary

20

09

9

USING THE MOUSE

Mouse buttons

The left mouse button

The right mouse button

The middle mouse button

The scroll wheel

Meta-keys

Pointing and clicking

Pointing

Clicking

Clicking and dragging

Double-clicking

Chord-clicking

Double-clicking and dragging

14

Ja

nu

ary

20

09

10

POINTING

14

Ja

nu

ary

20

09

11

CLICKING

14

Ja

nu

ary

20

09

12

CLICKING & DRAGGING

14

Ja

nu

ary

20

09

13

DOUBLE-CLICKING

14

Ja

nu

ary

20

09

14

EVENTS: MOUSE-UP & MOUSE-DOWN

Mouse-down over an object or data should select

the object or data

Mouse-down over controls means propose action;

mouse-up means commit to action.

14

Ja

nu

ary

20

09

15

POINTING AND THE CURSOR

Pliancy and Hinting Object hinting

Cursor hinting

Wait cursor hinting

14

Ja

nu

ary

20

09

16

SELECTION

The act of choosing an object or

control is referred to as selection

Command ordering and selection

verb-object and object-verb order

Discrete and contiguous selection

Mutual exclusion

Additive selection

Group Selection

Insertion and replacement

Visual indication of selection

14

Ja

nu

ary

20

09

17

PLAYTIME

http://www.dontclick.it/

14

Ja

nu

ary

20

09

18

DRAG-N-DROP

Very powerful idiom!

Efficient: Connects 2 locations with a function (in 1 move)

source -> target

Drag to new position / window / function icon (trash)

Interior vs. external

External difficult: interface standards, target capabilities,

formats, DnD data vs. functions, no standard

Long time little use online. That changes…

e.g. http://www.bbc.co.uk/

14

Ja

nu

ary

20

09

19

DRAG-N-DROP - INTERACTIONS

Forgiving interaction: Encourage users to try out DM

Indicate drop targets

not by cursor-hinting!

No drop-target = no action

Hinting at drag pliancy difficult

Assume: Everything is dragable

Show visual indication while dragging

Use insertion targets, when necessary

Indicate completion

14

Ja

nu

ary

20

09

20

DRAG-N-DROP - ISSUES

Auto-scroll! (where to begin,

how fast?)

Not automatic scrollbar

insertion, but automatic

scroll-movement

Selecting vs. dragging

need drag-threshold after

click

Fine scrolling

Exact positioning of dragged

data not always easy

Use meta-keys for fine

resolution

Or arrow-keys for positioning

14

Ja

nu

ary

20

09

21

DRAG-N-DROP – MOBILE DEVICES

One hand problem!

Past: Only with stylus…

Currently: Revival with touch screens / multi touch

Future: Important!

14

Ja

nu

ary

20

09

22

http://assets.gearlive.com/blogimages/gallery/iphone-113-preview/002-iphone-113-drag-calc_medium.jpg

CONTROL MANIPULATION

Menus used to be difficult click-n-drag motion

Beginners use menus often, but they are difficult!

Menus now often “click-based”

Palette Tools:

Modal

Not too many!

Charged cursor

Not for functions!

Good for objects

With mobile devices:

Not yet that important

Will get more important with more tools available

14

Ja

nu

ary

20

09

23

http://z.about.com/d/graphicssoft/1/0/5/P/1/cs2-workspace-2menus.gif

OBJECT MANIPULATION

14

Ja

nu

ary

20

09

24

Repositioning

Click-n-drag in certain area, constrain drags

with meta-keys, snapping

Resizing & reshaping

Where to grab, vertex handles, constrains,

3D object manipulation

Display, input, movement, indication,

shadow, etc., problems

With mobile devices:

Very difficult…

Not often used. Difficult motion. High

potential.

OBJECT MANIPULATION 3D1

4 J

an

ua

ry 2

00

9

25

http://www.free3dtutorials.com/userimages/dorian/11)-wireframe_view-fig.jpg

http://www.radoxist.com/download/my_works/worht_enough_shade_wire.jpg

OBJECT CONNECTION

Click-n-drag to connect objects

rubberbanding

Find valid target locations

Indicate with handles

Useful to indicate connection /

flow, but no insertion / function

execution

With mobile devices:

Until now hardly ever used…

14

Ja

nu

ary

20

09

26

EFFICIENCY STATS

14

Ja

nu

ary

20

09

27

Interaction Design for Handheld

Computers

Jesper Kjeldskov and Nikolaj Kolbe

CURRENT TRENDS

Direct Manipulation Web-Applications (AJAX)

Rich visual feedback

New unsuspected uses

Video Browsing by DirectManipulation http://www.aviz.fr/dimp/dimp-

chi08.pdf

Virtual physical directmanipulation

BumpTophttp://www.youtube.com/watch?v=M0ODskdEPnQ

Devices: Motion, alignment, position

14

Ja

nu

ary

20

09

28

FUTURE – PHYSICAL DIRECT MANIPULATION

14

Ja

nu

ary

20

09

29

http://cache.gizmodo.com/assets/resources/2007/08/405587346_ba45

30bdbd.jpg

3D FUTURE & PERSPECTIVE PRESENCE1

4 J

an

ua

ry 2

00

9

30

http://www.cs.cmu.edu/~johnny/projects/wii/

http://medien.informatik.uni-

bremen.de/research/hci/mrt/

FUTURE – DM VS. INTERFACE AGENTS

Direct Manipulation will stay

important

Become more intuitive

Faster with better feedback

Limits with huge data sets

Agents can help handling that

or: SoundTorch http://vimeo.com/groups/digmed/videos/2446867

Be polite to users and give

them controls that let them

focus on their jobs

14

Ja

nu

ary

20

09

31

http://farm1.static.flickr.com/8/7157243_a5cf2853bc.jpg?v=0

QUESTIONS 14

Ja

nu

ary

20

09

32