Visual Design for 3D User Interfaces Daniel C. Robbins [email protected] @ somewhere…

Visual Design for 3D User Visual Design for 3D User InterfacesInterfacesVisual Design for 3D User Visual Design for 3D User InterfacesInterfacesDaniel C. Robbins Daniel C. Robbins [email protected]@microsoft.comDaniel C. Robbins Daniel C. Robbins [email protected]@microsoft.com

@ somewhere…

Back Back Good UIGood UI ChannelsChannelsFocus+Focus+ MetaphorMetaphorProbs.Probs.

The Ideal?The Ideal?The Ideal?The Ideal?• Why is it so hard to use Why is it so hard to use

computers?computers?


Structure of This TalkStructure of This TalkStructure of This TalkStructure of This Talk• Who am I?Who am I?• Problems with current UIsProblems with current UIs• What to do about it?What to do about it?• Examples from our researchExamples from our research• Lessons we learnedLessons we learned• MetaphorsMetaphors• Future directionsFuture directions


Who Am I?Who Am I?Who Am I?Who Am I?• Fine-arts majorFine-arts major• Furniture, sculpture, jewelry, etc.Furniture, sculpture, jewelry, etc.• UI Designer for 13 yearsUI Designer for 13 years

– 7 years at MS, 6 in MSR7 years at MS, 6 in MSR– Combining academic and product Combining academic and product

focusfocus


Contact InfoContact InfoContact InfoContact Info• [email protected]@microsoft.com• http://research.microsoft.com/http://research.microsoft.com/

~dcr~dcr

mailto:[email protected]


What Is a User-What Is a User-interface?interface?What Is a User-What Is a User-interface?interface?• Means by which a person uses a Means by which a person uses a

computercomputer• The sensory connection between a The sensory connection between a

user and the functionality of a user and the functionality of a computer-based applicationcomputer-based application– HumanHuman-computer interaction-computer interaction– HumanHuman-computer--computer-humanhuman

interactioninteraction


UI PuzzlersUI PuzzlersUI PuzzlersUI Puzzlers• Has the Web killed innovative UI?Has the Web killed innovative UI?• Video: what is truth?Video: what is truth?


Problems w/ Current Problems w/ Current UIsUIsProblems w/ Current Problems w/ Current UIsUIs• Too hard to useToo hard to use• Too many optionsToo many options• UnreliableUnreliable• InconsistentInconsistent

Too VagueToo Vague


Specific ProblemsSpecific ProblemsSpecific ProblemsSpecific Problems• Multiple representations for an Multiple representations for an

objectobject– Objects look completely different in Objects look completely different in

different placesdifferent places

• Low-level interactionLow-level interaction– Manipulation centric vs. task-centricManipulation centric vs. task-centric– Should be:Should be:

““Just do what I wanted you to do!”Just do what I wanted you to do!”


Specific Problems (2)Specific Problems (2)Specific Problems (2)Specific Problems (2)• Discretely shifting spatial Discretely shifting spatial

configurationsconfigurations– The rug is pulled out from under the userThe rug is pulled out from under the user

• Multiple simultaneous viewsMultiple simultaneous views– Divided attentionDivided attention

• Homogeneous representationsHomogeneous representations– Everything looks the sameEverything looks the same

• No contextNo context– How do changes to one thing affect How do changes to one thing affect

another?another?


So What? Why Fix It?So What? Why Fix It?So What? Why Fix It?So What? Why Fix It?• Increase customer baseIncrease customer base

– Different agesDifferent ages– Different abilitiesDifferent abilities

• New ways of using computersNew ways of using computers– On-the-goOn-the-go– One-function-at-a-timeOne-function-at-a-time– Step-up useStep-up use– Over the NetOver the Net

How to Fix It?How to Fix It?How to Fix It?How to Fix It?


Mantra: Cause and Mantra: Cause and EffectEffectMantra: Cause and Mantra: Cause and EffectEffect

What can the user do?What can the user do?What can the user do?What can the user do?

What is the user What is the user doing?doing?

What is the user What is the user doing?doing?

PastPastPastPast

PresentPresentPresentPresent

FutureFutureFutureFutureWhat has the user What has the user

done?done?What has the user What has the user

done?done?

FutureFutureFutureFuture

PresentPresentPresentPresent

PastPastPastPast


SpecificsSpecificsSpecificsSpecifics• Take advantage of Take advantage of human human

perceptionperception– 3D, animation, constancy, richer cues3D, animation, constancy, richer cues

• Take advantage of human experienceTake advantage of human experience– Appropriate use of Appropriate use of metaphorsmetaphors

• In-situ assisted learning / “hard-fun”In-situ assisted learning / “hard-fun”– Video games, apprenticeship, sportsVideo games, apprenticeship, sports– Not the office-assistant!Not the office-assistant!


What is a 3D UI?What is a 3D UI?What is a 3D UI?What is a 3D UI?NOTNOT

Examples From Our Examples From Our ResearchResearchExamples From Our Examples From Our ResearchResearchData Mountain, TaskGallery, Data Mountain, TaskGallery, PhotoMountainPhotoMountainData Mountain, TaskGallery, Data Mountain, TaskGallery, PhotoMountainPhotoMountain

The TaskGalleryThe TaskGalleryThe TaskGalleryThe TaskGallery““Let’s change everything”Let’s change everything”““Let’s change everything”Let’s change everything”


Task Gallery: Task Gallery: FunctionalityFunctionalityTask Gallery: Task Gallery: FunctionalityFunctionality• UI exploration for new desktop UIUI exploration for new desktop UI

1)1) Windows management Windows management– Comparing multiple windows is hardComparing multiple windows is hard

2)2) Task management Task management– Switching between user tasks is hardSwitching between user tasks is hard– Use rooms concept, but use 3D to Use rooms concept, but use 3D to

engage spatial memoryengage spatial memory

• Push the technologyPush the technology


The Task Gallery: VideoThe Task Gallery: VideoThe Task Gallery: VideoThe Task Gallery: Video• Runs on commodity hardwareRuns on commodity hardware

• Uses unmodified applicationsUses unmodified applications

• Runs on Windows 2000 with “Runs on Windows 2000 with “

Application RedirectionApplication Redirection””

• ResultsResults


Good UI PracticesGood UI PracticesGood UI PracticesGood UI Practices• Show cause-and-effectShow cause-and-effect• Combine Combine related user actions, related user actions,

params.params.• Distinguish Distinguish UI from dataUI from data• Combine UI and data (Combine UI and data (Direct Direct ManipManip.).)• Use appropriate “Use appropriate “channelschannels””• Appropriate metaphorAppropriate metaphor• Support current functionalitySupport current functionality


Spatial Memory: the Spatial Memory: the Bet!Bet!Spatial Memory: the Spatial Memory: the Bet!Bet!• EvolutionEvolution: perceptual abilities: perceptual abilities

– Remember (generally) where stuff is Remember (generally) where stuff is (?)(?)

• Engage navigational memoryEngage navigational memory• Reinforce with Reinforce with landmarkslandmarks: :

proximityproximity• Requires spatial Requires spatial

stability/constancystability/constancy


Spatial Cues #2Spatial Cues #2Spatial Cues #2Spatial Cues #2• Kevin Lynch: Kevin Lynch: The Image of the The Image of the

CityCity– Urban DesignUrban Design

• Boundaries, neighborhoods, Boundaries, neighborhoods, topographytopography

The DataMountainThe DataMountainThe DataMountainThe DataMountainBack to basicsBack to basicsBack to basicsBack to basics


The Data MountainThe Data MountainThe Data MountainThe Data Mountain• What is the role of landmarks?What is the role of landmarks?• Is the arrangement of the data Is the arrangement of the data

itself enough of a cue?itself enough of a cue?


Cues: NeighborhoodCues: NeighborhoodCues: NeighborhoodCues: Neighborhood


BoundariesBoundariesBoundariesBoundaries

The PhotoMountainThe PhotoMountainThe PhotoMountainThe PhotoMountainAdapting the DataMountain to Adapting the DataMountain to PhotographsPhotographsAdapting the DataMountain to Adapting the DataMountain to PhotographsPhotographs


PhotoMountain PhotoMountain FunctionalityFunctionalityPhotoMountain PhotoMountain FunctionalityFunctionality• Import / Add photosImport / Add photos• Categorize photosCategorize photos

– Auto, manualAuto, manual

• Find photos byFind photos by– TimeTime– PeoplePeople– PlacePlace

• Create slide showsCreate slide shows


Design RefinementsDesign RefinementsDesign RefinementsDesign Refinements

Lessons We LearnedLessons We LearnedLessons We LearnedLessons We LearnedHow do we actually design this stuff?How do we actually design this stuff?How do we actually design this stuff?How do we actually design this stuff?


PrinciplesPrinciplesPrinciplesPrinciples• Get the “Big Picture”Get the “Big Picture”• ChannelsChannels• GeometryGeometry• MetaphorMetaphor


Focus + ContextFocus + ContextFocus + ContextFocus + Context• Scarcest resource is human Scarcest resource is human

attention!attention!• Hierarchical regions for Hierarchical regions for

organizationorganization• BroadenBroaden or or narrow narrow attentionattention

– Vs Vs switchingswitching attention attention– Parallel attentionParallel attention– 3D is a natural way to do this 3D is a natural way to do this

““Get interactions cheaply”Get interactions cheaply”


High-level InteractionHigh-level InteractionHigh-level InteractionHigh-level Interaction• Make one click do Make one click do lots of stufflots of stuff• Free-form Free-form Navigation vs. High-Navigation vs. High-

levellevel– Doom vs. “chauffer”Doom vs. “chauffer”


Channels: Bag of TricksChannels: Bag of TricksChannels: Bag of TricksChannels: Bag of Tricks• Animation Animation • Rendering styleRendering style , , TransparencyTransparency• AbstractionAbstraction• AudioAudio• HarderHarder in 3Din 3D

– Channels already taken (size, color, Channels already taken (size, color, etc.)etc.)

MethodologyMethodologyMethodologyMethodologyHow to free up designHow to free up designHow to free up designHow to free up design


Basic GeometriesBasic GeometriesBasic GeometriesBasic Geometries• CarouselCarousel• P-WallP-Wall• BowlBowl• LandscapeLandscape• 3D Star (chandelier)3D Star (chandelier)• LayeringLayering

– Embedding multiple dimensionsEmbedding multiple dimensions• Endocentric vs. ExocentricEndocentric vs. Exocentric


CarouselCarouselCarouselCarousel


Perspective WallPerspective WallPerspective WallPerspective Wall


BowlBowlBowlBowl• Natural Natural fisheyefisheye


LandscapeLandscapeLandscapeLandscape


3D Star3D Star3D Star3D Star


Data Layering (1)Data Layering (1)Data Layering (1)Data Layering (1)


Data Layering (2)Data Layering (2)Data Layering (2)Data Layering (2)


Other?Other?Other?Other?Axis 1Axis 1Axis 1Axis 1

Axis 2Axis 2Axis 2Axis 2

# of tics# of tics# of tics# of tics

amount amount of bendof bend

amount amount of bendof bend

# of spines# of spines# of spines# of spines

axis of axis of bendbend

axis of axis of bendbend


All at OnceAll at OnceAll at OnceAll at Once

Using These PrinciplesUsing These PrinciplesUsing These PrinciplesUsing These PrinciplesThe next version of the PhotoMountainThe next version of the PhotoMountainThe next version of the PhotoMountainThe next version of the PhotoMountain


Axes / DimensionsAxes / DimensionsAxes / DimensionsAxes / Dimensions• Identify Axes / DimensionsIdentify Axes / Dimensions• Prioritize dimensions (rankings)Prioritize dimensions (rankings)

– Which is primary?Which is primary?

• Bounded vs. UnboundedBounded vs. Unbounded– If bounded, how big?If bounded, how big?

•Extra Credit ->Extra Credit ->

•Does there have to be a Does there have to be a Primary Primary AxisAxis??


Axes / Dimensions for the Axes / Dimensions for the PhotoMountainPhotoMountainAxes / Dimensions for the Axes / Dimensions for the PhotoMountainPhotoMountain

Time People Place Composite: “Places I

have Lived”

Bounded No (within cluster)

? Yes

Hierarchy Depth

Year, Month,

Day, etc…

Friends -> College -> Close -

> Girlfriend

s

USA -> RI ->

Providence ->

George St.

USA -> MI…, RI…, ME…, WA…

Hierarchy

Breadth

Friends, Family,

Co-workers,

etc…

Alaska, Alabama,

George St., Brook St., Hope St.,

etc…


•Tents for collections

•Time slice through Map

•clusters•Slice through Geography

•Blimp shows selection

•floats and gives Slide-show

Draw Some GeometriesDraw Some GeometriesDraw Some GeometriesDraw Some Geometries

•Dual Perspective Walls

•Indexed Cylinder


Cleanup an Image…Cleanup an Image…Cleanup an Image…Cleanup an Image…


Try a First Pass Try a First Pass MappingMappingTry a First Pass Try a First Pass MappingMapping

Time

Width = # of items

Current Time

Cluster based on?


Add Some Real DataAdd Some Real DataAdd Some Real DataAdd Some Real Data

MetaphorsMetaphorsMetaphorsMetaphorsIs there anything new?Is there anything new?Is there anything new?Is there anything new?


Why Use a MetaphorWhy Use a MetaphorWhy Use a MetaphorWhy Use a Metaphor• Capitalize on previous knowledgeCapitalize on previous knowledge• Make a UI Make a UI seemseem friendlier (more friendlier (more

approachableapproachable• Provide a set of rules that Provide a set of rules that explainexplain

and and predictpredict the behavior of the behavior of objectsobjects– Under computer controlUnder computer control– Under user controlUnder user control


Inspiration for Inspiration for Metaphors?Metaphors?Inspiration for Inspiration for Metaphors?Metaphors?• NatureNature• The machineThe machine• The spiritualThe spiritual

• Is everything in our world Is everything in our world designed?designed?


Example MetaphorsExample MetaphorsExample MetaphorsExample Metaphors


Problems With Problems With MetaphorsMetaphorsProblems With Problems With MetaphorsMetaphors• Mapping can be arbitraryMapping can be arbitrary• ScalabilityScalability• False impressionsFalse impressions• High expectationsHigh expectations• Question:Question:

– Can we use “real-world” objects as Can we use “real-world” objects as landmarks in an abstract world?landmarks in an abstract world?

Future DirectionsFuture DirectionsFuture DirectionsFuture Directions


Other Geom #1Other Geom #1Other Geom #1Other Geom #1

Future DirectionsFuture DirectionsFuture DirectionsFuture Directions3D UI and the connected world3D UI and the connected world3D UI and the connected world3D UI and the connected world


Wall DisplaysWall DisplaysWall DisplaysWall Displays


Working TogetherWorking TogetherWorking TogetherWorking Together


Working from HomeWorking from HomeWorking from HomeWorking from Home


Dynamic TrackingDynamic TrackingDynamic TrackingDynamic Tracking


Sharing and RevisingSharing and RevisingSharing and RevisingSharing and Revising


Shared EnvironmentsShared EnvironmentsShared EnvironmentsShared Environments

The Way-outThe Way-outThe Way-outThe Way-outLets get wackyLets get wackyLets get wackyLets get wacky


Public Kiosks #1Public Kiosks #1Public Kiosks #1Public Kiosks #1


Public Kiosk #2Public Kiosk #2Public Kiosk #2Public Kiosk #2


Reading ListReading ListReading ListReading List• Principles of Three-Dimensional Principles of Three-Dimensional

Design, LueckingDesign, Luecking• Principles of Three-Dimensional Principles of Three-Dimensional

Design, WongDesign, Wong• The Image of the City, LynchThe Image of the City, Lynch• Design and Form (Bauhaus), IttenDesign and Form (Bauhaus), Itten• Metaphors We Live By, Lakoff et al.Metaphors We Live By, Lakoff et al.• Tufte, Shneiderman, Laurel, Wurman, Tufte, Shneiderman, Laurel, Wurman,

etc…etc…

ENDENDENDEND


Window ComparisonWindow ComparisonWindow ComparisonWindow Comparison• RearrangeRearrange• ““Peel-back”Peel-back”• TransparencyTransparency Flash Mock

up


User StudiesUser StudiesUser StudiesUser Studies– Four iterative Four iterative

usability tests usability tests completedcompleted

– Numerous Numerous usability problems usability problems identified and identified and fixedfixed

– Very high Very high satisfaction satisfaction ratingsratings


What Did Users Like?What Did Users Like?What Did Users Like?What Did Users Like?• Transparency Transparency (to deal with (to deal with

occlusion)occlusion)• Glancing (spatial awareness)Glancing (spatial awareness)• Context menusContext menus• Overview of tasksOverview of tasks


ProblemsProblemsProblemsProblems• OcclusionOcclusion• DiscoverabilityDiscoverability• Can it Can it scalescale??• Target audience?Target audience?


Future WorkFuture WorkFuture WorkFuture Work• Other metaphorsOther metaphors• Other geometric arrangementsOther geometric arrangements• Peripheral notificationPeripheral notification• Semantic zooming for thumbnailsSemantic zooming for thumbnails


Multiple Multiple RepresentationsRepresentationsMultiple Multiple RepresentationsRepresentations

Back Back


Low-level interactionLow-level interactionLow-level interactionLow-level interaction


Discrete ChangesDiscrete ChangesDiscrete ChangesDiscrete Changes


Multiple ViewsMultiple ViewsMultiple ViewsMultiple Views

5R x 3C


Homogeneous Reps.Homogeneous Reps.Homogeneous Reps.Homogeneous Reps.


Context and Context and RelationshipsRelationshipsContext and Context and RelationshipsRelationships

Back Back


AnimationAnimationAnimationAnimation• No animation #1, No animation #1, #2#2??• ““If it’s in the way, make it go If it’s in the way, make it go

away”away”• Shift load from cognitive to Shift load from cognitive to

perceptualperceptual• Don’t force user to figure out what Don’t force user to figure out what

changedchanged– Animation stylesAnimation styles


Discrete SwitchingDiscrete SwitchingDiscrete SwitchingDiscrete SwitchingBut But

with animwith animationation!!


Discrete BroadenDiscrete BroadenDiscrete BroadenDiscrete Broaden

But But with animation!

Animation


Rendering Styles #1Rendering Styles #1Rendering Styles #1Rendering Styles #1

• 2D 2D Wireframe Wireframe for navigation for navigation UI elementsUI elements


Rendering Styles #2Rendering Styles #2Rendering Styles #2Rendering Styles #2• Icons for Window Icons for Window

Manipulation Manipulation controlscontrols

• Different stylesDifferent styles

SelectSelect

Add to Add to SelectionSelection

X-ray X-ray VisionVision

Send Send BackBack

ToToPodiumPodium

ToToSpaceSpace

MoveMove

CloseClose

Widgets

mockup

Widgets

mockup


AbstractionAbstractionAbstractionAbstraction• Icons vs. ThumbnailsIcons vs. Thumbnails• Depends on:Depends on:

– Degree of attentionDegree of attention– PerceptibilityPerceptibility– ConfusabilityConfusability– Content (high freq. Vs. low freq.)Content (high freq. Vs. low freq.)

• Need semantic zooming!Need semantic zooming!• Tied to metaphor? Tied to metaphor?


2D vs. 3D Channels: 2D vs. 3D Channels: SizeSize2D vs. 3D Channels: 2D vs. 3D Channels: SizeSize

1

2

3

41

2

3

4


2D vs. 3D Channels: 2D vs. 3D Channels: ColorColor2D vs. 3D Channels: 2D vs. 3D Channels: ColorColor

1

2

3

41

2

3

4


Criteria: Fixing the Criteria: Fixing the ProblemsProblems

• Spatial memorySpatial memory• Context: human attentionContext: human attention• Visual recognitionVisual recognition• StabilityStability• ScalabilityScalability• Good UI practicesGood UI practices

PerceptioPerception n

vs. vs.

CognitionCognition


Output RedirectionOutput RedirectionOutput RedirectionOutput Redirection


Input RedirectionInput RedirectionInput RedirectionInput Redirection


The GutsThe GutsThe GutsThe Guts• Commodity hardwareCommodity hardware

• Unmodified applicationsUnmodified applications

• Windows 2000 with “Application Windows 2000 with “Application

Redirection”Redirection”


Audio ComparisonAudio ComparisonAudio ComparisonAudio Comparison• No audioNo audio • With audioWith audio


ScalabilityScalabilityScalabilityScalability

A few tasksA few tasksMany tasks


Spatial Cues: Spatial Cues: LandmarksLandmarksSpatial Cues: Spatial Cues: LandmarksLandmarks

Shadows aid in Shadows aid in depth depth perceptionperception

Landmarks aid in spatial memory


Visual Recognition (1)Visual Recognition (1)Visual Recognition (1)Visual Recognition (1)• Objects Objects

representerepresented by d by thumbnailsthumbnails

• User User selectable selectable wallpaperwallpaper


Visual Recognition (2)Visual Recognition (2)Visual Recognition (2)Visual Recognition (2)• Live Live

applicatioapplications ns – (Dynamic (Dynamic

updating)updating)


Task Gallery: EvaluationTask Gallery: EvaluationTask Gallery: EvaluationTask Gallery: Evaluation• Different interactionsDifferent interactions• User StudiesUser Studies• ProblemsProblems• Future workFuture work


Body CentricBody CentricBody CentricBody Centric• Left, Right HandsLeft, Right Hands• FeetFeet• Look UpLook Up• GlancingGlancing

– Lightweight, ephemeral interactionLightweight, ephemeral interaction


Spatial StabilitySpatial StabilitySpatial StabilitySpatial StabilitySort on sizeSort on sizeSort by NameSort by Name


Transparency VideoTransparency VideoTransparency VideoTransparency Video


Animation VideoAnimation VideoAnimation VideoAnimation Video


New Task VideoNew Task VideoNew Task VideoNew Task Video


Drag Task Video (Direct Drag Task Video (Direct manip)manip)Drag Task Video (Direct Drag Task Video (Direct manip)manip)


Rename Task VideoRename Task VideoRename Task VideoRename Task Video


Switch Tasks VideoSwitch Tasks VideoSwitch Tasks VideoSwitch Tasks Video


Show All Tasks VideoShow All Tasks VideoShow All Tasks VideoShow All Tasks Video


Transparency VideoTransparency VideoTransparency VideoTransparency Video


Wallpaper VideoWallpaper VideoWallpaper VideoWallpaper Video


Freeform Nav VideoFreeform Nav VideoFreeform Nav VideoFreeform Nav Video


Hands VideoHands VideoHands VideoHands Video


Glancing VideoGlancing VideoGlancing VideoGlancing Video


TG Full VideoTG Full VideoTG Full VideoTG Full Video


DM Short VideoDM Short VideoDM Short VideoDM Short Video


Broaden VideoBroaden VideoBroaden VideoBroaden Video


Narrow VideoNarrow VideoNarrow VideoNarrow Video

Visual Design for 3D User Interfaces Daniel C. Robbins [email protected] @ somewhere…

Documents

Transcript of Visual Design for 3D User Interfaces Daniel C. Robbins [email protected] @ somewhere…