Post-Wimp Interaction With Desktop Post-WIMP sketching-based 3D modelling interface n MaggLite [Huot

download Post-Wimp Interaction With Desktop Post-WIMP sketching-based 3D modelling interface n MaggLite [Huot

of 99

  • date post

    08-Jul-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of Post-Wimp Interaction With Desktop Post-WIMP sketching-based 3D modelling interface n MaggLite [Huot

  • 1

    Post-Wimp Interaction With Desktop Computers

    Pierre Dragicevic

    New Tools and Techniques

  • 2

    WIMP Interaction

    n Standard Interaction Techniques:

    n Windows, widgets, icon n Text input, click, double-click, d&d

    +

    n WIMP = Windows, Icons, Menus, Pointer

    n Standard Devices:

    n Mouse n Keyboard

  • 3

    What Next and Why?

    n WIMP system (Xerox Star, 1979)

  • 4

    Post-WIMP Interaction Paradigms

    Better exploit our motor abilities:

    n Bimanual Interaction

    Map Zooming [Hinckley et al 98]

    SmartSkin [Rekimoto 2002]

    n « Multifinger » Interaction

  • 5

    Post-WIMP Interaction Paradigms

    Better exploit our perceptual abilities:

    n Information visualization

    TreeMaps [Schneiderman et al]

    Multiple screens & large displays

    n Enhanced displays

  • 6

    Post-WIMP Interaction Paradigms

    Make interaction more « direct »:

    n Gestural interaction

    Toolglasses [Bier et al 93]

    n See-through tools

  • 7

    Two Research Directions

    n Explore Post-WIMP interaction further:

    n Propose innovative solutions to known problems n Investigate alternative interaction paradigms

    n Disseminate existing research:

    n Integrate Post-WIMP paradigms into GUI toolkits n Provide new design guidelines

  • 8

    My Research

    n Explore Post-WIMP interaction further:

    n Propose innovative solutions to known problems n Investigate new interaction paradigms

    n Disseminate existing research:

    n Integrate Post-WIMP paradigms into GUI toolkits n Provide new design guidelines

    n Support the exploration of new approaches:

    n Study more usable and more flexible tools n Advocate a vision of Post-WIMP with sound examples

  • 9

    My Research

    New tools

    New techniques

  • 10

    New techniques

    Introduction

    Conclusion

    II. Artistic Resizing

    IV. Fold n’ Drop

    I. Input Configurator

    III. SpiraClock

    New tools

  • 11

    I. Input Configurator

    Pierre Dragicevic

    Jean-Daniel Fekete

    An Input Interaction Model for Highly Configurable Multi-Device

    Interactive Systems

    Introduction

    Conclusion

    II. Artistic Resizing

    IV. Fold n’ Drop

    I. Input Configurator

    III. SpiraClock

  • 12

    Goal: Support Alternative Input

    n WIMP system (Xerox Star, 1979)

    n Input-Open System

  • 13

    Why Using Alternative Input?

    the task the user the environnement

    n Augmenting the bandwith

    n Optimizing interaction according to:

  • 14

    Current State of the Art

    n Very poor support for alternative input devices in:

    n Current operating systems n Current applications n Current toolkits

    n Little research on input management

    n Post-WIMP tools are specialized towards:

    n A fixed set of input devices n A fixed set of interaction techniques

    n No approach deals with:

    n Multiple and heterogeneous input (enriched / impoverished) n Flexibility and configurability

  • 15

    The Input Configuration Model

    n Approach:

    n Fully separate physical devices from applications

    n Provide a flexible way to map devices to applications

  • 16

    The Input Configuration Model

    n Device:

    n Parametrizable black box n Processes input into output

    n I/O Slots:

    n Simple types n + compound types

    n Implicit I/O:

    n User input n Feedback

    Input slots

    Output slots

    Implicit input

    Implicit output

  • 17

    The Input Configuration Model

    n Device types: n System

    (hardware ressources) n Adapter

    (software processes) n Application

    (specific to each app)

    n Input Configuration: n A set of system and

    application devices connected through adapter devices

    System devices

    Application devices

    Adapter devices

    Input Configuration

  • 18

    The ICON Toolkit

    n System Devices:

  • 19

    The ICON Toolkit

    n Adapter devices:

    n Data processing

  • 20

    The ICON Toolkit

    n Adapter devices:

    n Data processing n Graphical feedback

  • 21

    The ICON Toolkit

    n Adapter devices:

    n Data processing n Graphical feedback n Interaction techniques

  • 22

    The ICON Toolkit

    n Application Devices: n Example: IConDraw tools

  • 23

    The ICON Toolkit

    n Graphical Toolkit Devices:

    n Example: Swing devices

  • 24

    The ICON Toolkit

    n The input configuration editor:

    Available devices

    Available devices

    Editing pane

    Editing pane

  • 25

    The ICON Toolkit

    n The input configuration editor

  • 26

    The ICON Toolkit

    n The input configuration editor n Dynamic wire drawing algorithm

  • 27

    The ICON Toolkit

    n The input configuration editor n Fluid expand/collapse

  • 28

    The ICON Toolkit

    n The input configuration editor n Fisheye-drag

  • 29

    Examples

    1. Exploiting rich input

    n Pressure-sensitive drawing inside IConDraw

  • 30

    1. Exploiting rich input

    Examples

  • 31

    Examples

    1. Exploiting rich input

    n Symmetric bimanual interaction in IConDraw

  • 32

    Examples

    1. Exploiting rich input

  • 33

    Examples

    1. Exploiting rich input

    n Integral control of the Responsive Face [Perlin 97]

  • 34

    Examples

    1. Exploiting rich input

    n Integral control of the Responsive Face [Perlin 97]

  • 35

    Examples

    1. Exploiting rich input

  • 36

    Examples

    2. Improving Accessibility

    n Keyboard ? Pointing

    2nd order control

  • 37

    Examples

    2. Improving Accessibility

    n Pointer ? Text editing

    « Floating QuikWriting » [Perlin]

  • 38

    Examples

    2. Improving Accessibility

    n Pointer ? Text editing

    « Floating QuikWriting » [Perlin]

  • 39

    Examples

    3. Augmenting WIMP applications

    n « Augmented pointing »

    Filter Events

  • 40

    Examples

    3. Augmenting WIMP applications

    n « Augmented pointing » : smoothed cursor

  • 41

    Examples

    3. Augmenting WIMP applications

    n « Augmented pointing » : throwable cursor

  • 42

    Examples

    3. Augmenting WIMP applications

    n « Augmented pointing » : speech cursor

    rightrightNoise…Noise… downdown

    stopstoppresspressupup slowerslower releaserelease

  • 43

    Examples

    3. Augmenting WIMP applications

    n « Generalized Drag and Drop »

  • 44

    Projects using ICON

    n Marina II (2001) Photograph-based architectural modelling tool

    n Svalabard [Huot et al 2003] Post-WIMP sketching-based 3D modelling interface

    n MaggLite [Huot et al 2004] Post-WIMP UIMS based on ICON

    n LRI, University of Orsay [Appert et al 2003] Experimental comparison of WIMP / Post-WIMP techniques

    n Smart Home and Impairment (INT/ENST/EDF 2005) Configurable software architecture for accessibility

    n Student projects

  • 45

    II. Artistic Resizing

    Pierre Dragicevic

    Stéphane Chatty

    David Thevenin

    Jean-Luc Vinot

    A Technique for Rich Scale-Sensitive Vector Graphics

    Introduction

    Conclusion

    II. Artistic Resizing

    IV. Fold n’ Drop

    I. Input Configurator

    III. SpiraClock

  • 46

    n Graphic designers are given more and more importance in GUIs

    n Exploitation of vector graphics (e.g, Scalable Vector Graphics)

    n IntuiKit [Chatty et al 2004]

    Graphically Rich User Interfacs

  • 47

    IntuiKit: Example of Team Work

    Low-fi prototyping

    Graphic design

    Programming

    Integration

  • 48

    The Resizing Problem

    n Fixed size

    n Naive scaling

    n Common resizing

  • 49

    Expressing Resizing

    n Resizing is commonly described using formulae

    n These formulae are:

    n Translated into code by the programmer n Or used as an input to constraint-solving systems

    w

    h

    wL

    hL yL

    xL r

    hB wB

    • xL = (w-wL) / 2 • yL = (h-hL) / 2 • wL = 20 • hL = 10

    • wB = 5 • hB = 5

    • r = 20

  • 50

    Expressing Resizing

    n But designers think visually

    n They are used to produce visual variants

    n Variants