Ken Hinckley Koji Yatani

Pen + Touch = New Tools

• Handhelds• Slates• Drafting Boards• Tabletops• Wall Displays

• Touch• or Pen…

DIRECT INPUT A SHIFT TO

ISN’T THE FUTURE ABOUT TOUCH?

WHY THE PEN?

To help explain, let me introduce my nemesis.

MR. I.M.A. BIGBODY, CORPORATE DENIZEN,THIRD RATE INC.

“I CAN TYPE FASTERTHAN I CAN WRITE.”

TYPETHIS.

OR THIS.

OR MAYBETHIS.

YOU CAN’T TYPE THE SOUL OFA NEW IDEA.

BUT IT’S NOT ALL ABOUTTHE PEN.

FIRST YOU NEED TO GRAB THE NAPKIN.

“JUST TELL ME WHICH IS BEST, TOUCH OR PEN.”

OF HANDSAND

HANDEDNESS

PAPER NOTEBOOK OBSERVATIONS

B1: Tuck the pen, interleave stylus and fingers

B2: Hold Clippings while Writing

B3: Specific Roles for Pen and Fingers

B4: Two-handed actions

B5: Drawing along Edges

B6: Hold Page while Flipping

PAPER NOTEBOOK OBSERVATIONS

BE EVOCATIVE OF PAPER WITHOUT BEING BEHOLDEN TO IT.

Q: Which hand do you write with, Right or Left?

A: Both!

• Guiard, 1987: NPH defines frame of reference.• Right vs. Left is the wrong question to ask.

What is the logic of the division of labor

between pen and touch in interface design?

• Video– search on

“manual deskterity”

• Users easily interleaved pen & touch – obvious• Staple, Xacto Cut, and Peel off Copy

particularly resonated with users• The particular tools were not always of interest• Some example comments:

REACTIONS FROM USERS

“The way it works is just like the way I already work in my notebook.”

“I wouldn’t have guessed the gestures worked that way, but once I tried it, it felt pretty natural.”

DESIGNPROPERTIES

OF PEN + TOUCH

27

• Pen + Touch: distinct interaction states• Touch + Touch ambiguous : pinch• Zero-force, incidental Contact is an

inherent property of (soft) Touch– GREAT! Just touch it to do something.– TERRIBLE! Just touch it, and it might do

something.

WHY NOT ALL MULTITOUCH?

• Minimal on-screen UI, widgets, & buttons.• Holding touch integrates (1) Object

Selection with (2) Pen Mode Switching.• And also…

• (3) provides salient feedback of temporary state. • (4) unambiguously phrases together

multiple elementary pen and touch inputs.

SOME INTRIGUING PROPERTIES…

• Differentiated vs. Interchangeable– E.g. Common controls

• Mobile vs. Stationary – Hand-held use vs. on-desk use, with graceful

degradation between the two– One-handed interactions for core actions– Two-handed interactions to take full

advantage of expressiveness of pen + touch

DESIGN DIMENSIONS

• Interleaved vs. Simultaneous – Pen + touch has the most novelty– Touch-pen-touch has tremendous value

• Elementary Inputs vs. Phrases – Not just assign device/hand to individual

interactions – – Workflow across multiple subtasks…

DESIGN DIMENSIONS

SUMMARY

IF WE’RE ONLY THINKING ABOUT TOUCH, WE’RE BLOWING IT.

EXPERIENCES TO STAND OUT…

IF WE WANT OUR

Not pen. Not touch.

Not pen OR touch.Pen AND Touch,

designed in concert

“TOUCH OR PEN ARE JUST NEWWAYS TO CONTROL THE MOUSE,

SO WHAT’S THE BIG DEAL?”

YOU’RE FIRED!

YOU’RE FIRED!

YOU’RE FIRED!

YOU’RE FIRED!

YOU’RE FIRED!

“TOUCH OR PEN ARE JUST NEWWAYS TO CONTROL THE MOUSE,

SO WHAT’S THE BIG DEAL?”

Pen WritesTouch Manipulates

Pen + Touch Yields New Tools

PEN + TOUCH GESTURES

staple

straightedge stamp

Xacto cut

brush

peel off copy

Pen WritesTouch Manipulates

fluidity of annotation + manipulation navigation zooming object creation

Pen + Touch Yields New Tools

PROPERTIES SHARED BY

PEN AND TOUCH

PEN IS BEST!

Mechanical intermediary: familiar tool, lever arm, affords precision!

TOUCH IS BEST!

No mechanical intermediary!

Nothing to lose!

No time to unsheathe!

PEN IS BEST!

Point at a specific spot on the coin

TOUCH IS BEST!

Grab the whole coin

EVERY COIN HAS TWO SIDES.