1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.
-
date post
19-Dec-2015 -
Category
Documents
-
view
222 -
download
2
Transcript of 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.
![Page 1: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/1.jpg)
1
Pen-CentricShorthand Interfaces
Charles C. TappertSeidenberg School of CSIS, Pace
University
![Page 2: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/2.jpg)
2
Themes of Presentation
Online Handwriting Recognition and Pen Computing Tutorial
Historical Research – undertaken for the Palm-Xerox Patent Infringement Lawsuit
Recent Research - Enhanced Pen-Centric Shorthand Interfaces can have benefits DPS dissertation could extend M.S. thesis
![Page 3: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/3.jpg)
3
Enhanced Pen-Centric Shorthand Interfaces
Can use word/phrase shorthand to speed text input
Can provide critical infrastructure for many pen-centric applications
Can enhance natural pen-centric interactions for many applications
Will have greatest impact on the utility of applications running on small mobile devices
![Page 4: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/4.jpg)
4
Part 1: Online (Pen-Centric)Handwriting Recognition
Written Languages and Handwriting Properties The Fundamental Property of Writing Handwriting Recognition Difficulties Online (Pen-Centric) Handwriting Recognition Online more accurate than Offline Recognition Online Info Can Complicate Recognition
Process Design Tradeoffs / Design Decisions Computer Problems in English
![Page 5: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/5.jpg)
5
Written Language and Handwriting Properties
Alphabet Letters, digits, punctuation, special
symbols Writing is a time sequence of strokes
Stroke – writing from pen down to pen up Usually complete one character before
beginning the next Spatial order – e.g., in English left to right
![Page 6: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/6.jpg)
6
Fundamental Property of Writing
Differences between different characters are more significant than differences between different drawings of the same character
This makes handwritten communication possible
Can there be exceptions – say, different characters written identically?
![Page 7: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/7.jpg)
7
Fundamental Property of Writing
in English
Property holds within subalphabets of uppercase, lowercase, and digits, but not across them
“I”, “l”, and “1” written with single vertical stroke
“O” and “0” written similarly with an oval
![Page 8: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/8.jpg)
8
Handwriting Recognition Difficulties
Shape, size, and slant variation Similarly shaped characters – U and
V Careless writing
in the extreme, almost illegible writing Resolving difficult ambiguities
requires sophisticated recognition algorithms, syntax/semantics
![Page 9: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/9.jpg)
9
Electronic tablets invented in late 1950s Digitizer and display in separate surfaces
Pen Computers arrived in 1980s Combined digitizer and display Brought input and output into one surface Immediate feedback via electronic ink Created paper-like interface
Online (Pen-Centric) Handwriting Recognition
![Page 10: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/10.jpg)
10
Tablet Digitizers – Dynamic Information
Pen down – indication of inking X-Y coordinates as function of time
Sampling rate: 100 points per second Resolution: 200 points per inch
![Page 11: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/11.jpg)
11
Early Pen-Centric Interface
Different surfaces for input and output
Rand system, about 1959
![Page 12: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/12.jpg)
12
Pen Computers
IBM vision Paper-like interface,
1992
Microsoft Tablet PC Launched, 2001
![Page 13: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/13.jpg)
13
Pen-Centric PDAs
Early Palm Pilot
Palm Tungsten T3 and Sony Clié TH55
![Page 14: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/14.jpg)
14
Online (Pen-Centric) Handwriting Recognition
Machine recognizes the writing as the user writes
Digitizer equipment captures the dynamic information of the writing
Stroke number, order, direction, speed A stroke is the writing from pen down to pen up
![Page 15: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/15.jpg)
15
Online (Pen-Centric) more accurate than Offline (Static)
Recognition
Can use both dynamic and static information
Can often distinguish between similarly shaped characters E.g., 5 versus S where the 5 is usually
written with two strokes and the S with one stroke
![Page 16: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/16.jpg)
16
Online Information Can Complicate Recognition Process
Large number of possible variations E can be written with one, two, three, or four
strokes, and with various stroke orders and directions
A four-stroke E has 384 variations (4! stroke orders x 24 stroke directions)
![Page 17: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/17.jpg)
17
Online Information Can Complicate Recognition Process
Other variations
![Page 18: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/18.jpg)
18
Online Information Can Complicate Recognition Process
Segmentation ambiguities character-within-character problem lowercase d might be recognized as a cl if
drawn with two strokes that are somewhat separated from one another
![Page 19: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/19.jpg)
19
Design Tradeoffs/Decisions
No constraints on the user Machine recognizes user's normal
writing User severely constrained
Must write in particular style such as handprint
Must write strokes in particular order, direction, and graphical specification
![Page 20: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/20.jpg)
20
English Writing Styles Handprint
Uppercase – about 2 strokes per letter Lowercase- about 1 stroke per letter
Cursive Script Usually less than 1 stroke per letter Delayed crossing and dotting strokes
![Page 21: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/21.jpg)
21
Computer Problems in English
Constrained Handprint Printing one symbol per box – form filling Printing on lines – symbols can touch or
overlap Unconstrained Handprint
No lines and symbols can touch or overlap
Cursive Script Mixed Printing and Cursive
![Page 22: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/22.jpg)
22
Computer Problems in English
![Page 23: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/23.jpg)
23
Part 2Shorthand in Pen-Centric
PDAs
Famous Uses of Shorthand Historical Shorthand Alphabets Pen-Centric Shorthand Alphabets Pen-Centric Word/Phrase Shorthand Allegro/Chatroom Shorthand System
M.S. thesis that could be extended into a DPS dissertation
![Page 24: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/24.jpg)
24
Background
Famous writings throughout history were effectively written in a style of shorthand Cicero’s orations Martin Luther’s sermons Shakespeare’s and George Bernard Shaw’s
plays Samuel Pepys’ diary Sir Isaac Newton’s notebooks
![Page 25: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/25.jpg)
25
Historical Shorthand Alphabets
We first review the history of shorthand systems prior to pen computing
Shorthand is “a method of writing rapidly by substituting characters, abbreviations, or symbols for letters, words, or phrases”
Shorthand can be traced back to the Greeks in 400 B.C.
![Page 26: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/26.jpg)
26
Historical Shorthand Alphabets
We focus on shorthand alphabets that might be appropriate for PDAs
We review two types of shorthand Geometric shorthand
Small number of basic shapes Shapes reused in multiple orientations
Non-geometric shorthand shorthand
![Page 27: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/27.jpg)
27
Historical Shorthand Alphabets
Ancient Greeks – 400 BC Tironian Alphabet – 63 BC John Willis’s Stenography – 1602 Gabelsberger Alphabet – 1834 Moon Alphabet – 1845
![Page 28: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/28.jpg)
28
Tironian Alphabet, 63 B.C.Non-Geometric
![Page 29: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/29.jpg)
29
Stenography Alphabet, 1602
![Page 30: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/30.jpg)
30
Stenography Alphabet, 1602
Basic Shapes and Orientations
![Page 31: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/31.jpg)
31
Gabelsberger Cursive-Style, 1834
Non-Geometric Alphabet
![Page 32: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/32.jpg)
32
Moon Geometric Alphabet, 1845
![Page 33: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/33.jpg)
33
Other Historical Shorthand Systems
Phonetic alphabets Pitman (1837), was popular in UK Gregg (1888), was popular in USA
Systems for the blind Braille (1821)
![Page 34: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/34.jpg)
34
Pen-Centric Shorthand Alphabets
Some of the earliest were for CAD/CAM symbols represent graphical items and
commands Others developed for text input on
small consumer devices like PDAs that have limited computing power
We review geometric and non-geometric shorthands appropriate for small devices
![Page 35: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/35.jpg)
35
Pen-Centric Shorthand Alphabets
Historical alphabets presented above could be used for machine recognition symbols drawn with a single stroke (except “K” in Tironian and “+” in
Stenography) In addition to shape and orientation,
online systems can use stroke direction to differentiate among symbols
![Page 36: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/36.jpg)
36
Pen-Centric Shorthand Alphabets
Geometric Pen-Centric Shorthands Organek – 1991 Allen – filed 1991, patent 1993 Goldberg (Xerox) – filed 1993, patent 1997
Non-Geometric Pen-Centric Shorthands Graffiti (Palm Computing) – 1995 Allegro (Papyrus) – 1995
![Page 37: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/37.jpg)
37
Organek Alphabet, 1991
![Page 38: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/38.jpg)
38
Organic Alphabet, 1991 Basic Shapes and
Orientations
One shape in 4 orientations.
This gives 8 directions that together with 3 lengths provide 24 symbols.
A second wheel provides additional symbols.
![Page 39: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/39.jpg)
39
Allen patent, filed 1991
![Page 40: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/40.jpg)
40
Allen patent, filed 1991 Basic Shapes and
Orientations
![Page 41: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/41.jpg)
41
Goldberg patent, filed 1993(“unistroke symbols”)
![Page 42: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/42.jpg)
42
Goldberg patent, filed 1993 Basic Shapes and
Orientations
![Page 43: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/43.jpg)
43
Goldberg patent, filed 1993
5 Basic shapes4 Orientations
2 Stroked Directions40 Possible Symbols
Designed for Speed of Input and Maximum Symbol Separation
![Page 44: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/44.jpg)
44
Shorthand Alphabet Design
How would you design a shorthand alphabet?
What would be the design criteria?
![Page 45: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/45.jpg)
45
Design of Graffiti Alphabetfor the Palm Pilot
Small alphabet Uppercase, digits, special symbols
One stroke per symbol to avoid segmentation difficulty
Separate writing areas for letters and digits to avoid same-shape confusions
![Page 46: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/46.jpg)
46
Graffiti Alphabet, 1995
![Page 47: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/47.jpg)
47
Graffiti Mimics Keyboard Input
Character by character input Mode shifts for
Uppercase Special characters
Eyes can focus on application’s insertion point rather than on input area
![Page 48: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/48.jpg)
48
Graffiti Alphabet Design
What was the additional design criterion?
![Page 49: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/49.jpg)
49
Graffiti Alphabet Design
Designed for ease of learning 20 letters exactly match the Roman
alphabet 6 remaining ones match partially
![Page 50: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/50.jpg)
50
Graffiti Alphabet: 11 of 26 characters
have alternate variations
![Page 51: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/51.jpg)
51
Frequently Confused Characters
![Page 52: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/52.jpg)
52
Other Low Performance Characters
![Page 53: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/53.jpg)
53
Symbol Overlap Comparison
![Page 54: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/54.jpg)
54
Graffiti Recognition Accuracy Study
![Page 55: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/55.jpg)
55
Allegro Alphabet (Papyrus), 1995 (now Microsoft)
![Page 56: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/56.jpg)
56
Simplified Design Tradeoffs/Decisions
for Graffiti and Allegro PDA Alphabets
Small alphabet one case rather than both upper and lowercase
One stroke per character (character = stroke) allows machine to recognize each character upon pen lift
Small number of writing variations per letter preferably only one
Separate writing areas for letters and digits avoids confusion of similarly shaped letters and digits
High correspondence to Roman alphabet for ease of learning
non-geometric, might not actually qualify as shorthand
![Page 57: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/57.jpg)
57
Commercially Successful Shorthands
Similar to the Roman alphabet Easy to learn Graffiti used in Palm OS devices
notably the Palm Pilot & Handspring models Allegro used in Microsoft Windows devices
Geometric alphabets not successful
![Page 58: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/58.jpg)
58
Current Commercial Systems
Company/System Writing Style
Palm Computing/Graffiti*
Special Shorthand Alphabet
Microsoft/Papyrus Allegro
Special Shorthand Alphabet
CIC/Jot Relatively Unconstrained Handprint
Microsoft Relatively Unconstrained Handprint and Cursive
*A few years ago Palm switched from Graffiti to Graffiti2, Graffiti2 is basically Jot licensed from CIC.
![Page 59: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/59.jpg)
59
Jeff Hawkins, innovator 1979 BSEE Cornell, 1979-1986 worked at Intel and GRiD 1986-1987 ABD BioPhysics doctoral program, U.C. Berkeley 1987- back at GRiD he created GRiDPAD, first pen computer 1992 formed Palm Computing, 1993 created first handwriting reco
software product for a mobile handheld - Casio’s Zoomer 1995 Palm Computing bought by U.S. Robotics 1996 created PalmPilot, first PDA with Graffiti shorthand alphabet
(over a million shipped in 18 months, a 66% market share, and the fastest growth of any computing product in history, faster than the TV and the VCR)
1997 U.S. Robotics bought by 3Com (sued by Xerox for patent infringement)
1998 left Palm to form Handspring, 1999 launched the Visor handheld 2000 Palm Computing spun off by 3Com 2002 created what is now the
Redwood Center for Theoretical Neuroscience 2003 Handspring (with Hawkins, et al.) acquired by Palm Computing 2005 Founded Numenta to build the ultimate brain-like machine
![Page 60: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/60.jpg)
60
Palm-XeroxPatent Infringement Lawsuit
The nine-year old battle between Palm and Xerox over handwriting recognition ends in 2006, see article.
Palm pays Xerox $22.5 million for a fully paid-up license for Xerox patents covering its text input Unistrokes technology
Xerox first sued Palm predecessor Palm Computing back in April 1997, claiming that the Graffiti text-entry system used in its PDAs infringed on patents for Unistrokes, which allows users to input letters and numbers into personal data units with basic, one stroke movements.
![Page 61: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/61.jpg)
61
ConclusionsPalm-Xerox Patent Infringement
Lawsuit
Invalidity Historical research showed that Goldberg
alphabet not so unique Even though the patent was accepted as valid,
these arguments narrowed the scope of the patent
Infringement Analyses and comparisons of the Goldberg and
Graffiti alphabets showed major differences Result was favorable settlement for Palm
![Page 62: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/62.jpg)
62
Pen-Centric Word/Phrase Shorthand
such as Chatroom Shorthand
Further increase speed of text entry
Potential applications Where input speed important Where word/phrase abbreviations
occur frequently – e.g., email
![Page 63: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/63.jpg)
63
Chatroom Shorthand Examples
CU See you, or Cracking up
CM Call me
@TEOTD At the end of the day
^5 High five
2nite Tonight
LOL Laughing out loud
ASAP As soon as possible
B/C or BC Because
![Page 64: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/64.jpg)
64
Allegro/Chatroom Shorthand System
Developed for M.S. dissertation Student was hearing impaired Developed as output component of
communication system Handwriting to text to speech
Two input writing areas One for Allegro (all-purpose) One for chatroom-like or user-defined
words/phrases
![Page 65: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/65.jpg)
65
Allegro/Chatroom Shorthand System
Stroke acquisition GUI
allegro strokerecognition
alphabet
sentence accumulator
Sentence display and spoken output
allegro strokelibrary
user-defined stroke library
a single stroke
other strokerecognition
word/phrasecharacter
done?no
yes
meaning
is it
![Page 66: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/66.jpg)
66
Allegro/Chatroom Shorthand System
![Page 67: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/67.jpg)
67
Allegro/Chatroom Shorthand System
M.S. Thesis Experimental Results
Allegro/Chatroom pen-centric shorthand input is faster than typing text and is comparable to typing text and chatroom shorthand characters
![Page 68: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/68.jpg)
68
ConclusionsPen-Centric Shorthands
Pen-centric interfaces should use shorthand, and especially word/phrase shorthand where appropriate, for fast text input
Benefit of shorthand interfaces Provides critical infrastructure for many pen-
centric applications Enhances natural pen-centric interactions for
many applications Has greatest impact on the utility of
applications running on small mobile devices
![Page 69: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/69.jpg)
69
ConclusionsHandwriting Recognition
Graffiti and Allegro greatly simplified the recognition problem
Handprint problem not completely solved Even with IBM’s ThinkWrite, CIC’s Jot, and
Microsoft products Cursive script problem clearly not
solved
![Page 70: 1 Pen-Centric Shorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University.](https://reader030.fdocuments.net/reader030/viewer/2022032704/56649d3a5503460f94a151ec/html5/thumbnails/70.jpg)
70
References W.B. Huber, S.-H. Cha, C.C. Tappert, and V.L. Hanson, "Use of
Chatroom Abbreviations and Shorthand Symbols in Pen Computing," Proc. 9th Int. Workshop on Frontiers in Handwriting Recognition, IWFHR 2004, Tokyo, Japan, October 2004.
W. Huber, V. Hanson, S. Cha, and C.C. Tappert, "Common Chatroom Abbreviations Speed Pen Computing," Proc. 11th Int. Conf. Human-Computer Interaction, Las Vegas, NV, July 2005.
C.C. Tappert and S. Cha, "Handwriting Recognition Interfaces," Chapter 6, pp. 123-137, in Text Entry Systems, Scott MacKenzie and Kumiko Tanaka-Ishii (Eds.), Morgan Kaufmann, 2007.
C.C. Tappert, C.Y. Suen, and T. Wakahara, "The state-of-the-art in on-line handwriting recognition," IEEE Trans. Pattern Analysis Machine Intelligence, Vol. PAMI-12, pp. 787-808, August 1990.
C.C. Tappert and J.R. Ward, "Pen-Centric Shorthand Handwriting Recognition Interfaces," Proc. 1st Int. Workshop on Pen-Based Learning Technologies, Catania, Italy, May 2007.