Nomadic Computing with PCsNomadic Computing with PCs

Bert KeelyArchitect, Mobile PCs and Tablet TechnologyWindows Client orgMicrosoft Corporation

Bert KeelyArchitect, Mobile PCs and Tablet TechnologyWindows Client orgMicrosoft Corporation

Nomadic Computing ideasNomadic Computing ideas

Steve Roberts’ bicycles

Mark Weiser’s many-per-person nomadic ‘pc peripherals’ (part of ubicomp)Invisible

Wearable

I’m after less of a head rush

Steve Roberts’ bicycles

Mark Weiser’s many-per-person nomadic ‘pc peripherals’ (part of ubicomp)Invisible

Wearable

I’m after less of a head rush

A device you can carry and use for visual informationA device you can carry and use for visual information

Books & magazines don’t fit in pockets

A full PC with the handiness of paper

1M sold, but this device is still under construction!

Books & magazines don’t fit in pockets

A full PC with the handiness of paper

1M sold, but this device is still under construction!

The handiness of paperThe handiness of paper

You can use it directly with one hand, even while carrying it.

You can capture thoughts and express yourself naturally using digital ink.

It’s comfortable wherever you go - around the home and office, on the go, and in meetings.

You can use it directly with one hand, even while carrying it.

You can capture thoughts and express yourself naturally using digital ink.

It’s comfortable wherever you go - around the home and office, on the go, and in meetings.

Where we are today: finishing the beginningWhere we are today: finishing the beginning1M Tablet PCs

fold face open

interact using a pen

How good are they?

dependable?

thin & light?

instant access?

battery life?

automatic networking?

easy to interact with?

1M Tablet PCs

fold face open

interact using a pen

How good are they?

dependable?

thin & light?

instant access?

battery life?

automatic networking?

easy to interact with?

Where we are todayWhere we are today

Hardware evolvingSlates for vertical markets

Ultra-portable laptops that convert to tablet mode are still a bit clumsy, still too heavy to hold

Inaccurate pens rely on cursor for ‘indirect direct’ input

Displays look washed-out due to extra surface reflections and extra viewing angle sensitivity in portrait

Few tablets have portable DVD options

Hardware evolvingSlates for vertical markets

Ultra-portable laptops that convert to tablet mode are still a bit clumsy, still too heavy to hold

Inaccurate pens rely on cursor for ‘indirect direct’ input

Displays look washed-out due to extra surface reflections and extra viewing angle sensitivity in portrait

Few tablets have portable DVD options

Where we are todayWhere we are today

Underdeveloped application softwarePen as inking device needs consistent UI for mode switching, selection & manipulation

Ink as datatype needs accurate analysis to identify text flows, drawings, annotations, math equations etc.

Ink markup requires anchoring and transformation to keep meaning when the underlying content reflows

Poor online/offline transitions

Underdeveloped application softwarePen as inking device needs consistent UI for mode switching, selection & manipulation

Ink as datatype needs accurate analysis to identify text flows, drawings, annotations, math equations etc.

Ink markup requires anchoring and transformation to keep meaning when the underlying content reflows

Poor online/offline transitions

Where we are todayWhere we are today

System software evolving slowlyMany refinements needed for mobile use

simple on/off

all day batteries

automatic networking including WWAN

scalable UI

dependable simple interaction

Pen as pointing device needs accuracy, clear feedback, and reliable gestures

Pen as text input device needs quick mode switching, user-specific vocabulary, acceleration via auto-completion

System software evolving slowlyMany refinements needed for mobile use

simple on/off

all day batteries

automatic networking including WWAN

scalable UI

dependable simple interaction

Pen as pointing device needs accuracy, clear feedback, and reliable gestures

Pen as text input device needs quick mode switching, user-specific vocabulary, acceleration via auto-completion

What we’re doing about hardwareWhat we’re doing about hardware

Collaborating w/ OEMs and key component manufacturers

Optimized Display Initiative

Pen vendor collaborations to improve accuracy, reduce cost

Project Kinesis

Collaborating w/ OEMs and key component manufacturers

Optimized Display Initiative

Pen vendor collaborations to improve accuracy, reduce cost

Project Kinesis

Optimized Display InitiativeOptimized Display Initiative

Goals Being Pursued In Phases • Widen viewing angle for portrait use• Reduce reflections and sparkling• Increase resolution at 12” sweet spot• Improve optical efficiency to allow auto-dim to

save power during typical use• Adjust white balance using ambient sensors• Further improve outdoor readability

Goals Being Pursued In Phases • Widen viewing angle for portrait use• Reduce reflections and sparkling• Increase resolution at 12” sweet spot• Improve optical efficiency to allow auto-dim to

save power during typical use• Adjust white balance using ambient sensors• Further improve outdoor readability

1. Why is viewing angle limited?1. Why is viewing angle limited?There are materials through which light passes at different speeds in different directions. Liquid crystals stacked diagonally have this ‘birefringence’

Over long distances, birefringence can create a double image (naturally occurring calcite crystals, common in Iceland, are sometimes called ‘TV rock’)

Over short distance, birefringence can put waves 180 degrees out of phase, canceling the image

In a typical twisted nematic (TN) LCD, light from secondary path is strong enough at some viewing angles to substantially weaken the image

There are materials through which light passes at different speeds in different directions. Liquid crystals stacked diagonally have this ‘birefringence’

Over long distances, birefringence can create a double image (naturally occurring calcite crystals, common in Iceland, are sometimes called ‘TV rock’)

Over short distance, birefringence can put waves 180 degrees out of phase, canceling the image

In a typical twisted nematic (TN) LCD, light from secondary path is strong enough at some viewing angles to substantially weaken the image

Ways to widen viewing angleWays to widen viewing angle

The LCD manufacturer can choose polarizer films that include opposite birefringence, to ‘undo’ the misalignment caused by the crystals. A ‘Strong Compensation Film’ achieves the needed improvement with twisted nematics: 20:1 contrast ratio viewed 50° off of perpendicular

LCD manufacturers can avoid twisted nematic structure and instead keep all the crystals horizontal (‘in-plane switching’ = IPS), or force the top ones to go completely vertical (‘vertical alignment’ = VA)

The LCD manufacturer can choose polarizer films that include opposite birefringence, to ‘undo’ the misalignment caused by the crystals. A ‘Strong Compensation Film’ achieves the needed improvement with twisted nematics: 20:1 contrast ratio viewed 50° off of perpendicular

LCD manufacturers can avoid twisted nematic structure and instead keep all the crystals horizontal (‘in-plane switching’ = IPS), or force the top ones to go completely vertical (‘vertical alignment’ = VA)

IPS (e.g. FFS from Hyundai Display)IPS (e.g. FFS from Hyundai Display) VA (e.g. Samsung)VA (e.g. Samsung)

Status of wide view pushStatus of wide view push

This year all standard Tablet PC displays will include wide view technology

Any display manufacturer can add portrait viewing support to an existing display, by sourcing a Strong Compensation Film. They can further optimize long-term by adopting VA or IPS

The same approaches will be useful to improve photo viewing

See samples after the talk

This year all standard Tablet PC displays will include wide view technology

Any display manufacturer can add portrait viewing support to an existing display, by sourcing a Strong Compensation Film. They can further optimize long-term by adopting VA or IPS

The same approaches will be useful to improve photo viewing

See samples after the talk

2. Why is there glare? (a.k.a. reflections)2. Why is there glare? (a.k.a. reflections)

Light entering a solid slows downPer Snell’s law, most incident light is refracted, but some is reflected

The eye sees an unwanted reflected image, whose strength is proportional to the change in speedAir refractive index = 1, Plastic = 1.4, boundary reflects 5.5%

Light entering a solid slows downPer Snell’s law, most incident light is refracted, but some is reflected

The eye sees an unwanted reflected image, whose strength is proportional to the change in speedAir refractive index = 1, Plastic = 1.4, boundary reflects 5.5%

Traditional solution = antiglareTraditional solution = antiglare

Roughen the surface to scatter the reflections – they are still there, but no longer ‘specular’ (keeping image intact, like a mirror)

Anti-glare is common on laptop & desktop LCDs The eye sees no reflected objects, but sees lower contrastThe reflections combine with the display image, making black less dark

Darker blacks are important when watching video, so antiglare is now becoming out of vogue

Some marketers are now are now oddly featuring screens without antiglare as “glare type”

Roughen the surface to scatter the reflections – they are still there, but no longer ‘specular’ (keeping image intact, like a mirror)

Anti-glare is common on laptop & desktop LCDs The eye sees no reflected objects, but sees lower contrastThe reflections combine with the display image, making black less dark

Darker blacks are important when watching video, so antiglare is now becoming out of vogue

Some marketers are now are now oddly featuring screens without antiglare as “glare type”

LCD complication LCD complication

Liquid crystal stacks are easily disturbed by pen pressure, which slightly deforms front glass. Visible liquid crystal flow is called ‘pooling’

LCD vendors can now more tightly control cell gap by switching to column spacers.

But polarizer is also easily scratched.

Front glass Front glass

Back glass Back glass

Spacer Spacer

Typical tablet adds protective overlay, triples reflections!Typical tablet adds protective overlay, triples reflections!

Protective overlay is added to prevent disturbing crystals and scratching soft polarizer. This creates not just one air/solid boundary but three:Protective overlay is added to prevent disturbing crystals and scratching soft polarizer. This creates not just one air/solid boundary but three:

16.5% of incident light is reflected 16.5% of incident light is reflected

Display Display

Protective Overlay Protective Overlay 5.5% 5.5%

5.5% 5.5%

5.5% 5.5%

The magic of anti-reflection (AR) The magic of anti-reflection (AR)

Place two reflective surfaces a quarter-wavelength apart. Reflections are ½ wave out of phase, thus cancel each other

One layer of Magnesium Fluoride broadly cuts visible reflections in half

Multilayer combinations can eliminate 90% of reflections

For green (average visible light),For green (average visible light), ¼ wave = 550 nanometers thick ¼ wave = 550 nanometers thick

Alternate tablet solution Alternate tablet solution

OEMs who want to keep overlay can reduce reflections at all air-gap surfaces by using anti-reflection coatings (AR)

OCLI is developing optimized overlays

Nitto Denko offers anti-reflective polarizer ‘ARC 150’

4.5% total is reasonable4.5% total is reasonable

Display Display

Protective Overlay Protective Overlay 2% 2%

0.5 % 0.5 %

2% 2%

Ultimate tablet solution Ultimate tablet solution Laminate a scratch-resistant, anti-reflective (AR) material directly onto display. This reduces first surface reflections and eliminates air gap reflections

Flexible material can be laminated during LCD mfg processNitto Denko developed with MS encouragement, but deployment awaits better cell gap control by LCD manufacturers

Rigid material can be bonded afterwardsMotion Computing is first to ship with bonded glass in a Tablet PC. See their M1400 in Microsoft booth.

Optical Coating laboratories (OCLI) has developed an optimized first surface on glass (see sample table)

DuPont is developing cost-effective lamination process

Protective Laminate Protective Laminate

Display Display

Fingerprints interfere with first surface ARFingerprints interfere with first surface AR

When finger oils thicken the plate, unwanted reflections return

3-part solution for use indoors: mild AR with hydrophobic coat plus fine anti-glare

Because water is polar with great surface tension, a non-polar top coat will reduce collection of finger oils

Because mild AR (e.g., single layer Magnesium Fluoride) only cuts reflections in half, fingerprints are less visible

Because anti-glare diffuses reflections, fingerprints are even less visible

Status of reflection reduction pushStatus of reflection reduction push

1. Most OEMs/ODMs starting to optimize overlay and display surfaces

Overlay approach requires AR on 3 surfaces: front of overlay, rear of overlay, front of display

OCLI supplying high quality overlays

Nitto Denko supplying AR polarizers

2. A few OEMs/ODMs are eliminating overlay by laminating protection

Add scratch resistant layer directly to front polarizer

Requires tightly controlling cell gap to eliminate pooling

Requires fine AG coating on laminate (helps hide fingerprints without sparkling)

1. Most OEMs/ODMs starting to optimize overlay and display surfaces

Overlay approach requires AR on 3 surfaces: front of overlay, rear of overlay, front of display

OCLI supplying high quality overlays

Nitto Denko supplying AR polarizers

2. A few OEMs/ODMs are eliminating overlay by laminating protection

Add scratch resistant layer directly to front polarizer

Requires tightly controlling cell gap to eliminate pooling

Requires fine AG coating on laminate (helps hide fingerprints without sparkling)

3. Increasing resolution at 12” sweet spot 3. Increasing resolution at 12” sweet spot

V1 Tablets gathered around a sweet spot, but didn’t nail it10.4” at 1024x768 resolution gives 123 ppi, which gets sharp text (with ClearType rendering). But size is is too small to view a whole document page

12.1” portrait is large enough to view a document page, but at 1024x768 resolution, 106 ppi text is too grainy

Two displays are emerging to fit the sweet spot:12.1” 1400x1050 (SXGA+) is now shipping from Toshiba Matsushita Display (TMD). At 145 ppi, it can take full advantage of Longhorn scaling. Wide view will become standard on this display in June (Strong Compensation Film).

12.1” 1280x800 (Wide XGA) tablet-ready display under development by BOE Hydis. 124 ppi fits sweet spot, wide aspect ratio is good both for documents in portrait, and movies in landscape.

See samples table after the talk

V1 Tablets gathered around a sweet spot, but didn’t nail it10.4” at 1024x768 resolution gives 123 ppi, which gets sharp text (with ClearType rendering). But size is is too small to view a whole document page

12.1” portrait is large enough to view a document page, but at 1024x768 resolution, 106 ppi text is too grainy

Two displays are emerging to fit the sweet spot:12.1” 1400x1050 (SXGA+) is now shipping from Toshiba Matsushita Display (TMD). At 145 ppi, it can take full advantage of Longhorn scaling. Wide view will become standard on this display in June (Strong Compensation Film).

12.1” 1280x800 (Wide XGA) tablet-ready display under development by BOE Hydis. 124 ppi fits sweet spot, wide aspect ratio is good both for documents in portrait, and movies in landscape.

See samples table after the talk

Traditional font rendering: one pixel’s not enough, two is too manyTraditional font rendering: one pixel’s not enough, two is too many

Unlocking hidden resolutionUnlocking hidden resolution

Notebook PC displays use RGB sub-pixelsWhite is an optical illusion

Real resolution is 3X higher on one axisEdges between sub-pixels equal those between the pixels

ClearType’s advantage –Addresses edges within the pixel

Diffuses color error

New sampling theory

Works with existing hardware

Uses existing Windows font technology

Notebook PC displays use RGB sub-pixelsWhite is an optical illusion

Real resolution is 3X higher on one axisEdges between sub-pixels equal those between the pixels

ClearType’s advantage –Addresses edges within the pixel

Diffuses color error

New sampling theory

Works with existing hardware

Uses existing Windows font technology

ClearTypeClearType

Red

Green

Blue

Red

Green

Blue

Further resolution increases have diminishing returns Further resolution increases have diminishing returns

Shrinking the display aperture ratio loses brightness (consumes more power)

Increasing total resolution demands more graphics rendering and data transfer (consumes more power)

Today, finer pixel pitch makes pixel-based text too small to read, makes icons without scaling too small to target

Longhorn enables arbitrarily high resolutions, but mobile PC sweet spot seems likely to remain 120-145 ppi due to other factors above

Shrinking the display aperture ratio loses brightness (consumes more power)

Increasing total resolution demands more graphics rendering and data transfer (consumes more power)

Today, finer pixel pitch makes pixel-based text too small to read, makes icons without scaling too small to target

Longhorn enables arbitrarily high resolutions, but mobile PC sweet spot seems likely to remain 120-145 ppi due to other factors above

Optimal resolution and PPIfor a range of mobile sizesOptimal resolution and PPIfor a range of mobile sizes

XGA

1024x768

WideXGA

1280x800

SXGA+

1400x1050

UXGA

1600x1200

WideSXGA+

1680x1050

8.4 152

10.4 123 145

12.1 106 124 145

14.1 91 124 142

15.4 128

Sub-Optimal

Optimal

Optimal with advantages in Longhorn

Tablets at 12.1” need Wide XGA or SXGA+ resolution

Larger notebooks benefit from SXGA+ or UXGA resolution

Project ‘Kinesis’Project ‘Kinesis’Based on belief that breakthrough ergonomics and aesthetics

will help Tablet functionality appeal to mainstream PC users

Project "Kinesis" Goals:

Share hardware design feedback from customers and partners

Assist our partners with hardware design by sharing CAD drawings, usability feedback, etc.

Increase convenience without increasing BOM cost

Background Information:

Microsoft and IDEO collaborated on physical design embodying new ideas in response to customer input

Based on belief that breakthrough ergonomics and aesthetics

will help Tablet functionality appeal to mainstream PC users

Project "Kinesis" Goals:

Share hardware design feedback from customers and partners

Assist our partners with hardware design by sharing CAD drawings, usability feedback, etc.

Increase convenience without increasing BOM cost

Background Information:

Microsoft and IDEO collaborated on physical design embodying new ideas in response to customer input

Overview – Project “Kinesis”Overview – Project “Kinesis”Key Features:

1. Thin and light

2. Direct hinge

3. Navigation control

4. Media slice

5. Docking stand

6. 12.1” wide (1280x800) display

7. Portrait stand

8. Ergonomic pen and pen wells

9. Convenient carrying case

10. Retractable AC adapter

Key Features:

1. Thin and light

2. Direct hinge

3. Navigation control

4. Media slice

5. Docking stand

6. 12.1” wide (1280x800) display

7. Portrait stand

8. Ergonomic pen and pen wells

9. Convenient carrying case

10. Retractable AC adapter

Thin and Light + Direct HingeThin and Light + Direct HingeThin and Light Convertible

21.5MM (.85”) thin

1.25KG (2.75 lbs)

Direct Hinge:

Quick, simple access to keyboard

Electrically and mechanically reliable interconnect

Hinge costs less to manufacture than typical Tablet PC hinge.

Thin and Light Convertible

21.5MM (.85”) thin

1.25KG (2.75 lbs)

Direct Hinge:

Quick, simple access to keyboard

Electrically and mechanically reliable interconnect

Hinge costs less to manufacture than typical Tablet PC hinge.

Navigation ControlNavigation Control5 way button rolls up and down for scrolling documents

Rocks left and right for back and forward navigation

Center depresses cleanly for enter

Control rotates for portrait and landscape use.

When user rotates the button, Windows switches orientation.

Adjacent window switch and hotkey buttons

5 way button rolls up and down for scrolling documents

Rocks left and right for back and forward navigation

Center depresses cleanly for enter

Control rotates for portrait and landscape use.

When user rotates the button, Windows switches orientation.

Adjacent window switch and hotkey buttons

Portrait Landscape

Dock and Media SliceDock and Media Slice

Media Slice• Media slice contains extra

battery, DVD, and speakers.

Dock• Tablet can be placed in dock

with or without media slice• Simple pivot and hinge allows

for use as 2nd monitor or as Tablet

Media Slice• Media slice contains extra

battery, DVD, and speakers.

Dock• Tablet can be placed in dock

with or without media slice• Simple pivot and hinge allows

for use as 2nd monitor or as Tablet

12.1” wide display + portrait stand12.1” wide display + portrait stand

12.1” Wide XGA 1280x800 Display• Great for DVDs with media slice• Ideal aspect ratio for a full size page

plus toolbars in portrait mode• 124 DPI ideal for reading• N-Trig digitizer works with standard

12.1” display

Portrait Stand• Simple part, big ergonomic

improvement• Great ergonomics for reading and

writing

12.1” Wide XGA 1280x800 Display• Great for DVDs with media slice• Ideal aspect ratio for a full size page

plus toolbars in portrait mode• 124 DPI ideal for reading• N-Trig digitizer works with standard

12.1” display

Portrait Stand• Simple part, big ergonomic

improvement• Great ergonomics for reading and

writing

Pen and pen well + accessoriesPen and pen well + accessoriesPen and Pen Well• Recessed barrel button design

reduces accidental right click• Store & grab pen wells ease

pen use as pointing device while typing (for both right and left handed users)

Accessories:• Nylon ‘portfolio’ case conceals

and protects Tablet, also holds AC adapter

• A pocket-size AC adapter with retractable cord is easy to carry.

Pen and Pen Well• Recessed barrel button design

reduces accidental right click• Store & grab pen wells ease

pen use as pointing device while typing (for both right and left handed users)

Accessories:• Nylon ‘portfolio’ case conceals

and protects Tablet, also holds AC adapter

• A pocket-size AC adapter with retractable cord is easy to carry.

Status of application softwareStatus of application softwareGive presentations while adding ink and highlighter for emphasisStay connected, using pen to communicate quietly (Outlook 2003 cached mode, Send/Receive over the internet via RPC over HTTP, MSN Messenger)Check calendar, contacts, agenda & attachments while walking to next meetingTake notes naturally using ink and audio in OneNoteMarkup & share web pages via Snipping tool Read and mark up documents without pushing print (Office 2003)Sketch, diagram & draw freely, with undo Enhance and show digital photosShow a map of any street in the USUse the pen to interact directly with any PC application or game

Give presentations while adding ink and highlighter for emphasisStay connected, using pen to communicate quietly (Outlook 2003 cached mode, Send/Receive over the internet via RPC over HTTP, MSN Messenger)Check calendar, contacts, agenda & attachments while walking to next meetingTake notes naturally using ink and audio in OneNoteMarkup & share web pages via Snipping tool Read and mark up documents without pushing print (Office 2003)Sketch, diagram & draw freely, with undo Enhance and show digital photosShow a map of any street in the USUse the pen to interact directly with any PC application or game

Steady progress in system softwareSteady progress in system softwareImproved text input in XPSP2

Writing pad appears in-place, grows, disappears

Reco uses context

Next major release in ’06 codenamed LonghornSimple On/off (incl. stand-by default)

Auto-config of Extended Desktop and Conf. monitors

Single-tap to open

Visuals (cursors, feedback, fonts, icons etc.)

Flick gestures and complementary button software

More reco improvements incl. user vocabulary & auto-complete

Ink Analysis APIs

Improved text input in XPSP2Writing pad appears in-place, grows, disappears

Reco uses context

Next major release in ’06 codenamed LonghornSimple On/off (incl. stand-by default)

Auto-config of Extended Desktop and Conf. monitors

Single-tap to open

Visuals (cursors, feedback, fonts, icons etc.)

Flick gestures and complementary button software

More reco improvements incl. user vocabulary & auto-complete

Ink Analysis APIs

Longhorn favorite featuresLonghorn favorite features

Thank You!

© 2005 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.