Gesture This! Using the Gesture Interface to Create User-Friendly Touch DevicesDouglas BolingPresidentBoling Consulting Inc.

SESSION CODE: WEM304

About Douglas BolingIndependent consultant specializing in Windows Mobile and Windows Embedded Compact (Windows CE)

On-Site InstructionConsulting and Development

AuthorProgramming Embedded Windows CE

Fourth Edition

AgendaIntroducing Gestures

Basic Gesture Input

AutoGesture support

Physics Engine

Historic Touch ModelTouch panel produced mouse message

WM_LBUTTONDOWNWM_LBUTTONUPWM_MOUSEMOVE Only while touching panel

“Gestures” used for right clickTap and Shift KeyTap and Hold

These still work

GesturesGesture support enhances basic mouse interpretation

Nothing more than could be done before but done for youProvides consistent look and feel across all applications

All Windows controls enhanced to use gestures

Only need to use for custom windows where mouse manipulation was used before

Basic Gesture APIGestures are interpreted touch input

Either single or multiple touch input

Allows flick, pan, select gestures by defaultCustom gestures can be registered

Windows Embedded Compact Gesture API is Win32-basedFollows Windows 7 format

Gesture ActionsPan User presses moves finger and lifts

Scroll (flick) User presses, moves, then lifts finger while moving

Select User presses and lifts

Rotate Two finger press and rotate

Zoom Two finger pinch or spread

Enabling Gesture SupportTo enable gesture support call:

EnableGestures (HWND hWnd, ULONGLONG ullFlags, UINT wScope);Parameters:

hWnd Handle of window that supports gesturesullFlags Flags indicating which gestures you would like

PanHoldSelectDoubleSelect<Custom gestures>All

uScope Flags indicating gestures for specific window or entire application

EnableGestures typically called in WM_CREATE of main window

Using EnableGestures

//-------------------------------------------------// DoCreateMain - Process WM_CREATE message//LRESULT DoCreateMain (HWND hWnd, UINT wMsg, WPARAM wParam, LPARAM lParam) {

EnableGestures (hWnd, TGF_GID_ALL, TGF_SCOPE_PROCESS);return 0;

}

The WM_GESTURE MessageSent to a window when a gesture is recognized by the system

wParam Contains gesture command (Pan, Scroll, Select…)lParam Contains handle to gesture information

If WM_GESTURE not passed to DefWindowProc, you must call CloseGestureInfoHandle while processing the message

DefWindowProc handles WM_GESTURE by passing to parent windowDON’T forward WM_GESTURE to one of your child windows!

GetGestureInfoCalling GetGesutureInfo returns a GESTUREINFO structure

typedef struct tagGESTUREINFO { UINT cbSize; // Size of structure DWORD dwFlags; // GF_BEGIN/GF_INERTIA/GF_END DWORD dwID; // Gesture ID HWND hwndTarget; // Target Window handle POINTS ptsLocation; // Location of gesture DWORD dwInstanceID; // Reserved DWORD dwSequenceID; // Timestamp of Gesture ULONGLONG ullArguments; // Args. See next slide UINT cbExtraArguments; // Possible extra data} GESTUREINFO, *PGESTUREINFO;

GESTUREINFO dwFlagsdwFlags

GF_BEGIN Beginning of gestureGF_END End of gestureGF_INERTIA Indicates “Flick” has happened at end of Pan

Flags can be combined or zero

Multiple End flags can occurOne for each gesture recognized

GESTUREINFO ulArgumentsullArguments Depends on message

Notification: Interim Pan notificationsHIWORD () X delta from previous messageLOWORD () Y delta from previous message

Notification: GF_END + Pan or ScrollGID_SCROLL_ANGLE() Angle of gesture in radiansGID_SCROLL_DIRECTION() Direction of gesture (Up/Down/Left/Right)GID_SCROLL_VELOCITY() Speed in Pixels per second

WM_GESTURE messagesDouglas BolingPresident Boling Consulting Inc.

DEMO

AutoGestureSimple way to enable window scrolling by pan/flick gestures

Enable by adding Horizontal and/or Vertical scroll bars to your window

No need to handle WM_GESTURESimply respond to WM_VSCROLL and WM_HSCROLL messages

OS needs to be built with AutGesture component

Configuring AutoGestureWhile AutoGesture works ‘by default’ it can be configured

Used to customize drawing and presentation given pan and flick gestures

FunctionsSetWindowAutoGesture Configure gesture response and redraw

requirements for a windowGetWindowAutoGesture Query Autogesture settingsGetAnimateMessageInfo Get information about an AutoGesture

message

WAGINFO StructureGet/SetWindowAutoGesture use WAGINFO structure

typedef struct tagWAGI { size_t cbSize; // Size of structure DWORD dwFlags; // See next slide UINT nOwnerAnimateMessage; // ID of WM_USER msg or 0 UINT nAnimateStatusMessage; // ID of WM_USER msg or 0 HBRUSH hExtentBrush; // Reserved UINT nItemHeight; // Used for snapping to item UINT nItemWidth; // Used for snapping to item BYTE bHorizontalExtent; // Scroll ext past last item BYTE bVerticalExtent; // Scroll ext past last item} WAGINFO, *LPWAGINFO;

WAGINFO FlagsFlags

WAGIF_OWNERANIMATE Must set to enable custom responseWAGIF_VSCROLLABLE Enable vertical scrollingWAGIF_HSCROLLABLE Enable horizontal scrollingWAGIF_LOCKAXES Lock gestures to horizontal and verticalWAGIF_IGNOREPAN Don’t send Pan gesturesWAGIF_IGNORESCROLL Don’t send Scroll gestures

AutoGesture Custom MessagesOwnerAnimateMessage

WM_USER+x messages notifying window of gesture actionRedraw window when received. Data queried with GetAnimateMessageInfo()

AnimateStatusMessageWM_USER+x message notifying of gesture status change

WAG_STATUS_ANIMATION_START Start of animationWAG_STATUS_ANIMATION_END End of animationWAG_STATUS_ANIMATION_CHANGE Change from Pan to flick/scroll

Either message can be disabled by setting field to 0 in WAGINFO

The Physics EngineMakes movement look ‘natural’

Things bounce, spring back, and slow down naturally

How is this done?Physics!

The physics engine computes a series of point/time combinations to allow the application to animate movement in a natural looking way

Using the Physics EngineThe physics engine is controlled by 4 functions

CreatePhysicsEngine Create and initialize the engineQueryPhysicsEngine Query position and motion parameters

given a time

SetPhysicsEngineUserTime Set time for queryDestroyPhysicsEngine Destroy engine and free resources

Initializing the Physics Enginetypedef struct tagPHYSICSENGINEINIT{ DWORD cbSize; // Size of the structure DWORD dwEngineType; // Reserved, set to zero DWORD dwFlags; // Specifies system or User time base LONG lInitialVelocity; // Initial velocity in Pixels/sec DWORD dwInitialAngle; // Angle in radians BYTE bXAxisMovementMode; // Horz movement action (Decelerate only opt) BYTE bXAxisBoundaryMode; // Horz boundary action. Bounce/rubber band/None BYTE bYAxisMovementMode; // Vert movement action (Decelerate only opt) BYTE bYAxisBoundaryMode; // Vert boundary action. Bounce/rubber band/None RECT rcBoundary; // Entire movement rectangle SIZE sizeView; // Rectangle of visible region POINT ptInitialPosition; // Initial position in region SIZE sizeItem; // Size of item. Used to snap to item boundary} PHYSICSENGINEINIT;

HRESULT CreatePhysicsEngine(PHYSICSENGINEINIT* pEngineInit, HPHYSICSENGINE* phResult);

Using the Physics EngineInitialize Physics engine with values returned by Flick (Scroll) gesture

Units match so easily done

Create timer to send messages back to window to animateSetTimer()

When WM_TIMER message received, query physics informationUse QueryPhysicsEngine()Redraw with returned Physics position data

When physics status indicates complete, kill timer message

Using the Physics Engineif (gi.dwFlags & (GF_INERTIA | GF_END)) { PHYSICSENGINEINIT pei;  memset (&pei, 0, sizeof (pei)); pei.cbSize = sizeof (pei);

pei.dwFlags = 0;pei.lInitialVelocity = GID_SCROLL_VELOCITY (gi.ullArguments);pei.dwInitialAngle = GID_SCROLL_ANGLE (gi.ullArguments);pei.bXAxisMovementMode =

PHYSICSENGINE_MOVEMENT_MODE_DECELERATE; pei.bXAxisBoundaryMode =

PHYSICSENGINE_BOUNDARY_MODE_RUBBERBAND;pei.bYAxisMovementMode =

PHYSICSENGINE_MOVEMENT_MODE_DECELERATE; pei.bYAxisBoundaryMode =

PHYSICSENGINE_BOUNDARY_MODE_RUBBERBAND;pei.rcBoundary = g_rcWork;pei.sizeView.cx = rectImg.right;pei.sizeView.cy = rectImg.bottom;pei.ptInitialPosition.x = gi.ptsLocation.x;pei.ptInitialPosition.y = gi.ptsLocation.y;pei.sizeItem.cx = 1;pei.sizeItem.cy = 1;

  hr = CreatePhysicsEngine (&pei, &g_hPEAnimate);SetTimer (hWnd, ID_ANIMATE, 100, NULL);

}}

WM_TIMER with Physics Enginecase WM_TIMER: if (wParam == ID_ANIMATE) { // Query physics engine

PHYSICSENGINESTATE state;memset (&state, 0, sizeof (state));state.cbSize = sizeof (state);QueryPhysicsEngine (g_hPEAnimate, &state);

// Update imageMoveImg (hWnd, state.ptPosition);

// If animation complete, clean upif (state.fComplete){

DestroyPhysicsEngine (g_hPEAnimate);KillTimer (hWnd, 1);

} } break;

Managed SupportNo direct support through managed class library today

AutoGesture works by default

Physics is a simply P/Invoke

Managed framework sample available on codeplex

SummaryGesture support fairly straightforward

Model consistent with desktop API

Autogesture gives simple gesture support for most controls by defaultCustomization available for controls that need more than simply scrolling

Physics engine pretty simple to useCool effects

QuestionsDouglas Bolingwww.bolingconsulting.comdboling @ bolingconsulting.com

Track Resourceshttp://www.WindowsEmbedded.comhttp://msdn.microsoft.com/en-us/windowsembeddedhttp://social.msdn.microsoft.com/Forums/en-US/category/embeddedwindows/http://social.msdn.microsoft.com/Forums/en-US/category/windowsembeddedcompacthttps://connect.microsoft.com/windowsembeddedce

Related ContentBreakout Sessions

WEM201 | Discover Windows Embedded Standard 7 as Your Next Application PlatformWEM301 | Deploying Windows Embedded Standard 7 with StyleWEM302 | Explore the Multimedia Potential of Windows Embedded Standard 7WEM303 | Gamechanger: Using Microsoft Silverlight for Windows Embedded to Create an Amazing Embedded UIWEM305 | How to Choose a Windows Embedded Operating SystemWEM306 | Using the Sensor & Location API on Windows Embedded Standard 7 to Create Exciting Connected ApplicationsWEM307 | Windows Embedded Compact: New Tools and Developer StoryWEM308 | Windows Embedded Overview: Demos of the Latest and Upcoming ReleasesWEM309 | Programming Microsoft Silverlight for Windows Embedded Using Microsoft .NET

Interactive SessionsWEM01-INT | Build a Secure Device with Windows Embedded Standard 7WEM02-INT | Delivering Flexible Peripheral Support for Point of SaleWEM03-INT | How Windows Embedded Solutions Help to Protect the EnvironmentWEM05-INT | What a Desktop Developer Needs to Know to Develop for Windows EmbeddedWEM06-INT | Windows Embedded Compact CompeteWEM07-INT | Server Appliances with Windows Embedded ServersWEM08-INT | Roundtable: Windows Embedded @ Tech·Ed 2011 - Tell Us What You Want to Learn

Related ContentHands-on Labs

WEM01-HOL | Build Your Own Embedded SystemWEM04-HOL | Porting Third-Party Drivers into Image Configuration Editor

Product Demo Stations (all on Windows Embedded booth)TLC-46 | Get Your Hands on Windows EmbeddedTLC-47 | Powered by Windows Embedded POSReady – Touch ScreenTLC-48 | The Intel® Intelligent Digital Signage Proof of ConceptTLC-49 | Windows Embedded AutomotiveTLC-50 | Windows Embedded Device Showcase

Resources

www.microsoft.com/teched

Sessions On-Demand & Community Microsoft Certification & Training Resources

Resources for IT Professionals Resources for Developers

www.microsoft.com/learning

http://microsoft.com/technet http://microsoft.com/msdn

Learning

Complete an evaluation on CommNet and enter to win!

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st

http://northamerica.msteched.com/registration

You can also register at the

North America 2011 kiosk located at registrationJoin us in Atlanta next year

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to

be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

JUNE 7-10, 2010 | NEW ORLEANS, LA