ITCamp 2012 - Tim Huckaby - Keynote

itcampro @ itcamp12 # Premium conference on Microsoft technologies

The Future of Software Now: NUI - A New Genre of User

Experience

Tim Huckaby Microsoft MVP & RD

InterKnowlogy Chairman/Founder

Actus Interactive Software CEO


Mobile &

Development ITCamp 2012 sponsors


NUI: The Natural User Interface

• Common Gestures of Touch

• Physical Object Interaction

• Virtual Object Interaction

Multi-Touch Capable

• Motion Based Interface

• Tracking of full-body movement, facial expression, and voice with precision

Gesture Capable

• Multi-array Microphone

• Uniquely Distinguish Audio & Voices Voice Capable

• Conscious and non-conscience interfaces

• Non Invasive

Neural Capable


Tim Huckaby Founder / Chairman, InterKnowlogy Founder / CEO, Actus Software Microsoft RD & MVP


A little bit about me and why I am here…


About Tim, InterKnowlogy & Actus…

InterKnowlogy (www.InterKnowlogy.com) • Tim Huckaby, Founder/Chairman ([email protected])

– 13+ year old services firm – Custom Application Development

– focused on building NUI solutions

– Web, mobile, and desktop

– windows 7/8, Surface, Windows phone 7, azure, Kinect,

•Largest Client: Microsoft

Actus Software (www.Actus-Software.com) • Tim Huckaby, Founder/CEO ([email protected])

•Software Product Company

– Interactive Kiosk and Active Digital Signage

– Multi-Touch, Gesture, and Voice Driven Solutions


We have come a long way…

The “Green-Screen”


Image courtesy of Computer History Museum


Touch Capable Computing Devices Are Not New

Past Future (Now) Present

1972: PLATO IV Touch Screen Terminal

The Samsung SUR40 for Microsoft Surface

Windows 8 - Metro

Apple iPad


What is Relatively New is:

Inexpensive Touch Capable, Non

Proprietary Hardware

A Windows OS designed for Multi-

Touch

A Consistent Multi-Touch API for .NET

across all Windows Hardware/Software

Platforms

Developer Productivity in Touch

Capable Apps

The Forecast of Multi-Touch Hardware

Capability in Every New Computing

Device in the Short Term


• HP TouchSmart TM2

• Rates in Windows as 5.0

• Price in October-2010: Starting from: $1499.99*

• Price in March-2011: Starting from: $799.99*

• Price Today (HP TouchSmart 320m ): $579.99

About the Windows 7 Computer I am Using Today…


HTML 5 Touch Events Spec, Version 2

W3C Editor's Draft 14, November 2011

Defines a set of low-level events that represent

one or more points of contact with a touch-

sensitive surface

Changes of those points with respect to the

surface and any DOM elements displayed upon

it (e.g. for touch screens) or associated with it

(e.g. for drawing tablets without displays).

Addresses pen-tablet devices, such as drawing

tablets, with consideration toward stylus

capabilities.

Source: https://dvcs.w3.org/hg/webevents/raw-file/tip/touchevents.html


HTML 5 Touch Examples



What is also Relatively New: Multi-Touch User Interfaces Combined with Object

recognition of Physical Objects

The Microsoft Surface is expensive

because it has 5 cameras looking up

that do Object Recognition

Based on a “byte tag” (not a QR

code)

Object Recognition in Windows

The Use Cases are limitless…


Surface Gaming - Craps



Windows 8 - Designed for Multi-Touch

Metro - Perfect for Gesture


Windows 8:

• Metro mode / Desktop Mode

• Actus Interactive Kiosk – Metro

• Semantic Zoom



• Learning made Fun / Entertaining

• Proven as a learning tool beyond just

Children

• Computer based Games are a significant part

of popular culture

• Educational Video Game Curriculums

– MIT, UC Irvine, Minnesota, Mich State, etc.

NUI: Games as a Tool for Learning


NASA: Mars Bound



World's Fastest-Selling Consumer Electronics Device

… a Future Beyond Gaming

Combination of RGB camera, depth sensor and multi-array microphone

RGB camera delivers three basic color components

Depth sensors “sees” in 3-D

Microphone locates voices by sound and excludes ambient noise

The sensor provides eyes and ears - but it needs a brain

Raw data from the Kinect is just “a whole bunch of noise that someone needs to

take and turn into signal”

Our software finds the signal

Software makes all the magic possible

Skeletal Tracking

Face, Gesture Recognition

Audio Echo cancellation

Audio Beam Forming

Speech Recognition

Machine Learning

What is Kinect?

http://en.wikipedia.org/wiki/File:Guinness_World_Records_logo.png


Kinect for Windows

Combination of RGB camera, depth sensor and multi-array microphone

RGB camera delivers three basic color components

Depth sensors “sees” in 3-D

Microphone locates voices by sound and excludes ambient noise

Consumer Priced: $249.99

Tracks full-body movement, facial expression, and voice with precision

Kinect sensor bar scans and monitors 1 million points on the body

Kinect for Windows SDK

Skeletal Tracking

Face, Gesture Recognition

Audio Echo cancellation

Audio Beam Forming

Speech Recognition

Machine Learning


The Good News: • The Kinect SDK for Windows

The Not So Good News: • Windows Only Device; K4W does not work on your

Xbox; you cannot build apps for 18 million Xbox Kinect owners with it.

XBOX Kinect


The XBOX Kinect for Windows

Touch-less Operating Room



Neural Interfaces


Conscious and Non-Conscious Interfaces

• High fidelity EG acquisition device • No conductive gel, etc. • Non-invasive • Use Case Examples:

• Conscious: mind manipulation of objects • Non-Conscious: Ranking music

• First target market: gaming • $300!!!


Direct API access to all Emotiv detection suite inputs and controls,

including training, profile management and gyro outputs

Program in C++ and compatible languages, .NET (VB.NET, C#), Java

Developer plug-in available for Unity3D

Active developer community (exclusive Developer user forum)

C++ Header files and Import Libraries

…which means…

Wrap the SDK in managed C++ thereby exposing a the API to a C#

application

That is very powerful…

Emotiv Software Development Kit (SDK)


We are within a decade of building the Holodeck

“The Ultimate NUI Application”


• We are at the cusp of a revolution in software & hardware

• Hardware and software platforms will ultimately synchronize

• Using Natural User Interface (NUI) technologies improve the User Experience

• The hardware, tools, plumbing and platforms for NUI computing are powerful and available now.

– …and will just continue to get better, cheaper, powerful and more available

• Within a realistic time frame NUI computing will be the norm

• The concepts of NUI change the way we design and build software

Summary



HoloDesk - Direct 3D Interactions with a Situated See-Through Display


Tim Huckaby

• More info on InterKnowlogy:

www.InterKnowlogy.com Silverlight.InterKnowlogy.com www.vimeo.com/InterKnowlogy

• More info on Actus Software:

www.Actus-Software.com

• About Tim Huckaby… – Founder/Chairman, InterKnowlogy – Founder/Chairman, Actus Software – Microsoft® Regional Director – Southern

California – Microsoft® .NET Partner Advisory Council

Founder / Member – Microsoft® MVP - .NET – INETA Speaker – International .NET Users

Group Association – Board Member: Association for Competitive

Technology – Board Member: Dev Connections – Author / Speaker

• Contact me: Tim Huckaby

E-mail: [email protected]

[email protected]

• Phone:

Office: 760-444-8640 Mobile: 619 990 9200

Blog: http://team.interknowlogy.com/blogs/timhuckaby

FaceBook: http://www.facebook.com/Tim.Huckaby.Carlsbad

LinkedIn: http://www.linkedin.com/in/timhuckaby

Twitter: @TimHuckaby


Mobile &

Development ITCamp 2012 sponsors


Mobile &

Development

Agenda


Mobile &

Development


Q & A

ITCamp 2012 - Tim Huckaby - Keynote

Technology

Transcript of ITCamp 2012 - Tim Huckaby - Keynote