@michaelpoltorak

Michael Poltorak Nielsen@michaelpoltorak

Alternative SmartphoneInteractions Based on Sensor Data

@michaelpoltorak

@michaelpoltorak

I work at Statsbiblioteket

@michaelpoltorak

What did he say?

@michaelpoltorak

It’s a library

@michaelpoltorak

Statsbiblioteket, in Aarhus, Denmark

University research library and national libraryHuge search index (solr, 1 BN records)Keeps a physical copy of all Danish stuff (books, newspapers, flyers, etc.) for cultural, preservational purposesHarvests .dk domain for Danish web archiveStores Danish radio and TV (daily 1 TB, so far 1.5 PB)Newspaper digitisation: 32 M pages, 600 TB

@michaelpoltorak

Loading more slides

DONE

@michaelpoltorak

Mobile javascript sensor API’s and events

@michaelpoltorak

@michaelpoltorak

Touch screen. Touch events All browsers

Location. Navigator.geolocation All browsers

Motion. DeviceOrientation, DeviceMotion All browsers

getUserMedia/StreamAPI FF, Ch for Android, Bb, OM

Speech. Web Speech API Ch for Android, iOS < 7

Proximity (UserProximityEvent, DeviceProximityEvent) FF

Ambient Light API (device's light sensor) FF

Battery Status API FF

@michaelpoltorak

Introducing the

Demo search engine

@michaelpoltorak

A closer look at:touch,

device motion,web speech API

@michaelpoltorak

TouchThree types:

touchstart, touchend, touchmoveno of fingers, position on screen, drag(how long)

IE: pointer events (mouse + pen + touch)

@michaelpoltorak

TouchEvent {type: "touchmove", //fired when draggingtarget: DOM element

touches: TouchList [// size: no fingers{

clientX: 97, //position on screenclientY: 259,radiusX: 11.5, //finger shaperadiusY: 11.5

}]}

@michaelpoltorak

1

$1 Touch gesture recognizer

@michaelpoltorak

var result = ui.recogniser.Recognize(touchPoints, false); //do touchPoints resemble a $1-gesture?

if (result.Score > 0.8) { touchName = (result.Name); touch.command(result.Name);

}

@michaelpoltorak

Device orientation event:

window.addEventListener('deviceorientation', function(eventData) {

eventData.gamma; //left-to-right tilteventData.beta; //front-to-back tilteventData.alpha; //compass direction

}, false);

IE10: Windows 8 Sensor API

@michaelpoltorak

@michaelpoltorak

3d gestures from device orientation event data:

if (delta_gamma > 30) { //positive gamma changereturn "TILT_RIGHT";

}if (delta_gamma < -30) {//negative gamma change

return "TILT_LEFT";}

@michaelpoltorak

Web Speech API //Android Chrome only

var recognizer = new webkitSpeechRecognition(); recognizer.onresult = function (data) {

var words,list; list=data.results.SpeechRecognitionResultList;words =[list.length - 1][0].transcript;

console.log("Heard words ", words); };

@michaelpoltorak

Search engine demoNow with:

custom touch gestures3d gestures

speech

@michaelpoltorak

So, we can augment the traditional paradigm

@michaelpoltorak

But is it useful?

@michaelpoltorak

Touch gestures: difficult on small/scrolling screen, no common set of gestures. 3D gestures: more intuitive, but hard to do. No common language, hard to code (noise, hard to keep apart).Speech: easy to speak, but hard to decode. Mistakes will occur. Disturbing other people. You look foolish speaking to your device.

@michaelpoltorak

Very important:Evaluate any new features,

test on users,and, as always:

@michaelpoltorak

Kill your darlings

@michaelpoltorak

Thank you!

@michaelpoltorakmichaelpoltorak.dk