Alternative smartphone interactions
-
Upload
michaelpoltorak -
Category
Technology
-
view
232 -
download
0
description
Transcript of Alternative smartphone interactions
![Page 1: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/1.jpg)
@michaelpoltorak
Michael Poltorak Nielsen@michaelpoltorak
Alternative SmartphoneInteractions Based on Sensor Data
![Page 2: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/2.jpg)
@michaelpoltorak
![Page 3: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/3.jpg)
@michaelpoltorak
I work at Statsbiblioteket
![Page 4: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/4.jpg)
@michaelpoltorak
What did he say?
![Page 5: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/5.jpg)
@michaelpoltorak
It’s a library
![Page 6: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/6.jpg)
@michaelpoltorak
Statsbiblioteket, in Aarhus, Denmark
University research library and national libraryHuge search index (solr, 1 BN records)Keeps a physical copy of all Danish stuff (books, newspapers, flyers, etc.) for cultural, preservational purposesHarvests .dk domain for Danish web archiveStores Danish radio and TV (daily 1 TB, so far 1.5 PB)Newspaper digitisation: 32 M pages, 600 TB
![Page 7: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/7.jpg)
@michaelpoltorak
Loading more slides
DONE
![Page 8: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/8.jpg)
@michaelpoltorak
Mobile javascript sensor API’s and events
![Page 9: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/9.jpg)
@michaelpoltorak
![Page 10: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/10.jpg)
@michaelpoltorak
Touch screen. Touch events All browsers
Location. Navigator.geolocation All browsers
Motion. DeviceOrientation, DeviceMotion All browsers
getUserMedia/StreamAPI FF, Ch for Android, Bb, OM
Speech. Web Speech API Ch for Android, iOS < 7
Proximity (UserProximityEvent, DeviceProximityEvent) FF
Ambient Light API (device's light sensor) FF
Battery Status API FF
![Page 11: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/11.jpg)
@michaelpoltorak
Introducing the
Demo search engine
![Page 12: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/12.jpg)
@michaelpoltorak
A closer look at:touch,
device motion,web speech API
![Page 13: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/13.jpg)
@michaelpoltorak
TouchThree types:
touchstart, touchend, touchmoveno of fingers, position on screen, drag(how long)
IE: pointer events (mouse + pen + touch)
![Page 14: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/14.jpg)
@michaelpoltorak
TouchEvent {type: "touchmove", //fired when draggingtarget: DOM element
touches: TouchList [// size: no fingers{
clientX: 97, //position on screenclientY: 259,radiusX: 11.5, //finger shaperadiusY: 11.5
}]}
![Page 15: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/15.jpg)
@michaelpoltorak
1
$1 Touch gesture recognizer
![Page 16: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/16.jpg)
@michaelpoltorak
var result = ui.recogniser.Recognize(touchPoints, false); //do touchPoints resemble a $1-gesture?
if (result.Score > 0.8) { touchName = (result.Name); touch.command(result.Name);
}
![Page 17: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/17.jpg)
@michaelpoltorak
Device orientation event:
window.addEventListener('deviceorientation', function(eventData) {
eventData.gamma; //left-to-right tilteventData.beta; //front-to-back tilteventData.alpha; //compass direction
}, false);
IE10: Windows 8 Sensor API
![Page 18: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/18.jpg)
@michaelpoltorak
![Page 19: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/19.jpg)
@michaelpoltorak
3d gestures from device orientation event data:
if (delta_gamma > 30) { //positive gamma changereturn "TILT_RIGHT";
}if (delta_gamma < -30) {//negative gamma change
return "TILT_LEFT";}
![Page 20: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/20.jpg)
@michaelpoltorak
Web Speech API //Android Chrome only
var recognizer = new webkitSpeechRecognition(); recognizer.onresult = function (data) {
var words,list; list=data.results.SpeechRecognitionResultList;words =[list.length - 1][0].transcript;
console.log("Heard words ", words); };
![Page 21: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/21.jpg)
@michaelpoltorak
Search engine demoNow with:
custom touch gestures3d gestures
speech
![Page 22: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/22.jpg)
@michaelpoltorak
So, we can augment the traditional paradigm
![Page 23: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/23.jpg)
@michaelpoltorak
But is it useful?
![Page 24: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/24.jpg)
@michaelpoltorak
Touch gestures: difficult on small/scrolling screen, no common set of gestures. 3D gestures: more intuitive, but hard to do. No common language, hard to code (noise, hard to keep apart).Speech: easy to speak, but hard to decode. Mistakes will occur. Disturbing other people. You look foolish speaking to your device.
![Page 25: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/25.jpg)
@michaelpoltorak
Very important:Evaluate any new features,
test on users,and, as always:
![Page 26: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/26.jpg)
@michaelpoltorak
Kill your darlings
![Page 27: Alternative smartphone interactions](https://reader033.fdocuments.net/reader033/viewer/2022052901/55643e58d8b42adb258b547d/html5/thumbnails/27.jpg)
@michaelpoltorak
Thank you!
@michaelpoltorakmichaelpoltorak.dk