UbiCom Book Slides
description
Transcript of UbiCom Book Slides
![Page 1: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/1.jpg)
UbiCom Book Slides
1Ubiquitous computing: smart devices, environments and interaction
Chapter 5
Human Computer Interaction
Stefan Poslad
http://www.eecs.qmul.ac.uk/people/stefan/ubicom
![Page 2: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/2.jpg)
HCI: Overview
This part (a) first discusses:• What is Human Computer Interaction or Interfaces (HCI) and
why we need good HCI for human interactive systems?• What is a sub-type of HCI, implicit HCI (iHCI), how is it
differentiated from conventional explicit HCI (eHCI) and why do we need this to enhance pervasive computing?
• How to use eHCI in some common types of device?• How to use iHCI in (mobile and static) devices that are not
permanently attached to humans?• How to use iHCI in (mobile and static) devices that accompany
humans through being surface-mounted (wearable) or embedded (implants)
Ubiquitous computing: smart devices, environments and interaction 2
![Page 3: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/3.jpg)
Chapter 5 Related Links
• iHCI is a type of context-awareness for the human environment (Chapter 7)
• Human behaviour models of intelligence (Chapter 8)• Social & other consequences of making devices more
human and more intelligent (Chapter 12)
Ubiquitous computing: smart devices, environments and interaction 3
![Page 4: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/4.jpg)
HCI: OverviewThe slides for this chapter are also expanded and split into
several parts in the full pack
Part A: eHCI Use in some common smart device types
Part B iHCI for accompanied smart devices
Part C: iHCI for wearable & implanted smart devices
Part D: Human Centred Design
Part E: User Models and iHCI Design
Ubiquitous computing: smart devices, environments and interaction 4
![Page 5: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/5.jpg)
HCI: Overview
• HCI, eHCI & iHCI • eHCI use in 4 Widely Used Devices• iHCI use in accompanied smart devices• iHCI use in wearable and implanted smart devices• Human Centred Design (HCD)• User Models: Acquisition & Representation• iHCI Design
Ubiquitous computing: smart devices, environments and interaction 5
![Page 6: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/6.jpg)
6Ubiquitous computing: smart devices, environments and interaction
Mobile Devices
Handheld WIMPS
Usability
Basic Device eHCI
Input
User Modelling
Hidden iHCI
User Context
Games Console
Wearables & Implants
Telepresence
Smart Devices
iHCI with Devices Smart Services
Affective
HCD
Multi-modal
WIMPS
Organic
Touch-screen
Clothes
Tangible Gesture
Reflective
VR&AR
HUD
Neural Implants
Interaction Design
VRD
Auditory
Natural Language
Personalisation
(in)direct
iHCI
EyetapVideo Recorder
Output
Goals versus Situation
stereotypeSofttap multitap T9 etc
Design patterns
![Page 7: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/7.jpg)
HCI: Introduction• Term HCI, widely used, since onset of Personal Computing
era in 1980s. • However groundwork for field of HCI started earlier, during
onset of the industrial revolution • Tasks became automated and powered-assisted• -> triggers an interest in studying human-machine
interaction • Some tasks require little human interaction during
operation, e.g., clothes-, dish- washing etc• Other tasks are very interactive, e.g., face washing,
playing the violin, etc
7Ubiquitous computing: smart devices, environments and interaction
![Page 8: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/8.jpg)
H,C & IBasic concepts of HCI are:• Humans
• Computers / devices
• Interaction
8Ubiquitous computing: smart devices, environments and interaction
![Page 9: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/9.jpg)
HCI: Motivation
• Machines (systems) aid human performance, but systems that interact poorly with humans will be a poor human aid.
• Need design models & process that are (user) interactive• The motivation for HCI is clear; to support more effective
use (Dix, 2004a) in three ways– Useful:– Usable:– Be used:
9Ubiquitous computing: smart devices, environments and interaction
![Page 10: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/10.jpg)
HCI: Usability vs. Usefulness
• Success of a product depends largely on ?• • Summarised as Heckel's law and Heckel's inverse
law:– Heckel’s law:– Heckel’s inverse law:
• What this law expresses ?
Ubiquitous computing: smart devices, environments and interaction 10
![Page 11: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/11.jpg)
Explicit HCI (eHCI)
• eHCI design: explicit interaction during a device’s normal operation.
• What are the Dominant eHCI UIs
Pure eHCI • Context-free
• Focus on H2C (Human-to-Computer) Interaction
Ubiquitous computing: smart devices, environments and interaction 11
![Page 12: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/12.jpg)
eHCI versus Natural Interaction
• Natural interaction
• Natural interaction and familiarity and expertise
• Familiarity with use of tool is cultural and subjective
• Note also Natural Interaction linked to use of iHCI
Ubiquitous computing: smart devices, environments and interaction 12
![Page 13: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/13.jpg)
iHCI
• Concept of implicit HCI (iHCI)• Proposed by Schmidt (2000)
– Defined as “an action, performed by the user that is not primarily aimed to interact with a computerized system but which such a system understands as input”.
• Our definition of iHCI bit different: – inputs with an implicit or implied context,
Ubiquitous computing: smart devices, environments and interaction 13
![Page 14: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/14.jpg)
iHCI
• iHCI is more about C2H (Computer to Human) Interaction• iHCI assumes Chas a certain Model of H user• Model of H used as additional input• Need to share implicit context between human and system• Implicit interaction naturally supports hidden device design.
Ubiquitous computing: smart devices, environments and interaction 14
![Page 15: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/15.jpg)
eHCI + iHCI or iHCI vs eHCI
• E.g.?? . • eHCI, usability design?
• Alternative iHCI design?
• Shift from eHCI design to also include iHCI design will be a key enabler for effective UbiCom systems
Ubiquitous computing: smart devices, environments and interaction 15
![Page 16: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/16.jpg)
iHCI: Challenges
• Complex to accurately and reliably determine user context. Why?
Ubiquitous computing: smart devices, environments and interaction 16
![Page 17: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/17.jpg)
Overview
• HCI, eHCI & iHCI • eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices• iHCI use in wearable and implanted smart devices• Human Centred Design (HCD)• User Models: Acquisition & Representation• iHCI Design
Ubiquitous computing: smart devices, environments and interaction 17
![Page 18: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/18.jpg)
How Device Interfaces & Interaction Varies
Devices can be characterized according to?:
Ubiquitous computing: smart devices, environments and interaction 18
![Page 19: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/19.jpg)
UI and HCI Designs for 4 Common Devices
• PC• Mobile Phone• Games Console but many sub-types• TV / Projectors
• How does the UI and HCI design differ between these?
Ubiquitous computing: smart devices, environments and interaction 19
![Page 20: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/20.jpg)
UI Type: Personal Computer Interface• ???
Ubiquitous computing: smart devices, environments and interaction 20
![Page 21: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/21.jpg)
PC UI use in Mobiles
• Using a conventional PC UI approach won’t be optimum for mobile computing & ubiquitous computing - need a different approach, Why?
21Ubiquitous computing: smart devices, environments and interaction
![Page 22: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/22.jpg)
UI Type: Mobile Device Interfaces
• PC / WIMPS models not so suitable for mobile (one handed) devices, Why not?
Ubiquitous computing: smart devices, environments and interaction 22
![Page 23: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/23.jpg)
Mobile Device Interface: Limited I/P
How to support mobile user and small size of input?
23Ubiquitous computing: smart devices, environments and interaction
![Page 24: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/24.jpg)
Mobile Device Interface: Limited O/P
How to overcome limited output?• Haptic interface use, e.g., vibration to signal incoming call• Maximising use of small screen: scrolling, switching screen• Peephole displays• Foldable displays• Filter information so receive and display less information,
e.g., using Personalisation(Chapter 7) Personal Agents (Chapter 8)
Ubiquitous computing: smart devices, environments and interaction 24
![Page 25: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/25.jpg)
UI Type: Games Console Interfaces
• Games consoles: an important driver and can contribute to UbiCom in a number of ways.
• Computer games have often acted as an incubator for many innovations driving computing. How?
• Many different types of Games Console Interface
Ubiquitous computing: smart devices, environments and interaction 25
![Page 26: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/26.jpg)
Games Console Interfaces: D-pad
Ubiquitous computing: smart devices, environments and interaction 26
• How does the D-pad controller work?
![Page 27: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/27.jpg)
Games Console Interfaces: 3D Gesture-Based
Ubiquitous computing: smart devices, environments and interaction 27
• How does the 3D Gesture-Based controller work?
• Use of MEMS/ Sensors (Chapter 7)
• Use of gesture recognition (see later)
![Page 28: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/28.jpg)
UI Type: Control (Panel) Interfaces
• Different Types of remote controllers depending on how remote the controller is:
• User approx. co-located with device being controlled
• User not co-located with device being Controlled
Ubiquitous computing: smart devices, environments and interaction 28
![Page 29: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/29.jpg)
UI Type: Localised Remote Control Interfaces
Characteristics• Input controller and device separation
• Input device interfaces
• Wireless link between input control device and device
Ubiquitous computing: smart devices, environments and interaction 29
![Page 30: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/30.jpg)
UI Type: Localised Remote Control Interfaces
• But profusion of remote control devices which have overlapping features
• Is it necessary to have a specialised controller per consumer device?
• Problems?
• How to solve this?
Ubiquitous computing: smart devices, environments and interaction 30
![Page 31: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/31.jpg)
31Ubiquitous computing: smart devices, environments and interaction
Radio
17:38
1
GHI4
PQR7
ABC2
JKL5
STU8
DEF3
MNO6
WXYZ9
* 0 + #
Play channel
Record channel Channel 4
Radio TV DVD-W
Channel 5
![Page 32: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/32.jpg)
Localised Remote Control Interface Design
• Instructors can add more detail about the discussion and design of universal controller here or delete this slide.
• (Section 5.2.5)
Ubiquitous computing: smart devices, environments and interaction 32
![Page 33: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/33.jpg)
Overview
• HCI, eHCI & iHCI• eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices • iHCI use in wearable and implanted smart devices• Human Centred Design (HCD)• User Models: Acquisition & Representation• iHCI Design
Ubiquitous computing: smart devices, environments and interaction 33
![Page 34: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/34.jpg)
iHCI use in Accompanied Smart Devices: Topics
• Single vs. Multi-Modal Visual Interfaces• Gesture Interfaces• Reflective versus Active Displays• Combining Input and Output User Interfaces
– ???
• Auditory Interfaces• Natural Language Interfaces
Ubiquitous computing: smart devices, environments and interaction 34
![Page 35: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/35.jpg)
Single vs. Multi-Modal Visual Interfaces
• Mode of human interaction uses human senses? Which
• Interactive ICT systems have modalities that mimic human senses. What?
Ubiquitous computing: smart devices, environments and interaction 35
![Page 36: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/36.jpg)
Computer input & output modalities
36Ubiquitous computing: smart devices, environments and interaction
Computer Sensor
Computer
Output
1
1
HumanInterface
![Page 37: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/37.jpg)
Single vs. Multi-Modal Visual Interfaces
Many interactive ICT systems use single visual mode of output interaction. Problems?
Solutions?
Ubiquitous computing: smart devices, environments and interaction 37
![Page 38: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/38.jpg)
Multi-Modal Interaction Design: challenges
Integrating multiple modes is complex. Why?
Ubiquitous computing: smart devices, environments and interaction 38
![Page 39: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/39.jpg)
Multi-Modal Interaction: Design
Two main approaches• Data for each modality can be processed separately, then
combined at the end.
• Data for each modality can be processed & combined concurrently
Ubiquitous computing: smart devices, environments and interaction 39
![Page 40: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/40.jpg)
Gesture Interfaces
What are Gestures?• Expressive, meaningful body motions • Involving physical movements. Which?
• With the intent of conveying meaningful information about interacting with the environment.
Ubiquitous computing: smart devices, environments and interaction 40
![Page 41: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/41.jpg)
Gesture Interfaces
• What are the main types of human gestures?
• How can gestures be sensed?
Ubiquitous computing: smart devices, environments and interaction 41
![Page 42: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/42.jpg)
Gesture Interfaces: Classification
Gestures can also be classified into • 2D versus 3D
• Contactful versus Contactless–
• Directly sensed versus indirectly sensed•
Ubiquitous computing: smart devices, environments and interaction 42
![Page 43: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/43.jpg)
Gesture Interfaces: Applications
• 1st basic contact based gesture interfaces?
• From the mid 2000s, contact less gestures being used in several types of games consoles, mobile phones, cameras, etc.
Ubiquitous computing: smart devices, environments and interaction 43
![Page 44: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/44.jpg)
Gesture Interfaces: Applications
• ?????
Ubiquitous computing: smart devices, environments and interaction 44
![Page 45: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/45.jpg)
Gesture: Rotate or flip handAction: Rotate or flip image
Gesture Interfaces: Applications
45Ubiquitous computing: smart devices, environments and interaction
![Page 46: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/46.jpg)
Gesture: tilt display awayAction: Menu selection moves up
Gesture Interfaces: Applications
46Ubiquitous computing: smart devices, environments and interaction
Navigation Options
Find cinemasFind RestaurantsFind CafesFind NewsagentFind Bookshop
Tilt Navigation Options
Find cinemasFind RestaurantsFind CafesFind NewsagentFind Bookshop
![Page 47: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/47.jpg)
Gesture: Two finger stretchAction: Stretch image
Gesture Interfaces: Applications
47Ubiquitous computing: smart devices, environments and interaction
![Page 48: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/48.jpg)
Gesture Interfaces: HCI->HPI->HHI->HCI
48Ubiquitous computing: smart devices, environments and interaction
Human to virtual device interaction
Human to physical artefact interaction
Human to human physical interaction
Human to human physical interaction triggers machine to machine interactions
![Page 49: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/49.jpg)
Gesture Design: Challenges
• ???.
Ubiquitous computing: smart devices, environments and interaction 49
![Page 50: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/50.jpg)
Reflective vs Active Displays
• Which is more pervasive today and which will be more pervasive in the future: paper or active display devices?
• What are inherent characteristics of paper versus active displays and how do these effect their ability to become truly pervasive?
Ubiquitous computing: smart devices, environments and interaction 50
![Page 51: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/51.jpg)
Reflective versus Active Displays
• Can we produce ICT displays that support more of the properties of physical paper?
• Display design mimics paper
• Epaper display design differs from actual paper
Ubiquitous computing: smart devices, environments and interaction 51
![Page 52: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/52.jpg)
ElectroPhoretic Displays or EPDs
52Ubiquitous computing: smart devices, environments and interaction
- - +- ++
Clear Fluid
Micro-Capsule
Positively charged while particles
Negatively charged white particles
Transparent Electrode
Charged Electrode
![Page 53: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/53.jpg)
Combining Input and Output User Interfaces
• UIs discussed so far, input devices are separated from the output devices
• State of the input is available as a visual cue only.• How can we combine / link input and output better?
Ubiquitous computing: smart devices, environments and interaction 53
![Page 54: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/54.jpg)
Touchscreen
What are touchscreens?• Displays where position of contact with screen is detected• Via pointed physical objects such as pens, fingers, etc• Events can then be generated for an associated visual
object at that position and • Associated actions can then be triggered.
Ubiquitous computing: smart devices, environments and interaction 54
![Page 55: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/55.jpg)
Touchscreen
• Touchscreen behaves as 2D, planar smart skin.• Wherever it is touched, a virtual object can be activated.• Types of touchscreens ?
– Resistive– Capacitive– Surface acoustic waves etc.
• Touch screen can behave as a:– soft control panel and user interface – that is reprogrammable– which can be customised to suit a range of applications and users
Ubiquitous computing: smart devices, environments and interaction 55
![Page 56: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/56.jpg)
Touchscreen: Benefits
What are the benefits?
These characteristics make them ideal for many workplaces and public spaces.
Ubiquitous computing: smart devices, environments and interaction 56
![Page 57: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/57.jpg)
Touchscreen: Applications• Touchscreens sed routinely in many applications & devices
– ??
• To ease use of pointing• To ease use of gestures• Single versus multiple finger gestures
Ubiquitous computing: smart devices, environments and interaction 57
![Page 58: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/58.jpg)
Tangible User Interface (TUI)
• (TUI) is a UI that augments the real physical world by coupling digital information to everyday physical objects and environments.
• Tangible user interfaces are also referred to as– passive real-world props, – graspable user interfaces,– manipulative user interfaces – embodied user interfaces
Ubiquitous computing: smart devices, environments and interaction 58
![Page 59: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/59.jpg)
Tangible User Interface (TUI)
How do Tangible Interfaces work?
• Attach micro sensors and actuators (Section 6.4) to physical objects
• Used as input devices to allow their manipulation to generate data streams in an output device or virtual view in a related virtual environment, (Section 6.2).
Ubiquitous computing: smart devices, environments and interaction 59
![Page 60: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/60.jpg)
Tangible User Interface (TUI)
• Taxonomy of TUIs based upon embodiment and metaphors – –
• Four types of embodiment can be differentiated– Full embodiment e.g.,??
– Nearby embodiment e.g. ??
– Environmental embodiment e.g., ???
– Distant embodiment, e.g., ???
Ubiquitous computing: smart devices, environments and interaction 60
![Page 61: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/61.jpg)
Tangible Bits Project
• Instructors can explain in more detail how this works or delete this slide
Ubiquitous computing: smart devices, environments and interaction 61
![Page 62: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/62.jpg)
DataTiles Project
• Allows users to manipulate data in form of tangible “tiles”• Combinations of data streams and functions make it
possible to create new applications62Ubiquitous computing: smart devices, environments and interaction
![Page 63: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/63.jpg)
DataTiles Project
• Instructors can explain in more detail how this works or delete this slide
Ubiquitous computing: smart devices, environments and interaction 63
![Page 64: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/64.jpg)
Organic Interfaces
• Similar to Tangible Interfaces
• 3 characteristics which characterize organic UIs.
• Typically use Organic Light-Emitting Diode (OLED) type materials
Ubiquitous computing: smart devices, environments and interaction 64
![Page 65: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/65.jpg)
Organic Interfaces
• Instructors can add more detail about this or delete this slide
Ubiquitous computing: smart devices, environments and interaction 65
![Page 66: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/66.jpg)
Auditory Interfaces
What are the Benefits?
Design challenges?•
Ubiquitous computing: smart devices, environments and interaction 66
![Page 67: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/67.jpg)
Auditory Interfaces: Non-Speech Based
2 basic auditory interfaces: • Speech based• Non-speech based
Non-speech auditory interfaces:• ?????
Ubiquitous computing: smart devices, environments and interaction 67
![Page 68: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/68.jpg)
Auditory Interfaces: Speech Based
• ????.
Ubiquitous computing: smart devices, environments and interaction 68
![Page 69: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/69.jpg)
Natural Language Interfaces
• Natural language interaction with machines can occur in a variety of forms. Which?
Ubiquitous computing: smart devices, environments and interaction 69
![Page 70: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/70.jpg)
Natural Language Interfaces
• Generally, interaction can be more easily processed and understood if it defined using an expressive language that has a well-defined syntax or grammar and semantics– requires that users already know the syntax.
Benefits in using NL in HCI?
Ubiquitous computing: smart devices, environments and interaction 70
![Page 71: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/71.jpg)
Natural Language Interfaces: Challenges
• What are the challenges in using NL Interfaces (NLI)?
Ubiquitous computing: smart devices, environments and interaction 71
![Page 72: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/72.jpg)
Overview
• HCI, eHCI & iHCI• eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices• iHCI use in wearable and implanted smart
devices • Human Centred Design (HCD)• User Models: Acquisition & Representation• iHCI Design
Ubiquitous computing: smart devices, environments and interaction 72
![Page 73: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/73.jpg)
Hidden UI via Wearable and Implanted Devices
• In the Posthuman model, technology can be used to extend a person's normal conscious experience and sense of presence, across space and time.
There are 3 types of post-human technology: • Accompanied
– e.g. ???
• Wearable– e.g., ???
• Implants – E.g., ???
Ubiquitous computing: smart devices, environments and interaction 73
![Page 74: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/74.jpg)
Wearable computers
• Wearable interfaces include a combination of ICT devices & modalities
• Wearable computers are especially useful when?
• Focus is on multi-modal interaction which includes visual interaction.
Ubiquitous computing: smart devices, environments and interaction 74
![Page 75: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/75.jpg)
Wearable computers
• Visual modal systems are divided according to how humans interact with the system: – ??
• Visual interaction can be classified into– command– non-command interfaces.
• Non-command vision-based (human motion) analysis systems generally have four stages:
• motion segmentation
• object classification
• tracking
• interpretation.
Ubiquitous computing: smart devices, environments and interaction 75
![Page 76: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/76.jpg)
Wearable Computer: WearComp and WearCam
• Many researchers contributed to the advancement of wearable computing
• Perhaps the most important Pioneer of Wearable Computing is Steve Mann –
• His 1st early main application focussed on recording personal visual memories that could be shared with other via the Internet.
Ubiquitous computing: smart devices, environments and interaction 76
![Page 77: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/77.jpg)
Wearable Computer: WearComp and WearCam
Ubiquitous computing: smart devices, environments and interaction 77
Photo courtesy of Wikimedia Commons, http://en.wikipedia.org/wiki/Wearable_computing)
![Page 78: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/78.jpg)
Wearable computing: Mann’s definition
Mann (1997): 3 criteria to define wearable computing. • Eudaemonic criterion
• Existential criterion
• Ephemeral criterion
Ubiquitous computing: smart devices, environments and interaction 78
![Page 79: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/79.jpg)
Wearable computing: Types
• Some different type of wearable computers??
• N.B. Not all these meet Mann’s criteria
Ubiquitous computing: smart devices, environments and interaction 79
![Page 80: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/80.jpg)
Head(s)-Up Display or HUD:
• presents data without blocking the user's view• pioneered for military aviation - now used in commercial
aviation and cars. • 2 types of HUD
– Fixed HUD:– Head-mounted HUD
Ubiquitous computing: smart devices, environments and interaction 80
![Page 81: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/81.jpg)
EyeTap & Virtual Retinal Display
• Instructors can add more detail about these here or delete this slide.
Ubiquitous computing: smart devices, environments and interaction 81
![Page 82: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/82.jpg)
Brain Computer Interface (BCI) or Brain Machine Interfaces (BMI)
• HCI focuses on indirect interfaces from human brain via human actuators–
• BCI are direct functional interfaces between brains and machines
• BCI represents ultimate natural interface
• Would you choose to make use of one when they become available in the future?
Ubiquitous computing: smart devices, environments and interaction 82
![Page 83: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/83.jpg)
Brain Computer Interface (BCI) or Brain Machine Interfaces (BMI)
• Direct vs. Indirect coupling design choices ??–
– See also BANs in Chapter 11
• Brain versus nerve direct coupling design choices??
Ubiquitous computing: smart devices, environments and interaction 83
![Page 84: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/84.jpg)
Computer Implants
• Opposite of wearing computers outside the body is to have them more directly interfaced to the body.
• Many people routinely use implants– ????
• Of specific interest is developing devices that can adapt to signals in the human nervous system.
• By connecting electronic circuitry directly to the human nervous system,– ???
Ubiquitous computing: smart devices, environments and interaction 84
![Page 85: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/85.jpg)
Cyborg 2
Electrode array surgically implanted into Warwick’s left arm and interlinked into median nerve fibres is being monitored.
85Ubiquitous computing: smart devices, environments and interaction
![Page 86: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/86.jpg)
BCI
• Instructors can add more detail about experiments here or delete this slide
Ubiquitous computing: smart devices, environments and interaction 86
![Page 87: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/87.jpg)
PostHuman Model
• Use of alterative technology mediated realities
• A feeling of presence in the experience provides feedback to a person about the status of his or her activity.
• The subject perceives any variation in the feeling of presence and tunes its activity accordingly.
Ubiquitous computing: smart devices, environments and interaction 87
![Page 88: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/88.jpg)
PostHuman Model and Reality
• People can experience alternative realities depending on:– the type of environment people are situated in– on their perception of the environment.
• Reality can be:– Technology mediated, e.g., ???– Chemically mediated, , e.g., ???– Psychologically mediated, , e.g., ???
Ubiquitous computing: smart devices, environments and interaction 88
![Page 89: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/89.jpg)
Realities: VR, AR and MR
• (Revision of Section 1.2.3.3)
Ubiquitous computing: smart devices, environments and interaction 89
![Page 90: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/90.jpg)
Virtual Reality (VR)
• VR seeks to immerse a physical user in a virtual 3D world• VR uses a computer simulation of a subset of the world
and immerses the user in it using UIs based upon:– ??
–
• VR seeks to enable humans to interact using a more natural interaction that humans use in the real world – ??
Ubiquitous computing: smart devices, environments and interaction 90
![Page 91: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/91.jpg)
Augmented Reality (AR)
• Electronic images are projected over the real world so that images of the real and virtual world are combined.
• VR considered as a subset of AR?
• Early E.g. head-mounted display by Sutherland (1968). • Similar systems are in use today in types of military aircraft.
Ubiquitous computing: smart devices, environments and interaction 91
![Page 92: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/92.jpg)
Telepresence & Telecontrol
• Telepresence allow a person in 1 local environment to:– ??
– .
• Telecontrol refers to the ability of a person in 1 place to– ???
Ubiquitous computing: smart devices, environments and interaction 92
![Page 93: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/93.jpg)
Overview
• HCI, eHCI & iHCI• eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices• iHCI use in wearable and implanted smart devices • Human Centred Design (HCD) • User Models: Acquisition and Representation• iHCI Design
Ubiquitous computing: smart devices, environments and interaction 93
![Page 94: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/94.jpg)
Conventional Design versus HCD
Ubiquitous computing: smart devices, environments and interaction 94
![Page 95: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/95.jpg)
Conventional Functional System Design
95Ubiquitous computing: smart devices, environments and interaction
Validate
Implement
Design
Requirements analysis
Final product
New product need
![Page 96: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/96.jpg)
Human Centred Design (HCD)
• Focus on types of UbiCom System & environments:
• Need to make the type of user explicit: human users
• In contrast, automatic / autonomous systems
Ubiquitous computing: smart devices, environments and interaction 96
![Page 97: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/97.jpg)
Human Centred Design (HCD)
ISO standard human centred design life-cycle involves 4 main sets of activities:
1. Define context of use
2. Specify stake-holder and organisational requirements
3. Multiple alternative (UI) designs need to be built.
4. Designs need to be validated against user requirements.
Ubiquitous computing: smart devices, environments and interaction 97
![Page 98: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/98.jpg)
Human Centred Design (HCD)
98Ubiquitous computing: smart devices, environments and interaction
Understand &specify use context
Validate designs against requirements
Produce Design solutions
identify need for interactive design
Systemsatisfies requirements
Identify stakeholder & Organisationalrequirements
Human-centered design
![Page 99: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/99.jpg)
A Fuller Range of System & User Requirements / Use Contexts
• HCD System & User requirements • -> Wider requirements than back-end functional
requirements• HCD Methodologies are a powerful way to get the
wide range of environment requirements & use contexts for UbiCom systems
• What is the fuller ranges of UbiCom / HCD requirements?
Ubiquitous computing: smart devices, environments and interaction 99
![Page 100: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/100.jpg)
A Fuller Range of System & User Requirements / Use Contexts
• System– ???
• Physical Environment– ???
• Users– Types– Task & goals– User interface
• Social – ???
• Usability & User experience: – Usability:– User experiences
Ubiquitous computing: smart devices, environments and interaction 100
![Page 101: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/101.jpg)
HCD: Use Context / Requirements
101Ubiquitous computing: smart devices, environments and interaction
Human
UsabilitySocial
Physical
Physical operating context: dark versus light conditions etc.
ICT
UbiComp System
ICT
Virtual
ICT
Functional <.- -> TasksNon-functional <.- -> System Spec.
External: Storage, QoS, network. etc Virtual
Environment Requirements
User
Physical
Human
Internal: Storage, display. etc
![Page 102: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/102.jpg)
HCD: Usability as a User Requirement
• Usability is defined as ??
–
• Usability is not a single, one-dimensional property of a user interface.
• Usability is a combination of factors. ISO-940-11 explicitly mentions no. of factors– ???
• These usability factors can often be expanded further sub-properties,
Ubiquitous computing: smart devices, environments and interaction 102
![Page 103: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/103.jpg)
HCD: Stake-Holders
• End-user is obvious stake-holder in HCD Design• Who are the other stake-holders in the personal memory
scenario?• Are there additional stake-holder requirements?
Ubiquitous computing: smart devices, environments and interaction 103
![Page 104: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/104.jpg)
HCD: Acquiring User Context/User Requirements
Several dimensions for get user requirements during HCD life-cycle
• In Controlled conditions (Lab) vs. in the field • Direct user involvement (e.g., interview,
questionnaire) vs. indirect (e.g., observations)
• Individual users vs. user groups, HCI / domain experts vs. predictive user models (no users)
104Ubiquitous computing: smart devices, environments and interaction
![Page 105: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/105.jpg)
HCD: Methods to Acquire User Requirements
Which Methods?• ?????
Analysis of data gathered depends on:• Amount of time, level of detail, uncertainty etc• Knowledge the analysis requires
105Ubiquitous computing: smart devices, environments and interaction
![Page 106: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/106.jpg)
Usability Requirements & Use Contexts Examples
• For each of the scenarios in chapter 1, e.g., the personal video memories, define the use context and usability requirements.
Ubiquitous computing: smart devices, environments and interaction 106
![Page 107: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/107.jpg)
HCI / HCD versus User Context Awareness
• Are these the same or similar concepts?
See User context awareness (Chapter 7)
See HCI / HCD (Chapter 5)
Ubiquitous computing: smart devices, environments and interaction 107
![Page 108: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/108.jpg)
HCD: System Model for Users vs. Users’ Model of System
• What model of the system does it project to the user?
• What model does the user have of the system?
• What if these models differ?
Ubiquitous computing: smart devices, environments and interaction 108
![Page 109: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/109.jpg)
HCD Design: Conceptual Models & Mental Models
• Amazing number of everyday things & objects – ????
• Very challenging for people to learn to operate and understand many devices of varying degrees of complexity if the interaction with each of them is unique.
• Complexity of interacting with new machines cm be reduced. How?
Ubiquitous computing: smart devices, environments and interaction 109
![Page 110: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/110.jpg)
HCD Design: Conceptual Models
• Discuss some example conceptual models
Ubiquitous computing: smart devices, environments and interaction 110
![Page 111: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/111.jpg)
HCD Design: Affordances
• Complexity of interacting with new systems is reduced if:– they have parts that provide strong clues on how to
operate themselves.
• These are referred to as affordances• What are examples of physical UI affordances?
Ubiquitous computing: smart devices, environments and interaction 111
![Page 112: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/112.jpg)
HCD Design: Virtual Affordances
• Many analogue physical objects being replaced by virtual computer UIs
• Virtual UI affordances are being increasing important.• How to design virtual UI affordances?• Can link virtual objects or widgets in it to related & familiar
physical world objects• Challenges in linking widgets to familiar physical objects?
Ubiquitous computing: smart devices, environments and interaction 112
![Page 113: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/113.jpg)
HCD: Multiple Prototype Designs
• Example: Consider PVM Scenario (Chapter 1)• What type of design?• Is there only 1 type of design for recording / playing /
transmitting multimedia?– ?
• Consider the requirements:
113Ubiquitous computing: smart devices, environments and interaction
![Page 114: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/114.jpg)
HCD: Evaluation
• Summative versus Formative Evaluation• Summative
– Conventional – To verify Design
• Formative– HCD
Ubiquitous computing: smart devices, environments and interaction 114
![Page 115: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/115.jpg)
HCD: System Evaluation Methods
• Can use similar techniques to gathering user requirements.
Ubiquitous computing: smart devices, environments and interaction 115
![Page 116: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/116.jpg)
Overview
• HCI, eHCI & iHCI• eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices• iHCI use in wearable and implanted smart devices • Human Centred Design (HCD)• User Models: Acquisition & Representation • iHCI Design
Ubiquitous computing: smart devices, environments and interaction 116
![Page 117: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/117.jpg)
User Modelling: Design Choices
• Implicit vs. explicit models• User instance (Individual) modelling versus user
(stereo)type modelling• Static versus dynamic user models• Generic versus application specific models• Content-based versus collaborative user models
Ubiquitous computing: smart devices, environments and interaction 117
![Page 118: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/118.jpg)
User Modelling Design: Implicit vs. Explicit models
• Systems can either use– Explicit feedback– Implicit feedback
• Often these can be combined. How?
• Some specific techniques for acquiring a user model are described in more detail elsewhere (Section 5).
• Hybrid user models may also be used.
Ubiquitous computing: smart devices, environments and interaction 118
![Page 119: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/119.jpg)
Indirect User Input and Modelling
Benefits?
Methods?• See Previous Slides• Accuracy & precision?• Handing inaccuracy & imprecision
Ubiquitous computing: smart devices, environments and interaction 119
![Page 120: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/120.jpg)
Direct User Input and Modelling
Benefits versus Challenges?
User requirements & user model built using:• Single-shot versus Multi-shot user input• Static versus Dynamic input
Also need to consider user model maintenance
Ubiquitous computing: smart devices, environments and interaction 120
![Page 121: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/121.jpg)
User Stereotypes
• Challenge in bootstrapping user model / behaviour leads to use of group behaviour
• Stereotype: infers user model from small number of facts using a larger set of facts from a group user model.
• Used by collaborative type user model, e.g., recommender systems
• Challenges?
Ubiquitous computing: smart devices, environments and interaction 121
![Page 122: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/122.jpg)
Modelling Users’ Planned Tasks and Goals
• Users often interact purposely with a system in a task-driven way, to achieve a particular goal.
• Several ways to analyse and model user tasks:– Hierarchical Task Analysis or HTA – Etc
• Consider each scenario in Chapter 1, e.g., PVM scenario, give a user task / goal model (next slide)
Ubiquitous computing: smart devices, environments and interaction 122
![Page 123: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/123.jpg)
HCD: Functional Requirements
123Ubiquitous computing: smart devices, environments and interaction
0: Record a physical world scene
1: Switch on camera
2: Set camera task mode
3.2: View Scene
4: Configure camera shot of scene
5: Compose Scene
6: Record Scene
8: Switch off camera
4.1: set zoom
3: Select Scene
3.1: Move towards Scene
3.3: Fix Scene
4.2: set lightingcorrection
4.2.1: set Flash
4.2.2: set under exposure
4.2.3: set over exposure
Plan 0: Do 1..2; Repeat 3..7 until no more recordings needed or no more power then do 8
7: Check recording
Plan 1: Repeat 3.1, 3.2 until satisfied, do 3.3
Plan 2: Repeat 4.1 until satisfied, do 4.2
Plan 3: do 4.2.1 or 4.2.2 or 4.2.3
For Photographer in PM scenario (Chapter 1)
![Page 124: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/124.jpg)
Multiple User Tasks and Activity Based Computing
• Use tasks as part of activities that require access to services across multiple devices,
• Devices can be used by different types of people• Users are engaged in multiple concurrent activities• Users are engaged in activities which may occur across
multiple physical environments,• Activities may be shared between participants • Activities on occasion need to be suspended and resumed.• (See Chapter 12)
Ubiquitous computing: smart devices, environments and interaction 124
![Page 125: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/125.jpg)
Situation Action versus Planned Action Models
• 2 basic approaches to task design
• Planned actions: – ????
• Situated action:– ???
Ubiquitous computing: smart devices, environments and interaction 125
![Page 126: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/126.jpg)
Models of Human Users: HCI vs. AI
• Field of HCI proposes models of humans that focus on supporting high-level usability criteria and heuristics – Focus is less on explicit computation models of how humans think
and act.
• Field of AI proposes models of humans that make explicit computation models to simulate how humans think, act and interact– (Chapters 8 and 9)
Ubiquitous computing: smart devices, environments and interaction 126
![Page 127: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/127.jpg)
Overview
• HCI, eHCI & iHCI• eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices• iHCI use in wearable and implanted smart devices • Human Centred Design (HCD)• User Models: Acquisition & Representation• iHCI Design
Ubiquitous computing: smart devices, environments and interaction 127
![Page 128: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/128.jpg)
iHCI
• iHCI Model Characteristics• User Context Awareness• Intuitive and Customised Interaction• Personalisation• Affective Computing• iHCI Design Heuristics and Patterns
Ubiquitous computing: smart devices, environments and interaction 128
![Page 129: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/129.jpg)
Types of User Model
• Several related terms & kinds of user model are differentiated
• User Models• Personal Profiles• User contexts• Application / User requirements• System Models• Mental Models• Conceptual Models
Ubiquitous computing: smart devices, environments and interaction 129
![Page 130: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/130.jpg)
User Context Awareness
• User context aware can be exploited to beneficially lessen the degree of explicit HCI needed.
• User context-awareness is a sub-type of general context-awareness (Chapter 7)
User context-awareness can include:• Social environment context• Users’ physical characteristics and capabilities for HCI• User presence in a locality or detected activity• User identity (Section 12.3.4).• User planned tasks and goals (Section 5.6.4).• Users’ situated tasks (Sections 5.6.5, 5.6.6).• User emotional state (Section 5.7.5)
Ubiquitous computing: smart devices, environments and interaction 130
![Page 131: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/131.jpg)
Intuitive and Customised Interaction
Are current computer systems dominated by MTOS based devices & use of desktop UI metaphor intuitive?
• E.g., ??• E.g., ??• E.g., etc
Ubiquitous computing: smart devices, environments and interaction 131
![Page 132: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/132.jpg)
Intuitive and Customised Interaction
Moran & Zhai propose 7 principles to evolve desktop model into more intuitive model for UbiCom
• From Office Container to Personal Information Cloud• From desktop to a diverse set of visual representations• From Interaction with 1 device to interaction with many• From Mouse & Keyboard to Interactions & modalities• Functions may move from Applications to Services• From Personal to Interpersonal to Group to Social• From low-level tasks to higher level activities
Ubiquitous computing: smart devices, environments and interaction 132
![Page 133: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/133.jpg)
Personalisation
• Personalisation: tailoring applications & services specifically to an individual’s needs, interests, preferences
• Adaptation of consumer product, electronic or written medium, based on person profile
• Applications of personalisation– targeted marketing– product & service customisation including information filtering – CRM
Ubiquitous computing: smart devices, environments and interaction 133
![Page 134: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/134.jpg)
Personalisation: Benefits
• ???
Ubiquitous computing: smart devices, environments and interaction 134
![Page 135: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/135.jpg)
Personalisation: Challenges (Cons)
• ???
Ubiquitous computing: smart devices, environments and interaction 135
![Page 136: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/136.jpg)
Personalisation
• Personalisation: a more complete model of user-context that is more reusable and persists: – ????
• 2 key issues:– design of model so that it can be distributed and shared– dynamic vs. static task-driven user preference contexts
Ubiquitous computing: smart devices, environments and interaction 136
![Page 137: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/137.jpg)
Personalisation: Mechanisms
• Instructors can add more slides about how personalisation mechanisms, e.g., recommender systems, work here or delete this slides
Ubiquitous computing: smart devices, environments and interaction 137
![Page 138: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/138.jpg)
Affective Computing: Interactions using Users’ Emotional Context
• Affective computing: computing relates to, arises from, or influences emotions.
• Applications include: – ???
• Design challenges for affective computing with those for: – determining the user context – developing more complex human-like intelligence models
Ubiquitous computing: smart devices, environments and interaction 138
![Page 139: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/139.jpg)
Affective Computing
Pickard (2003) identified six design challenges: • Range & modalities of emotion expression is broad• People’s expression of emotion is idiosyncratic & variable• Cognitive models for human emotions are incomplete• Sine qua non of emotion expression is the physical body
but computers not embodied in the same way • Emotions are ultimately personal and private• No need to contaminate purely logical computers with
emotional reactiveness
Ubiquitous computing: smart devices, environments and interaction 139
![Page 140: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/140.jpg)
iHCI: Design Heuristics and Patterns
• Many different higher-level HCI design usability / user experience criteria have been proposed by different HCI designers to promote good design of HCI interaction.
• Many different HCI heuristics (rules of thumb derived from experience) have proposed to support HCI criteria
• Specific guidance is needed to engineer UIs to comply with these usability & user experience HCI principles.
• UI design patterns can support HCI usability principles and then be mapped into lower-level more concrete design patterns
Ubiquitous computing: smart devices, environments and interaction 140
![Page 141: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/141.jpg)
iHCI: Design Heuristics and Patterns
Example iHCI patterns include:
Ubiquitous computing: smart devices, environments and interaction 141
![Page 142: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/142.jpg)
iHCI: Design Patterns & Heuristics
• Instructors can propose many more examples here or delete this slide.
Ubiquitous computing: smart devices, environments and interaction 142
![Page 143: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/143.jpg)
iHCI: Engineering iHCI Design Patterns
• Can propose simplify design models along 2 dimensions that are interlinked– Organisation / structural models versus time-driven interaction
models– Front-end / Presentation (UI) interaction versus back-end system
actions that support this interaction
• Need to organise UI widgets or objects at UI• Need to organise and link presentation to actions• Need to design interaction with these widgets• (see next slide as an example)
Ubiquitous computing: smart devices, environments and interaction 143
![Page 144: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/144.jpg)
Image SearchiHCI: Engineering iHCI Design Patterns
144Ubiquitous computing: smart devices, environments and interaction
MVC Pattern Toolkit
Model
View
Query String
Object Model Interaction Model
Query
Get-Input Query
Back-end Front-end (UI)
UI Presentation Model
Desktop
Search App
Get image
Locate Access
UI Task Model
TextfieldButton
Clear entry points Pattern
Button
StartEnter description here Advanced Search
Control
Defined in GUI Toolkit API
GUI Event handlers
![Page 145: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/145.jpg)
Overview
• HCI, eHCI & iHCI • eHCI use in 4 Widely Used Devices • iHCI use in accompanied smart devices • iHCI use in wearable and implanted smart
devices • Human Centred Design (HCD) • User Models: Acquisition & Representation • iHCI Design
Ubiquitous computing: smart devices, environments and interaction 145
![Page 146: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/146.jpg)
Summary
• A human centred design process for interactive systems specifies four principles of design: the active involvement of users and a clear understanding of user and task requirements; an appropriate allocation of function between users and technology based upon the relative competence of the technology and humans; iteration is inevitable because designers hardly ever get it right the first time; a multi-disciplinary approach to the design.
• Human centred design life-cycle involves user participation throughout four main sets of activities: defining user tasks and the (physical, ICT) environment context; defining user and organisational requirements; iterative design prototyping and validation against the requirements.
Ubiquitous computing: smart devices, environments and interaction 146
![Page 147: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/147.jpg)
Summary
• To enable humans to effectively interact with devices to perform tasks and to support human activities, systems need to be designed to support good models of user interfaces and processes of human computer interaction.
• Users can be modelled directly and indirectly. User task models can be modelled as task plans or as situated actions. iHCI design concerns three additional concerns: support for natural (human computer) interaction; user models including models of emotions which can be used to anticipate user behaviour and user context awareness including personalisation.
• Some design patterns and heuristics oriented towards iHCI are described.
Ubiquitous computing: smart devices, environments and interaction 147
![Page 148: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/148.jpg)
Summary & Revision
For each chapter• See book web-site for chapter summaries, references,
resources etc.• Identify new terms & concepts• Apply new terms and concepts: define, use in old and
new situations & problems• Debate problems, challenges and solutions• See Chapter exercises on web-site
148Ubiquitous computing: smart devices, environments and interaction
![Page 149: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/149.jpg)
Exercises: Define New Concepts
• Touchscreen, etc
Ubiquitous computing: smart devices, environments and interaction 149
![Page 150: UbiCom Book Slides](https://reader035.fdocuments.net/reader035/viewer/2022081420/56815971550346895dc6b410/html5/thumbnails/150.jpg)
Exercise: Applying New Concepts• What is the difference between touchscreen and a normal display?
Ubiquitous computing: smart devices, environments and interaction 150