Rapid prototyping of ubiquitous interaction

79
A comprehensive framework for the rapid prototyping of ubiquitous interaction Candidate: Andrea Bellucci Supervisor: Dr. Ignacio Aedo Co-supervisor: Dr. Alessio Malizia Doctoral degree in Ciencia y Tecnología Informática Departamento de Informática, Universidad Carlos III de Madrid Leganés, October 21st, 2013

description

These are the slides I have used for the oral dissertation of my Ph.D. in Computer Science at Universidad Carlos III de Madrid, Spain. The subject is the development of a framework to ease the rapid prototyping of digitally-augmented room setups.

Transcript of Rapid prototyping of ubiquitous interaction

Page 1: Rapid prototyping of ubiquitous interaction

A comprehensive framework for the rapid prototyping of ubiquitous interaction

Candidate: Andrea Bellucci

Supervisor: Dr. Ignacio AedoCo-supervisor: Dr. Alessio Malizia

Doctoral degree in Ciencia y Tecnología InformáticaDepartamento de Informática, Universidad Carlos III de MadridLeganés, October 21st, 2013

Page 2: Rapid prototyping of ubiquitous interaction

REsearch QUestion

Page 3: Rapid prototyping of ubiquitous interaction

REsearch QUestionHow to lighten the prototyping of ubiquitous interactive systems?

Page 4: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

SynopsisPart I: How things are• Framing• State of the art• Exploration• Open issues

Part II: How THINGS COULD BE• Hypothesis• Objective • Research methodology• The proposal• Evaluation

PART III: CONCLUSIONs• Contributions• Potential for future research

Page 5: Rapid prototyping of ubiquitous interaction

PART I: HOW THINGS ARE• Framing• State of the art• Exploration• Open issues

Page 6: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

• Texto

UbiComp: merging the physical and the digitalFraming

Page 7: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Digitally-augmented environments

Page 8: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

• Texto

Technology-mediated human-to-human interactionPEOPLE

Page 9: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

• Texto

Device EcologiesTEchnology

Page 10: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

[...] a massive noncentralized agglomeration of devices, connectivity and electricity means, applications, services, and interfaces.

[The] infrastructure is not homogenous or seamless, but fragmented into several techniques that the user has to study and use.

Oulasvirta, A. (2008). When users “do” the ubicompTHE REAL UBICOMP

Page 11: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 12: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 13: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 14: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 15: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 16: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 17: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

cumbersome design and developmentUBICOMP TECHNOLOGIES IN 2013

Page 18: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

EXPERIENCE

The exploration has been carried out within three research projects:1. INTEGRA (national, AMPER)2. TIPEx (national, MINECO)3. meSch (european, FP7)

Page 19: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

EXPERIENCE (2)Don’t touch me

Page 20: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.TESIS: Turn Every Surface into an Interactive SurfaceEXPERIENCE (3)

Page 21: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

LESSONS LEARNT

1. Difficult knowledge reuse2. High technical expertise

Page 22: Rapid prototyping of ubiquitous interaction

CONTEXTTechnological perspective

Page 23: Rapid prototyping of ubiquitous interaction

OPEN ISSUES

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Page 24: Rapid prototyping of ubiquitous interaction

Issue #1. Integration of hetherogeneous devices and interaction modalities

Issue #1

OPEN ISSUES

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Page 25: Rapid prototyping of ubiquitous interaction

Issue #2. Lowering the technical expertise

Issue #2

OPEN ISSUES

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Page 26: Rapid prototyping of ubiquitous interaction

Issue #1. Integration of hetherogeneous devices and interaction modalities

Issue #2. Lowering the technical expertise

Issue #2

OPEN ISSUES

Issue #1

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Page 27: Rapid prototyping of ubiquitous interaction

PART II: HOW THINGS COULD BE• Hypothesis• Objective• Research methodology• The proposal• Evaluation

Page 28: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

RESEARCH QUESTION

How to lighten the prototyping of ubiquitous interactive systems?

Page 29: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Encompassing heterogeneous devices into a unique design can reduce users efforts to develop ubiquitous interactive systems

HYPOTHESIS

Issue #1Issue #2

Hypothesis

Page 30: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

The definition of a conceptual and software framework that provides users with a comprehensive environment for managing interactions between heterogeneous networked devices...

...to support the rapid prototyping of ubiquitous interactive systems

OBJECTIVE

Page 31: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

The six steps of Design and Development ResearchHasan, H. “Information systems development as a research method”, (2003)

METHODOLOGY

Page 32: Rapid prototyping of ubiquitous interaction

METHODOLOGY

Page 33: Rapid prototyping of ubiquitous interaction

PART II: HOW THINGS COULD BE• Hypothesis• Objective • Research methodology• The proposal• Evaluation

Page 34: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Materialization of design ideasRAPID PROTOTYPING

Exploration of the design space

The output is not the artifact, but the feedbacks

Rapid prototyping allows to test: appearance, functionality and user experience

Page 35: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Why does it matter?

Desktop environments have the same I/O capabilities, while UbiComp devices vary in shape, size and I/O design

Usability and UX cannot be evaluated with simulations on a screen

PROTOTYPING UBIQUITOUS INTERACTION

Page 36: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

1. Interaction Developerslong time/high ceiling; high eXperience; coding

2. Interaction Designersmedium/rapid operation; medium X; design

3. End-Usersrapid operation; X?; usage

STAKEHOLDERS

Page 37: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

REQUIREMENTS

Page 38: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

Literature Survey

Personal Experience

REQUIREMENTS

Page 39: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

Initial Set of Requirements

REQUIREMENTS

Page 40: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

Interviews

REQUIREMENTS

Page 41: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

User-Generated Requirements

REQUIREMENTS

Page 42: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

Categories (Card Sorting)

REQUIREMENTS

Page 43: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Understand and define users needs

Literature Survey Personal Experience

Initial Set of Requirements

Categories (Card Sorting)

Interviews

User-Generated Requirements

Final Set of Requirements

Final Set of Requirements

REQUIREMENTS

Page 44: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Six categoriesREQUIREMENTS (2)

1. Input/Output Hardware: Support Heterogeneous I/O; Device Abstraction

2. Interaction Modalities: Support Heterogeneous Interaction Modalities

3. Interactive Space: Spatial Awareness; Multi-display environment

4. Architectural Traits: Distributed Architecture; Agnosticism of Legacy Middlewares

5. Developing/Coding: Low Viscosity; Programming Alternatives; Hide Low-Level Details

6. Application/User Interface: Cross-Device UI; Easy Configuration of I/O Devices

Page 45: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

INTERACTION MODEL

Page 46: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

Environment

INTERACTION MODEL

Page 47: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

Environment

PObject

INTERACTION MODEL

Page 48: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

Environment

PObject

TInputTObject

INTERACTION MODEL

TOutputTObjectTInput

Page 49: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

Environment

PObject

TInputTObject

INTERACTION MODEL

DInput

TOutputTObjectTInput

Page 50: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

Environment

PObject

TInputTObject

INTERACTION MODEL

DInput

Network

DObjectDObject

TOutputTObjectTInput

Page 51: Rapid prototyping of ubiquitous interaction

PHYSICAL

DIGITAL

Environment

PObject

Model Model

TInputTObject

INTERACTION MODEL

DInput

Network

DObjectDObject

TOutputTObjectTInput

Page 52: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

AN EXAMPLE

Page 53: Rapid prototyping of ubiquitous interaction

DINPUT

Page 54: Rapid prototyping of ubiquitous interaction

DINPUT

Support Heterogeneous Input/Output Hardware;

Device Abstraction

Page 55: Rapid prototyping of ubiquitous interaction

Support Heterogeneous Interaction Modalities

Agnosticism of LegacyMiddlewares

DINPUT

Page 56: Rapid prototyping of ubiquitous interaction

Spatial Awareness

DINPUT

Page 57: Rapid prototyping of ubiquitous interaction

DINPUT

Page 58: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

For developers: an Application Programming Interface for the direct coding of devices behaviors.

For designers/researchers: an XML-based language for defining object creation, management, communication and interaction within a multiuser, multidevice environment.

PRACTICAL FEASIBILITY

Page 59: Rapid prototyping of ubiquitous interaction

PART II: HOW THINGS COULD BE• Hypothesis• Objective• Research methodology• The proposal• Evaluation

Page 60: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

FramingA COMPREHENSIVE EVALUATION

1. Use case

2. USER STUDY

Page 61: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

USE CASEDigitally-augmented product shelf

Page 62: Rapid prototyping of ubiquitous interaction

Depth Camera

RGB Camera

I/O Hardware MAGIC MOUSE(FIDUCIAL MARKER)

TESIS

Hardware Abstraction

Pico-Projector

USER’S FINGER

RGB CAMERA

IR CAMERA

+PICO PROJECTOR

IR EMITTER

Page 63: Rapid prototyping of ubiquitous interaction

Depth Camera

RGB Camera

I/O Hardware MAGIC MOUSE(FIDUCIAL MARKER)

TESIS

Hardware Abstraction

Pico-Projector

USER’S FINGER

RGB CAMERA

IR CAMERA

IR EMITTER

PICO PROJECTOR

Page 64: Rapid prototyping of ubiquitous interaction

Gesture Interpreter

Depth to Touch

Input Interpretation

Application

FIDUCIAL MARKER

FINGER

DPANELDPANELDPANEL DBUTTONDBUTTONDBUTTON

Page 65: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Null hypothesis

The framework does not affect positively the efficiency in the development of prototypes that make an integrated use of different devices.

USER STUDY

Page 66: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Independent variables

Hardware Technology (2)Wiimote or iPad

Software Technology (3) wrj4P5 library, oscP5 library or the framework

Interaction Component (3)real button (the physical button of the Wiimote), virtual button (a button widget on the iPad) or accelerometer

USER STUDY (2)

Page 67: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Dependent variables

Time to complete the task

USER STUDY (3)

Page 68: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

• Texto

USER STUDY (4)Experiment setup

Page 69: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

TIME TO COMPLETE THE TASK

Page 70: Rapid prototyping of ubiquitous interaction

TIME TO COMPLETE THE TASKAverage times for each task under each condition

Wiimote w/ framework

Wiimote w/o framework

iPad w/ framework

iPad w/o framework

Page 71: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Results

The null hypothesis has been rejected using the non-parametric test Wilcoxon Rank Sum for Large Samples

USER STUDY (5)

Page 72: Rapid prototyping of ubiquitous interaction

PART III: CONCLUSIONS• Contributions• Potential for future research

Page 73: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Objective. The definition of a conceptual and software framework that provides u s e r s w i t h a c o m p r e h e n s i v e environment for managing interactions between heterogeneous networked devices

MAIN CONCLUSION

Page 74: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Objective. The definition of a conceptual and software framework that provides u s e r s w i t h a c o m p r e h e n s i v e environment for managing interactions between heterogeneous networked devices

MAIN CONCLUSION

Page 75: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Objective. The definition of a conceptual and software framework that provides u s e r s w i t h a c o m p r e h e n s i v e environment for managing interactions between heterogeneous networked devices

Hypothesis. Encompassing heterogeneous devices into a unique design can reduce user efforts to develop ubiquitous interactive systems

MAIN CONCLUSION

Page 76: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

Objective. The definition of a conceptual and software framework that provides u s e r s w i t h a c o m p r e h e n s i v e environment for managing interactions between heterogeneous networked devices

Hypothesis. Encompassing heterogeneous devices into a unique design can reduce user efforts to develop ubiquitous interactive systems

MAIN CONCLUSION

Page 77: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

DISSEMINATION

Journals and MagazinesIEEE Pervasive Magazine 2010,

ACM Computing Surveys 2013

ConferencesTEI 2014, Physicality 2012, ITS 2011, AVI 2010

Page 78: Rapid prototyping of ubiquitous interaction

D. Andrea Bellucci. Doctoral degree in Ciencia y Tecnología Informática. Leganés, October 21st, 2013.

FUTURE WORK

Implementation of a wide range of sensors, actuators and platforms

1.

Web-based user interface for the definition of sensors-actuators rules

2.

In which ways a tool influences the exploration and understanding of a problem space?

3.

Page 79: Rapid prototyping of ubiquitous interaction

Thanks for your attention!¡Gracias por su atención!Grazie per l’attenzione!