Internet of Things: From Sensing to Interaction

Sean Follmer Stanford University

Sensing -> Data

Consumer Supply Chain Manager

Building Engineer

Designers

What are their needs?

Consumer Supply Chain Manager

Building Engineer

Designer

What are their goals?

Smart Objects

Sensing -> Interaction

Saul Greenberg

ControlVisualization Activity Based Interaction Toolkits /

FabricationPhysical Output

Visualization

Seeing the big picture

Seeing the invisible

Seeing behavior change

Visualization: Seeing Everything

Seeing the invisible

Seeing the big picture

Seeing behavior change

Visualization: Seeing the Ivisible

Seeing the invisible

Seeing the big picture

Seeing behavior change

Remote Visualization

Paradiso et al., Responsive Environments Group MIT Media Lab,

2007-present

Doppel Lab

ASTOR - Spatial AR For CNC Machining

Olwal et al., KTH, 2008

Co-located Visualization (AR)

iLampsRaskar et al., MERL, 2004

Co-located Visualization (Projected - AR)

UbiFitConsolvo et al., UW/Intel Labs Seattle,

2008

Personal/Persuasive Visualization

Control

Interacting with Small Devices

Interacting with Many Devices

Interacting in situ

Interacting with Small Devices

Poupyrev et al., Google ATAP, 2015Project Soli

Spatial Control of Many Devices

Haller et al., Media Interaction Lab, 2010

Crystal

In Situ Interaction

Maes et al., Fluid Interfaces Group, MIT Media Lab, 2014

Reality Editor

In Situ Interaction

Hartmann et al., UC Berkeley, 2014

HOBS: Head Orientation-Based Selection in Physical Spaces

Activity Based Interaction

Sensing Activity at Different Scales

Proxemic Interaction

Sensing Activity: Device Scale

Poupyrev et al., Disney Research, 2012Touché

Sensing Activity: Building Scale

Patel et al., UW, 2010

electric, water, gas

Disaggregated End-Use Energy Sensing

Proxemic Interaction

Smart Objects

Greenberg et al., Calgary, 2010Proxemic Interaction

Proxemic Interaction

Greenberg et al., Calgary, 2010Proxemic Interaction

Toolkits and Fabrication

Designing New Hardware

Toolkits and Programming for IoT

Toolkits for Prototyping Hardware

Steimle et al., Embodied Interaction Group, 2013

Print Sense

Toolkits for Prototyping Hardware

Paradiso et al., Responsive Env. Group, MIT Media Lab 2015

Sensor Tape

Toolkits for Prototyping Hardware

Follmer et al., BID, UC Berkeley 2015

Makers Marks

Toolkits for Multidevice Interaction

Chi and Li, Google, 2015Weave

Toolkits for Multidevice Interaction

Marquardt et al. , Calgary, 2011Proxemic Toolkit

Physical Output

Touching the invisible

Shape Change

Ubiquitous Robotics

Mid Air Haptics

Poupyrev et al., Disney Research, 2013Aireal

Physical Output

Subramanian et al., Univ. Bristol Interaction Group, 2015

Ultrasonic Haptics

Shape-Change

Ishii et al., Tangible Media Group, 2013PneUI

Physical Output

Follmer et al., Tangible Media Group, 2013

inFORM

Physical Output

Follmer et al., Tangible Media Group, 2014

TRANSFORM

Thanks!