Pre-Cognitive Human Extension as the Future of IoT!

http://resenv.media.mit.edu! 7/2017!Salish-IoT!

Prof. Joe Paradiso!Responsive Environments Group!

MIT Media Lab!

ResEnv Research Update 4/16! Joe Paradiso!

Header Helvetica Bold 65/65 pt!

3!

ResEnv Research Update 4/16! Joe Paradiso!

CHAIN-API applies where sensor data touches the ‘net’!

Not meant to run on low-powered nodesDoesn’t specify database standard

Transitory Phase – Smart environments will become an extension of self

The’DigitalButler’ArisesNow

Scientific AmericanCover ArticleJuly 2014

http://doppellab.media.mit.eduGershon Dublon, Nick Joliat, Brian Mayton

ResEnv Research Update 4/16! Joe Paradiso!

ResEnv Research Update 4/16! Joe Paradiso!

Building the Networked Sensory Landscape

Mayton, B., et al, ‘The Networked Sensory Landscape: Capturing and!Experiencing Ecological Change Across Scales,’ to appear in Presence, MIT Press Journal,

Special issue on Arts, Aesthetics, and Performance in VR and Telepresence, 2018!

600 acres!

‘Perpetual’ Solar power!100 now, soon >300,!

Cameras, 30 Audio streams!

The Mayton 2.0 Wireless Tidmarsh Node!

Easily Expandable for!Ubiquitous Wireless Sensing!

Can host a variety of sensors!

-  Soil Moisture!-  Soil Redox/Conductivity!-  Soil Temperature-  Air/Water Quality-  Etc…!

Uses cheap, vintage Telco crimp connectors (waterproof!)!

Resynthesized Reality - Flora, Ground Cover & Weather based on camera feeds

DonDerekHaddad

Sensors drive animations & spatialized audio

New DDH Environment!

Animals Embody Sensor History!

D. Haddad, K. Perlin, J. Paradiso, ‘Resynthesizing Reality: Driving Vivid Virtual Environments from Sensor Networks,’ in SIGGRAPH 2017

H e a r T h e r e v 2

T o u c h + h e a d t r a c k e r B o n e c o n d u c t i o n h e a d p h o n e

E y e t r a c k e r , P u p i l ø

C h e s t s t r a p s e n s o r ( h e a r t r a t e + r e s p i r a t i o n )

Attention-Driven Sensory Prosthetic

UWB/GPS Location

E E G / E M G

Rebecca Kleinberger & Gershon Dublon Presented at NIME 2015

Tes-ngatEPAMonitorIn Boston

LearnAir–DavidRamsay

D. Ramsay, J. Paradiso, “Automated Characterization of Consumer-Grade Sensor Accuracy from Supporting Data in Heterogeneous Air Quality Monitoring Networks,” NEMC 2017

IoT & the Built Environment

Ligh5ngControlisBroken

Commercial lighting control panel in the new (2010) Media Lab Building

Nan Zhao et al, ‘A Multidimensional Continuous Contextual Lighting Control System Using Google Glass,’ in Proc. of BuildSys’15 - Best presentation award

Nan Zhao at al.

1/27/15

ResEnv Research Update 4/16! Joe Paradiso! 27!

Name of presentation. Enter text on main master page.!

Name of presenter Helvetica Regular 18/18!

28!

Mediated Atmospheres!

Understanding perceptual and physiological effects of multimodal mediated environments!

Feature comparison !

•  Facial Expressions, Head Orientation, Gaze •  Bio-Signals: ECG, GSR, Respiration, Temperature, etc.

•  Basic Posture and Motion

Perception and Experience Physical Reaction Cognitive Performance

Nan Zhao, Robert Richer Asaf Azaria!!

Restorative!

Focu

s!

Zhao, N., Azaria, A., Paradiso, J.A., “Mediated Atmospheres: a Multimodal Mediated Work Environment,” To appear at Ubicomp 2017 !

Name of presentation. Enter text on main master page.!

Name of presenter Helvetica Regular 18/18!

29!

Rovables!

ArtemDementyev(ResEnv)CindyKao(LivingMobile)Sean Follmer (Stanford)

Best Paper @ UIST 2016!

Name of presentation. Enter text on main master page.!

Name of presenter Helvetica Regular 18/18!

31!

Summary•  SensorsaregeFngoutthere,piggybackingoncommercialproducts

• Onceaffordancesaresharedacrossdevices,we’relivinginanecologyofdevices&applica5ons– Thiswillhappenfastoncecommonprotocolsappear– Phasetransi5onintotrueUbicomp/IoT

• Howwillhumanpresencegeneralize?

• Wheredoes‘self’stopand‘other’begin?

Resenv – Spring 2017