HFES 2016, Human Factors and Ergonomics in...

Hugh Dubberly, Dubberly Design OfficeStephen B. Wilcox, Design Science

HFES 2016, Human Factors and Ergonomics in Healthcare: Shaping the FutureSan Diego April 15, 2016

From Devices to PlatformsImplications of the Internet of Thingsfor Healthcare

2Dubberly Design Office · From Devices to Platforms · April 15, 2016

...we are in the middle of a dramatic and broad technological and economic shift in which software companies are poised to take over large swathes of the economy...

Health care and education, in my view, are next up for fundamental software-based transformation.”

— Marc Andreessen, founder, Netscape and Andreessen-Horowitz

Wall Street Journal OpEd www.wsj.com/articles/SB10001424053111903480904576512250915629460

“...software is eating the world.


“If you went to bed last night thinking you’re an industrial company, you’re going to wake up this morning as a software and analytics company.”

— Jeff Immelt, Chairman and CEO, General Electric

Minds + Machines 2014www.ge.com/stories/industrial-internet

1981 1995 2016Personal Computers The Internet Internet of Things (IoT)

Social

Mobile

Voice UIs

SaaSDrones Robots

Virtual realityAugmented reality

Big dataLinked dataOpen data

GlassesGogglesWearables

AI 2.0Machine learningPredictive analytics

CNC3D printing

Cloud computingon-demand


Technological change comes in waves,which interact to create “combinatorial innovation.”[1]

– Work has gone digital. – People got connected. – Now, things are connecting, too.

[1] Hal Varian


2016 is like 1981 and 1995 all over again.You can see the next wave coming.

It goes by many names: – Internet of Things (IoT) – Internet of Everything, Cisco – Industrial Internet, GE – Smarter Planet, IBM – Living Services, Accenture – Platform World, Sapient.Publicis – Social CRM or Social Business – Digital Engagement – Digital Transformation


The change has already begun,for example

Amazon announced AWS IoT, also testing pop-up stores in malls featuring IoT devices and Echo.

GE hired 2,000 engineers in San Ramon, builds Predix platform, GE Digital now has 30,000 employees.

Sears opens a series of “Connected Home” showcase stores.

Target launches “Open House” IoT showcase store in San Francisco.

Fixedcommunications

Mobilecommunications

Computers

Consumer electronicsand medical devices

Industrial devices(military and aerospace)

Automotive

2013 2014 2015 2016 2017 2018 2019 2020

10b

20b

30b

40b

50b


By 2020, ~50 billion devices will be connected to the Internet;today, ~7 billion computers and tablets are connected.

Sources: The Economist and Cisco

Product, Service, and Ecologies Diagrams

Product Service Ecologies

Product Service Product Service


Products no longer stand alone.Increasingly, they exist in complex service webs.

[1] Source: Automotive Sensors and Electronics Expo 2015http://www.automotivesensors2015.com/

The average vehicle includes 60–100 sensors;that figure may grow to 200 by 2020. [1]

[2] Source: The New York Timeshttp://www.nytimes.com/2010/02/05/technology/05electronics.html

In 2010, basic cars included 30 microprocessors;luxury cars had as many as 100. [2]

1. Product2. Smart product

Software


Products are becoming “smart”.Product

+ Sensor + Computer = Smart Product

—Michael Porter and James Heppelmann, How Smart, Connected Products Are Transforming CompetitionHarvard Business Review, November 2014https://hbr.org/2014/11/how-smart-connected-products-are-transforming-competition


Software

3. Smart, connected product

Remotedevices

Software


Sets of smart products are connecting.Smart Product

+ Network + Cloud Service = Smart, Connected Product



Software


Remotedevices

Software

4. Product system

Planters

Combineharvesters

Tillers

Tractors

FarmEquipment

System

Remotedevices

Software


Sets of connected products form product systems.Smart, Connected Product

+ other Smart, Connected Products = Product System



Software


Remotedevices

Software

4. Product system

Planters

Combineharvesters

Tillers

Tractors

FarmEquipment

System

Remotedevices

Software

5. Systems of systems

IrrigationSystem

Irrigation nodes

Irrigationapplication

Fieldsensors

Weather DataSystem

Weather dataapplication

Weather forecastsWeather maps

Rain, humidity,temperature sensors

FarmManagement

System

SeedOptimization

SystemSeed database

Farm performancedatabase

Seed optimizationapplication

Planters

Combineharvesters

Tillers

Tractors

FarmEquipment

System

Remotedevices

Software


Systems connect to other systems, forming ecologies.Product Systems

+ other Product Systems= Product-Services Ecology



Farms are becoming automated factories.Plants are attached to sensors, connected to networks, generating data.


Macro view: processed satellite images of crop growth over time,e.g., central Iowa, March 29 to October 23, in 8 day increments.Algorithms automatically align images,remove clouds,and detect vegetation.


Daily weather data can augment machine learning.Precipitation,temperature,wind direction and speed,snow cover,and cloud covercan aid forecasting.

Music Listener iPod

Hardware Software Networked Service Marketplace

iTunes iTunes Store Record Companies Artists

MENU


iPod was the proto-IoT device—an integrated systemof hardware, software, and networked services.

Ride Seeker Driver

Database

Matching Service

GPS

MapsMembers

Drivers

CarPhone Phone


VCs are no longer funding stand-alone apps;Uber isn’t just an app; it’s an IoT platform for logistics.

Customerdatabase

Batteries

Microprocessor Cloud

ConnectedMobile App

Speaker

AImachine-learningsystem

Microphone

Wi-Fi


Last fall, Mattel relaunched Barbie, as a smart, connected product.Barbie + sensor + computer + network + cloud services = Eliza 2016

– Recognizes you and what you say – Learns about you – Converses with you,

using 8,000 pre-recorded phrases – Extends up to 120 exchanges

https://en.wikipedia.org/wiki/ELIZAhttp://www.nytimes.com/2015/09/20/magazine/barbie-wants-to-get-to-know-your-child.htmlhttp://www.nytimes.com/2015/10/16/business/mattel-aims-to-reanimate-sales-with-interactive-barbie.html

Business

Marketing

Mar-com

CRM MMDB

Member MarketingDatabase

Business

New products + servicesbuilt on the MMDB.

Marketing

Mar-com

CRM

MMDB

Business

Marketing

Mar-com

CRM MMDB

Member MarketingDatabase

Business

New products + servicesbuilt on the MMDB.

Marketing

Mar-com

CRM

MMDB


Now imagine Barbie’s AI connected to a CRM system.“Have you told Grandma what you’d like for your birthday?”


What does the IoT mean for healthcareand healthcare usability?


Consumer tracking devices (AKA “wearables”)have become common.

Apple Watch Basis B1 Fitbit Flex

Narrative Clip

Microsoft Band

Misfit Shine Nike Fuel

Jawbone Up

Moto 360

Garmin Vivofit

Pebble Classic Samsung Gear

Cloud

Accelerometer

Phone App

RAM

Analytics engine

Friends

Wi-Fi

Website


Sensors collect data and pass it to web-based apps;users set goals and share data with friends and family.Feedback loops can promote change and help manage health.


Advanced sensors will change the meaning of “wearables”.Google’s Project Jacquard is creating touch-sensitive fabric.

https://www.google.com/atap/project-jacquard/

HCP

Wi-Fi

Website

Sensor

RAM

Phone

PatientFamily

Cloud

Blood

App

Analytics engine


Medical devices are connecting to the cloud, too,including pacemaker-defibrillators, autoinjectors, and glucometers.Smart, connected glucometers can alert family and HCPs about extreme lows;they also collect data automatically and help users see trends and make correlations.

Cartridge(disposable)

Plunger(disposable)

Cartridge Cap(disposable)

Needle(medical sharps,

disposable)

Needle Cap(disposable)

Insulin Vial(disposable)

Syringe Filled cartridgesyringe disassembled &cartridge cap connected.

Filled cartridgecasket cap threaded onto tubing &infusion kit connected to cartridge.(cartridge and tubing disposable)

Animas® Inset®

(medical sharps,disposable)

ReservoirPackaging

(disposable)

InsulinPackaging

(disposable)

Animas® Inset®

Packaging(disposable)

Reservoir Kit(disposable)

Inset Kit(disposable)

contains 10single use

contains

containscontains 10single use

use inserterto apply

consists of creates load into

fill thesyringe

attachinfusionkit tubingto cartridge

attachinfusion

kit tubing

patientrecieves

patientrecieves

patientrecieves

flexibleplastictube

OneTouch® Ping®

Insulin Pump

Infusion Set(disposable)


And they are connecting to other smart, connected devices.A smart glucometer + a smart insulin pump = artificial pancreas

Endocrinologists may be able to set doses remotely.Systems could enable direct contact; e.g., “Do you feel nausea?”

Remote monitoring+ intervention DB

EMR

Nurse

Wi-Fi

Website

Pumps

Phone

Patient

On-board drug library

IV bags + lines

Cloud

Analytics engine


Similar changes are beginning in clinical settings, too.Large volume infusion pumps (LVPs) used to stand-alone.

Nurse

Wi-Fi

Cloud

Air mattresssensors

Patient

Website

Phone App


Even hospital beds will become smart, connected devicesand join the medical product-services ecology.Already, some beds can measure:

– Patient weight – Heart rate – Breathing rate – Intensity and duration of movement – Bed entrance + exit – Sleep

http://smartbed.goodmarkmedical.com/smart-bed-technology-new/learn-smart-bed-solution/

Customer database

BatteriesMicroprocessor

Cloud

SpeakerAImachine-learningsystem

Microphone

Wi-Fi


Catalia Health is developing Mabu, a personal healthcare companion.Robot + touch-screen + network + cloud services = Mabu

– Recognizes you and what you say – Learns about you – Imports data from health trackers – Converses with you about your condition – Reminds you to take medications – Connects with HCPs


10 11 12PM

1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11 12PM

1 2 3 4 5 6 7 8 9

HEART R ATE (BEATS PER MINUTE)

ACCELER ATION (G FORCE)

ELECTRODERMAL ACTIVIT Y (MICRO-SIEMENS) Gray = OriginalBlue = Processed

0

5

10

15

20

25

30

0

20

40

60

80

100

120

140

160

180

200

220

0

0.25

0.50

0.75

1.00

1.25

9Day 1

12Day 2

IoT devices will generate huge amounts of data,requiring monitoring, analysis, and visualization.Stress data from a RWJF study of family caregiving,over ~24 hours, 20 participants with a dozen sensors, generated 5GB of data.

Gra

y =

Ori

gina

lB

lue

= Pr

oces

sed

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

Front doorFay’sbedroom

Josephine’sbedroom

BathroomLiving roomB

KitchenLiving roomA9

Day

110

1112 PM

12

34

56

78

910

1112

Day

21

23

45

67

89

1011

12 PM1

23

45

67

89

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

30 20 10 0405060708090100

NO

ISE

(DEC

IBEL

S)

CO

2 (P

AR

TS

PER

MIL

LIO

N)

HU

MID

ITY

(P

ERC

ENT

)

30 20 10 0405060708090100

MO

TIO

N

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

AC

TIV

ITIE

S

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

9D

ay 1

1011

12 PM1

23

45

67

89

1011

12D

ay 2

12

34

56

78

910

1112 PM

12

34

56

78

9

HE

AR

T R

AT

E (B

EA

TS

PER

MIN

UT

E)

AC

CEL

ERA

TIO

N (

G F

OR

CE

)

ELEC

TR

OD

ERM

AL

AC

TIV

ITY

(M

ICR

O-S

IEM

ENS

)

051015202530 020406080100

120

140

160

180

200

2200

0.25

0.50

0.75

1.00

1.25

Prim

ary

unit

in li

ving

roo

m B

Seco

ndar

y un

itin

bat

hroo

m

Prim

ary

unit

in li

ving

roo

m B

100 0

200

300

400

500

600

700

800

900

1,00

01,

100

1,20

01,

300

1,40

01,

500

Care

givi

ngSt

ress

leve

l 5St

ress

leve

l 4St

ress

leve

l 3St

ress

leve

l 2St

ress

leve

l 1St

ress

leve

l 0

Self

Car

e

Leis

ure

Wor

k

Oth

er

Slee

p

Closing the windows before leaving the house causes the CO2 to increase, and noise to decrease.

Spike in humidity matches self-reported shower.

Motion detectors show that Fay’s mother Josephine went to the bathroom in the middle of the night.

Spike in acceleration may be driving to the grocery store. Spike in EDA may indicate driving was stressful.

Spikes in acceleration, EDA, and heart rate match self-reported shower.

Sel

f-re

po

rted

sle

epS

elf-

rep

ort

ed g

roce

ry s

ho

pp

ing


Comparing data sets enables us to recognize patternsand becomes the basis for ongoing machine learning.

1. Product

Glucometer + processor = computer that can run apps; e.g., bolus calculator, calorie estimator and tracker.

2. Smart product

Software


Remotedevices

Software

When a glucometer connects to a smart-phone, cost can come down,because the meter can build on the phone's processor and display.Plus data can be shared with family and HCPs.

4. Product system

Monitor

Syringe

Cartridge

Vial

Pump

GlucoseManagement

System

A near continuous glucose monitor can be coupled with an insulin pump,forming a glucose management system.

Remotedevices

Software

5. Systems of systems

The glucose management system can connect with many other systems,such as EMRs, remote alerting, patient population management,drug interaction monitoring, pharmacy order management, and billing.

Monitor

Syringe

Cartridge

Vial

Pump

GlucoseManagement

System

HealthManagement

System

PharmacyOrder

ManagementSystem

EMRSystem

RemoteAlertingSystem

DrugInteractionMonitoring

SystemBillingSystem

PatientPopulation

ManagementSystem

Remotedevices

Software


In the future, medical products will no longer stand alone.Increasingly, they will exist in complex service webs.


As medical products connect with service webs,they are also connecting with drug development systems.

– Systems for funding research and development, creating and protecting “intellectual” property, and rewarding investment

– The drug-knowledge-package system

– Compound sourcing, manufacturing, and distribution systems

– Drug delivery devices

– Systems for educating physicians and patients

– Systems for helping patients integrate the drug into their lives

– Insurance and government payment systems

– Government regulatory systems and professional association practices

Patient Web + mobile

Bio-med engineers

Researchers

Designers

HCPs

Individual view

Individual view

Aggregate view

Individual view

Devices

Cloud

Integrateddatabases

Company views

Patient view- My Record- Messages- Appointments- Billing- Support

Patient ID#

Patient ID#

Patient ID#

All Patient Data

My Record


Unified patient and device data will afford useful views to many constituents.

Integrateddatabases

Company views

Patient ID#

Patient ID#

Patient ID#

All Patient Data

Patient

Bio-med engineers

Researchers

Designers

HCPs

Individual view

Individual view

Aggregate view

Individual view

Devices

Cloud

Benefit = better experience

Web + mobile Patient view- My Record- Messages- Appointments- Billing- Support

- Clearer picture of health status- Alerts on possible problems- Personalized communications- Record of appointments

My Record


Patients can know much more about what’s happeningand can share information with family, friends, and HCPs.

Integrateddatabases

Company views

Patient ID#

Patient ID#

Patient ID#

All Patient Data

Patient

Bio-med engineers

Researchers

Designers

HCPs

Benefit = higher patient sat

Individual view

Individual view

Aggregate view

Individual view

Devices

Cloud

Web + mobile Patient view- My Record- Messages- Appointments- Billing- Support

My Record


HCPs can receive a more holistic view of each patientand can manage groups of patients more efficiently.

Patient

Bio-med engineers

Researchers

Designers

HCPs

Benefit = lower costs

Individual view

Individual view

Aggregate view

Individual view

Devices

Cloud

Web + mobile Integrateddatabases

Company views


Patient ID#

Patient ID#

Patient ID#

All Patient Data

My Record


Bio-med engineers can better manage equipment,improve service, and reduce support costs.

Patient

Bio-med engineers

Researchers

Designers

HCPs

Benefit = more efficient ops

Individual view

Individual view

Aggregate view

Individual view

Devices

Cloud


Company views


Patient ID#

Patient ID#

Patient ID#

All Patient Data

My Record


Researchers can learn from aggregate data,to improve procedures and care-facility operations.

Patient

Bio-med engineers

Researchers

Designers

HCPs

Benefit = improved next gen products

Individual view

Individual view

Aggregate view

Individual view

Devices

Cloud


Company views


Patient ID#

Patient ID#

Patient ID#

All Patient Data

My Record


Designers can get detailed usage data,to improve next generation products.


Questions raised by the Internet of Things:

Of course, more UIs will have to be validated, but will FDA require the whole interconnected system to be validated together or treat each component as a separate device?

Will all parts of the system be considered Class 2 or 3, or will parts of it be Class 1?

How will usability engineering and usability testing change?

What other skills will usability professionals need?

What changes in methods will be required?


The IoT signals a larger cultural shiftFrom

Industrial Age

Cathedral

Age of Enlightenment

Machine metaphors

Tree structures (e.g., the tree of life)

Direct causality

Patient coordination of specialists

“Mechanism of disease”

To

Information Age

Bazaar

“Age of Entanglement”

Ecology metaphors

Network structures(e.g., web of life)

Systemic causality

“Patient medical home” or “patient population management”

“Context of living in which disruptions arise” or other “integrative approaches”

—Eric Raymond

—Danny Hillis

—Manual Lima

—George Lakoff

—Fritjof Capra

[email protected]@dscience.com

Presentation posted athttp://presentations.dubberly.com/HFES_health.pdf

Special thanks to Lynn StrotherAnthony AndreSean HägenRyan Reposar

HFES 2016, Human Factors and Ergonomics in...

Documents

Transcript of HFES 2016, Human Factors and Ergonomics in...