HEC 2016 Panel: Putting User-Generated Data in Action: Improving Interpretability for Consumer...

90
AN INTERNATIONAL HELLO Brazil - Opa Chinese – nin hao Dutch – Hallo, Goededag French – Bonjour German - Guten Tag Hawaiian - Aloha Indonesian -Selamat Japan – konnichiwa Korean – annyeonghaseyo Norwegian - Goddag Portugese –’Ola Spanish - ¡Hola! Swedish - Hej / Hallå Thailand - sà-wàt-dee Russian - Allo Turkey - Alo, Efendim Italian – Ciao Israel-Shalom Africa – Hallo Polish – HALO/SLUCHAM Arabic – As salam ‘alakum

Transcript of HEC 2016 Panel: Putting User-Generated Data in Action: Improving Interpretability for Consumer...

AN INTERNATIONAL HELLO

Brazil - Opa

Chinese – nin hao Dutch – Hallo, Goededag

French – Bonjour German - Guten Tag Hawaiian - Aloha Indonesian -Selamat

Japan – konnichiwa Korean – annyeonghaseyo

Norwegian - Goddag Portugese –’Ola� Spanish - ¡Hola!

Swedish - Hej / Hallå�

Thailand - sà-wàt-dee Russian - Allo Turkey - Alo, Efendim

Italian – Ciao Israel-Shalom

Africa – Hallo

Polish – HALO/SLUCHAM

Arabic – As salam ‘alakum

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Pa#ent  generated  data  –  The  transi#on  from  “more”  to  “be6er”    

 HEC  2016  Workshop    

WS  884  Pu(ng  User-­‐Generated  Data  in  Ac8on:  Improving  Interpretability  for  Clinical  and  Consumer  Informa8cs  

Aug  30  16:30  -­‐  18:00    

Panelists:  Thomas  WETTER,  Ying-­‐Kuen  CHEUNG,  Sanjoy  DEY  ,  XinXin  ZHU,  Bian  YANG    

Moderator:  Pei-­‐Yun  Sabrina  Hsueh      

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics  HEC/MIE  2016  Workshop:  PuJng  User-­‐Generated  Data  in  Ac#on:  Improving  Interpretability  for  Clinical  and  Consumer  Informa#cs       Katie Zhu, PhD MD

(IBM TJ Watson Research, USA)

Sanjoy Dey, PhD

(IBM TJ Watson Research, USA)

Ken Cheung, PhD

(Columbia University, USA)

Bian Yang

(Norwegian University of Science of Technology, Norway)

Thomas Wetter, PhD (Panel Discussant)

(University of Heidelberg, Germany

University of Washington, USA)

Pei-Yun Sabrina Hsueh, PhD (Moderator)

(IBM T.J. Watson Research, USA)

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Agenda    •   16:30-­‐16:40    Opening  Remark    by  Dr.  Sabrina  Hsueh  

•  EMERGING  HEALTHCARE  LANDSCAPE  SHIFT  WITH  PATIENT-­‐GENERATED  DATA      

•   16:40-­‐17:20    Presenta#ons  –  Dr.  Xinxin  Zhu:  So  we  got  sensor  data,  now  what?  –  Dr.  Sanjoy  Dey:  Enhancing  interpretability  of  computa#onal  model  

–  Dr.  Ken  Cheung:  SMART-­‐AR  to  evaluate  health  apps  for  outcome  op#miza#on  

–  Dr.  Bian  Yang:  The  need  for  addressing  privacy  issues  with  be6er  interpretable  rules  •   17:20-­‐18:00    Discussant  summary  presenta#on  &  Panel  

discussion/audience  Q&A    –  Dr.  Thomas  We6er:  Pa#ent  generated  data  –  The  transi#on  from  “more”  to  “be6er”  

–  Panel  discussion  (moderated  by  Dr.  Sabrina  Hsueh)  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  on  Workshop  Theme  (1)  

•  (1)  Iden#fy  immediate  ac#on  items  to  start  ini#a#ng  proposal  for  enabling  evidence-­‐based  conversa#on  with  pa#ents/physicians/providers  in  the  loop  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  on  Workshop  Theme  (1)  

•  2.  Implica#ons  and  lessons  learned  from  the  case  studies  -­‐-­‐  especially  the  gaps  you  perceived  as  barriers  of  entry  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  on  Workshop  Theme  (2)  

•  3.  Requirements  for  successful  redesign  of  healthcare  systems  to  accommodate  pa#ent-­‐generated  informa#on  (with  a  sub-­‐goal  of  iden#fying  the  areas  where  such  informa#on  can  make  most  impacts).  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Ques#ons  •  1.  What  is  the  state-­‐of-­‐the-­‐art?  •  2.  What  are  the  benefits  of  improving  interpretability  in  PGHD  

in  ac#on?  •  3.  What  the  key  dimension  of  interpretability  of  PGHD?    What  

are  the  barriers?  Technical/social?  •  4.  What  is  our  defini#on  of  interpretability?  What  are  the  

likely  measures?  •  5.  What  is  the  opportunity  area  going  forward?  •  6.  What  are  the  likely  ac#on  items  to  be  suggested  to  the  

community  to  further  the  discussion  about  improving  interpretability  for  PGHD?      –  In  the  field  of  consumer  health  informa#cs  or  beyond?  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

INTRODUCTION  

EMERGING  HEALTHCARE  LANDSCAPE  SHIFT  WITH  PATIENT-­‐GENERATED  DATA    

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Pei-Yun (Sabrina) Hsueh, PhD

Wellness  Analy8cs  Lead  Global  Technology  Outlook  Healthcare  Topic  co-­‐Lead  Healthcare  Informa8cs  PIC  co-­‐Chair            Computa8onal  Behavioral  and  Decision  Science  Group            Health  Informa8cs  Research  Dept.            IBM  T.  J.  Watson  Research  Center      •   Research  focus:  Pa8ent-­‐genera8on  info  from  wearables  and  biosensor  

devices/implants,  Personaliza8on  analy8cs,  Pa8ent  engagement  &  Adherence  risk  mi8ga8on,  Interpretable  machine  learning  

Opening Remark

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Source:  Based  on  McGinnis  et  al,  The  Case  for  More  Active  Policy  Attention  to  Health  Promotion,  Health  Affairs,  2002.  

Health  Determinants  Mismatches  Today’s  Spending  “We  need  to  invest  in  addressing  all  determinants  of  health…”  

BIG DATA Clinical + behavior

driven Wellness Management

It’s Big Data! It is also not just Big Data!

SOURCE: Barbara J. Sowada, A Call to Be Whole: The Fundamentals of Health Care Reform, July 30, 2003, Praeger.

IBM Watson // ©2015 IBM Corporation

NOISY, LARGE VOLUME, UNCONTROLLED

Need minimum description & quality/validity study

Solutions Population Health

Management

Condition Specific Care

Health and Wellness

Social Programs

Discovery Solutions

Real World Evidence

Ecosystem Population Health

Management

Condition Specific Care

Health and Wellness

Social Programs

Discovery Solutions

Real World Evidence

Individual

Social Programs

Education

Governments

Home Health Agencies

Practitioners

Hospitals

Therapists

Health Plans

Family

Public Health

Medical Devicesand Diagnostics

Bio-Pharma

Employers

Payers

DataInsight

IBM Watson // ©2015 IBM Corporation

To tap into the potential of DTR in open deployment, accessing a vast amount of untapped data could have a great impact

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

14

PGHD: Beyond Capturing Social/Behavioral Determinants from EHR

Institute of Medicine report (2016)

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

15

•  R.W. White, R. Harpaz, N.H. Shah, W. DuMouchel, and E. Horvitz. Toward Enhanced Pharmacovigilance using Patient-Generated Data on the Internet, Nature CPT, April 2014.

Success Story: PGHD for Pharmacovigilance

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Success Story: PGHD for Personalized Communication

Palmquist, A.E.L., Koehly, L.M., Peterson, S.K. et al. J Genet Counsel (2010) 19: 473. doi:10.1007/s10897-010-9299-8

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Success Story: PGHD for Diagnosis

Identify the onset and progression of disease states e.g., depression, Parkinson’s, PTSD

Assist with decision making in ER (e.g., FITBIT CHARGE HR)

Source: 1. http://www.androidauthority.com/fibit-charge-hr-save-patient-685205/ 2. M. Sung, C. Marci, and A.S. Pentland, Objective Physiological and Behavioral Measures for Identifying and Tracking Depression State in Clinically Depressed Patients, MIT Technical Report, 595 (2005): 1-20. 3. S. Arora, V. Venkataraman, S. Donohue, K.M. Biglan, E.R. Dorsey, M.A. Little, High accuracy discrimination of Parkinson’s disease participants from healthy controls using smartphones, IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), 3641–3644.

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Success Story: PGHD for Care Coordination

IBM Taiwan Collaboratory

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

19

Promoting patient activation for behavioral change (Dietary intake: Burke et al., 05; Physical activity: Prestwich et al., 09; Michie et al., 09) Preventing lifestyle-related chronic diseases, e.g., Type II Diabetes

Helmrich et al, 1991;Bailey, 2001; Scottish Intercollegiate Guidelines Network, 2001; Finland National Type II Diabetes Prevention Programme, 2007; American Diabetes Prevention Program, 2008).

Increase awareness to self-monitoring (Prestwich et al., 09; Burke et al., 05)

Triggering reminders to care plans (Consolvo et al. 09; Hurling et al., 07) Personalizing communication messages and education materials (Thaler and Sustein, ‘08)

Making  Sense  of  PGHD  for  Individuals  

Nudge: Improving Decisions About Health

PERSONAL INFORMATICS TOOLS (auto PGHD capturing + manual input)

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

20

The Failure of Scripps Trial

Patients who monitored their health were less likely to attribute health outcomes to chance than those who didn’t monitor their health

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Where do we meet in the middle?

???

Unsustainable, ill-supported health consumers

Healthcare Triple aim

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

22

Reference Story: Kaiser Permanente – Improved Outcome and Reduced Co Individualized Guideline Improved Clinical Outcomes §  Reduce 5-year CVD risk 2.4 times more than EHR+panel support tool alone (≈ 13% absolute risk reduction)

§  ≈ 6,000 myocardial infarctions (MIs) and strokes prevented annually if applied throughout KP (≈43% increase over JNC7 guideline for the same cost)

Individualized Guideline Reduced Operational Costs §  ≈ $7,000 cost savings per MI and stroke §  ≈ $420M annual net savings if applied throughout KP

Source: Eddy, et al. (2011). Individualized Guidelines: The Potential for Increasing Quality and Reducing Costs. Annals of Internal Medicine, vol. 154, no. 9, p.627-634. http://www.annals.org/content/154/9/627.abstract

22

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

23

Kaiser Permanente – Improved Patient Motivation and Adherence, Increased Clinician Confidence

(Respondents were)“…more likely to report that they have been asked to change their medication, diet and exercise habits. ”

—Patient Survey

“…helped the doctor to motivate them and helped them participate in their treatment choices, i.e., making lifestyle changes and understanding the rationale for their suggested interventions.”

— Patient Focus Group

“All doctors agreed that it helps them to make the best clinical decisions for their patients.”

— Clinician Survey

23

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Adding High Touch by Lay Care Guides

•  Parallel-group randomized trial (2010-2012). –  6 primary care clinics in Minnesota. –  Adults with hypertension, diabetes, or heart failure. –  Assigned in a 2:1 ratio to a care guide or usual care.

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics  HEC/MIE  2016  Workshop:  PuJng  User-­‐Generated  Data  in  Ac#on:  Improving  Interpretability  for  Clinical  and  Consumer  Informa#cs       Katie Zhu, PhD MD

(IBM TJ Watson Research, USA)

Sanjoy Dey, PhD

(IBM TJ Watson Research, USA)

Ken Cheung, PhD

(Columbia University, USA)

Bian Yang

(Norwegian University of Science of Technology, Norway)

Thomas Wetter, PhD (Panel Discussant)

(University of Heidelberg, Germany

University of Washington, USA)

Pei-Yun Sabrina Hsueh, PhD (Moderator)

(IBM T.J. Watson Research, USA)

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Agenda    •   16:30-­‐16:40    Opening  Remark    by  Dr.  Sabrina  Hsueh  

•  EMERGING  HEALTHCARE  LANDSCAPE  SHIFT  WITH  PATIENT-­‐GENERATED  DATA      

•   16:40-­‐17:20    Presenta#ons  –  Dr.  Xinxin  Zhu:  So  we  got  sensor  data,  now  what?  –  Dr.  Sanjoy  Dey:  Enhancing  interpretability  of  computa#onal  model  

–  Dr.  Ken  Cheung:  SMART-­‐AR  to  evaluate  health  apps  for  outcome  op#miza#on  

–  Dr.  Bian  Yang:  The  need  for  addressing  privacy  issues  with  be6er  interpretable  rules  •   17:20-­‐18:00    Discussant  summary  presenta#on  &  Panel  

discussion/audience  Q&A    –  Dr.  Thomas  We6er:  Pa#ent  generated  data  –  The  transi#on  from  “more”  to  “be6er”  

–  Panel  discussion  (moderated  by  Dr.  Sabrina  Hsueh)  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

SO  WE  GOT  SENSOR  DATA,  NOW  WHAT?  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

•  MD  (Anesthesiologist)  from  China  Medical  University  

•  PhD  in  Biomedical  Informa#cs  from  Columbia  University    

•  Past  Experience  –  Chief  Medical  Informa#on  Officer  at  Kforce  Government  Solu#ons,  U.S.A.  

–  Associate  Medical  Director,  Pfizer  Health  Solu#ons,  U.S.A.  

–  Senior  Manager,  Pfizer  Health  Solu#ons,  U.S.A.    

–  Clinical  Program  Manager,  Philips  North  America  Research  Center,  U.S.A.    

–  Healthcare  Informa#cs  Subject  Ma6er  Expert,  Veterans  Affairs  Medical  Center,  U.S.A.  

 

Xinxin (Katie) Zhu •  Telehealth lead at IBM

Watson •  External Advisory Board

member to Columbia Univ. Center of Advanced Technology

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

So  we  got  sensor  data,  now  what?  

•  What  sensor  data  could  help  with  care?    •  How  to  determine  when  to  use  what?    •  Are  the  sensor  data  reliable?  •  What  is  the  context  when  data  were  collected?  •  How  to  interpret  data  in  context?  •  Clinicians’  concerns    

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

What  sensor  data  could  help  with  care?  Use  case:  stress  management  

Subjec#ve  Stressors  

Psychological  Response  

Physiological  Response  

Stress Hormones

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Many  sensors  are  out  there…    Tinke  

31  

Approach •  Plug into a smartphone •  Scan finger •  Provide stress/relax index

Data Tracked •  Heart rate variability •  Respiration rate •  Blood oxygen level

Tinke Website

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Spire  

32  

Data Tracked •  Breathing pattern •  Steps

Approach •  Consistent breaths à Calmness •  Uneven breaths à Tension •  Fast and consistent breaths à Focus •  Guided meditation

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Pip  

33  

Data Tracked •  Skin conductance (EDA)

Approach •  Hold device between the

thumb and index fingers •  Stress level via audio/

visual feedback

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Many  sensors  are  out  there  

Brain Wave (EEG sensor)

Skin Conductance (EDA sensor)

Blood Volume Pulse (PPG sensor)

Skin Temperature (Infrared Thermophile)

Heart Rate (PPG sensor)

Heart Rate Variability (ECG sensor)

Respiration Rate/Volume (RIP sensor)

RR Interval Distribution (ECG sensor)

Image Source: Neurosky, Empatica, Hexoskin

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Hexoskin  V.S.  BioSens  Holter  ECG  Valida#on  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Brain  Wave  

36  

Relaxed Reading a paper with a time limit

Delta - Adult slow wave sleep Theta - Drowsiness, idling, inhibition Alpha - Relaxed, reflecting Beta - Alert, busy, anxious, thinking Gamma - Short term memory usage Mu - Rest state motor neuron activity

-  Produced by electrical pulses from neuron communication

-  Frequency bands associated with different behaviors and emotions

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

How  can  people  make  sense  of  these?  

37  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Clinicians’  concerns    

Information overload Unreliable data à false alarms

Clinical workflow Context, context, context!

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  on  Workshop  Theme  (1)  

•  (1)  Iden#fy  immediate  ac#on  items  to  start  ini#a#ng  proposal  for  enabling  evidence-­‐based  conversa#on  with  pa#ents/physicians/providers  in  the  loop  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  on  Workshop  Theme  (2)  

•  2.  Implica#ons  and  lessons  learned  from  the  case  studies  -­‐-­‐  especially  the  gaps  you  perceived  as  barriers  of  entry  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  on  Workshop  Theme  (3)  

•  3.  Requirements  for  successful  redesign  of  healthcare  systems  to  accommodate  pa#ent-­‐generated  informa#on  (with  a  sub-­‐goal  of  iden#fying  the  areas  where  such  informa#on  can  make  most  impacts).  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Ques#ons  (preliminary)  •  1.  What  is  the  state-­‐of-­‐the-­‐art?  •  2.  What  are  the  benefits  of  improving  interpretability  in  PGHD  

in  ac#on?  •  3.  What  the  key  dimension  of  interpretability  of  PGHD?    What  

are  the  barriers?  Technical/social?  •  4.  What  is  our  defini#on  of  interpretability?  What  are  the  

likely  measures?  •  5.  What  is  the  opportunity  area  going  forward?  •  6.  What  are  the  likely  ac#on  items  to  be  suggested  to  the  

community  to  further  the  discussion  about  improving  interpretability  for  PGHD?      –  In  the  field  of  consumer  health  informa#cs  or  beyond?  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

ENHANCE  INTERPRETABILITY  WITH  PRIOR  KNOWLEDGE  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Sanjoy  Dey  PhD.    

Postdoctoral Research Scientist, Center of Computational Health, IBM T. J. Watson Research Center, Yorktown Heights, NY 10598

Sanjoy  Dey’s  research  interests  lie  in  the  areas  of  health  care  informa#cs,  data  mining  

and  machine  learning,  especially  in  building  interpretable  models  by  integra#ng  mul#ple  healthcare  datasets.  .  In  par#cular,  Sanjoy  is  interested  in  building  models  

which  aim  to  incorporate  domain  knowledge  at  mul#ple  stages  of  model  development  (e.g.,  feature  selec#on,  cohort  selec#on  and  study  design)  so  that  these  models  can  infer  knowledge  that  are  complementary  to  the  already  known  clinical  prac#ces  and  

guidelines.  Prior  to  this  posi#on,  he  earned  his  Ph.  D.  from  the  department  of  computer  science  at  university  of  Minnesota.  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Improving Interpretability of Patients Generated Data

45

Dis

ease

H

ealth

y

Dataset 1 Dataset 2

Class  label  

Relation across the datasets

 

Analysing  the  obtained  results  from  Complex  Models  •  Interpret  the  model  parameters  so  that  they  

can  be  used  to  infer  meaningful  knowledge  •  Visualize  the  obtained  informa#on  from  a  

model  in  a  meaningful  way  

Taking  prior  knowledge  into  account  

•  Many  #mes,  medical  knowledge  are  available  containing  useful  rela#onships  among  clinical  events  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Dataset

Interpre#ng  Complex  Computa#onal  Models  

•  Complex  model  parameters  can  be  converted  to  metrics  that  are  easily  understandable  by  domain  experts  

–  Logis#c  Regression    –  LASSO  with  regulariza#on  to  perform  

simultaneous  variable  selec#on  

•   Logis#c  loss  func#on  can  be  used  as  Log  Odds,  which  can  be  converted  to  Odds  Ra#o  -­‐      where  β0  the  log  odds  for  smoking  for  men    

•  Probabili#es  of  an  event  can  be  viewed  as  clinical  uncertainty  

Class Level

Liu et al. 2011, Jeiping Ye et al., 2012

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Visualiza#on  of  the  obtained  model  

Decision  Boundaries  of  Logis#c  Regression  

Rule  based  representa#on  of  Decision  Tree  and  Cart  based  Models  

Graphical  Models  for  Disease  Models    

Work   Environment     Gene    

Disease    

Symptoms  

Westra  et  al.  2011,  Manzi  et  al.    2013  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Integra#ng  Prior  Knowledge  with  Mul#-­‐source  EHR  data  for  Enhancing  Interpretability    

Diagnosis  Codes    (ICD-­‐9)  

Admission  Assessment  

Survey  

Discharge  Assessment  

Survey  

Home Healthcare

Dey et al. AMIA 13, Dey et. al., SDM 14, Westra et al. 11

Demographic, behavioral, pathological, psycho-social factors, outcome variables.

Problem Formulation:

48

260,000 patients

Data source: CMS OASIS dataset

Outcome prediction

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Enhancing Interpretability of Patterns

49

Predictive Power

Inte

rpre

tabi

lity

•  Interpretability  (Relevance)  and  predic#on  power  are  different  goals  

•  Prior  rela#onships  present  in  the  data  can  be  incorporated  into  model  

ICD-­‐9  Group  1   ICD-­‐9  Group  2  250.6:  Neurological  manifesta#on  

401.1:  Benign  hypertension  

290:  Demen#a   838:  Disloca#on  of  foot  331:  Alzheimer’s  disease     692.71:  Sunburn  331.9:  Cerebral  degenera#ons  

V58.42:  Hip  joint  replacement  

Neural disorders No common underlying disease

Interpretable   Predic8ve  

•  Which  group  of  pa#ents  are  likely  to  improve  ambula#on?  

•  Are  those  factors  clinically  interpretable  and  make  a  homogeneous  group?  

G1  

G2  

Ideal  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Proposed Approach Key  Steps:  •  Integrate  both  survey  data  from  EHR  and  

ICD-­‐9  diagnoses  codes  to  predict  the  improvement  of  urinary  incon#nence  

•  Use  clinical  prior  knowledge  such  as  Clinical  Classifica#on  Sotware  (CCS)  into  account  to  increase  the  interpretability  

•  Develop  a  sta#s#cal  technique  called  Sparse  Hierarchical  Canonical  Correla#on  Analysis  (SHCCA)  to  address  these  challenges  

50

X Y

Algorithm:  •  Take  the  hierarchy  of  the  CCS  tree  into  account  to  

define  a  similarity  matrix  called  H  among  the  ICD-­‐9  codes  

•  Trade-­‐off  between  the  data-­‐driven  and  prior  knowledge  driven  similarity  of  ICD-­‐9  codes  using    λh    

•  Converted  into  convex  formula#ons  

•  Solve  the  final  equa#on  based  on  gradient  descent  formula#on  

Prior  Knowledge  

λh trades off between domain-driven and data-driven knowledge

Dey et. al., SDM 14

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Sparse Hierarchical CCA (SHCCA) Parameter  Selec8on:  •  Op#mize  the  parameters  using  cross-­‐valida#on  such  that  it  

op#mizes  the  correla#on  on  valida#on  data    Evalua8on:  Predic8on  power:  how  well  the  selected  group  of  ICD-­‐9  codes  can  predict  the  improvement  of  outcome  Interpretability:  –  I-­‐score  based  on  the  co-­‐occurrences  of  the  ICD-­‐9  terms  

belonging  to  a  group  C  in  PubMed  ar#cles  –  Domain  knowledge  by  physicians  and  nurses  

 51

ti is the set of articles found with ICD-9 code i

I-­‐score(C)=∑𝑖ϵ𝐶↑▒∑𝑗ϵ𝐶↑▒| 𝑡↓𝑖   ⋂ 𝑡↓𝑗 |/| 𝑡↓𝑖   ∪ 𝑡↓𝑗 |     

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Results •  SHCCA  has  similar  performances  as  

the  baseline  methods,  but  with  fewer  components  

•  It  enhances  the  interpretability  significantly    

 

Predictive power of SHCCA

52

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Components from SHCCA

Survey  data  1   ICD-­‐9  codes  1   Survey  data  2   ICD-­‐9  codes  2  

Age,  Prior  Memory  Loss,  

Poor  Speech,  Poor  Cogni8ve  

Func8on,    High  Confusion,  

Memory  Deficiency,  Frequent  Behavioral  

Problem    

         

Demen8as,  Persistent    

mental  disorders,  Alzheimer's  disease,  Cerebral  

degenera8ons                    

Surgical  Wound,  Fully  

granulated  Surgical  Wound  

                         

Acercare  for  healing  fracture  of  hip,  

Knee  joint  replacement,  Hip  joint  replacement,  

Acercare  following  surgery  of  the  musculoskeletal  system,  Acercare  following  joint  

replacement,  Acercare  following  surgery  for  

neoplasm,  Acercare  following  surgery  of  

the  circulatory  system       53

Component relevant to Mental health

Component relevant to surgical treatment

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Summary  &  Limita#ons  •  Summary  

–  Predic#on  power  and  interpretability  are  two  different  goals,  which  are  oten  hard  to  achieve  by  computa#on  models  simultaneously  

 –  Predic#ve  models  can  be  post-­‐processed  and  visualized  to  make  them  more  

interpretable  

–  Leveraging  clinical  prior  knowledge  such  as  Clinical  Classifica#on  Sotware  (CCS)  into  account  can  increase  the  interpretability  substan#ally  

 

•  Limita#ons  –  The  defini#on  of  interpretability  is  oten  subjec#ve  and  oten  requires  domain  

exper#se  

–  Prior  knowledge  about  a  par#cular  problem  is  not  oten  readily  available  in  many  clinical  applica#ons  

 –  Use  of  prior  knowledge  into  the  model  op#miza#on  is  oten  not  straighvorward  

54

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

SMART-­‐AR  to  evaluate  health  apps  for  outcome  op#miza#on  

Ken  Cheung  Columbia  University  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Ying  Kuen  (Ken)  Cheung  

•  PhD  in  Sta#s#cs  (U  Wisconsin,  Madison  WI,  USA)  

•  Professor  of  Biosta#s#cs,  Columbia  University,  New  York  NY,  USA  

•  General  interest:  Transla#onal  research  in  all  phases  •  Specific  areas  

•  Dose  and  treatment  selec#on  in  adap#ve  clinical  trials  •  Op#mal  behavioral  interven#on  for  secondary  stroke  preven#on  •  Analysis  of  high-­‐dimensional  physical  ac#vity  data    •  N-­‐of-­‐1  trial  designs    •  Evalua#on  and  dissemina#on  of  mobile  technologies  for  mental  

health  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Objec#ve  &  Reinforcement  Learning  

•  Data  sequence:  (X,  A1,  U1,  A2,  U2,  …,  AK,  Y)  –  X  =  Individual  characteris#cs  –  At  =  Apps  downloaded  (Ac#on)  at  #me  t  –  Ut  =  response  and  use  pa6ern  between  At  and  At+1  

–  Y  =  Final  outcome  (depression  reduc#on)  

•  Objec#ve:  Iden#fy  the  sequence  At  based  on  X  and  Ut  so  as  to  maximize  Y  (on  average)  

•  Reinforcement  learning:  Q-­‐learning,  OWL,  etc.  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

SMART  Design  

•  SMART  (Sequen#al  Mul#ple  Assignment  Randomized  Trial)  

App  1  

App  2  

Ac#ve  use  

Ac#ve  use  

Non-­‐use  

Non-­‐use  

App  1  +  App  3  

App  2  

App  2  +  App  3  

App  2  +  reminder  

App  1  

App  1  +  reminder  

App  2  

App  3  

Depression  reduc8on  at  6  months,  Y    

Enrichment  based  on  intermediate  use  paeern,  U  

P  =  2/3  

P  =  1/3  

P  =  0.3  

P  =  0.7  

P  =  0.6  

P  =  0.4  

P  =  0.6  

P  =  0.4  

P  =  0.3  

P  =  0.7  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

SMART-­‐AR  Design  (Cheung  et  al,  2015  Biometrics)  

•  SMART  Design  ü  Allows  learning  ✗  No  feedback  to  system  ✗  Curse  of  dimensionality:  many  apps  

in  prac#ce  

•  SMART-­‐AR  •  AR  =  Adap#ve  randomiza#on  •  Assign  more  users  to  more  

promising  branches  •  Curse  of  dimensionality:  Sot  

elimina#on  of  poor  performing  apps  à  Improve  signal-­‐noise  ra8o,  hence  interpretability  of  the  recommender    

 

0 20 40 60 80 100

910

1112

1314

15

Enrollment number

BD

I red

uctio

n

o o o o o o o o o o o o

+ + + + + + + + + + + +

p p p p p p p p p p p p

m m m m m m m m m m m m

Scenario 1

CODIACSBalanced

Application in conventional depression program

Cheung et al, 2015 Biometrics

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Some  Simula#on  Illustra#on  Non-­‐adap8ve  SMART   SMART-­‐AR  

Balanced  randomiza#on  

CODIACS  randomiza#on  

Scenario  1  

Probability  of  iden#fying  the  op#mal  sequence  

0.91   0.94   0.95  

Expected  adjusted  value*   0.98   0.99   0.99  

Variance  of  adjusted  value   3.1   2.3   1.3  

Scenario  3  

Probability  of  iden#fying  the  op#mal  sequence  

0.53   0.51   0.51  

Expected  adjusted  value*   0.95   0.95   0.96  

Variance  of  adjusted  value   8.5   11.0   7.4  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Health  informa#cs  support  issues  

•  SMART-­‐AR  requires  real  #me  transmission  between  data  site,  apps  cura#on  site,  compu#ng  site  –  Large  volume:  Use  data  pre-­‐processing  

–  Privacy  &  security  •  Health  outcomes  

–  Valida#on  of  outcomes  

0 10 20 30 40

Days since download first app

Ap

p ID

12

34

56

78

910

11

12

13

14

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

ADDRESSING  PRIVACY/SECURITY  CONCERN  WITH  INTERPRETABILITY  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Bian’  intro  goes  here….  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

- February 4, 2015 -  Hacker broke into the medical

insurance database -  80 million records stolen in

plaintext -  Insurance company’s database

are not required to be encrypted by HIPAA

-  administrator's credentials were compromised

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Now  …  the  hot  term  for  2016  –  «ransomware»    

-  More than half of hospitals (in US) hit with ransomware in last 12 months

(HealthcareITNews, April 07, 2016)

-  Good business model for the hackers -  Low risk -  Good cost-benefit efficiency -  Easy to build "reputation" for

the service –

(https://www.theguardian.com/technology/2016/feb/17/los-angeles-hospital-hacked-ransom-bitcoin-hollywood-presbyterian-medical-center)

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  A6ack  vectors  to  Pa#ent  Data  Security    

More attacking vectors opened due to … the shifts of healthcare patterns - now and future

•  hospital -> home / cyber space (telemedicine, IoT, mobile technologies, care research)

•  in-hospital treatment -> prevention (big data, health analytics, health electronics, e-drugs)

•  doctor-centered -> patient-centered (telemedicine, big data, machine intelligence, cloud storage and computing)

•  health care organizations -> associated business partners in liability (law and regulations, e.g., HIPAA -> HITECH (2009))

•  Local service -> global service (service across the borders) •  the “SafeHarbor” agreement •  Facebook fined by the Belgian court

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

What  does  security  mean  for  eHeath/mHealth’s  future  

-  new breaches and "business models for hacking" would continue to come… (but take it easy)

-  More liability to IT tech enablers and business associates (e.g, HIPAA ->HITECH)

-  cloud / SDN makes "security as a service" that can be outsourced

-  IT Tycoons (Microsoft, IBM, Google, etc.) could finally take it over (capable to take risk, more resources, global threat intelligence, etc.)

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Future  Solu#ons  

-  Data ownership re-definition -  Generating incentives for industry to migrate from

data silos to data sharing -  Patients’ awareness of their interests in their own

data -  Patients’ convenience in accessing their own data -  Legal support -  Technology: security / privacy by design

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

DISCUSSANT  SUMMARY  PRESENTATION  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Prof.  Dr.  Thomas  Weeer  •  MSc  /  PhD  in  mathema8cs  from  Aachen  Technical  U,  

Germany  •  PostDoc  with  IBM  Scien#fic  Center  Heidelberg,  Germany  •  Since  1997  Prof.  of  Medical  Informa8cs,  Heidelberg  U  

–  Interna8onal  assignments  to  Boca  Raton  (FL),  Aus#n  (TX),  Salt  Lake  City  (UT),  Sea6le  (WA)  

–  Affil.  Faculty  with  Dept.  BIME,  U  of  Washington,  Sea6le  –  Author  of  textbook    

Consumer  Health  Informa#cs:  New  Services,  Roles  and  Responsibili#es;  Heidelberg  (Springer)  2015  (eBook)  resp.  2016  (Hardcover)  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Pa#ent  generated  data  –  The  transi#on  from  “more”  to  “be6er”    

Thomas  We6er  

May be obsolete here with the title slide already

using this paraphrase of the workshop title

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Growth  is  everywhere  •  More  modali#es  to  collect  and  store  data  are  offered  •  More  communica#on  media  carry  health  info  •  More  condi#ons  suggest  to  be  monitored  •  More  ins#tu#ons  consider  usage    •  More  consumers  buy  in  

•  Does  this  make  sense?  •  How  can  we  move  towards  meaningful  ac#on?    •  How  can  we  protect  against  unethical  exploita#on?  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Growth  is  everywhere  •  More  modali#es  to  collect  and  store  data  are  offered  •  More  communica#on  media  carry  health  info  •  More  condi#ons  suggest  to  be  monitored  •  More  ins#tu#ons  consider  usage    •  More  consumers  buy  in  

•  Does  this  make  sense?  •  How  can  we  move  towards  meaningful  ac#on?    •  How  can  we  protect  against  unethical  exploita#on?  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  is  more  op#ons  

   .  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  is  more  op#ons  

 

Period  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

But  beware  

Date  are  not  the  world;  data  map  the  world  –  truthfully?  What  is  the  ci#zen‘s  contribu#on  to  the  mapping?  •  Carrier  of  implanted  sensors  •  Operator  of  a6ached  and  mobile  sensors    •  Witness  of  health  signs    •  Interpreter  of  health  signs  •  Self  therapist  •  Health  plan  contractor  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  is  more  tempta#ons    

Ci#zens  •  Trust  more  than  warranted  •  Shit  focus  from  senses  to  data    Clinicians  •  Shit  focus  from  senses  to  data  Researchers,  public  health  •  Urge  to  find  something  Big  business  •  More  big  business    

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  is  more  tempta#ons:  Ci#zens  

Percep#on  of  the  presumably  unfailable  objec#ve  givens  as  proxy  for  truth  •  Mental  fixa#on  on  data  •  Unwarranted  trust  as  decision  aid  •  Adverse  reac#on  to  contradictary  data  •  Overreac#on  upon  alarming  data  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  is  more  tempta#ons:  Researchers,  Public  health  Percep#on  that  regarding  the  massive  volume  of  data  there  cannot  be  no  effects  •  Do  the  big  data  mechanics  •  Spot  peculiari#es  •  Publish  results  Knowing  that  5%  of  significant  studies  are  not  substan#ated  through  an  effect  

Curb: Complexity reduction –

Sanjoy Rey, Ken CheungRisk: Blindfolded actionism

Curb: Plausibility, context – Katie ZhuRisk: Funding agency expectations Curb: Research ethics

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  is  more  expecta#ons  

If  scien#st  dispose  of  more  data  their  methods  are  challenged:    •  Profound  interpreta#on  and  predic#on  –  Sanjoy  Dey  •  Parsimony,  wise  selec#on  –  Sanjoy  Dey  •  Secure    storage/communica#on  –  Bian  Yang  •  Insight  –  Ken  Cheung,  Sabrina  Hsueh  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  is  more  expecta#ons  

If  ci#zens  volunteer  their  data,  they  expect  services:    •  Serious  PGHD  into  PHR  into  EHR  integra#on  •  No  data  leakage  •  Explana#ons  of  the  unexplainable  •  Emergency  rescue  in  response  to  alarming  data  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

More  data  can  be  the  hays#ck  

•  where  we  don‘t  find  the  needle  •  while  being  distracted  by  –  hay  •  but  someone  needs  the  needle  –  now  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Claude  Shannon  1948  1)  

 „Informa#on  is  that  which  reduces  uncertainty“    Which  the  needle  in  the  hays#ck  does  not  do    

1)  A mathematical theory of communication Bell Systems Technical Journal

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Bring  forth  the  signal  from  the  noise  

•  Concentrate  trials  on  treatments  with  emerging  posi#v  prognosis  (Ken  Cheung)  

•  Select  data  with  high  interpreta#ve  or  predic#ve  power  (Sanjoy  Dey)  

•  Regard  context  to  detect  noise  (Ka#e  Zhu)  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Are  we  achieving  quality  that  sa#sfies  doctors?  

•  Not  a  ma6er  of  taste  •  Doctors‘  code  of  conduct  regulates  that    when  trea#ng  diagnosed  pa#ents  he  –  assumes  responsibility  for  correct  recordings  of  devices  he  hands  to  the  pa#ents  

–  has  to  waive  liability  for  data  generated  through  other  pa#ent  solicited  devices  

while,  when  coaching  for  healthy  lifestyle  –  anything  goes  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Are  we  achieving  quality  that  sa#sfies  doctors?  

•  Under  a  treatment  contract  a  doctor  is  held  responsible  for  medical  errors.  

•  Morally,  he  cannot  be  held  responsible  for  decisions  based  on  false/faked  data  from  outside  his  control  

•  Pa#ents  want  their  data  used  •  They  cannot  guarantee  correct  data  

•  A  classical  gridlock  1)  

1) In NY/NY en.wikipedia.org/wiki/Gridlock

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

 Who  can  do  what    

to  solve  the  gridlock?  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Are  we  achieving  full  transparency?  Do  we  want  it?  

Imagine  that  a  certain  set  of  sensor  data  is  so  characteris#c  of  you  that  you  need  not  register,  just  

deliver  a  sample  and  they  know  who  you  are.  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Are  we  suppor#ng  personalized  medicine?  

•  If  the  wealth  of  our  data  is  so  large  that  we  can  iden#fy  data-­‐twins  –  A  treatment  for  the  second  twin  should  work  if  it  did  for  the  first  

–  The  end  of  clinical  trials  

Putting User Generated Data in Action: Improving Interpretability for Clinical and Consumer Informatics

Thank You

Merci Grazie

Gracias

Obrigado Danke

Japanese

English

French

Russian

German

Italian

Spanish

Brazilian Portuguese Arabic

Traditional Chinese

Simplified Chinese

Hindi Tamil

Thai Korean

Hebrew