Boabab Health, March 2014 Harry Hochheiser, harryh@pitt.edu

Human-Computer Interaction - Observations

Harry Hochheiser University of Pittsburgh Department of Biomedical Informatics !harryh@pitt.edu !+1 410 648 9300

Attribution-ShareAlike CC BY-SA

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Research Setup

● User(s) with clinical problems

● Current approach not optimal

● How do we know what to build?

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Important Claim #1

● Understanding work in context – goals, motivations, priorities, behavior, difficulties, etc. is necessary for building better systems

!● Most go beyond just talking about computer systems to

address bigger picture questions

!

● What if we build a pharmacy prescription tracking system…

● ..but the pharmacy never has any medication…

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Important Claim #2 ● Successful implementations may require work redesign

● Translating the same old methods and procedures to computers may not help much

● Must go beyond what we are doing?

● Instead, ask why?

!

● In-depth qualitative research needed to inform these efforts

● Not numbers. Rather, goals, methods, activities..

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Important Claim #3 ● In-depth qualitative research needed to inform these

efforts

● Not numbers. Rather, goals, methods, activities..

● Watch, listen , interpret

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

What are the factors that influence usability?

Is a given tool usable for all users?

For some users?

!

Is an interface full of text usable… for someone who can’t read?

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Determinants of Usability

Context: social, organizational, etc.

Tool

Task Data analysis? Writing? Graphing?

usability= f(user,tool, task, context,system)

System: desktop, mobile, tablet, etc.

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Systems Engineering for Patient Safety

Lowry, et al. Integrating Electronic Health Records into Clinical Workflow: An Application of Human Factors Modeling Methods to Ambulatory Care National Institute of Standards and Technology NISTIR 7988 http://dx.doi.org/10.6028/NIST.IR.7988

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Broad Goal

• Use study of users to understand how to build usable systems

• What is their work?

• What struggles are they facing?

• What information do they need?

• Why do they do what they do?

• How can IT help? Where can’t it help?

• What will be required for an IT tool to work?

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Approaches• Engage with users to see how work is done

• Interview

• Observe

• Collaborate

• Build models describing work

• Validate models with users

• Use models to design appropriate interactions.

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Some Methodologies

Participatory Design

Contextual Design

Scenario-Based Design

Cooperative Inquiry

Participatory Action

Research

Methodologies

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Key Questions & Tradeoffs

● Who to involve?

● When to involve users?

● How to collect information?

● How to interpret?

● How to inform design?

● How to evaluate success?

!● Usual tradeoffs apply: Never enough time or money

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Stakeholder AnalysisRosson & Carroll 2002

● Identify stakeholder groups

● Background

● Expectations

● Needs

● Preferences

● Concerns

● Values

!● An important, but often overlooked step

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Stakeholders

● Anyone who has an interest in the outcome of a system

● Work, play, or some other aspect of life

● Customer – those who pay for the work

● User – those who work with the system

● Others – perhaps those who are described by data in the system

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Stakeholders - Challenges

● Defined by roles, not by person

● Billing clerk for the hospital system is likely also a health-care consumer

● Must identify people who can speak to different roles?

● How can we meaningfully integrate understanding of needs of diverse users?

● Patients, practitioners, financial people, bureaucrats?

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

A Spectrum of Possibilities for Engaging Stakeholders

Focus Groups

Traditional Written Requirements

Interviews

ObservationContextual InterviewsEthnography/ Participatory Design

Low Cost, Low Fidelity

High Cost, High Fidelity

Surveys

Diaries/ActivityRecording

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Observation: When• Understanding work in complex environments

• Often used when interviews with users are not possible

• busy clinical environments

• Even if possible, interviews might not be optimal

• Ecological validity: observations must be as close a match to reality as is practical.

• Least demanding on users and context

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Observation• Watch users as they complete tasks

• Who is involved

• What they are trying to accomplish

• How they meet their goals

• Look for

• struggles

• breakdowns

• workarounds

• contextual factors

• improvisation

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Methodology• Take notes - lots

• Pictures

• Diagrams

• Audio recordings (if privacy concerns allow)

• No interruption

• possibly debrief with user afterwards

• Multiple methodologies - triangulation

• Multiple observers - validation

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Guidelines• Get out of the way

• particularly in busy clinics

• Respect participant privacy

• no recording or photos in clinic

• Consider observing from multiple perspectives

• clinician and patient

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

After the observation• Debrief

• Summarize notes

• Transcribe any recordings

• Analyze

• build models (more on this later)

• identify new questions - what did you not ask

• ask these at subsequent interviews

• Do this as soon as possible!

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Iterative process

Data Collection

Analysis New Questions

Models/ Theories Convergence

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Drawbacks of observations• Receptive only

• Potential for misinterpretation

• Observation of work as it is done

• not as management or others think it is done.

• Solutions:

• multiple observers

• Debrief/review with participants post fact

• “I think this is what I saw…”

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Iterative process

Data Collection

Analysis New Questions

Models/ Theories Convergence

Validation with Users

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

“All models are wrong…”•Many possible ways to try to model user work

•Remember the goal:

• to do develop rich and valid understanding of work processes

• to use this understanding to drive design of appropriate software

• The particular methods may matter less than the process

• iteration

• validation

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Key Points• Understanding how work is done is vital for building tools

to support that work.

• Try to understand goals

• Redesign work if necessary

• Observations can provide useful data for driving these processes

• Need appropriate participants, triangulation, and validation.

Baobab Health, March 2014Harry Hochheiser, harryh@pitt.edu

Tomorrow: Bwaila ANC• Meet at Bwaila at 8:15

• Observe ANC clinic

• Return to Baobab: 9:45

• Discuss/review observations: 10:15

• Contextual inquiry presentation: 10:45

• Lunch: 11:45

• Arrive at Bwaila for contextual interviews with ANC clinicians and mock patients: 1:15