Design and evaluation of an interactive proof-of-concept dashboard for general practitioners
-
Upload
robin-de-croon -
Category
Healthcare
-
view
352 -
download
0
Transcript of Design and evaluation of an interactive proof-of-concept dashboard for general practitioners
DESIGN AND EVALUATION OF AN INTERACTIVE PROOF-OF-CONCEPT
DASHBOARD FOR GENERAL PRACTITIONERS Robin De Croon, Joris Klerkx, Erik Duval
IEEE International Conference on Healthcare Informatics 2015, Dallas
Patient follow-up
Patients• forget follow-up meetings• do not consider a follow-up necessary• do not want to spend additional money• have the impression treatment is not working
General practitioners• hard to find patients in need of follow-up• are too busy to accommodate a prompt visit• do not have enough time
Wednesday, October 21, 2015 2
How to improve follow-up quality?
Patient oriented
• Stimulate patient empowerment
General practitioner driven
• Provide better tools to augment workflow
Wednesday, October 21, 2015 3
Wednesday,October21,2015 4
{=,<,>,>=,<=,<>}|value|{and,or}|value
Research question
“How do we help general practitioners identify patients in need of follow-up using interactive visualizations?”
Wednesday, October 21, 2015 5
Research question
“How do we help general practitioners identify patients in need of follow-up using interactive visualizations?”
Wednesday, October 21, 2015 6
Research question
“How do we help general practitioners identify patients in need of follow-up using interactive visualizations?”
Wednesday, October 21, 2015 7
Research question
“How do we help general practitioners identify patients in need of follow-up using interactive visualizations?”
Wednesday, October 21, 2015 8
User centered, rapid prototyping
Wednesday, October 21, 2015 9
Initial design
Prototype 2- Content- Missing data
ResultFinal Design
- Updated table- Consistency- bugs
Prototype 3- Internal changes
Academic CenterGeneral Practice 3 GPs 12 infovis students 9 GPs
design phase usability phase evaluation phase
User centered, rapid prototyping
Wednesday, October 21, 2015 10
Initial design
Prototype 2- Content- Missing data
ResultFinal Design
- Updated table- Consistency- bugs
Prototype 3- Internal changes
Academic CenterGeneral Practice 3 GPs 12 infovis students 9 GPs
design phase usability phase evaluation phase
Visualization concepts
• Overview first, zoom and filter, then details-on-demand
• Data ink ratio
• Stevens’ model
Wednesday, October 21, 2015 11
BenShneiderman, TheEyesHaveIt:ATaskbyDataTypeTaxonomyforInformationVisualizations.In ProceedingsoftheIEEESymposium onVisual Languages, pages336-343,Washington.IEEEComputerSocietyPress, 1996
EdwardTufte. TheVisualDisplay ofQuantitativeInformation. 1983
S.Stevens, “Onthetheoryofscalesofmeasurement.”Science(NewYork,N.Y.),vol.103,no.2684,pp.677–680,1946
Demographics
Wednesday, October 21, 2015 12http://www.cdc.gov/nchs/data/databriefs/db81.htm
Populationpyramid Mostcommondiseases Religion
Highestdegree Maincaregiver
#dayssince lastvisit income systolicbloodpressur diabolicbloodpressure weight sugar age
Picture Name Condition Gender Age Degree Maincaregiver
Parallel coordinates
Wednesday, October 21, 2015 14
https://syntagmatic.github.io/parallel-coordinates/
User centered, rapid prototyping
Wednesday, October 21, 2015 15
Initial design
Prototype 2- Content- Missing data
ResultFinal Design
- Updated table- Consistency- bugs
Prototype 3- Internal changes
Academic CenterGeneral Practice 3 GPs 12 infovis students 9 GPs
design phase usability phase evaluation phase
Academic Center for General Practice
What• 3 general practitioners + 2 electronic medical record experts
• already perform audits on general practices
• Perceived usefulness
Result
• Too much demographics• Medication (groups) more important
• primary and secondary condition
• Deal with noise in data• outliers, missing elements
Wednesday, October 21, 2015 16
User centered, rapid prototyping
Wednesday, October 21, 2015 18
Initial design
Prototype 2- Content- Missing data
ResultFinal Design
- Updated table- Consistency- bugs
Prototype 3- Internal changes
Academic CenterGeneral Practice 3 GPs 12 infovis students 9 GPs
design phase usability phase evaluation phase
Usability iteration
What• 12 information visualization students• 10 tasks
• time-to-task & error-rate & perceived difficulty
• ± 60 minutes• Questionnaires
• initial & closing & System Usability Scale
Wednesday, October 21, 2015 19
Results usability evaluation
• System Usability Score: 68
• 7/10 tasks < 60s
Visual Updates:
• Visual update
• Consistent navigation
• Patient table
Wednesday, October 21, 2015 20
User centered, rapid prototyping
Wednesday, October 21, 2015 22
Initial design
Prototype 2- Content- Missing data
ResultFinal Design
- Updated table- Consistency- bugs
Prototype 3- Internal changes
Academic CenterGeneral Practice 3 GPs 12 infovis students 9 GPs
design phase usability phase evaluation phase
Final evaluation
• 9 general practitioners• 7 male + 2 females
• Concurrent Think Aloud protocol• short introduction, free interaction, 15 ~ 20 minutes
C. Lewis, Using the "thinking Aloud" Method in Cognitive Interface Design, (ibm resea ed., ser. Researchreport. Yorktown Heights, NY: IBM T.J. Watson Research Center
• System Usability ScaleBrooke, John. "SUS-A quick and dirty usability scale."Usability evaluation in industry 189.194 (1996): 4-7
• Questions from O’Leary et al. à likert questionsO’Leary, P., Carroll, N., & Richardson, I. (2014). The Practitioner’s Perspective on Clinical Pathway Support Systems. In 2014 IEEE International Conference on Healthcare Informatics (pp. 194–201)
• SWOT analysis
Wednesday, October 21, 2015 23
System Usability Scale
Wednesday, October 21, 2015
74
Bangor,A.,Kortum,P.,&Miller,J.(2009).Determiningwhatindividual SUSscoresmean:Addinganadjectiveratingscale.JournalofUsabilityStudies, 4(3),114–123
System Usability Scale
Wednesday, October 21, 2015 25
1 2 3 4 5
Learn a lot of things
Confident using the system
Very cumbersome to use
Learn to use this system very quickly
Too much inconsistency
Functions well integrated
Need the support of a technical person
Easy to use
Unnecessarily complex
Like to use this system frequently
Likert questions
Wednesday, October 21, 2015 26
1
1
2
3
5
1
3
6
4
2
2
3
3
5
6
3
3
4
2
5
2
4
2
0% 20% 40% 60% 80% 100%
Need for follow-up tool
Follow best practice
Train GPs
Correct level of detail
Recognize follow-up patients
Reducing mistakes
Useful feedback
Right kind of tool
totally disagree disagree neutral agree totally agree
O’Leary, P., Carroll, N., & Richardson, I. (2014). The Practitioner’s Perspective on Clinical Pathway Support Systems. In 2014 IEEE International Conference on Healthcare Informatics (pp. 194–201)
SWOT analysis
2722
34
34
5
2
3
4
4
2
2
3
3
4
5
Control systemPrivacy
Too little timeAverages can be dangerous
Can be improved with patient collected dataTriggers self-reflection
Ideal for research
Not needed oftenNot clear which content too show
Not much structured data availableMap uses too much screen estate
No pseudo code neededImproves team communication
Visual overviewAugment work
Check with guidelinesEase to select patients
Technologies
Wednesday, October 21, 2015 28
Limitations
• Sample size• total 15 GPs à exploratory study
• Semi realistic data• anonymous & assess perceived usefulness
• Requirements gathering• user centered, rapid prototyping
• Evaluation setting• limited time à perceived usefulness
Wednesday, October 21, 2015 29
Conclusion
• Powerful, yet easy to use
• Reduces burden to analyze patient records
• Recognize patients in need of follow-up
• Query patient with visual filters
• Needs to adapt to specific use cases
• Biggest opportunity is in research
Wednesday, October 21, 2015 30
http://cdn.makeuseof.com/wp-content/uploads/2012/12/3D-Man-Presenting-Intro-Image.jpg?a53b57
Looking for collaborations!
• Make the dashboard available in general practice• evaluate impact on collaboration• evaluate if self-reflection is triggered
• Continue working with academic GPs
Wednesday, October 21, 2015 31
Acknowledgements
Wednesday, October 21, 2015 32
Research grand: IWT 120896
Access to their products and expertise
For their experienced feedback
+ All participants!
Thank you!
Wednesday, October 21, 2015 33
http://2.bp.blogspot.com/-gZjNR3XVULs/T_ZOVgE-5lI/AAAAAAAAAg8/6YVmd5Q064o/s1600/questions11.jpg
information seeking mantra
34
BenShneiderman, TheEyesHaveIt:ATaskbyDataTypeTaxonomyforInformationVisualizations.In ProceedingsoftheIEEESymposiumonVisual Languages, pages336-343,Washington.IEEEComputerSocietyPress, 1996.
data ink ratio
Wednesday, October 21, 2015 35
EdwardTufte. TheVisualDisplay ofQuantitativeInformation. 1983
target user
Target Audience
Wednesday, October 21, 2015 36
• General Practitioners (in Belgium)
Independent of:
• Experience & age
• Individual or group practice
• ICT-knowledge
• Current medical software
http://marketingyoucanuse.com/wp-content/uploads/2010/12/HittingTarget.jpg
Related work: Eventflow
Wednesday, October 21, 2015 37
Monroe, M., Lan, R., Plaisant, C., Shneiderman, B. (May 2013) Temporal Event Sequence Simplification In IEEE Trans. Visualization and Computer Graphics, 19, 12 (2013), 2227-36. HCIL-2013-11
AS-IS
• Audit by Academic Center for General Practice• perceived as control
• Limited tools available:
Wednesday, October 21, 2015 38query window from Medidoc