WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and...
-
Upload
wcit-2014 -
Category
Healthcare
-
view
243 -
download
3
description
Transcript of WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and...
![Page 1: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/1.jpg)
Human Computer Interaction Evaluation, usability testing,
study design and methods
Presented by: Peter L. Elkin, MD, MACP, FACMI
With Studies from EvaLAB
Marie-Catherine Beuscart-Zéphir, PhD
1
![Page 2: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/2.jpg)
![Page 3: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/3.jpg)
Purpose
The purpose of this workshop is to discuss why human factors engineering is important for safe and effective clinical systems and to learn the Usability methodology and how to develop and run a Usability study
3
![Page 4: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/4.jpg)
Learning ObjectivesIdentify when to employ Usability TestingDiscern how to apply Usability testing to the
development of clinical systemsDescribe how to run a Usability studyDescribe how to interpret the results of a
usability studyDecide when not to apply Usability testingRecognize the linkage between patient safety
and usable systems
4
![Page 5: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/5.jpg)
UsabilityUsability is the “effectiveness, efficiency and
satisfaction with which a specified set of users can achieve a specified set of tasks in a particular environment.” (International Standard Organization: ISO 9241)
Usability is the EXTENT to which a product, application, is « usable »A product is more or less usableUsability can be measured
5
![Page 6: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/6.jpg)
Human-centred designdevelopment cycle (ISO 13407)
Plan the human-centredprocess
Understand and specifythe context of use
Specify the user and organizational requirements
Producedesign solutions
Evaluate designs againstrequirements
Meetsrequirements?
6
![Page 7: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/7.jpg)
ISO 13407 & ISO/IEC 12207ISO/IEC-12207 primary lifecycle
Acquisit. & supply
DevelopmentOperation & mainten.Require.
anal.Architect. Design
Qualific. testing
ISO13407
Context of use N/A N/A N/A
User & org. requir. N/A N/A N/A
Design solutions N/A N/A
Evaluate
7
![Page 8: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/8.jpg)
UsabilityEffectiveness: accuracy and completeness with which users
achieve specific goals
Efficiency: resources expended in relation with effectiveness
Satisfaction: comfort and acceptability of use
Context of use: users, goals, tasks, equipment, physical and
social environment
Task: a goal (what’s to be done) and the conditions
(constraints) to reach the goal
Activity: the set of actions (mental and behavioural) carried out
by the user to perform the task
8
![Page 9: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/9.jpg)
Usability (cont’d)The evaluation of the usability of a product can
have different objectives:Improve the usability of an application (formative
evaluation): Usability EngineeringDuring the design and development phase of
this application (traditional usability engineering lifecycles)
Via the re-engineering of an existing applicationAcquire or not the product, choose the most usable
product (normative evaluation)
9
![Page 10: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/10.jpg)
Usability engineering« Usability engineering provides structured
methods for achieving usability in user interface design during product development » (Deborah Mayhew, 1999)
Usability Engineering is intertwined with Software Engineering and integrated in the Project lifecycle
10
![Page 11: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/11.jpg)
Usability engineering
Understand the project (goals, expected benefits)
Modelof the currentwork situation
Model of there-engineeredwork situationUsability goals
Iterative usability assessment
Usability problemsrecommendations
Analysis of the current work situation
• Analysis, description and model of the context of use
• Analysis, description and model of users, their tasks and activity
• Usability assessment of existing systems
Project lifecycleSystem engineering
Goals, Expected benefits (quality, safety, investment)
Users needsRequirements
SpecificationDevelopment of a concept
Implementation of theconcept
Release
Requirements verification
Specification verification
Product verificationIterative testing
Validation
11
![Page 12: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/12.jpg)
The usability engineering toolboxTheories, conceptual frameworks, models,
methods, techniques, devices …
From various fields: cognitive psychology, cognitive ergonomics, cognitive engineering, usability, ethnology, sociology, software engineering …
12
![Page 13: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/13.jpg)
COGNITIVE psychology, ergonomics, engineering
Theories and modelsDecision making, process control, human error
Methodologies, frameworksCognitive Task Analysis (CTA); activity analysis
MethodsInterviews, questionnaires, protocol analysis,
documents analysis, audio/video-recorded observations, …
13
![Page 14: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/14.jpg)
UsabilityMethods:
Usability inspection: heuristic evaluation, cognitive walkthrough
Usability testsFocus groups
Techniques, devices:Usability labs
14
![Page 15: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/15.jpg)
Rationale for the use of HFESo What?
Who Cares?
What is in it for you?
15
![Page 16: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/16.jpg)
So What?IBM – “2/3 of all healthcare dollars spent on IT
are spent on Systems which are never used or are used for less than 3 months.”
Patient Safety can be compromised by poorly designed systemsHas led to deaths Is a source of unnecessary liability
16
![Page 17: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/17.jpg)
Who Cares?Administration
Limiting Costs (Mayo saved 1.5M, day 1)Physicians
Limiting ErrorsIncreased Efficiency
PatientsGreater Safety
17
![Page 18: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/18.jpg)
What is in it for you?Greater ReliabilityGreater AcceptanceLower Cost of ImplementationLower Cost for TrainingIncreased Patient SafetyImproved Clinical Reputation of your
Organization
18
![Page 19: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/19.jpg)
Reasons for Hard-to-Use Products
19
![Page 20: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/20.jpg)
Reasons for Hard-to-Use ProductsDevelopment Emphasis was on the Machine or
System, Rather than the User
20
![Page 21: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/21.jpg)
Reasons for Hard-to-Use ProductsDevelopment Emphasis was on the Machine or
System, Rather than the UserTarget Audience is a Moving Target
21
![Page 22: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/22.jpg)
Reasons for Hard-to-Use ProductsDevelopment Emphasis was on the Machine or
System, Rather than the UserTarget Audience is a Moving TargetLack of Design Verification (Science vs.
“Common Sense”)
22
![Page 23: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/23.jpg)
Reasons for Hard-to-Use ProductsDevelopment Emphasis was on the Machine or
System, Rather than the UserTarget Audience is a Moving TargetLack of Design Verification (Science vs.
“Common Sense”)Development Teams are not well Integrated
23
![Page 24: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/24.jpg)
Reasons for Hard-to-Use ProductsDevelopment Emphasis was on the Machine or
System, Rather than the UserTarget Audience is a Moving TargetLack of Design Verification (Science vs.
“Common Sense”)Development Teams are not well IntegratedSkills necessary for the interface Design are
Different than those necessary for the Technical Implementation of Systems!
24
![Page 25: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/25.jpg)
User-Centered Design
25
![Page 26: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/26.jpg)
User-Centered DesignRequires an early focus on users and tasks
26
![Page 27: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/27.jpg)
User-Centered DesignRequires an early focus on users and tasksRequires empiric measurement of product usage
27
![Page 28: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/28.jpg)
User-Centered DesignRequires an early focus on users and tasksRequires empiric measurement of product usageRequires iterative design cycles
DesignTestingModificationRe-Design
28
![Page 29: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/29.jpg)
Human Factors Analysis
29
![Page 30: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/30.jpg)
Human Factors AnalysisContextual Inquiry
Understanding End-User Needs
30
![Page 31: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/31.jpg)
Human Factors AnalysisContextual Inquiry
Understanding End-User NeedsCompetitive Usability Evaluations
Expose Gaps in Existing Systems
31
![Page 32: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/32.jpg)
Human Factors AnalysisContextual Inquiry
Understanding End-User NeedsCompetitive Usability Evaluations
Expose Gaps in Existing SystemsLow-Fidelity Prototyping
Rapid turnaround of Identified Program Changes
32
![Page 33: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/33.jpg)
Human Factors AnalysisContextual Inquiry
Understanding End-User NeedsCompetitive Usability Evaluations
Expose Gaps in Existing SystemsLow-Fidelity Prototyping
Rapid turnaround of Identified Program Changes
Modified Focus GroupsUnderstand Relevant Workflow and Processes
33
![Page 34: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/34.jpg)
Human Factors AnalysisContextual Inquiry
Understanding End-User NeedsCompetitive Usability Evaluations
Expose Gaps in Existing SystemsLow-Fidelity Prototyping
Rapid turnaround of Identified Program Changes
Modified Focus GroupsUnderstand Relevant Workflow and Processes
34
![Page 35: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/35.jpg)
Usability Studies
35
![Page 36: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/36.jpg)
Usability StudiesDevelopers and Evaluators
36
![Page 37: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/37.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without Intervention
37
![Page 38: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/38.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users Pain
38
![Page 39: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/39.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users PainCome to Grips with Imperfection
39
![Page 40: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/40.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users PainCome to Grips with ImperfectionResist the Impulse to Blame the User
40
![Page 41: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/41.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users PainCome to Grips with ImperfectionResist the Impulse to Blame the UserResist the Impulse to Blame Yourself
41
![Page 42: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/42.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users PainCome to Grips with ImperfectionResist the Impulse to Blame the UserResist the Impulse to Blame YourselfAcceptance
42
![Page 43: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/43.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users PainCome to Grips with ImperfectionResist the Impulse to Blame the UserResist the Impulse to Blame YourselfAcceptanceProgress
43
![Page 44: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/44.jpg)
Usability StudiesDevelopers and EvaluatorsObserve without InterventionFeel the Users PainCome to Grips with ImperfectionResist the Impulse to Blame the UserResist the Impulse to Blame YourselfAcceptanceProgress
44
![Page 45: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/45.jpg)
Realities of Human Factors Engineering ~Jakob NielsenYour Best Guess Is Not Good Enough
It is impossible to design an optimal user interface just by giving it your best try.
Users will make unexpected misinterpretations of the interface and perform tasks differently than you expect.
45
![Page 46: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/46.jpg)
The User is Always RightAccept the need to make modifications to fit the
users’ needs and expectations.If users have problems with the software, it is not
their fault.
46
![Page 47: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/47.jpg)
The User is Not Always RightCaution: But it does not follow that the most
usable interfaces can be designed just by asking users what they would like. Users may not understand how changes/enhancements could be beneficial.
47
![Page 48: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/48.jpg)
Designers Are Not UsersDesigners have a great deal of computer
experienceDesigners have an inherent enthusiasm about
computer applicationsDesigners know the conceptual foundation for
the design of the interface“Knowing about a system is a one-way street.
One cannot go back to knowing nothing.”
48
![Page 49: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/49.jpg)
Vice Presidents Are Not Typical UsersCompany executives typically have very different
characteristics than the user population.
49
![Page 50: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/50.jpg)
Usability Engineering is a ProcessThe Usability Engineering process is well refined
and the activities needed to arrive at a good result are fairly constant
However, each project is different and will have different objectives.
50
![Page 51: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/51.jpg)
Why GUI Design Fails
51
![Page 52: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/52.jpg)
Why GUI Design FailsProgrammers/Designers
52
![Page 53: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/53.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patterns
53
![Page 54: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/54.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptions
54
![Page 55: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/55.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptionsWant the application to control user tasks
55
![Page 56: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/56.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptionsWant the application to control user tasksAssume all users will be “Trained”
56
![Page 57: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/57.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptionsWant the application to control user tasksAssume all users will be “Trained”Plan that users will refer to documentation
57
![Page 58: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/58.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptionsWant the application to control user tasksAssume all users will be “Trained”Plan that users will refer to documentationProvide all features at the top levels
58
![Page 59: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/59.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptionsWant the application to control user tasksAssume all users will be “Trained”Plan that users will refer to documentationProvide all features at the top levelsAre unaware of consistency issues or standards
59
![Page 60: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/60.jpg)
Why GUI Design FailsProgrammers/Designers
Design to their work patterns, not user work patternsDesign to their metaphor, not user perceptionsWant the application to control user tasksAssume all users will be “Trained”Plan that users will refer to documentationProvide all features at the top levelsAre unaware of consistency issues or standards
60
![Page 61: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/61.jpg)
Bird Flu
61
![Page 62: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/62.jpg)
Usefulness: What Does This Mean?
Utility - does the system encompass needed functionality
Easy to learn
Efficient to use
Easy to remember
Few Errors
Subjectively pleasing
Usefulness -can system be used to achieve desired goals?
Usability -how well users can use functionality?
62
![Page 63: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/63.jpg)
What is a Usability Study?A usability study is a replicable study of
consumer/product or consumer/process interactions conducted in a controlled, simulated environment.
63
![Page 64: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/64.jpg)
Essential ComponentsTo accomplish a valid study, we follow a specific
protocol and have multiple participants interact with the same situations (scenarios).
It is important to observe several participants interacting with the software in order to identify trends and prioritize issues. The goal is to improve the process or product, not to train the participants.
64
![Page 65: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/65.jpg)
What can be studied?Products or Processes True classical experiments with large sample
sizes and complex test designs Informal, less complex studies designed for
rapid processing of results
Note: Each type of study has different objectives, as well as different time and resource requirements.
65
![Page 66: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/66.jpg)
What can be learned?
66
![Page 67: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/67.jpg)
What can be learned?Features/Functions that work
67
![Page 68: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/68.jpg)
What can be learned?Features/Functions that workFeatures/Functions that need improvement
68
![Page 69: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/69.jpg)
What can be learned?Features/Functions that workFeatures/Functions that need improvementUser/Consumer work patterns and mental
models
69
![Page 70: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/70.jpg)
What can be learned?Features/Functions that workFeatures/Functions that need improvementUser/Consumer work patterns and mental
modelsUser/Consumer expectations and desires
70
![Page 71: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/71.jpg)
What can be learned?Features/Functions that workFeatures/Functions that need improvementUser/Consumer work patterns and mental
modelsUser/Consumer expectations and desiresPrioritize areas for improvement - unique
advantage
71
![Page 72: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/72.jpg)
What can be learned?Features/Functions that workFeatures/Functions that need improvementUser/Consumer work patterns and mental
modelsUser/Consumer expectations and desiresPrioritize areas for improvement - unique
advantage
72
![Page 73: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/73.jpg)
Objectives Addressed Through the Study DesignWhat are the general navigational weaknesses? Ease of use / Intuitive nature of application
interface (Do controls and functions make sense?)
User satisfaction?Efficient use of screen
real estate (space)? Are fields and widgets placed intuitively on the
screen?
73
![Page 74: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/74.jpg)
Objectives Addressed Through the Study DesignWill the application make users’ jobs easier? /
Will it make users more productive? Is the system responsive/ fast enough?Is the complexity of the application appropriate
for the defined / intended user group(s)? What On-line Help functions are needed?What type(s) / degree of training is needed?
74
![Page 75: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/75.jpg)
Limitations of Usability Testing
75
![Page 76: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/76.jpg)
Limitations of Usability TestingTesting is always an artificial situation
76
![Page 77: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/77.jpg)
Limitations of Usability TestingTesting is always an artificial situationTest Results do not Prove that a Product Works
77
![Page 78: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/78.jpg)
Limitations of Usability TestingTesting is always an artificial situationTest Results do not Prove that a Product WorksParticipants are rarely truly representative of
their target Population
78
![Page 79: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/79.jpg)
Limitations of Usability TestingTesting is always an artificial situationTest Results do not Prove that a Product WorksParticipants are rarely truly representative of
their target PopulationTesting is not Always the Best Technique
(To Test or Not to Test?)vs. Expert EvaluationFocus Groups
79
![Page 80: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/80.jpg)
When Usability Study
80
![Page 81: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/81.jpg)
When Usability StudyWhen Studying Complex Processes without
option to Study Components Separately
81
![Page 82: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/82.jpg)
When Usability StudyWhen Studying Complex Processes without
option to Study Components SeparatelyLab Environment is Vastly different than User
Environment
82
![Page 83: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/83.jpg)
When Usability StudyWhen Studying Complex Processes without
option to Study Components SeparatelyLab Environment is Vastly different than User
EnvironmentLab Space will not accommodate the Test
83
![Page 84: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/84.jpg)
When Usability StudyWhen Studying Complex Processes without
option to Study Components SeparatelyLab Environment is Vastly different than User
EnvironmentLab Space will not accommodate the TestWhen feedback will not be Utilized
84
![Page 85: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/85.jpg)
When Usability StudyWhen Studying Complex Processes without
option to Study Components SeparatelyLab Environment is Vastly different than User
EnvironmentLab Space will not accommodate the TestWhen feedback will not be UtilizedSubjects cannot be found (Typical Users)
85
![Page 86: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/86.jpg)
When Usability StudyWhen Studying Complex Processes without
option to Study Components SeparatelyLab Environment is Vastly different than User
EnvironmentLab Space will not accommodate the TestWhen feedback will not be UtilizedSubjects cannot be found (Typical Users)
86
![Page 87: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/87.jpg)
87
![Page 88: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/88.jpg)
Usability Lab
88
![Page 89: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/89.jpg)
89
![Page 90: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/90.jpg)
90
![Page 91: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/91.jpg)
91
![Page 92: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/92.jpg)
92
![Page 93: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/93.jpg)
93
![Page 94: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/94.jpg)
94
![Page 95: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/95.jpg)
95
![Page 96: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/96.jpg)
96
![Page 97: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/97.jpg)
97
![Page 98: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/98.jpg)
98
![Page 99: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/99.jpg)
Illustration
Overview of the re-engineering project of a CPOE application carried out at the EVALAB
Focus on usability tests of the system
99
![Page 100: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/100.jpg)
The projectPartners:
University Hospital of LilleSyndicat Inter-hospitalier de Bretagne (SIB)
Project and context:Re-engineering of an existing Computerized
Physician Order Entry (CPOE) application: Génois
100
![Page 101: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/101.jpg)
The project (cont’d)
Medication Ordering, [dispensing], and administration process
Hospital setting, survey of the therapeutic process during the patient’s stay
101
![Page 102: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/102.jpg)
Medication orderingDaily medical rounds
At bedside
Follow-up of Medication
administration
Medication decision making
Medication ordering
Preparation of 24h pillboxes
Administrationvalidation
patient’s medication orders list
![Page 103: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/103.jpg)
Usability engineering
Understand the project (goals, expected benefits)
Modelof the currentwork situation
Model of there-engineeredwork situationUsability goals
Iterative usability assessment
Usability problemsrecommendations
Analysis of the current work situation
• Analysis, description and model of the context of use
• Analysis, description and model of users, their tasks and activity
• Usability assessment of existing systems
The project: Re-engineering of an existing CPOE application: Génois (SIB)
103
![Page 104: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/104.jpg)
Usability engineering
Understand the project (goals, expected benefits)
Modelof the currentwork situation
Model of there-engineeredwork situationUsability goals
Iterative usability assessment
Usability problemsrecommendations
Analysis of the current work situation
• Analysis, description and model of the context of use
• Analysis, description and model of users, their tasks and activity
• Usability assessment of existing systems
Goals and Expected Benefits
104
Company (SIB): Improve the usability of the
CPOE application (Génois) Support the re-engineering of
the application (re-design HCI)
Hospital: Assess the feasibility of the
installation of this CPOE Improve the quality and
safety of the medication process
![Page 105: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/105.jpg)
Usability engineering
Understand the project (goals, expected benefits)
Modelof the currentwork situation
Model of there-engineeredwork situationUsability goals
Iterative usability assessment
Usability problemsrecommendations
Analysis of the current work situation
• Analysis, description and model of the context of use
• Analysis, description and model of users, their tasks and activity
• Usability assessment of existing systems
Identify the current work situation
105
![Page 106: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/106.jpg)
Identify the Work SituationUsers: physicians, nursesSystems: paper-based / computer-based
106
![Page 107: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/107.jpg)
Usability engineering
Understand the project (goals, expected benefits)
Modelof the currentwork situation
Model of there-engineeredwork situationUsability goals
Iterative usability assessment
Usability problemsrecommendations
Analysis of the current work situation
• Analysis, description and model of the context of use
• Analysis, description and model of users, their tasks and activity
• Usability assessment of existing systems
Analysis of the context of use
Analysis of Users’ tasks & activity
Usability assessment of existing systems
107
![Page 108: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/108.jpg)
Analysis of the Work Situation (1 of 3)
Usability tasks Methods
Analysis of the context of use Interviews: head nurses, managers, senior physicianAnalysis (inventory) of systems’ functionsOn site observations (ethno methodology)
Analysis of Users’ tasks & activity•Individual
•Collective (cooperation)
Cognitive Task AnalysisActivity analysisDocuments analysis (content analysis)Self confrontation interviews
Analysis of communications (oral, written, through the system)
Usability assessment of existing systems Computer-based (CPOE):Usability inspection Usability tests
Paper-based:Analysis of the format, content, availability, redundancy of supports / contentsUsers complaints and incidents observed or reported
108
![Page 109: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/109.jpg)
Analysis of the Work Situation (2 of 3)
Usability tasks Methods
Analysis of the context of use Interviews: head nurses, managers, senior physicianAnalysis (inventory) of systems’ functionsOn site observations (ethno methodology)
Analysis of Users’ tasks & activity•Individual
•Collective (cooperation)
Cognitive Task AnalysisActivity analysisDocuments analysis (content analysis)Self confrontation interviews
Analysis of communications (oral, written, through the system)
Usability assessment of existing systems Computer-based (CPOE):Usability inspection Usability tests
Paper-based:Analysis of the format, content, availability, redundancy of supports / contentsUsers complaints and incidents observed or reported
109
![Page 110: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/110.jpg)
Analysis of the Work Situation (3 of 3)Usability tasks Methods
Analysis of the context of use Interviews: head nurses, managers, senior physicianAnalysis (inventory) of systems’ functionsOn site observations (ethno methodology)
Analysis of Users’ tasks & activity•Individual
•Collective (cooperation)
Cognitive Task AnalysisActivity analysisDocuments analysis (content analysis)Self confrontation interviews
Analysis of communications (oral, written, through the system)
Usability assessment of existing systems
Computer-based (CPOE):Usability inspection Usability tests
Paper-based:Analysis of the format, content, availability, redundancy of supports / contentsUsers complaints and incidents observed or reported
110
![Page 111: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/111.jpg)
Outcomes of the analysis of the work situation: Illustration on two tasks
Physician: information gathering / patient’s current medications for therapeutic decision making
Nurse: information gathering / patient’s current medication for the preparation of medication administration (24 hours pill-box)
111
![Page 112: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/112.jpg)
Physician’s task: comprehend the patient’s list of meds
Usability assessment
Activity analysis• Doctors need the list of current
meds on one page• Need the duration of each
order immediately available (no calculation)
• Need the recent modifications of the treatment highlighted
Context of use• Never performs this task
alone (nurse or resident present)
• Frequency of encounters with the patient varies from twice per day to once in 2 weeks
Paper-based system• The medication order sheets (list
of orders written by the physician) are not usable
• The physicians prefer to use the nurses Medication Administration Record to update their representation of current treatments
CPOE system (inspection, on site observations)
• Obligation of scrolling through the screen page to read the entire list of meds
• Display of modifications of the treatments suffers from lack of intuitivity
• Duration is not directly available (computation required)
![Page 113: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/113.jpg)
Nurse’s task: prepare the pillboxes (24h administration)
Usability assessment
Activity analysis• Nurses need the list of current
meds on one page, sorted out by route
• Need to be informed immediately and very clearly of the medication changes ordered by the doctors along their rounds or visits
Context of use• No problem when the
nurses accompany the doctors along their rounds
• If not, great difficulties to update their knowledge about the patients current treatment and to remain aware of the modifications of orders
CPOE system (on-site observations)
• Difficulties with the 24h administration schedule
• The system requires the printing of the preparation plans: in case of frequent modifications of the treatment, there are a lot of printed plans, and a risk of confusion
Paper-based system (on-site observations)• The medication order list is not
usable• The only synthesized
document (MAR) is not available at the location where the pillboxes are prepared
![Page 114: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/114.jpg)
Usability tests (1) PhysiciansPhysicians
Test the usability of information display for the information gathering task supporting the decision making
Check for recent changes in the current therapeutic treatment
Test orders entry functions
114
![Page 115: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/115.jpg)
Usability tests (2) NursesNurses
Test the usability of the functions supporting the preparation of the 24h pillboxes (display of information, editing of preparation plans)
Test the usability of administration validation
115
![Page 116: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/116.jpg)
Usability testsThe usability labPreparation of the testsExcerpts (video)ConclusionsRecommendations
116
![Page 117: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/117.jpg)
Observers screens
Observers
Technician’scontrol
Facilitator control screens
Control roomDirect observation room
Main experimental&
simulation room
Experimental & simulation room
Experimental & simulation room
One-way mirrors
![Page 118: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/118.jpg)
Usability testsThe usability labPreparation of the testsExcerpts (video)ConclusionsRecommendations
118
![Page 119: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/119.jpg)
Create a realistic simulated environmentConfigure the system to accommodate the
characteristics of users activityCreate realistic patient’s records (from real
cases)Write precise scenarios
119
![Page 120: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/120.jpg)
RecommendationsItems in the Interface that worked well and safelyItems in the Interface that worked poorly or
inconsistentlyIdentify those features that are capable of causing
Harm.Recommendations for specific engineering
improvements to the interface design
120
![Page 121: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/121.jpg)
Conclusions
121
![Page 122: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/122.jpg)
ConclusionsUsability Studies
122
![Page 123: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/123.jpg)
ConclusionsUsability Studies
Useful Mechanisms for Objective Evaluation
123
![Page 124: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/124.jpg)
ConclusionsUsability Studies
Useful Mechanisms for Objective EvaluationDesigned to Answer Specific Questions
124
![Page 125: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/125.jpg)
ConclusionsUsability Studies
Useful Mechanisms for Objective EvaluationDesigned to Answer Specific QuestionsDesigned to Discover Questions to Answer
125
![Page 126: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/126.jpg)
ConclusionsUsability Studies
Useful Mechanisms for Objective EvaluationDesigned to Answer Specific QuestionsDesigned to Discover Questions to AnswerKeeping patients safe from software related medical
error
126
![Page 127: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/127.jpg)
“The Best Way to Predict the Future,
127
![Page 128: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/128.jpg)
“The Best Way to Predict the Future,
Is to Create It.”
128
![Page 129: WCIT 2014 Peter Elkin - Human computer interaction, evaluation, usability testing, study design and methods.](https://reader038.fdocuments.net/reader038/viewer/2022110309/558a30edd8b42a40138b45b7/html5/thumbnails/129.jpg)
Quote: Peter Drucker – Harvard University
“The Best Way to Predict the Future,
Is to Create It.”
129