Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When...
Transcript of Human Factors Considerations When Setting up/ Expanding … · Human Factors Considerations When...
Human Factors Considerations When Setting up/ Expanding a Telemedicine and
Telehealth program
Patrick Boissy Ph.D Université de Sherbrooke
American Telemedicine Association Quality Healthcare Through Telecommunications Technology
September 2013 Toronto, Ontario
Canada
What is Human Factors ?
PROCESS ‘Human factors engineering—the application of knowledge about human behavior, abilities, limitations, and other characteristics to the design of tools, machines, equipment, devices, systems, tasks, jobs, and environment to achieve productive, safe, comfortable, and efficient human use.’ DISCIPLINE AND A JOB ‘Human factors is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance’ . - International Ergonomics Association
From: Eric Shaver (2009). The Many Definitions of Human Factors and Ergonomics.
Human factors
From: http://www.doubleimage.co.uk/the_simpson_posters.html
© BOISSY September 2013 Human Factors SIG 1
Goals: Ensuring that human interactions with systems enhance performance (usually measured as efficiency), increase safety and increase user satisfaction
Scientific Disciplines & Methodological Approaches in HF
2 Human Factors SIG
So are the methodological approaches…
The field of human factors is diverse and multidisciplinary…
© BOISSY September 2013
HF in Telehealth and Telemedecine Programs?
3 Human Factors SIG
• Efficiency, safety and user satisfaction impact service delivery… and technology acceptance (TA)
• TA can be seen as behavioral intention mediated by two key variables (perceived usefulness and perceived ease of use)
• TA is a determining factor in use behavior and ultimately adoption
Degree to which a person has formulated conscious plans to perform or not perform some specified future behavior.
From : Venkatesh,V. and H.Bala (2008). Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences Volume 39 Number 2, p 280.
Technology acceptance model 3 (TAM3)
Use and adoption are requirements for a
sustainable program !
© BOISSY September 2013
Technological acceptance, use and adoption of EHR
4 Human Factors SIG
You can lead a horse to water, but you can't make it drink
• Longitudinal survey of US hospitals
show that 44 percent of hospitals report having and using what can be define as at least a basic EHR system.
• Although 42.2 percent meet all of the federal stage 1 "meaningful-use" criteria, only 5.1 percent could meet the broader set of stage 2 criteria.
• Having a basic EHR and using an EHR is not the same …
‘Meaningful-use stages and criteria’
Desroches, C. M., et al. (2013). "Adoption Of Electronic Health Records Grows Rapidly, But Fewer Than Half Of US Hospitals Had At Least A Basic System In 2012." Health Aff (Millwood) 32(8): 1478-1485.
© BOISSY September 2013
7 Human Factors SIG
RESEARCH
ARTIC
LE
Ope
nAccess
Barriers
totheacceptan
ceof
electron
icmed
ical
recordsby
physicians
from
system
atic
review
totaxono
myan
dinterven
tions
Albe
rtBo
onstra*†,M
anda
Broe
khuis†
Abs
tract
Backgrou
nd:T
hemainob
jectiveof
thisresearch
isto
iden
tify,catego
rize,andanalyzebarrierspe
rceivedby
physicians
totheadop
tionof
Electron
icMed
icalRecords(EMRs)in
orde
rto
provideim
plem
enterswith
bene
ficial
interven
tionop
tions.
Metho
ds:A
system
aticliteraturereview
,based
onresearch
pape
rsfro
m1998
to2009,con
cerningbarriersto
the
acceptance
ofEM
Rsby
physicians
was
cond
ucted.
Four
databases,“Scien
ce”,“EBSCO
”,“Pub
Med
”and“The
Cochrane
Library”,w
ereused
intheliteraturesearch.Studies
wereinclud
edin
theanalysisifthey
repo
rted
onph
ysicians’p
erceived
barriersto
implem
entin
gandusingelectron
icmed
icalrecords.Electron
icmed
icalrecordsare
defined
ascompu
terized
med
icalinform
ationsystem
sthat
collect,store
anddisplaypatient
inform
ation.
Results
:The
stud
yinclud
estw
enty-twoarticlesthat
have
considered
barriersto
EMRas
perceivedby
physicians.
Eigh
tmaincatego
riesof
barriers,includ
ingatotalo
f31
sub-catego
ries,wereiden
tified.
Theseeigh
tcatego
riesare:
A)Financial,B)
Technical,C)
Time,D)Psycho
logical,E)
Social,F)Legal,G)Organizational,andH)Ch
ange
Process.
Allthe
secatego
riesareinterre
latedwith
each
othe
r.In
particular,C
ateg
oriesG(Organizational)andH(Chang
eProcess)seem
tobe
med
iatin
gfactorson
othe
rbarriers.By
adop
tingachange
managem
entpe
rspe
ctive,we
developsomebarrier-related
interven
tions
that
couldovercometheiden
tifiedbarriers.
Conc
lusion
s:Despite
thepo
sitiveeffectsof
EMRusagein
med
icalpractices,the
adop
tionrate
ofsuch
system
sis
stilllow
andmeets
resistancefro
mph
ysicians.Thissystem
aticreview
revealsthat
physicians
may
face
arang
eof
barrierswhe
nthey
approach
EMRim
plem
entatio
n.Weconclude
that
theprocessof
EMRim
plem
entatio
nshou
ldbe
treatedas
achange
project,andledby
implem
entersor
change
managers,in
med
icalpractices.The
quality
ofchange
managem
entplaysan
impo
rtantrole
inthesuccessof
EMRim
plem
entatio
n.Thebarriersandsugg
ested
interven
tions
high
lighted
inthisstud
yareintend
edto
actas
areferenceforim
plem
entersof
Electron
icMed
ical
Records.Acarefuld
iagn
osisof
thespecificsituatio
nisrequ
iredbe
fore
relevant
interven
tions
canbe
determ
ined
.
Backgrou
ndElectronicMed
ical
Records
(EMRs)
areco
mpu
terized
medical
inform
ationsystem
sthat
collect,store
anddis-
play
patien
tinform
ation.
The
yareameans
tocreate
legiblean
dorganizedreco
rdings
andto
access
clinical
inform
ationab
outindividu
alpa
tien
ts.Fu
rthe
r,EMRs
areintende
dto
replaceex
isting(often
pape
rba
sed)
med
ical
reco
rds
which
are
alread
yfamiliar
to
practition
ers[1].Patient
reco
rdsha
vebe
enstored
inpa
perform
forcenturiesan
d,ov
erthis
period
oftime,
they
hav
eco
nsu
med
increasing
spacean
dnotab
lyde
layedaccess
toeffic
ient
med
ical
care
[2].In
contrast,
EMRsstoreindividu
alpatien
tclinical
inform
ationelec-
tron
ically
anden
able
instan
tavailabilityof
thisinform
a-tion
toallprov
idersin
thehe
althcare
chainan
dso
shou
ldassist
inprovidingcoherent
andconsistent
care.
Electronic
Med
ical
Records
(EMRs)
andElectronic
HealthRecords
(EHRs)
areview
edas
intercha
ngeable
synon
ymsin
mosthe
alth
inform
atics.
Other
similar
expr
ession
sex
istalbe
itwith
aso
metim
esslightly
restricted
focu
s.W
hile
EMRsha
veagene
ralfocu
son
*Co
rrespon
dence:albe
rt.boo
nstra@
rug.nl
†Co
ntrib
uted
equally
Faculty
ofEcon
omicsandBu
siness,University
ofGroning
en,G
roning
en,The
Nethe
rland
sFulllistof
author
inform
ationisavailableat
theen
dof
thearticle
Boon
stra
andBroe
khuisBM
CHealth
Services
Research
2010
,10:23
1http://www.biomed
central.com
/147
2-69
63/10/23
1
©2010
Boon
stra
andBroe
khuis;licen
seeBioM
edCe
ntralLtd.Thisisa
nOpe
nAc
cessarticledistrib
uted
unde
rthe
term
softhe
Creative
Common
sAttribu
tionLicense(http://creativecom
mon
s.org/licenses/by/2.0),w
hich
perm
itsun
restricteduse,
distrib
ution,
and
reprod
uctio
nin
anymed
ium,p
rovide
dtheoriginal
workisprop
erlycited.
HF variables are part of the the barriers most cited for technological acceptance of EHR
© BOISSY September 2013
expected to replace current product revenue by offering a better fit with current market needs.While convenient to diagram, decisions on whether to modify or retire products, when toretire products and which products to retire are difficult for firms to make.
Not all of the traits of artifacts are identical to those of living things. In contrast with liv-ing things, human-made systems do not typically show whether they are doing well or arehaving problems. It can be a challenge to know how well a system is performing. Are each ofthe elements that have been selected to be a part of the system performing at capacity? Is theconfiguration of those elements optimal? Can the system be optimized further, or is the sys-tem obsolete and actually living on borrowed time? The breadth, complexity andinterdependence of systems can make these difficult questions to answer.
1.4 The development process
Firms vary in the approach that each takes to research and development. Even so, there aresteps that account for the information and action that is typically necessary to accomplishdesign and human factors activity in the context of development.
1.4.1 Manufacturing
Figure 1.4 shows Robert Bateman’s view of the way that research and development activitiesfit into the life cycle of a system. Portions of the diagram which are shown in gray indicatemajor steps: define problem, gather data, determine feasibility, plan (define requirements,identify alternatives), analyze (evaluate alternatives), design (build prototype, test, install),implement, use (evaluate) and retire. The design, prototype and test cycle may be repeateduntil the solution meets requirements. Evaluation includes means to either plan a productmodification or to retire it. To create a follow-on product in time, it is necessary to completeplanning before retiring its predecessor.
16 Human factors practice
Figure 1.4 Development process—Robert Bateman describes the phases of product developmentwithin the context of product life cycles. To create a follow-on product in time, it is neces-sary to complete planning before retiring its predecessor. Major phases in the developmentcycle are shown as grey shaded arrows.
Source: Bateman, R. Reproduced by permission of the author.
DesignAnalyze
Plan
Use
Test
Install
Identify alternatives
Define requirements
Define problem Gather
data
Determine feasibility
Evaluate
Build prototype
Evaluatealternative
Implement
Retire
Contributions of Human Factors in dev. & Lifecycle of Product
DESIGN AND DEVELOPMENT
PHASE
USE OF PRODUCT IN REAL WORLD
6 Human Factors SIG
Who : Designers, users, managers, stakeholders When Design , procurement, Implementation and operation phases
From: Christopher P. Nemeth (2004). Human Factors Methods for Design Making Systems Human-Centered. CRC Press, Page 16.
© BOISSY September 2013
Context and Goals of HF Evaluations / program perspective
7 Human Factors SIG
• Understanding how people
work and what people will do with technology
• Fitting the technology to the users and their clinical workflow
• Supporting users in the field • Evaluating and monitoring
technology use to optimize outcomes
• Establish technological feasibility and requirement to scale up the activities
Real life use of the product in your context
HF
eval
uatio
ns to
opt
imiz
e te
chno
logy
us
e du
ring
key
phas
es
Goals of HF evaluations:
From: Christopher P. Nemeth (2004). Human Factors Methods for Design Making Systems Human-Centered. CRC Press, Page 16.
© BOISSY September 2013
HF Evaluations processes / program perspective
8 Human Factors SIG
How: • Contextual inquiries to
understand workflow of user and the factors involved
• Creating a use case scenario for the technology
• Identifying usability issues with the technology to find workarounds
• Monitoring real time use, reliability and satisfaction metrics
© BOISSY September 2013
• Fly on the wall - Observe and record behavior within its context, without interfering with people’s activities.
• A day in the life - Catalog the activities and contexts that users experience for an entire day.
• Personal inventory - Document the things that people identify as important to them as a way.
• Shadowing - Tag along with people to observe the use of similar devices in a similar scenario and understand their day-to-day routines, interactions, and contexts. The researcher asks questions to ensure that he or she understands what the users are doing and why they are doing it that way.
• Interview and focus group –Methods to elicit information and insights from users in response to carefully designed questions, either one informant at a time or through interaction between the moderator and a group of informants.
Contextual Inquiries /
“Know the User”
9 Human Factors SIG
Walking in someone else
shoes…
© BOISSY September 2013
• Use case = method for describing from the user point of
view the interactions between a user and a device to accomplish a functional objective in a specific context.
• Tasks are the fundamental unit of description of that interaction. A task is ‘a set of human actions that contributes to a specific functional objective and ultimately to the output goal of a system.’ (Drury et al. (1987:373).
• A use case has multiple “paths” that can be taken by any user at any one time.
• A use case scenario is a single path through the use case where a set of tasks that represent the optimal solution to accomplish a functional objective with a device in a specific context are identified and sequenced.
Creating a Use Case Scenario for the Technology
10 Human Factors SIG
The use case scenario is often used for usability testing and to create documentation for users… Task analysis is the foundation to establish a use case
scenario. © BOISSY September 2013
• There are various forms and levels of TASK ANALYSIS (TA) possible.
• The form and level of a TA depends on the purpose for which the analysis is to be conducted.
• For the creation of a use case scenario the most common form is HIERARCHICAL TASK ANALYSIS (HTA).
• HTA is a mean to break down the use of a device into a
hierarchy of goals, operation and plans needed to accomplish a functional objective. • Goals are the unobservable task goals associated
with operation of the device. • Operations are the observable behaviours or
activities which accomplish the goals. • Plans are unobservable decisions and planning on
behalf of the operator.
Creating a Use Case Scenario - Task analysis / HTA
11 Human Factors SIG © BOISSY September 2013
Ex. of HTA with Task Flow diagram- Making Coffee
From: Mark R. Lehto, James R. Buck (2008). Introduction to Human Factors and Ergonomics for Engineers . Taylor & Francis Group, pages 118,119.
task about the accuracy of the description. The latter verification helps if thetask includes sensory or mental activities that are not always obvious fromdirect observation.
When the observed activities do not correspond to the TFD, direct ob-servational procedures can be used to develop the task description. Theseforms of direct observation may be continuous or periodic, and they may beperformed by a human observer making on-site observations or by using somerecording device such as digital video or photography.22 Here again, it is agood idea to take the resulting TFD from the observational study and askhuman operators performing the tasks about its accuracy.
Another technique that is sometimes used is called a Talk Aloud proce-dure, which asks the human operator to talk about what he or she is doingwhile performing the task. Use of the talk aloud procedure is controversial be-cause the acts of talking and performing the task can interfere with each other,and the result may be misleading, especially if performance time measuresare taken at the same time. No matter which combination of procedures isused to determine an initial TFD, the primary purpose is to answer the ques-tions: who, what, where, when, why, and how about the tasks and job.
Hierarchical Task Analysis (HTA)
As should be apparent from the earlier example for making coffee, tasksand subtasks normally fit into a natural hierarchy. In some instances there are
CHAPTER THREE Design to Fit Tasks, Processes, and People 119
22. Some approaches for taking such observations are discussed in chapter 10. Making a video recording of thetask is often the easy part. Anyone who has tried to analyze video tapes of task activity knows well thatthis can be time-consuming, tedious, and difficult. Fortunately, computer tools, such as the Interact soft-ware available from Mangold International (http://www.mangold-international.com) are now availableto make it easier to analyze video recordings of task activity. The latter software allows the analyst toview digital video recordings of the task on a PC, and provides a convenient interface for marking the startand end of particular activities, which are then automatically recorded in a convenient spread sheet format.The interface also provides the analyst the ability to rapidly move back and forth at different speedsthrough the sequence of activities to specific frames of interest, which is a great improvement over tryingto figure out exactly where a task starts or ends using a standard video player or movie projector!
A more refined view of making coffee. ! FIGURE 3.15
COFFEE POT ACTIVITIES1. Pick up the pot2. Grasp the handle of the pot with right hand3. Move the pot off the warmer4. Transport the pot to the kitchen sink5. Place the pot under the cold water faucet6. Fill the pot with water7. Turn on cold water faucet and let the water fill to the mark8. Turn off the cold water faucet9. Transport the filled pot to the coffee maker
10. Empty the pot into the receiver of the coffee maker11. Tip the pot and pour the total contents into the water receiver12. Untip the pot13. Carry the pot to the lower level part of the coffee maker14. Insert the pot back onto the warmer
ch03_8129_Lehto_LEA 5/9/07 5:40 PM Page 119
Operations
A block diagram showing these three steps is shown in Figure 3.14. The read-ers’ likely feeling at this point is that the description is too coarse to be verybeneficial. An expanded list for the first set of activities might be as shown inFigure 3.15. Similarly expanded lists could be developed for the Filter holderactivities, and the Switch activities.
Note that each level of detail fits within the level of detail above, creat-ing a hierarchy of finer and finer details. If one used a hyper-link computerprogram, each level would fit within the one above so that a designer canmove down to the desired level of detail. Virtually any reader who has madecoffee with a similar type of coffeemaker will easily recognize how this par-ticular procedure works, although that reader may argue about the particularsubtasks, and he or she could easily point out that steps 1 and 2 could be donein either order.
Task Flow Diagrams
The activities performed in a task are often described with flow diagramsvery similar to those used to describe computer programs. When there is onlyone accepted way to perform the task, the Task Flow Diagram (TFD) is a sim-ple sequence of boxes denoting the sequence of activities (see Figure 3.14).More complex TFDs include decisions (see Figures 3.12 and 3.13). Whenthe design team starts with the currently existing procedure as the initial con-cept for finding improvements, the typical procedure is to first find the oldtask flow diagram (TFD). The degree this description fits the current situationis then evaluated. If there is enough time available, a person from the designteam may go to the site and observe people doing the task. If the observedactivities correspond to the description in the TFD, it can be assumed to becurrent. Further verification may involve asking the operator performing the
118 Introduction to Human Factors and Ergonomics for Engineers
! FIGURE 3.14 Task flow diagram for making coffee.
ch03_8129_Lehto_LEA 5/9/07 5:40 PM Page 118
Goals
12 Human Factors SIG © BOISSY September 2013
Ex. of use case scenario for teledermatology consult
13 Human Factors SIG
Patient presents with Derm
condition
Primary Care Physician
Primary Care nurse
Teledermatologist
Document background
in chart
Capture photos
Upload photos & route chart
Review photos & chart
Appointment
Needed?
Route chart to PCP
End Process Communicate to patient
MA deletes photos
Document Tx Plan
Request appt in Derm
ACME Teledermatology Flow Chart
© BOISSY September 2013
16 Human Factors SIG
Operations to accomplish goals #2 (capture clinical photo)
1. Obtain Camera & Setup Clinical Camera
2. Capture Clinical Photos 3. Setup Dermatoscope Camera &
Capture Dermatoscope Photos (if used)
4. Provide Patient Instructions for Rapid Response (Preferred) or Deferred Response
5. Enter Note in KPHC Visits Section 6. Connect the Clinical Camera 7. 1mport/Acquire Clinical Camera
lmages 8. Connect Dermatscope Camera 9. Import Derrnatscope lmages 10. Connect to PACScan 11. Validate Photo Upload (System1) 12. Validate Photo Upload (System 2)
List of goals for Primary care nurse to capture clinical photo and upload
photos & route chart
© BOISSY September 2013
Identifying Usability Issues to Find Workarounds
15 Human Factors SIG
• No new technology is perfect out of the box…
• Often a disconnect between procurement people versus people using, supporting and managing technology
• Will it be usable in
your context and environment with your users ?
© BOISSY September 2013
• "[Usability refers to] the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” - ISO 9241-11
• Usability has multiple components and is traditionally associated
with these five usability attributes: learnability, efficiency, memorability, errors, satisfaction. - Jakob Nielsen, Usability Engineering, 1993, p.26
What is Usability?
16 Human Factors SIG © BOISSY September 2013
Usability Inspection and Testing Methods
Usability testing
‘ Refers to an observational research technique where representative end users are recruited to participate in realistic task scenarios in a simulated environment to detect design flaws and deficiencies’ (Rubin, 1994). • Formative test: Focus = diagnose
problems and offer solutions, less formal data analysis techniques.
• Summative test: Tests conducted in order to generate metrics or measure formal improvements, such as benchmark studies require more formal data analysis methods
17 Human Factors SIG
Usability Inspection
• Heuristic evaluation & estimation • Cognitive walkthrough • Pluralistic walkthrough • Feature inspection • Consistency inspection • Standards inspection • Formal usability inspection
‘ Generic name for a set of methods that are all based on having evaluators inspect a user interface to identify usability problems in the design, though some methods also address issues like the severity of the usability problems and the overall usability of an entire system’ (Neilsen, 1995).
See http://www.nngroup.com/articles/summary-‐of-‐usability-‐inspection-‐methods/ © BOISSY September 2013
How is Usability Testing Done ?
• Step #1 - Define the audience and the goals of the usability test.
• Step #2 -Create and select task scenarios participants will have to realize to address goals of usability test.
• Step #3- Choose the right metrics and methods to collect usability outcomes during the task scenarios.
• Step #4- Establish qualification of test participants and recruit the right people to perform the task scenarios.
• Step #5- Watch test participants and measure usability outcomes as they perform the task scenarios or once completed.
• Step #6 Analyze usability outcomes in relation to usability goal.
18 Human Factors SIG © BOISSY September 2013
• Ease of learning - How fast can a user who has never seen the user interface before learn it sufficiently well to accomplish basic tasks?
• Memorability - If a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything?
• Subjective satisfaction - How much does the user like using the system?
• Efficiency of use - Once an experienced user has learned to use the system, how fast can he or she accomplish tasks?
• Error frequency and severity - How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors?
Metrics of Usability ?
19 Human Factors SIG © BOISSY September 2013
Capturing Usability Metrics ?
User interactions are studied using: • Live view with observation grid and
timing device (time and motion study) • Video and audio recordings • Data logging and screen capture of
the system operation • Gaze tracking
20 Human Factors SIG
Video camera
Gaze tracking system
Data logging
© BOISSY September 2013
Capturing Usability Metrics ?
21 Human Factors SIG © BOISSY September 2013
• Morea 3.3 Usability software suite http://www.techsmith.com/morae.html • Silverback Usability testing software http://silverbackapp.com • Online Screenshot Testing Software http://www.optimalworkshop.com/chalkmark.htm • Visual heatmap of clicks on a HTML page http://www.labsmedia.com/clickheat/index.html
http://www.usabilitynet.org/tools/ r_questionnaire.htm
Software resources for capturing usability metrics from PC based technology
http://www.epiphan.com
Questionnaire resources for capturing usability metrics
Hardware for screen capture, audio and video capture
• Key consideration when buying, deploying a product/ technology in healthcare processes.
• Part of the system development life cycle but it doesn’t mean that the product/technology you buy/deploy will work for all contexts and users.
• Buying or going to the field without considering human factors is like jumping head first in a lake without knowing the depth of the lake.
• Human factors is often a compromise between function and form.
Conclusions
Human factors :
21 Human Factors SIG © BOISSY September 2013
Conclusions
Usability evaluation is:
• Part of the triangulation process required to establish the complete picture of a product and its intended use in a specific environment.
• When put in context of other data, such as project goals, user goals, user an experts feedback, and usage metrics, usability evaluation can help identify specific high priority problems.
• Not an exact science. Recommendations and conclusions have to be interpreted with caution and confronted to the reality studied.
22 Human Factors SIG © BOISSY September 2013
1. http://www.usabilitynet.org/tools.htm 2. http://www.upassoc.org/usability_resources/index.html 3. Faulkner, L. (2003). Beyond the five-user assumption: benefits of increased sample sizes in usability
testing. Behav Res Methods Instrum Comput, 35(3), 379-383. 4. FDA. (2011). Applying human factors and usability engineering to optimize medical device design. 5. Jaspers, M. W. (2009). A comparison of usability methods for testing interactive health technologies:
methodological aspects and empirical evidence. Int J Med Inform, 78(5), 340-353. 6. Kaufman, D. R., Patel, V. L., Hilliman, et al. (2003). Usability in the real world: assessing medical
information technologies in patients' homes. J Biomed Inform, 36(1-2), 45-60. 7. Kushniruk, A., & Borycki, E. (2011). Exploring the relationship between usability and technology-induced
error: unraveling a complex interaction. Stud Health Technol Inform, 166, 48-56. 8. Kushniruk, A. W., Borycki, E. M., Kuwata, S., & Kannry, J. (2011). Emerging approaches to usability
evaluation of health information systems: towards in-situ analysis of complex healthcare systems and environments. Stud Health Technol Inform, 169, 915-919.
9. Kushniruk, A. W., & Patel, V. L. (2004). Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform, 37(1), 56-76
10. Liljegren, Erik. (2006). Usability in a medical technology context assessment of methods for usability evaluation of medical equipment. International Journal of Industrial Ergonomics, 36(4), 345-352.
11. Sears, Andrew , & Jacko, Julie A. (2007). The Human-computer interaction handbook: fundamentals, evolving technologies and 3merging applications: CRC; 1 edition (Sep 1 2002).
Usability evaluations
© BOISSY April 2013 23 Human Factors SIG
Web Resources & References