Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI
description
Transcript of Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI
![Page 1: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/1.jpg)
Usability and User-centered Design in Biomedical Informatics:Using computers to help patients! Not hurt them.
Katrina M. RomagnoliDr. Harry Hochheiser
7/26/13CoSSBI
![Page 2: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/2.jpg)
Who knows how to make a peanut butter and jelly sandwich?
• Write down the instructions for making a PB&J.
![Page 3: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/3.jpg)
Not so simple, right?
• Try doing that for a complicated computer system used by a hospital.
• Or a radiation delivery system.
• Or a medication delivery system.
![Page 4: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/4.jpg)
Challenge
• Computers in medicine can help people.
• They can also kill them.
• How do we build computer systems in healthcare that help clinicians better care for their patients?
![Page 5: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/5.jpg)
“Radiation Offers New Cures, and Ways to Do Harm”
Walt Bogdanich, NY Times, January 23 2010“As Scott Jerome-Parks lay dying, he clung to this wish:
that his fatal radiation overdose — which left him deaf, struggling to see, unable to swallow, burned, with his teeth falling out, with ulcers in his mouth and throat, nauseated, in severe pain and finally unable to breathe — be studied and talked about publicly so that others might not have to live his nightmare.”
“A New York City hospital treating him for tongue cancer had failed to detect a computer error that directed a linear accelerator to blast his brain stem and neck with errant beams of radiation. Not once, but on three consecutive days.”
![Page 6: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/6.jpg)
“Radiation Offers New Cures, and Ways to Do Harm”
Walt Bogdanich, NY Times, January 23 2010“Shortly after 11 a.m., as Ms. Kalach was trying to save her
work, the computer began seizing up, displaying an error message. The hospital would later say that similar system crashes “are not uncommon with the Varian software, and these issues have been communicated to Varian on numerous occasions.””
“An error message asked Ms. Kalach if she wanted to save her changes before the program aborted. She answered yes.”
“When the computer kept crashing, Ms. Kalach, the medical physicist, did not realize that her instructions for the collimator had not been saved, state records show. She proceeded as though the problem had been fixed.”
![Page 7: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/7.jpg)
Therac-25
• Medical linear accelerator• Radiation Delivery for Cancer Treatment
• Therac-25 - Third iteration• First that was completely computer controlled
![Page 8: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/8.jpg)
Therac-25• Treatment suspend – must reboot• Treatment pause – single key restarts
• Up to 5 times
• “Error messages provided to operator were cryptic, and some merely consisted of the word MALFUNCTION followed by a number from 1 to 64 denoting an analog/digital channel number”
• Manual did not supply or describe these codes• Operators did not understand errors• One clinic: average of 40 dose-rate malfunctions/day
![Page 9: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/9.jpg)
Therac-25 causes• Many – bad software, poor practices, lack of
defensive design• “Safe versus Friendly User Interfaces: Making the
machine as easy to use as possible may conflict with safety goals. Certainly, the user interface design left much to be desired, but eliminating multiple data entry and assuming that operators would check the values carefully before pressing the return key was unrealistic.”
• Speed of performance should not always trump error rate
![Page 10: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/10.jpg)
Fluorouracil Incident Root Cause Analysis
• Chemotherapy medication incident• 4-day dose of fluorouracil delivered in 4
hours• 28.8 mL entered as rate. - per hour
• 3/5 participants entered incorrect data when programming pump
• 5/5 confused with one or more aspect of pump operation
![Page 11: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/11.jpg)
Fluorouracil
![Page 12: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/12.jpg)
Who do we blame?
• “User error”: the user just shouldn’t have made the error.
• Simple, right?• ….Or, the manufacturers should make them
easier to use!• That’s simple, right?
![Page 13: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/13.jpg)
What are the goals?
• Understand the problems that a system must solve• what does it have to do?
• Understand the users who will use the system• What do they know? • What can they do?
![Page 14: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/14.jpg)
Fundamental Theorem of Biomedical Informatics
![Page 15: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/15.jpg)
Human-Computer Interaction
• Definition: “the discipline concerned with the design, evaluation, and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them”
- ACM SIGCHI Curriculum Development Group, 1992
• Understand human abilities, needs, goals in order to design systems that will help meet those goals and needs
![Page 16: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/16.jpg)
Usability
• Usable: one can use a system to successfully complete goals and tasks.
• Unusable: a system is unusable if it interferes with the successful completion of goals and tasks.
• A system that has crashed is unusable.
![Page 17: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/17.jpg)
What usability is not
• Not “user-friendliness”• Are you guys old enough to remember this
creature?• He was “friendly”• Not usable!• Being “cute” is not the same
as being helpful.
![Page 18: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/18.jpg)
Human Factors
• Ergonomics (or human factors) is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance.
--International Ergonomics Association
![Page 19: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/19.jpg)
Human abilities
• We are limited by– Perception– Attention– Memory– Language (reading, speaking, listening)– Problem solving (planning, reasoning, decision
making)– Learning– Mental models
![Page 20: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/20.jpg)
Visual Perception
• Color • Contrast
What is the time?
What is the time?
What is the time?
What is the time?
What is the time?
What is the time?
![Page 21: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/21.jpg)
Pre-attentive Processing: Color selection
![Page 22: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/22.jpg)
Pre-attentive processing: Shape Selection
![Page 23: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/23.jpg)
Not pre-attentive: conjunction of shape and color
![Page 24: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/24.jpg)
Motor abilities
• How quickly we can move around, with what parts of our bodies
• Cost of various actions– Mouse vs. keyboard
• Cut down on number of steps required
![Page 25: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/25.jpg)
Attention
• http://www.youtube.com/watch?v=47LCLoidJh4
• How do you alert user of a problem?– Each claim on attention has a cost– Context-shift takes time, loses focus
• When do you interrupt the user?
• How do you interrupt the user?
![Page 26: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/26.jpg)
Short term Memory
• Temporary recall– 7+/- 2 ‘chunks’ of information retained• George Miller- strings of numbers
– Rapid access, rapid decay– Repeat to send to long-term– Don’t overtax!
![Page 27: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/27.jpg)
User-centered Design
• Incorporate the user(s) (needs, goals, limitations) into the design and development process
• At every step, not just at the end, or just at the beginning!
![Page 28: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/28.jpg)
It’s not as obvious as it sounds.
• Requires an investment of time, money, and effort
• Many types of studies at various phases of development
![Page 29: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/29.jpg)
Seems like a pain! Why bother?
• Reduces implementation issues, which are more costly to fix later
• Reduces costs over the long run– Every dollar invested in usability returns $10-100!
(Karat, 1994)
![Page 30: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/30.jpg)
How does it happen?
• A variety of studies performed at various points during development– Starts before you even start writing code!– Continues the whole way!– Even after it’s implemented, you can still work on
usability!– I know it sounds Sisyphean, but remember the
previous slide: it’s worth it.
![Page 31: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/31.jpg)
Most important step
• Identify your users!– Remember: users are not just the people who will be
directly interacting with the software.– They could also be the people paying for it.– The people who will be keeping it running (IT
support staff)• Identify roles versus users– A clinician taking her child to the pediatrician is
interacting with the EMR as a patient, not a clinician.
![Page 32: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/32.jpg)
First Steps
• Contextual Inquiry– What task(s) will this resource support?– What task(s) do the users want/need it to
support?– What is the current environment?• Stakeholders?• Current workflow?• What works currently?• What doesn’t work currently?
![Page 33: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/33.jpg)
Contextual Inquiry
• Observations– Watch your users in the wild.
• Semi-structured interviews– Talk to your users, sort of in the wild.
• Surveys– Talk to your users, at a distance.
• Focus groups– Talk to groups of your users, sort of in the wild.
![Page 34: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/34.jpg)
Let’s try it!
![Page 35: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/35.jpg)
Build models
• Stories, use-cases, scenarios• Diagrams, pictures• Describe the work flow– Who does what? When? How?
• What can be improved? Where are the points that things can go wrong?
![Page 36: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/36.jpg)
Next step: Prototyping
• No, you may not begin programming yet. Soon, I promise.
• Draw a prototype.– Could be as simple as a paper drawing of the
possible interface, or Power Point slides.• Draw another prototype, and maybe another.– Multiple options are better than no options.
![Page 37: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/37.jpg)
Prototyping, cont.
• Show your prototypes off to some users!– Have them mimic doing some tasks. • Can they find the menus? Can they figure out where
they need to go, what they need to do?– Ask them what they like.– Ask them what they do not like.
• Incorporate changes into new prototypes!• Rinse and repeat, until you (and your users)
are satisfied enough.
![Page 38: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/38.jpg)
Interface Design in 5 min
• Art, not science• Interaction Style• “First, do no harm”– Build on familiar models– Metaphors– Don’t mess with convention!
• Less is always more– Get the basics right.
• Avoid 3D– Unless necessary
![Page 39: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/39.jpg)
Harry’s 8 Golden Rules (via Ben Shneiderman)
1. Strive for consistency2. Cater to universal usability3. Offer informative feedback4. Design dialogs to yield closure5. Prevent errors6. Permit easy reversal of actions7. Support internal locus of control8. Reduce short-term memory load
![Page 40: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/40.jpg)
Is this a good interface?
![Page 41: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/41.jpg)
Now you may program.
• Go ahead.
![Page 42: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/42.jpg)
Ok, stop!
• That’s plenty. • Don’t get too far into development without
showing it to some users and doing some evaluation.
• The bigger it is, the harder it will be to make changes.
![Page 43: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/43.jpg)
Lab evaluations: No users!
• Heuristic analysis (Nielsen, 1994): look at these throughout! – Learnability: should be easy to learn– Efficiency: an experienced user should be very
productive using it– Memorability: features should be easy to retain – Errors: System should minimize errors, support error
detection and recovery– Satisfaction: the user experience should be
subjectively satisfying (and not aggravating)
![Page 44: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/44.jpg)
Let’s do it!
![Page 45: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/45.jpg)
Lab evaluations: no users!
• Cognitive Walk-through
– Characterize the cognitive processes of users when engaged in a task
– Analysts (NOT users)• Go step by step through a task from beginning to end• Track mouse clicks, cognitive actions, responses from
the system, and screen transitions.
![Page 46: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/46.jpg)
Lab evaluations: no users!
• Questions to ask during a Cognitive Walk-through: – Are there too many mouse clicks? – Are the responses from the system clear? – What does the user have to know to reach their goal? – Where could they go astray? How likely is it that that could happen?– How can you improve these things?– What can you minimize to make it even simpler?
• Who should do lab analysis?– Ideally, not the person/people building the system. Having fresh, objective
eyes is a good thing.– Not always possible. But do your best to be objective and critical of your
own work if you have to do it yourself.
![Page 47: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/47.jpg)
Rinse. Repeat.
• Make improvements. • Do it again.• Make improvements.• Do it again.• This is known as iterative design.
![Page 48: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/48.jpg)
Analysis in the Wild
• It’s time to show your software off to the people who will use it! – Find out if they CAN use it.– Can they do their work on it?– Is there an improvement compared to the current
environment?– What problems do they experience?
![Page 49: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/49.jpg)
Task Analysis
• Recruit users from each stakeholder group• Give them a series of tasks that your software
supports.• Record what they do!– Screen recording – Video recording– Audio recording
• Ask them to “Think Aloud”.
![Page 50: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/50.jpg)
Think Aloud
• Think aloud means vocalizing what is going through your head while you do a task
• You want to know:– What they are thinking– Why they are doing what they are doing– Why did they click that? What did they expect to
happen? What are their assumptions about your system?• So if they have a problem, you can know better
what caused it.
![Page 51: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/51.jpg)
Analyze the data
• Someone (probably you) gets to watch a lot of video.
• Code every step, every mouse click, every selection, every error, every success.
• Measure the error rate.– How many users fail at doing a task? What tasks?
Where do they go wrong? Why?
![Page 52: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/52.jpg)
Experimental Deployment
• Deploy your software in the wild on a small scale
• Randomly (if possible) select some users to use that software for their work, and others the current software– Example: compare a new computerized physician
order entry (CPOE) tool to the old one• Measure the difference!
![Page 53: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/53.jpg)
Example: Avansino & Leu, 2013
• Effects of CPOE on Provider Cognitive Workload: A Randomized Crossover Trial
![Page 54: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/54.jpg)
Rinse. Repeat.
• Do this iteratively, too.• Make improvements again.• Evaluate it again.
![Page 55: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/55.jpg)
So which evaluations should you do?
• In an ideal world, all of them, repeatedly, with as many users as possible.
• We do not live in an ideal world.
• We have to make compromises.
![Page 56: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/56.jpg)
Lab studies (heuristics, CW)
• Pros:– Inexpensive– Easy– Will cover the vast
majority of problems– Do not have to recruit or
entice users to participate
– Not time consuming
• Cons– Do not use real users– Will miss some problems– Will not get as rich of
data– Data might not be
objective, especially if evaluation is done by developers
![Page 57: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/57.jpg)
Non-lab studies (Task analysis, experiments)
• Pros:– Rich data– Will find problems that
lab studies might not – More objective– Use real users
• Cons:– Expensive– Time-consuming– Difficult to recruit users– Lots of data to analyze
![Page 58: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/58.jpg)
At the very least…
• Do user needs inquiries.
• Do the inexpensive lab studies.
• If you can, do some evaluation with real users.
![Page 59: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/59.jpg)
Whatever you do…
• Try to understand the environment first. You are not the user, so you cannot know what the user needs intuitively.
• Don’t NOT evaluate. You will doom yourself to failure if you skip evaluation.
![Page 60: Katrina M. Romagnoli Dr. Harry Hochheiser 7/26/13 CoSSBI](https://reader036.fdocuments.net/reader036/viewer/2022062323/5681645c550346895dd62e01/html5/thumbnails/60.jpg)
Next up….
• Dr. Harry Hochheiser– Our local usability guru! (and source of much of
the material in this lecture!)
• Let’s do some evaluation together!– OpenMRS