ESCTAIC 20051 Evaluation of Graphical Symbols Used in ICU Comprehension among Users in Germany and...
-
date post
21-Dec-2015 -
Category
Documents
-
view
215 -
download
1
Transcript of ESCTAIC 20051 Evaluation of Graphical Symbols Used in ICU Comprehension among Users in Germany and...
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
1ESCTAIC 2005
Evaluation of Graphical Symbols Used in ICU
Comprehension among Users in Germany and in China
Long Liu, Uvo Hoelscher Muenster University of Applied Sciences
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 2
Muenster University of Applied Sciences
10.000 students 12 faculties Faculty of Engineering Physics
– Biomedical Engineering Group
– Centre of Ergonomics and Process Design in Healthcare
Use-Lab GmbH
Affiliated institutes– USA
– China
– Japan
Cooperations Projects
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 3
Motivation of the project
Symbols are widely used due to their advantages:– They transfer information without language barrier
– They occupy less space on devices
– They provide information more quickly and direct than text
Problems with symbol application:– Too many symbols may cause confusion
– Some symbols may be difficult to learn, remember or understand
– Even standardized symbols may be misunderstood
– How are symbols validated before use
– Medical personnel complains about symbols
– Misunderstanding of safety-related symbols may result in additional risk
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 4
According to ISO 14971 and EN 60601-1-6, all use-related risks should be analyzed and controlled
Although many efforts (e.g. standardization) have been exerted to improve symbol application on medical devices, there are examples where users obviously are confused and add descriptions
Symbol application on medical devices
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 5
Purpose of the study
Comprehension of some frequently used IEC 60878 symbols
Influencing factors on symbol comprehension
Intercultural aspects on symbol comprehension, e.g. China, Germany and Canada / USA
Evaluation method (e.g. influence of context information, etc.)
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 6
Criteria for effective symbol application
Noticeability If a symbols can be noticed by target users
LegibilityIf a symbol is legible (under all context conditions, for all target users)
Comprehensibility If a symbol can be comprehended correctly by target users
Learnability If a symbol can be learnt easily
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 7
Methods to evaluate symbol comprehension
Typical tasks that the participants conduct: – Matching a symbol to a specified meaning (matching test)
– Choosing a correct meaning of a symbol (multiple choice)
– Freely guessing the meaning of a symbol (open-ended free “definition”)
Presentation ways: – Symbol presentation using electrical media or on paper
– Symbol presentation with or without time limit
– Symbol presentation without any context, or with context in form of text, drawn picture, photos, etc.
Evaluation criteria:– The correct responses and the incorrect responses of the participants
– Execution time
– Subjective valuation like response certainty or user satisfaction
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 8
Methods suggested by ISO 9186:2001
The comprehensibility judgment test is NOT a test to evaluate the comprehension of the symbols but a test to evaluate the judged comprehensibility of the symbols. In the test a concept and a symbol are presented to the participants. They are asked to judge the percentage of the target population which would understand the meaning of the given symbol without problems.
The comprehension test is a test to evaluate the comprehension rate of the participants of the symbols. In the test the participants are presented with some symbols and are required to express the meaning (the response) of these symbols.
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 9
Method used in the study
Open-ended comprehension test
– Free answer while global context is presented by photo. Global context shows the general workplace where a device with the symbol is typically used
– Free answer while global and fine context are presented by photo. Fine context shows a device with the symbol
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 10
Evaluation criteria (ISO 9186: 2001)
Each response is assigned into one of 7 categories:1. Correct understanding of the symbol is certain (estimated probability of
correct understanding over 80%)
2. Correct understanding of the symbol is very probable (estimated probability of correct understanding between 66% and 80%)
3. Correct understanding of the symbol is probable (estimated probability of correct understanding between 50% and 65%)
4. The stated meaning is the opposite to the assigned
5. Any other response except: “Don’t know”
6. Response: “Don’t know”
7. No response
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 11
Evaluation criteria: scoring
Method suggested by ISO 9186:2001
Final score for the comprehensibility of a symbol:
100 % of category 1 (correct understanding certain)
+ 75 % of category 2 (correct understanding very probable)
+ 50 % of category 3 (correct understanding probable)
- 100 % of category 4 (stated meaning opposite)
--------------------------------------
= Final Score
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 12
Test
Symbols tested – 13 from IEC 60878– 3 from frequently used medical products
Participants were doctors or nurses. Participants had extensive experience with medical devices being marked with the such symbols.
Test has been conducted in their working locations
Participants in Germany– 20 from intensive care department / OR
Participants in China– 13 from intensive care department / OR
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 13
Selected symbols
Number Symbol Intended Meaning Number Symbol Intended Meaning
1 Date of manufacture 9 Trend information
2 Person identification 10 Bell cancel
3* Bellow connection 11 Do not re-use
4 Manual control 12 Nurse
5 Power plug 13* Infusion status
6 Display transfer 14 Locking (locked status)
7 Type BF applied part 15 Zero-point adjustment
8 Stand-by 16* Blood pressure measurement
Symbols with * are from manufacturers, the others are from IEC 60878
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 14
Pictorial Abstract
ICU
/OR
G
ener
alCategory of symbols
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 15
Test material - Symbol with context
answer: Fine context
answer: Global context
Global context
Fine context
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 16
Results – ICU / OR (Germany)
Mean (with global context) = 42,7%
Mean (with fine context) = 52,3%
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 17
Results – ICU / OR (China)
Mean (with global context) = 32,2%
Mean (with fine context) = 48,2%
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 18
Conclusions - 1
Average comprehension of symbols is poor in both countries– RISK ASSESSMENT for symbols used to establish an ESSENTIAL
REQUIREMENT is recommended
Some symbols may be evident for designers but may not evident for users
Fine context is essential for comprehension of those symbols that have lower comprehensibility
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 19
Results - Germany vs. China (ICU)
Special symbols with fine and global context
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
General symbols with fine and global context (China)
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
General symbols with fine and global context
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
Special symbols with fine and global context (China)
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
Global context
Fine context
General symbols Specialized symbols
Germany
China
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 20
Results - Germany vs. China (ICU)
Abstract symbols with fine and global context (China)
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
Global context
Fine context
Abstract symbols Pictorial symbols
Germany
China
Abstrat symbols with fine and global context
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
Pictorial symbols with fine and global context
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
Pictorial symbols with fine and global context (China)
0,0%
20,0%
40,0%
60,0%
80,0%
100,0%
1 2 3 4 5 6 7 8
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 21
Conclusions - 2
Differences in comprehension performance in Germany and China are not significant
Work experience has more influence on comprehension performance than cultural difference (general symbols are more frequently used, so they are better comprehended)
Pictorial symbols are much better comprehended than abstract symbols
FachhochschuleMünster University of
Applied SciencesCenter for Ergonomics and Usability Engineering
Z F ECenter for Ergonomics and Usability Engineering
Z F E
ESCTAIC 2005 22
Symbol evaluation by ISO 9186:2001
Rather subjective;
Difficulties to exactly assign responses to category 1-3;
The assignment between category 3 and 5 is not always clear;
The formula for the final scores should consider category 5
The formula for the final scores should checked
Evaluation process is not efficient.