Awake Institute Scientific Study 3-20-15
-
Upload
carol-setters -
Category
Documents
-
view
12 -
download
0
Transcript of Awake Institute Scientific Study 3-20-15
Awake Institute
Evaluation of Fatigue Management Systems
A Study of the PRISM System at the Kumba Kolomela Mine
June 4, 2012
Prepared by Anneke Heitmann, PhD, Awake Institute, LLC
1
CONTENTS
1. Background
1.1. Description of the Evaluated Fatigue Management System……………………. 02
1.2. Study Objectives……………………………………………………………………………………… 04
2. Methods
2.1. System Implementation……………………………………………………..…………………… 05
2.2. Study Design and Measures…………………………………………….………………………. 08
2.3. Study Participants……………………………………………………………………………………. 10
2.4. Statistical Analysis.…………………………………………………………………………………… 11
3. Results of Surveys
3.1. General Assessment …….………………………………………………………………………..…. 13
3.1.1. Benefits…………………………….………………………………………………..… 13
3.1.2. Feasibility/Practicability and Acceptability............................... 18
3.1.3. Sensitivity to Increased Fatigue………………………….…………………. 21
3.2. Fatigue Countermeasure Assessment.……………………………………………………… 22
3.2.1. Countermeasure Usage………………………………………………………... 22
3.2.2. Compliance with Countermeasure Recommendations………….. 24
3.2.3. Countermeasure Effectiveness………………………………………..……. 24
3.2.4. Countermeasure Practicability………………………………….………….. 26
4. Results of Alertness/Performance Testing and PRISM
4.1. PRISM Sensitivity to Increased Fatigue ….………………………………………………… 27
4.2. PRISM System Effectiveness …..….………….……………………………………..………… 38
4.2.1. Comparison of Data from Baseline and Post-Implementation 38
4.2.2. Countermeasure Usage and Compliance .……………………………. 40
5. Summary………………………………………………………………………….……………………………………..… 42
Appendix…..…………………………………………………………………………………………………………………….. 45
2
1. BACKGROUND
1.1. Description of the Evaluated Fatigue Management System
Kumba Iron Ore, LTD, is a recognized leader in mining safety among its peer mining
corporations within South Africa. Kolomela mine is a greenfield open pit iron ore operation set
to become one of the largest open mining sites in the world. The management team at
Kolomela has adopted a mission statement which includes becoming one of the safest mines in
the world. To that end, they are building a culture of safety from the ground up including the
utilization of the latest fatigue management technologies available. The Kolomela mine chose
to evaluate the PRISM fatigue management system because it offered the ability to predict
fatigue risk and monitor fatigue levels on an individual employee basis in a real time
environment and then provide job-specific countermeasures to mitigate the fatigue risk for
employees that exceed a predetermined fatigue threshold.
PRISM - Predictive Risk Intelligent Safety Module - links human fatigue risk prediction software
and validated alertness technologies to reduce schedule-specific risk. The system is interfaced
with common Time & Attendance systems to predict fatigue risk in real time to provide
practical, schedule-specific fatigue mitigation recommendations. PRISM can give a graphical risk
output to workers at clock-in and clock-out, and can also provide to workers and supervisors,
via various media, automatic notifications when specific fatigue thresholds are crossed. In
addition, PRISM data can be used to track worker fatigue in operational units over longer time
periods and it provides several data-driven management tools (e.g., statistical reports) to give a
detailed overview of the workforce fatigue status.
Fatigue risk prediction is based on bio-mathematical models rooted in science related to
circadian rhythms and sleep physiology. Alertness prediction models take into account
individual sleep patterns (duration, timing and quality of sleep), using actual sleep and/or
predicted sleep based on schedule-specific sleep opportunities. The well-established Three-
Process Model of sleep and alertness computes alertness taking into consideration homeostatic
factors (build-up of sleepiness during wakefulness and dissipation during sleep), circadian
factors (time-of-day alertness changes based on the phase of the biological clock and its
adjustment to changes in circadian sleep-wake patterns), and sleep inertia (the transitory
impairment of alertness after wake-up depending on duration of prior sleep and other factors).
The bio-mathematical model of PRISM used in this evaluation study is called FIRM, or Fatigue
Index Risk Measurement. FIRM provides a Fatigue Risk Index for each worker in real-time. The
scale of the Fatigue Risk Index is divided into several risk zones, ranging from Low to Severe,
and each risk zone calls for tailored fatigue mitigation actions. The risk zones can be adjusted to
specific job risk profiles. See Figure 1.
In this study, PRISM calculated risk scores directly from actual historic work hour records in real
time at clock in and clock out and predicted the individual fatigue level for upcoming shifts and
3
commutes based on projected scheduled hours. Notifications/warnings were delivered to
workers and supervisors/managers via SMS. Then fatigue countermeasure recommendations
were communicated after accessing a PRISM station. Details about the specific fatigue
countermeasures can be found in the Method section.
Signia Risk
5 Point
Signia
(Refined)
7 Point
Scale Range
Potential %
Exposed and/or
Diminished
Capacity
Strategy
5 Point
Strategy
7 Point
10 9.51 - 10+ 0%
9 8.51 - 9.50 5%
8 7.51 - 8.50 10%
7 6.51 - 7.50 15%
6 5.51 - 6.50 20%
5 4.51 - 5.50 25%
4 3.51 - 4.50 30%
3 2.51 - 3.50 35%
2 1.51 - 2.50 40%
1 .51 - 1.50 45%
0 .50 - (-.50) 50%
-1 -.51 - (-1.50) 55%
-2 -1.51 - (-2.50) 60%
-3 -2.51 - (-3.50) 65%
-4 -3.51 - (-4.50) 70%
-5 -4.51 - (-5.50) 75%
-6 -5.51 - (-6.50) 80%
-7 -6.51 - (-7.50) 85%
-8 -7.51 - (-8.50) 90%
-9 -8.51 - (-9.50) 95%
-10 -9.51 - (-10+) 100%
Protect
(Watchful/Attentive)
Guard
(Shield/Defense)
Protect
Low
Significant
Guarded
Risk Index Measurement Scale
Optimal
Maintain
(Preserve-Conserve)
Nominal
Low
Maintain
Extreme
Guarded
Significant
High
Guard
Proactive
Severe
Vigilant
Copyright 2001-2009
Vigilant
(Cautious/Alert)
Priority 1
Proactive
(Active/Control)
Severe
High
• These index values require immediate
action. Employee will be asked to
take counter measure before
reporting to work station or continuing
work
• At this index value, employee can continue to work but have awareness
of the risk and utilize counter
measures
• At this index value, employee can
safely report to his work station.
Employees need 1 or more days off to
recover to green status
-5 to -10 SEVERE
-2 to -4 HIGH
1 to -1 SIGNIFICANT
4 to 2 GUARDED
10 to 5 LOW
Critical Job Risk Index Legend
Scale is adjusted to job risk profile!
Figure 1: Zoning of the PRISM Fatigue Risk Index
4
1.2. Study Objectives
This study evaluates the PRISM fatigue management system in an around-the-clock operational
setting. Specifically, it investigates the effectiveness of the PRISM system as a fatigue
management tool, its impact on fatigue awareness, operational feasibility and acceptability of
the system, and validity and sensibility of the PRISM Fatigue Risk Index as a measure of
impaired alertness and performance.
.
5
2. METHODS
2.1. System Implementation
The PRISM evaluation study was conducted at the KIO Kolomela mine in South-Africa.
Figure 2: Images from the Kolomela mine
Testing was conducted at work units with around-the-clock operations. The shift schedule
involved blocks of three and four consecutive 12-hour day or night shifts, separated by two- or
three day breaks with a seven-day break every four weeks. See Figure 3.
WEEKS/ WEEKS/
CREWS CREWS
1 - - D D D - - 1
2 N N N N - - - 2
3 - - - - - D D 3
4 D D - - N N N 4
MON TUE WED THU FRI SAT SUN
WEEKS/ WEEKS/
CREWS CREWS
1 - - D D D - - 1
2 N N N N - - - 2
3 - - - - - D D 3
4 D D - - N N N 4
MON TUE WED THU FRI SAT SUN
Figure 3: Work schedule of the participating work units, involving 12-hour day and night shifts.
Shift changes were at 6:30 am and 6:30pm.
6
PRISM generated various fatigue status reports for the employees. See Figure 4 shows a typical
view of the ‘real time’ fatigue status of everyone clocked into the site. The first line managers
use this to track the status and response of their teams.
Figure 4: Sample of fatigue status report (screen shot)
The implemented fatigue countermeasures included primarily:
- energy drinks,
- exercise breaks,
- napping station (designated napping room).
Fatigue countermeasure recommendations also included drinking water and eating high-
protein snacks, dried fruit and meat. A bright light station was also available, but not used in
the study as there was no occurrence of the higher fatigue levels that called for that measure.
Figure 5: Fatigue countermeasures: Napping station, energy drink, bright light.
Countermeasure assignments were communicated via SMS and needed to be confirmed by the
worker on a PRISM unit within one hour. Depending on the actual fatigue risk level (Significant,
High or Severe), there were three different sets of fatigue countermeasures. See Figure 6. The
Significant level resulted in recommendations for an exercise break, drinking water and an
energy drink and eating a high protein snack. At the High level, recommendations also included
a 20-min nap as well as advisement to work with a partner with good fatigue status. At the
7
Severe level, recommended nap duration was increased to 30 min, and bright light use was
advised. The maximum nap duration of 30 min is based on scientific evidence that longer naps
(e.g., 45 or 60 min) induce increased sleepiness at wake-up (sleep inertia), and would also be
operationally less practical.
All participants attended a fatigue countermeasure training session prior to actual testing. The
training informed participant about the use of countermeasures and the notification process.
See Figures 6 and 7.
Figure 6: Sample slides from Fatigue Countermeasure Training – Mitigation strategies by risk level
� Drink between 11:pm –3:am
� Try to drink with food
� Try NOT to drink after 3:am
� 911 drink lasts 5-6 hours
� You can request 911 drink at shift start if tired
� Requires Supervisors permission – only he has a key to nap station
� Timing to use nap station agreed between you and supervisor
� Used for ‘High’ and ‘Severe’ Fatigue Status◦ At ‘High’, set alarm for 25 minutes
◦ At ‘Severe’, set alarm for 35 minutes
Figure 7: Sample slides from Fatigue Countermeasure Training – Countermeasure user instructions
8
2.2. Study Design and Measures
Data collection was conducted during two study phases, before and after PRISM implementation.
See Figure 8. Each study phase included one full cycle of the roster schedule of 28 days, with both
sets of multiple 12-hour day shifts and multiple 12-hour night shift blocks. Workers were asked to
record their activity pattern (hours of sleep, wake, work and commuting to/from work) in an
Activity Diary for about one month during each study phase. During about the same time period,
they also participated in testing before and after each shift. The testing included an alertness test
battery with Visual Analog Scales for subjective ratings of arousal level, mood, motivation,
concentration and physical fatigue; the Karolinska Sleepiness Scale; and a computerized four-choice
reaction time test (Wilkinson Test). In addition, workers completed a daily Shift Performance Log at
the end of each work shift. See Figure 9. Baseline (pre) testing with PRISM off occurred in June/July
2011 and Post-implementation testing with PRISM on in September/October 2011.
daily Activity Diary (records times of sleep/work/commuting)
Work Shift
Alertness Test Battery Alertness Test BatteryShift Performance Log
Surveys:
Fatigue and Health Surveys (during Baseline and Post-Implementation)
PRISM Evaluation Surveys (after study completion)
Pre/Post-Shift Testing:(for all shifts)
twostudy
phases
BaselinePRISM Off
Post-ImplementationPRISM On
1 2
Figure 8: Study design
9
Activity Log(Example)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Sleep*
Awake
Commuting**
Work I: Regular
Work II: Training
Time of Day
ID#: 28 Date: 06/03/11 Weekday: Friday
6
Wilkinson Response Test
Figure 9: Selected data collection tools
The Visual Analog Scales 1 2 3 were 100-mm lines (on which participants marked their ratings) with
the ends of the scales representing the opposite extremes for each parameter: arousal – very
sleepy / very alert; mood – very bad mood / very good mood; motivation – not motivated at all/
very motivated; concentration – unable to concentrate / able to concentrate very well; physical
fatigue – very fatigued / not fatigued at all. The Karolinska Sleepiness Scale (KSS) 4 is a 9-point
scale: 1 = very alert; 9 = very sleepy, great effort to keep awake, fighting sleep. The daily Shift
Performance Log included fifteen brief questions to assess the overall level of alertness and
performance during the shift, including potential nodding-off events and
accidents/incidents/injuries, and countermeasure use.
1 Folstein MF and Luria R (1973): Reliability, validity and clinical application of the Visual Analog Mood Scale.
Psychol Med 3, 479-486.
2 Lee Ka, Hicks G, Nino-Murcia G (1991): Validity and reliability of a scale to assess fatigue. Psychiatry Research 36
(3), 291-298.
3 Wewers ME, Lowe NK ( (1990): A critical review of visual analogue scales in the measurement of clinical
phenomena. Research in Nursing and Health 13, 227-236.
4 Kaida K, Takahashi M, Akerstedt T, Nakata A, Otsuka Y, Haratani T, Fukasawa K (2006): Validation of the
Karolinska sleepiness scale against performance and EEG variables. Clinical Neurophysiology 117(7), 1574-1581
10
In addition to the activity diary and on-shift testing described above, two types of surveys were
administered: a Fatigue and Health Survey and a PRISM Evaluation Survey.
The Fatigue and Health Surveys were used to describe the study population and to assess
comparability of any potential differences in the participant groups used for Pre and Post
testing. Questions included demographic information (age) and information on sleep, alertness,
medical issues, overtime, and fatigue management. A 52-question version of the Fatigue and
Health Survey was administered to workers, and a shorter version (21 questions) was used for a
smaller group of individuals who held supervisory or management positions (workers in
supervisory positions completed both versions). This survey was administered three times: at
the end of the pre-implementation and post-implementation study phases in 2011, as well as in
fall 2010, one year before the post-implementation testing (early baseline). This early baseline
survey was used for assessing the understanding of fatigue levels and fatigue monitoring before
participants had any knowledge of the PRISM system. The complete Fatigue and Health Survey
for workers with the results from Post testing is included in the Appendix.
The PRISM Evaluation Surveys were administered to workers (26 questions) and to supervisors
and managers (14 questions) at the end of the post-implementation study in fall 2011. The
purpose of this survey was to assess the general benefits and operational
feasibility/acceptability of the PRISM system and specific fatigue countermeasures. In addition
to multi-choice response options, many questions of the surveys included space for verbal
comments.
2.3. Study Participants
The study participants were workers of the Maintenance&Engineering group at Kumba’s
Kolomela mine. Table 1 shows the number of people participating in the survey data collection.
Number of
Participants
Health/Fatigue
Survey
Fall 2010
Health/ Fatigue
Survey
Summer 2011
Health/Fatigue
Survey
Fall 2011
PRISM Evaluation
Survey
Fall 2011
Workers 58 16 25* 25*
Supervisors/Managers 7 4 7 7
Table 1: Number of individuals participating in survey data collection. *Two additional workers who completed the surveys were not yet on the PRISM notification system and
were excluded from the analysis.
All the survey participants were male except for three workers who participated only in the
Health and Fatigue Survey during the earlier baseline in fall 2010. The worker surveys were
completed by shift-working mine workers including some with supervisors function, and the
11
supervisors/manager surveys were completed by shift-working supervisors and some section
managers who were dayshift supervisors.
The group of mine workers who were on PRISM and completed the PRISM Evaluation Survey at
the end of the study included 25 people, with 12 people, 9 people and 4 people in the age
brackets 20-29 years, 30-39 years and 40-49 years, respectively. See Appendix for complete
responses for all survey questions.
The survey results for the workers (Result section) are shown in percentage of workers, rather
than number of workers, to standardize result presentation and adjust for occasional
incidences where a worker skipped a question. The survey results for supervisors/managers,
however, are showing actual numbers of respondents as this study group was very small.
A smaller group of workers participated in the experimental testing involving Shift Logs and test
battery measures. Table 2 shows the number of individuals included in Baseline and Post-
Implementation results. The number of shifts for each participant per test phase varied
between individuals, ranging from just a few shifts in some cases to nearly 30 shifts. The data
from participants with only single or few shifts or with generally unreliable data (e.g.,
individuals who never/rarely changing default response settings) were excluded from the
analysis.
The participant numbers in Tables 1 and 2 are for workers who were monitored on PRISM. Shift
Log and test battery data from a few additional study participants, but who were not on PRISM,
were not included in this report in order to assure that the analysis is are based on comparable
data pools.
Shift Logs
Summer 2011
(Baseline)
Test Battery
Summer 2011
(Baseline)
Shift Logs
Fall 2011
(Post)
Test Battery
Fall 2011
(Post)
17 15 23 18
Table 2: Number of individuals participating in survey data collection
2.4. Statistical Analysis
The statistical analysis included the following tests:
- Chi-Square test for comparing frequencies (e.g., survey data),
- t-test or Mann-Whitney Signed Rank test for comparing means of independent measures (for
data passing or failing the normality test, respectively),
12
- paired t-test or Wilcoxon Signed Rank test for comparing means of paired records (repeated
measures) (for data passing or failing the normality test, respectively),
- correlation analysis (Pearson correlation coefficient) for analyzing intra-individual correlations
between PRISM data and the various experimental test data.
The results of the statistical analyses are shown in the graphs by indicating the level of statistical
significance: ‘***’, ‘**’ or ‘*’ for indicating a statistically significant difference at the significance
level p=<0.001, P=<0.01 or p=<0.05, respectively, ‘(*)’ for a trend just below the 0.05 significance
level, or by ‘n.s.’ for not significant differences. The expression “statistically significant” refers to a
result that is unlikely to have occurred by chance, with “p” indicating the probability that the
difference in the data between two groups may be due to random sampling variability (lower p
indicates stronger result).
13
3. RESULTS OF SURVEYS
3.1. General Assessment
3.1.1. Benefits
Most of the workers (76% - 19 people) and all of the supervisors/managers rated the potential
benefits of the PRISM fatigue monitoring system as “very” or “somewhat beneficial” on the
PRISM Evaluation Surveys, with all but one of the supervisors/managers rating it “very
beneficial”. While workers overall favored the system (no worker selected the response option
“not beneficial at all”), the percentage of individuals selecting the highest rating was higher in
the supervisor/manager group than in the worker group (52% of all worker survey respondents,
and 43% when only considering respondents who also had supervisor status). See Figure 10.
Perc
enta
ge
of W
ork
ers
0
20
40
60
80
100
not beneficialat all
slightlybeneficial
verybeneficial
somewhatbeneficial
not sure
How would you rate the potential benefits of PRISM fatigue monitoring system?
0
Nu
mb
er
of S
up
erv
iso
rs/M
an
ag
ers
0
1
2
3
4
5
6
7
not beneficialat all
slightlybeneficial
verybeneficial
somewhatbeneficial
not sure
How would you rate the benefits of PRISM fatigue monitoring system?
0 0 0
Figure 10: Ratings of the potential benefits of the PRISM fatigue monitoring system, assessed by
workers (left) and supervisors/managers (right)
Workers and supervisors/managers made very positive comments about the PRISM benefits.
Many of the verbal survey comments are included throughout the Result section, in blue and
green boxes for workers and supervisors/managers, respectively. The study team was positively
surprised about the many verbal comments received on the surveys, which in itself indicates
that people cared about the project. “Supervisors acknowledge that we are not robots” is
perhaps one of the most illustrative comments indicating the importance of fatigue
management.
Workers’ Quotes: Benefits of PRISM
“It will reduce the rate of fatigued employees.”
“[It]helps you by telling you what to do when you feel sleepy on the job. Making you aware of your status.”
“How to work against low alertness, i.e., stretching, right things to eat and all that.”
“Supervisors acknowledge that we are not robots”
“Makes you aware of your fatigue levels, in other words, you are not in denial about your fatigue status and
alertness status.”
14
One of the most important benefits of the PRISM fatigue monitoring system is increased safety
awareness. All of the supervisors/managers and most of the workers (84% - 21 people) agreed
that the system increases awareness of job safety and performance, with the remaining
workers not being sure except for one person who disagreed. See Figure 11.
Do you think that PRISM fatigue monitoring system
increases your awareness of job safety and performance?
Pe
rcen
tage
of W
ork
ers
0
20
40
60
80
100
not sureyes no
Do you think that PRISM fatigue monitoring systemincreases awareness of job safety and performance?
Num
be
r of
Mana
gers
0
1
2
3
4
5
6
7
not sureyes no
0 0
Figure 11: PRISM-related increase of awareness of job safety and performance, assessed by
workers (left) and supervisors/managers (right)
All of the supervisors/managers said on the PRISM Evaluation Surveys that PRISM gives them
the ability to manage their employees’ fatigue levels at work, and most of the workers (80% -
20 people) said that it gives them the ability to manage their own fatigue level. See Figure 12.
These questions were also asked on the Fatigue and Health Surveys during baseline and post-
implementation. It showed an increase in the percentage of workers who thought that a fatigue
monitoring system would give them the ability to manage their fatigue, from 73% (baseline one
year prior to post) to 88% (post), indicating that becoming familiar with an actual fatigue
monitoring system lead to more people supporting it. See Figure 13, top panel.
Supervisors/managers, on the other hand, were already quite confident about fatigue
monitoring during baseline, with all participants saying in both test phases that a fatigue
monitoring system would give them the ability to manage their employee’s fatigue.
Supervisors’/Managers’ Quotes: Benefits of PRISM
“People know their fatigue status and tend to be more careful.”
“[PRISM] reduces safety risk, it makes me always wanting to monitor the focus of each team member.”
“If [workers] know their status they can be able to manage the way they work.”
Workers’ Quotes: PRISM and Safety Awareness
“It makes a person to be alert all times and stay focused on your job”
“At least I know what to do to keep me awake even though I fail to be alert sometimes”
15
Do you think that PRISM fatigue monitoring system gives you
the ability to manage your fatigue level while on your shift?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
100
not sureyes no
Do you think that PRISM fatigue monitoring system gives youthe ability to manage your employees' fatigue level while on shift?
Num
be
r of M
an
agers
0
1
2
3
4
5
6
7
not sureyes no
00
Figure 12: PRISM-related ability to manage worker fatigue levels, assessed by workers (left) and
supervisors/managers (right)
Do you think that a fatigue monitoring system gives you the abilityto manage your fatigue level while on your shift?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
100
Baseline
Post-Implementation
yes not sureno
p<0.05
Do you think you have understanding of your fatigue levelsand have the ability to manage it properly while on your shift?
Pe
rce
nta
ge
of
Wo
rke
rs
0
20
40
60
80
100
Baseline
Post-Implementation
yes not sureno
0
p<0.01
Do you think that your manager has an understanding of fatigue levels of your joband provides the ability to manage it properly while on your shift?
Perc
en
tage
of
Wo
rke
rs
0
20
40
60
80
100
Baseline
Post-Implementation
yes not sureno
p<0.001
Figure 13: Pre/post comparisons of perceptions of fatigue and fatigue monitoring. Top: Effects of
fatigue monitoring on ability to manage own fatigue, assessed by workers. Bottom: Workers’ (left
panel) and supervisors’/managers’ (right panel) understanding of workers’ fatigue levels and ability
to manage it, assessed by workers. P-values indicate statistical significance level (Chi-Square test).
To evaluate any changes after the implementation of PRISM, the pre/post Fatigue and Health
Surveys asked participants also whether they had an understanding of fatigue levels and the
ability to manage them. The majority of the workers thought they had an understanding of their
16
own fatigue levels during the post-implementation phase (80% - 20 people) which was
significantly increased from baseline one year earlier (62% - 36 people). A smaller portion of the
workers (44% - 25 people) thought during baseline that their manager had an understanding of
the fatigue levels of the job and an ability to manage fatigue. After PRISM implementation this
percentage of workers being confident in their managers’ understanding of fatigue increased
significantly to 76% (19 people). See Figure 13 lower left and right panels.
This was also reflected in the responses of the supervisors/mangers. During baseline, only four of
the seven participants thought they had an understanding of their employees’ fatigue levels and
an ability to manage it, while almost all (n=6) made this statement on the post-implementation
Fatigue and Health Survey. Similar trends were observed when supervisors/managers were asked
if KIO Kolomela had an understanding of the fatigue levels. See Figure 14.
Do you think you have an understanding of your employee fatigue levels
and they have the ability to manage it properly while on your shift?
Num
ber
of S
uperv
iso
rs/M
ana
gers
0
1
2
3
4
5
6
7 Baseline
Post-Implementation
yes not sureno
Do you think KIO Kolomela has an understanding of fatigue levels of your joband provides the ability to manage it properly while on your shift?
Num
ber
of S
uperv
iso
rs/M
anagers
0
1
2
3
4
5
6
7 Baseline
Post-Implementation
yes not sureno
Figure 14: : Pre/post comparisons of the supervisors’/managers’ (left) and KIO Kolomela’s (right)
understanding of fatigue levels, assessed by supervisors/managers.
While workers and managers thought that a fatigue monitoring system would improve
understanding and management of fatigue (see above), workers interestingly also said that they
would feel better about their work environment when knowing that all employees around them
were monitored for alertness/fatigue. Almost all workers (24 people - all but one) said they
would feel much better or somewhat better, with the majority of them (80% - 20 people) saying
they would feel much better. See Figure 15.
Would you feel better about your work environment knowing
that all employees around you had been monitored for alertness/fatigue?
Pe
rcen
tage
of
Wo
rke
rs
0
20
40
60
80
100
would not mattermuch better somewhat better
Figure 15: Alertness monitoring and workers’ perception of their work environment
17
Fatigue monitoring can help improve working conditions. Almost all workers (24 people – all but
one) thought that PRISM would help management understand workers better (76% - “yes”, 20% -
“somewhat”), and all respondents thought it may encourage other actions by employers to
improve alertness (88% - “yes”, 12% “somewhat”). See Figure 16.
Do you think PRISM fatigue monitoring system may encourageother actions by employers to improve on-the job alertness?
Pe
rce
nta
ge
of
Wo
rkers
0
20
40
60
80
100
noyes somewhat
0
Figure 16: PRISM-related potential effects on management and employers, assessed by workers.
Left: management’s understanding of workers and improving working conditions. Right: Other
actions by employers to improve on-the-job alertness.
In the verbal comment section (see selected quotes below), one individual wrote that PRISM
helps acknowledge the role of fatigue in incidences. One worker commented that the system’s
fatigue assessment is based on work hours rather than “actual conditions”. This statement points
out that it is important to educate workers and management about the theoretical basis of
fatigue prediction and how fatigue models can estimate likely sleep based on sleep opportunities
within a given work schedule. In Section 4, we will compare the PRISM fatigue risk scores with
actual alertness data collect during this study.
Workers’ Quotes: Feeling Better About Work Environment With PRISM
“When working night shift you want to work with someone who is alert”
“If you know that all employees are alert, you will work freely knowing that no one will be injured due to fatigue
related matters.”
Workers’ Quotes: PRISM Helping Management Understand Workers
“It [PRISM] acknowledges that fatigue does play a huge role in the occurrence of incidents (HPI's).”
‘Not really because I think it [PRISM] doesn’t monitor [the] actual conditions [of the] worker, rather judges
[based on] worked hours.”
Do you think the PRISM fatigue monitoring system can helpmanagement understand workers better and improve working conditions?
Pe
rce
nta
ge
of
Wo
rke
rs
0
20
40
60
80
100
noyes somewhat
18
3.1.2. Feasibility/Practicability and Acceptability
Workers and supervisors/managers were asked how feasible or practical PRISM fatigue
monitoring is in their work environment. The majority of respondents (80% of the workers and six
of the seven supervisors/managers) said that it is “usually not a problem” or “not a problem at
all”. No one said that it was “most of the time a problem”, and only some participants (two
workers and one supervisor) thought it could be sometimes a problem. See Figure 17.
How practical would you say it is
to use PRISM fatigue monitoring system in your work environment?
Pe
rcenta
ge o
f W
ork
ers
0
20
40
60
80
100
most of the timea problem
sometimesa problem
not a problemat all
usuallynot a problem
not sure
0
How would you rate the feasibility of conducting PRISM fatigue monitoring system in your work environment?
Nu
mbe
r o
f M
ana
ge
rs0
1
2
3
4
5
6
7
most of the timea problem
sometimesa problem
not a problemat all
usuallynot a problem
not sure
0 0
Figure 17: Feasibility/practicability of the PRISM fatigue monitoring system, assessed by workers
(left) and managers (right).
Specifically, we asked how difficult or easy login and the SMS notification system were. Most of
the workers (84%) said on the survey that it was “not a problem at all“ or “usually not a problem”
(17 people and 4 people, respectively). See Figure 19. The verbal comment section (see selected
quotes below) included positive individual notes like “easy to use because it just requires to log
the card”. Only a few individuals had trouble with system access (e.g., prolonged login time), and
these technical issues can be easily corrected.
How difficult or easy was it for you tolog into the PRISM system during your shift?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
100
most of the timea problem
sometimesa problem
not a problemat all
usuallynot a problem
Figure 18: Easiness to log into the PRISM system, assessed by workers.
19
The SMS notification system was viewed by many workers as generally effective for letting
workers know their fatigue status during their shift or on their way home. For example, most of
the workers (84%) said “yes” it was effective (76% - 19 people) or “somewhat effective” (8% - 2
people) during the shift. Usefulness of SMS notifications for the way home were rated favorably
by 72% of the workers (18 people). See Figure 18. The verbal comment section of the survey
includes notes such as “every time” and “very useful”. However, a few individuals’ experience was
somewhat compromised due to setup issues and they commented that they were either not
registered on the system or the system had the wrong cell phone number.
Was the PRISM system SMS notification effective
to let you know your fatigue status during your shift?
Pe
rce
nta
ge
of W
ork
ers
0
20
40
60
80
100
noyes somewhat
Was the PRISM system SMS notification usefulin letting you know your fatigue status on your way home?
Pe
rce
nta
ge
of
Wo
rke
rs
0
20
40
60
80
100
noyes somewhat
Figure 19: Effectiveness of PRISM’s SMS notification system for letting workers know their
personal fatigue status during the shift (left) and on the commute home (right), assessed by
workers.
The general acceptance of the PRISM fatigue monitoring system among the workers was good.
Sixty percent of the workers (15 people) rated it favorably and about another 32% (8 people) said
they were willing to try, with only two workers rejecting the system. See Figure 20, left panel.
Interestingly, only one supervisor thought workers had a favorable attitude and all other
supervisors and managers thought workers were more neutral (marking on the survey either
“open to concept” or “neutral”), while none of them thought employees would reject the
concept. See Figure 20, right panel.
How would you rate your general acceptance of PRISM fatigue monitoring system?
Pe
rce
nta
ge
of
Wo
rke
rs
0
20
40
60
80
100
don't need itneutralfavorable willing to try
0
How would you rate employees' acceptance of PRISM fatigue monitoring system?
Nu
mbe
r o
f M
ana
gers
0
1
2
3
4
5
6
7
reject the conceptneutralfavorable open to concept
0
Figure 20: General acceptance of the PRISM fatigue monitoring system by workers (left) and
supervisors/managers (right)
20
However, all seven supervisors/managers rated the acceptance of PRISM by their own group as
either “favorable” (three respondents) or “willing to try” (four respondents). See Figure 21.
How would you rate supervisors'/managers' acceptance
of PRISM fatigue monitoring system?
Num
be
r of M
anagers
0
1
2
3
4
5
6
7
don't need itneutralfavorable willing to try
0 0
Figure 21: Supervisors’/managers’ acceptance of the PRISM fatigue monitoring system.
The verbal comments on the surveys about acceptance of PRISM were very positive, and some
workers simply summarized it with comments like “I like it very much”. One of the supervisors
made the suggestion that the actual monitoring of the PRISM output should be managed by
another department and not the foremen, indicating that it may be appropriate to have a
dedicated person who can fully focus on the issue of fatigue management. See selected quotes
below.
A potentially sensitive issue related to any kind of fatigue monitoring or fitness-for-duty testing is
privacy. A little over fifty percent of the workers said they were not worried about privacy issues
related to the PRISM fatigue monitoring system, while about one third was somewhat concerned,
with the remaining people being indifferent. Supervisors/managers seemed to worry less about
privacy issues, with only one of the seven respondents stating that he would be somewhat
worried. See Figure 22. Acknowledging a potential difference between workers and
supervisors/managers, the analysis for the workers was repeated without the four workers who
also had supervisory status, resulting in the percentage of not worried workers decrease from
52% to 43%.
Supervisors’/Managers’ Quotes: Acceptance/Feasibility
“Feasibility is good for everyone. Acceptance is good by all workers. All of us will benefit from it (change the
alertness and fatigue level of coworkers).”
“I just felt that it should be monitored by a separate section, somebody to make sure everybody uses the system,
other than the foreman…..Fatigue Center should be managed by another section (on its own). That person will
manage it better as it will be all he has to do.”
Workers’ Quotes: Acceptance of PRISM
“I like it”
“I like it very much.”
21
How concerned are you about privacy issuesrelated to the PRISM fatigue monitoring system?
Pe
rce
nta
ge
of W
ork
ers
0
20
40
60
80
100
would not matternot worried somewhat concerned
How concerned are you about privacy issuesrelated to the PRISM fatigue monitoring system?
Num
ber
of M
ana
gers
0
1
2
3
4
5
6
7
would not matternot worried somewhat concerned
0
Figure 22: PRISM-related privacy concerns of workers (left) and supervisors/managers (right)
3.1.3. Sensitivity to Increased Fatigue
While most aspects of the PRISM system were judged very positively by the workers, a more
controversial issue was PRISM’s sensitivity to increased fatigue. About two thirds of the workers
(67% - 17 people) thought it was very or somewhat sensitive (four workers rated it as very
sensitive). Most of the remaining respondents were not sure and only one respondent thought it
was not sensitive. See Figure 23. Workers expressed their uncertainty in the verbal comment
section, describing situations when they thought PRISM and their own fatigue assessment were
not quite in agreement (see selected comments below). The last of the selected comments
seemed to indicate that some doubt about PRISM’s sensitivity to reduced alertness may stem
from an uncertainty about how the alertness prediction works “by just swiping you card”.
A systematic comparison between the PRISM risk scores and the results from workers’ testing
during the study is detailed in Section 4 of this report. In this context, it is important to
acknowledge that alertness models are progressively becoming improved and can also be tailored
to specific populations based on actual data. While this development work is of a technical
nature, the main important prerequisite for the success of fatigue monitoring is a readiness of
workers, supervisors and managers to change culture and a willingness to accept new
technologies. The survey data presented above have shown that participants generally thought
the PRISM fatigue monitoring was beneficial, practical and acceptable, and it helped increase
fatigue awareness and understanding which in itself is an important accomplishment for
mitigating fatigue. This indicates also that it will be beneficial to include in the training sessions
more information about alertness modeling.
Workers’ Quotes: Privacy Issues
“Some employees would not reveal their fatigue status which is dangerous to other employees.”
“Don’t think it’s that invasive.”
22
From your own experience with the PRISM Fatigue Monitor System, how do you rate its sensitivity to reduced alertness or reduced performance levels
(i.e. did the PRISM system indicate low alertness levels when you felt sleepy)?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
100
notsensitive
verysensitive
somewhatsensitive
not sure
Figure 23: Workers’ perceptions of PRISM’s sensitivity to reduced alertness
3.2. Fatigue Countermeasure Assessment
3.2.1. Countermeasure Usage
The use of fatigue countermeasures was recommended for subjects who reached one of the
critical fatigue thresholds. At Clock Out, the percentage of shifts in the Significant, High and
Severe bracket were 43%, 2% and 0%, respectively (based on a total number of 497 shifts worked
by the study participants between September 25 and November 1, 2011).
The main fatigue countermeasures used in this study were energy drinks (Significant and higher),
exercise breaks (Significant and higher) and napping (High and Severe). Energy drinks were used
most frequently, with about two thirds of the workers stating that they had (at least) one on
almost every night shift, and about one third having one during almost every day shift. Exercise
breaks were taken less frequently with about one third of the workers taking one once during
their work shift series on night shifts and once on day shifts (a work shift series is a block of three
or four consecutive shifts). Less than twenty percent of the participants had an exercise break
during every shift (16% - 4 people during every night shift, 12% - 3 people during every day shift).
Naps were mostly taken during night shifts, and most people who had tried napping did it about
once during their work shift series, with only a few people taking naps somewhat more or less
frequently. On night shifts when fatigue is expected to be highest, all participants had an energy
Workers’ Quotes: PRISM’s Sensitivity to Reduced Alertness
“Sometimes when I am alert it says my fatigue level is high.”
“In most cases it said I'm in low whereas I was extremely tired/fatigued.”
“Sometimes I got an alert that I'm significant, but then I felt very high.”
“Sometimes it says I am in a fatigue zone whereas I'm active and all awake.”
“We seem to doubt as to what can this Prism monitor detect or measure your fatigue level by just swiping your
card how possible is it.”
23
drink at least once, however a considerable portion of the group said that they never had an
exercise break (40% - 10 people) or nap (60% -15 people) during night shifts. See Figure 24.
How often do you have an energy drink?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
Night Shifts
Day Shifts
neveronce per monthduring my
work shift series
almost everywork shift
onceduring my
work shift series
0 0
How often do you take an exercise break?
Pe
rce
nta
ge
of W
ork
ers
0
20
40
60
80
Night Shifts
Day Shifts
neveronce per monthduring my
work shift series
almost everywork shift
onceduring my
work shift series
How often do you take scheduled naps?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
100
Night Shifts
Day Shifts
neveralmost everywork shift
once per monthduring my
work shift series
onceduring my
work shift series
0
Figure 24: Frequency of workers’ fatigue countermeasure usage during night shifts and day shifts.
Energy drinks (top), exercise breaks (middle), napping (bottom)
24
3.2.2. Compliance with Countermeasure Recommendations
Compliance with PRISM’s fatigue countermeasure recommendations was reasonable, but not
optimal. Energy drinks had the highest compliance, with about two thirds of the workers who
received a recommendation saying they always or often followed the recommendations.
Compliance with recommendations for an exercise break or a nap was somewhat lower. See
Figure 25. Note, not all workers actually had received recommendations for each specific fatigue
countermeasure, and napping recommendations which were only issued for High or Severe
fatigue levels were received by only about half of the workers.
How often do you follow
PRISM fatigue countermeasure recommendations?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
Napping
Energy Drinks
Exercise Break
rarelysometimesalways often
Figure 25: Workers’ compliance with fatigue countermeasure recommendations. Percentages are based on the total number of workers who received the specific recommendations at least once.
3.2.3. Countermeasure Effectiveness
Workers and supervisors/managers were asked to rate the effectiveness of the fatigue
countermeasures and the majority (about three quarters of the respondents) thought they were
at least somewhat effective. See Figure 26. Energy drinks were rated as very effective by about
one third of the workers who used them. And nearly all of the supervisors and managers (6 out of
7) rated energy drinks and exercise breaks as very effective. Two supervisors/managers thought
that napping did not make any difference and two did not respond to the question on napping
effectiveness. One has to keep in mind that there was only a limited number of napping
recommendations issued and the study cannot provide a representative assessment of this
countermeasure.
25
How effective are the fatigue countermeasuresfor improving your alertness?
Perc
enta
ge o
f W
ork
ers
0
20
40
60
80
Napping
Energy Drinks
Exercise Break
makes no differencevery effective somewhat effective
0
How effective are the fatigue countermeasuresfor improving alertness?
Num
ber
of M
anagers
0
1
2
3
4
5
6
7 Napping
Energy Drinks
Exercise Break
makes no differencevery effective somewhat effective
0 0
Figure 26: Perception of effectiveness of individual fatigue countermeasures, assessed by workers
(left) and supervisors/managers (right). Percentages are based on the total number of workers who received the specific recommendations at least once.
The diverse opinions about the fatigue countermeasures were reflected in the survey comment
section (see selected quotes below). Particularly the fairly poplar energy drinks received several
comments, noting that the drinks are always available, and also indicating that there were
individual differences in the participants’ liking of the drinks. One individual cautioned against
potential overuse of the drinks.
Supervisors’/Managers’ Quotes: Exercise Breaks
“It helps to stretch when you drive a lot.”
Supervisors’/Managers’ Quotes: Energy Drinks
“Have no complaints from my … shift workers.”
“One needs to monitor members not to overuse energy drinks.”
“For some it works, others not.”
“They are always available.”
Workers’ Quotes: Energy Drinks
“This is a very interesting, helpful prism, I love it very much, but it’s the energy [drink] type that I don’t enjoy
because the drink should have a good taste as well.”
“It [PRISM] does work, but the only problem is the energy drinks, they are not making any difference.”
26
3.2.4. Countermeasure Practicability
Supervisors/managers were asked about the practicality of the individual fatigue
countermeasures in their environment. Most seemed to think that energy drinks and exercise
breaks did not pose a problem at all, while most of the respondents thought napping could be
some problem. No respondent said that any of the fatigue countermeasures were a big problem.
See Figure 27.
How practical would you say is it to use the fatigue countermeasure in your environment?
Nu
mbe
r of
Ma
na
ge
rs
0
1
2
3
4
5
6
7 Napping
Energy Drinks
Exercise Break
not sureit's abig problem
not a problemat all
could besome problem
00 0 0 0
Figure 27: Practicality of individual fatigue countermeasures, assessed by supervisors/managers
Napping elicited several verbal comments on the surveys (see selected quotes below), some
relating to nap duration and nap location, and others commenting on policy issues. Interestingly,
one manager (who was not working night shifts himself) was not supporting sleep/napping at
work at all, which underscores the complexities of implementing fatigue management and
related cultural change.
It is indicated that the implementation of the napping countermeasure may require further fine-
tuning (e.g., mitigation of sleep inertia, considerations for other locations) and a longer time to
gain more experience with more workers taking actual naps in order to better assess their
practicality and effectiveness.
Supervisors’/Managers’ Quotes: Napping
“[Policies should include] disciplining action when caught sleeping, controlled sleeping is permitted”
“Sleep required at work should result in an investigation. Bad habits at home must not be tolerated, allowing
people to "rest" at work…..You should sleep at home, work at work”
“People don't want to sleep in nap station, they prefer to have a nap in their ldv [light duty vehicle] in a safe
place.”
“People feel the nap time is too short (20min), should be around 40 min.”
27
4. RESULTS OF ALERTNESS/PERFORMANCE TESTING AND PRISM
4.1. PRISM Sensitivity to Increased Fatigue
One of the goals of this study is to investigate the relationship between the PRISM output and
test measures of alertness/fatigue and specifically PRISM’s sensitivity to impairment. In a first
step, we investigated PRISM’s sensitivity to alertness by comparing day and night shifts. As known
from the shiftwork literature and workers’ everyday experience, alertness tends to be impaired
during night shifts (e.g. see Akerstedt T (1988): Sleepiness as a consequence of shift work. Sleep 11, 11-34). This was
verified for this operation, using the data from alertness testing, and we investigated whether
this expected difference between day and night shifts would hold true in the PRISM data (see
below). The data from all subjects and shifts in a given test condition (day, night, baseline, post-
implementation) were pooled and treated as independent for statistical purposes, unless noted
differently in the text. No day-night differences were found in the Wilkinson four-choice reaction
time test. It was concluded that the variability of the data was too high to be sensitive to
alertness under the specific field study conditions (e.g., possible distractions during test, limited
practice time, etc.), and the Wilkinson data are therefore not included in this report.
The Shift Log analysis included a total of 196 logs for Baselines (104 night shifts, 92 day shifts),
and a total of 389 Shift logs for Post-Implementation (233 night shifts, 156 day shifts). Subjects
rated their overall on-shift alertness and performance on the daily Shift Performance Log which
they completed after every shift during Baseline and Post-Implementation testing. The Shift
Performance Log included a 5-point rating scale for alertness, ranging from ‘very alert’ to ‘very
sleepy’, and the results for this scale are shown in Figure 28. Day shifts and night shifts differed
(Chi Square test) during both test phases (Baseline and Post-Implementation), with a higher
percentage of shifts with ‘very alert’ ratings for day shifts. In both test phases, there were very
few ‘very sleepy’ ratings for day shifts as well as night shifts (1-3% of shifts).
Alertness During Shift (Baseline)
Perc
en
tag
e o
f S
hifts
0
20
40
60
80
Night Shifts
Day Shifts
Alertness Rating
**
veryalert
moderatelyalert
moderatelysleepy
neither alertnor sleepy
verysleepy
Alertness During Shift (Post-Implementation)
Perc
enta
ge
of
Shifts
0
20
40
60
80
Night Shifts
Day Shifts
Alertness Rating
***
veryalert
moderatelyalert
moderatelysleepy
neither alertnor sleepy
verysleepy
Figure 28: Alertness ratings (Shift Log) during night shifts and day shifts, shown separately for
Baseline (left) and Post-Implementation (right).
28
Participants also rated their ability to focus or concentrate on the daily Shift Performance Logs
using a four-point scale ranging from ‘excellent’ to ‘poor’ (see Figure 29). Similarly, night shifts
and day shifts were different with a higher percentage of shifts with ‘excellent’ ratings for day
shifts and few or no ‘poor ratings’. The daily Shift Performance Logs also indicated differences
between day and night shifts in ratings on mental exhaustion (for Baseline and Post-
implementation) and in ratings on overall performance (Post-Implementation only).
Concentration/Focus During Shift (Baseline)
Perc
en
tag
e o
f S
hifts
0
20
40
60
80
Night Shifts
Day Shifts
Concentration/Focus Rating
**
fairexcellent good poor
0 0
Concentration/Focus During Shift (Post-Implementation)
Perc
en
tag
e o
f S
hifts
0
20
40
60
80
Night Shifts
Day Shifts
Concentration/Focus Rating
***
fairexcellent good poor
Figure 29: Ability to concentrate/focus (Shift Performance Log) during night shifts and day shifts,
shown separately for Baseline (left) and Post-Implementation (right).
Perhaps the most revealing question of the daily Shift Performance Logs was asking participants
to report whether they struggled to remain awake or briefly nodded off during the shift by a
simple yes/no answer (see Figure 30).
Prevalence of Impaired Alertness:
Self-Reported Nodding Off
Perc
enta
ge o
f S
hifts
0
10
20
30
40
50
Night Shifts
Day Shifts
Baseline Post-Implementation
**
**
Figure 30: Prevalence of shifts with self-reported ‘nodding-off or struggling to remain awake’
(Shift Performance Log) during night shifts and day shifts for Baseline and Post-Implementation.
29
Alertness impairment (nodding off/struggling to remain awake) was significantly more frequent
for night shifts (29% in Baseline, 21% in Post-implementation) as compared to day shifts (10% in
Baseline and Post-Implementation).
The differences between day and night were also seen in the results of the test battery data
(Visual Analog Scales for Arousal, Mood, Motivation, Concentration and Physical Fatigue, and
Karolinska Sleepiness Scale) (see Figures 31 and 32). The analysis included a total of 157 shifts for
Baseline (83 night shifts, 74 day shifts) and a total of 305 shifts for Post-Implementation (184
night shifts, 121 day shifts).
Visual Analog Scales (Baseline)
VA
S S
cale
0
20
40
60
80
100
Night Shifts
Day Shifts
*** ***n.s.***
Arousal Mood Concen-tration
Motivation Physic.Fatigue(scale inverted)
Visual Analog Scales (Post-Implementation)
VA
S
Scale
0
20
40
60
80
100
Night Shifts
Day Shifts
*** ***
Arousal Mood Concen-tration
Motivation Physic.Fatigue(scale inverted)
**
Figure 31: Ratings of arousal, mood, motivation, concentration and physical fatigue (Visual Analog
Scales ranging from 0 to 100; inverted scale for Physical Fatigue) for night shifts and day shifts,
shown separately for Baseline (left) and Post-Implementation (right).
Karolinska Sleepiness Scale
KS
S
Sco
re
1
2
3
4
5
6
7
8
9
Night Shifts
Day Shifts
** ***
Baseline Post-Implementation
veryalert
verysleepy
Figure 32: Sleepiness scores (9-point Karolinska Sleepiness Scale) for night shifts and day shifts
during Baseline and Post-Implementation.
30
Arousal, Mood and Concentration were significantly lower, and Physical Fatigue and the
Karolinska Sleepiness Score were significantly higher on night shifts as compared to day shifts
during both Baseline and Post-Implementation. Statistically lower Motivation was found on Post-
Implementation night shifts as compared to day shifts. The results in Figures 31 and 32 are based
on data sets that, similar to the Shift Log analysis, pooled the data from all subjects and shifts in a
given test condition and treated them as independent for statistical testing.
Distribution of Arousal Data (Post-Implementation)
Visual Analog Scale Bins (Arousal)
0-9.
9
10.1
0.9
20-2
0.9
30-3
0.9
40-4
0.9
50-5
0.9
60-6
0.9
70-7
0.9
80-8
0.9
Perc
enta
ge o
f R
esponses
0
5
10
15
20
25
30
Night Shifts
Day Shifts
verysleepy
veryalert
90-1
00
Figure 33: Distribution of arousal ratings (Visual Analog Scale) for night shifts and day shifts during
Post-Implementation. Results are displayed in percentage of total responses.
Distribution of Karolinska Sleepiness Scale Data (Post-Implementation)
KSS Score
1 2 3 4 5 6 7 8 9
Pe
rce
nta
ge o
f R
esp
on
se
s
0
5
10
15
20
25
30
35
Night Shifts
Day Shifts
veryalert
verysleepy
Figure 34: Distribution of sleepiness ratings (Karolinska Sleepiness Scale) for night shifts and day
shifts during Post-Implementation. Results are displayed in percentage of total responses.
31
Figures 33 and 34 show the distribution of individual ratings for two selected parameters: arousal
(Visual Analog Scale) and sleepiness (Karolinska Sleepiness Scale). The figures illustrate the
differences between day and night shifts seen in Figures 31 and 32, but also demonstrate that the
majority of the responses were in the ‘alert’ half of the scale. The observation guided some of the
subsequent analysis to focus on impairment (e.g., shifts with nodding-off reports) and also limits
somewhat the potential for correlations between test parameters and PRISM data.
Having confirmed the expected differences between day shifts and night shifts in the test
measures, we were investigating whether these differences could be also found in the PRISM
records. Two sets of PRISM records were analyzed, one set corresponding to the shifts with
alertness testing (see Figure 35) and another, bigger data set including all shifts with PRISM
records during the timeframes of Baseline (n=357) and Post-implementation (n= 603), regardless
of presence of test data for these shifts (i.e., also including PRISM data from employees who did
not participate in alertness testing) (see Figure 36). For this bigger PRISM data set of study
participants and non-participants, day and night shifts were defined by the shift end time in the
PRISM records (5-8pm and 5-8am, respectively).
Figures 35 and 36 show the percentage of shifts and absolute shift numbers for each of the five
PRISM zones, ‘severe’ (-10 to -5), ‘high’ (-4 to -2), significant’ (-1 to +1), ‘guarded’ (+2 to +4) and
‘low’ (+5 to +10). For day shifts, PRISM Values were mostly in the ‘significant’ and ‘guarded’
zones. Data for night shifts show a shift towards ‘significant’ and ‘high’. Only two isolated records
were in the ‘severe zone’, (both at ‘-5’) and no records in the ‘low’ zone. The differences between
night shifts and day shifts were statistically significant (Chi-Square test).
PRISM Fatigue Zone
severe high signif. guarded low
Perc
enta
ge o
f S
hifts
0
10
20
30
40
50
60 Night Shifts
Day Shifts
06
47
39
0
PRISM Zone Frequency (Baseline)
2
23
59
21
0
***
includes only shifts withconcurrent alertness testing
PRISM Fatigue Zone
severe high signif. guarded low
Perc
enta
ge
of S
hifts
0
10
20
30
40
50
60 Night Shifts
Day Shifts
06
85
65
0
PRISM Zone Frequency (Post-Implementation)
1
30
139
63
0
***
includes only shifts withconcurrent alertness testing
Figure 35: Frequency of PRISM zones (severe, high, significant, guarded, low) for night shifts and
day shifts, shown separately for Baseline (left) and Post-Implementation (right). Bars show
percentage of shifts in each PRISM zone, and absolute shift numbers are indicated above bars.
Includes only PRISM records from study participants and shifts with concurrent Shift Performance
Log testing.
32
PRISM Fatigue Zone
severe high signif. guarded low
Pe
rce
nta
ge
of S
hifts
0
10
20
30
40
50
60 n=183
2
34
106
41
0
Night Shifts (Baseline)
PRISM Fatigue Zone
severe high signif. guarded low
Perc
en
tage
of S
hifts
0
10
20
30
40
50
60 n=174
0
17
7681
0
Day Shifts (Baseline)
PRISM Fatigue Zone
severe high signif. guarded low
Pe
rcen
tage o
f S
hifts
0
10
20
30
40
50
60 n=308
0
42
180
86
0
Night Shifts (Post-Implemenation)
PRISM Fatigue Zone
severe high signif. guarded low
Pe
rce
nta
ge
of
Sh
ifts
0
10
20
30
40
50
60 n=295
07
164
124
0
Day Shifts (Post-Implementation)
Figure 36: Frequency of PRISM zones (severe, high, significant, guarded, low) for night shifts (left)
and day shifts (right). Upper panel: Baseline. Lower panel: Post-Implementation. Bars show
percentage of shifts in each PRISM zone, and absolute shift numbers are indicated above bars.
Includes all PRISM records during the two study phases for both study participants and non-
participants.
The averages of the actual PRISM Values corresponding to Figures 35 and 36 are shown in Figure
37. PRISM Values for night shifts were significantly lower (indicating increased fatigue) as
compared to day shifts (Mann-Whitney test) for both Baseline and Post-Implementation data.
33
PRISM Value (Average)
PR
ISM
Valu
e
-1
0
1
2
3
Night Shifts
Day Shifts
Baseline(n=197)
Post-Implementation(n=389)
*** ***
includes only shifts withconcurrent alertness testing
PR
ISM
Va
lue
-1
0
1
2
3
Night Shifts
Day Shifts
Baseline(n=357)
Post-Implementation(n=603)
All Shifts(n=1780)
*** *** ***
PRISM Value (Average)
Figure 37: Average PRISM values for night shifts and day shifts during Baseline and Post-
Implementation. Lower PRISM values indicate lower alertness. Left panel: Includes only PRISM
records from study participants and shifts with concurrent alertness testing (Shift Performance
Log). Right panel: Includes all PRISM records for both study participants and non-participants
during Baseline, Post-Implementation, as well as for the entire PRISM recording time (Baseline,
Post-Implementation, and transition time).
As noted, the above statistical results for test parameters and PRISM Values are based on the
assumption of independency of the pooled data. To verify that the results were not due to
imbalances in the number of shifts contributed by each participant, the analysis was repeated
based on within-subject means across all shifts in a given test condition (day, night, Baseline,
Post-Implementation) of a given participant. This way each subject contributed only one value
per test parameter and test condition, and paired means from subjects who contributed data to
both conditions for the comparison were analyzed by tests for repeated measures (e.g., paired t-
test). Although this dramatically reduces the sample size for the statistical comparisons (10
subjects; subjects with only one or two shifts per test condition were excluded), and while
eliminating intra-individual variability between shifts, significant day-night differences were still
clearly seen in the PRISM data, Karolinska Sleepiness Scale and in most of the VAS measures.
The aim of ‘fitness-for-duty’ testing is to detect impairment. PRISM values in the ‘high’ and
‘severe’ zones were considered to indicate impairment. Figure 38 shows that impairment was
clearly more frequent during night shifts in both Baseline (over 20%) and Post-Implementation
(over 10%).
34
Prevalence of Impairment:
Percentage of Shifts with High and Severe PRISM Values
0
10
20
30
40
50
Night Shifts
Day Shifts
Baseline(n=197)
Post-Implementation(n=389)
Pe
rce
nta
ge
of
Sh
ifts
includes only shifts with concurrent alertness testing
Prevalence of Impairment:
Percentage of Shifts with High and Severe PRISM Values
Pe
rce
nta
ge
of
Sh
ifts
0
10
20
30
40
50
Night Shifts
Day Shifts
Baseline(n=357)
Post-Implementation(n=603)
All Shifts(n=1780)
Figure 38: Frequency of impaired PRISM values (PRISM zones ‘High’ and ‘Severe’) for night shifts
and day shifts during Baseline and Post-Implementation. Left panel: Includes only PRISM records
from study participants and shifts with concurrent Shift Performance Log testing. Right panel:
Includes all PRISM records for both study participants and non-participants during Baseline, Post-
Implementation, as well as for the entire PRISM recording time (Baseline, Post-Implementation,
and transition time).
Shifts with low (fatigue-indicating) PRISM Values (PRISM zones ‘severe’ and ‘’high’; n=68) were
associated with more frequent reports of ‘nodding-off/struggling to remain awake’ on the Shift
Logs (32% of the shifts) as compared to shifts with high PRISM Values (PRISM zones ‘low’ and
‘’guarded’; n=188) (13% of the shifts) (Figure 39). For this analysis, the data from all day and night
shifts, from both Baseline and Post-Implementation were combined.
Prevalence of Impaired Alertness:
Self-Reported Nodding Off for Low and High PRISM Values
Pe
rcen
tag
e o
f S
hifts
0
10
20
30
40
50
PRISM Value '+2' and Higher (high)
PRISM Value '-2' and Lower (low)
Figure 39: Prevalence of shifts with self-reported ‘nodding-off/struggling to remain awake’ (Shift
Log). Comparison between shifts with medium/high PRISM Values (PRISM zones ‘low’, ‘guarded’
and ‘significant’ – black bar) and low PRISM Values (PRISM zones ‘severe’/‘high’ - red bar). Data
from both shift types (day, night) and both study phases (Baseline, Post-Implementation) were
combined.
35
When using Shift Log data to define impairment (questions on ‘nodding-off/struggling to remain
awake’), it was found that increased fatigue in the Shift Log data corresponded to lower PRISM
Values (see Figure 40, left panel). The analysis also focused on certain ‘high-risk’ subjects who had
more incidences of impaired alertness (e.g., more shifts with reports of ‘nodding-off/struggling to
remain awake’) than their fellow-workers. For example, one subject who had three shifts (out of
28 shifts) with reported ‘nodding-off/struggling to remain awake’ showed lower (impaired) mean
PRISM Values (mean=-1.33) for these shifts as compared to the mean PRISM Values for the shifts
without these reports (mean=+0.6). The example of another subject who reported reduced
alertness on the Shift Performance Log (choosing response options ‘moderately sleepy’ and ‘very
sleepy’) more frequently (5 shifts out of 32) than most co-workers clearly showed a reduced
mean PRISM Value (mean=-2.4, ranging from -5 to -1) for these five shifts (see Figure 40, right
panel).
Average PRISM Value
for Shifts With and Without Reported Nodding-Off
PR
ISM
Va
lue
-1
0
1
2
3
Shifts Without Reported Nodding-Off
Shifts With Reported Nodding-Off
Baseline Post-Implementation
* *
PR
ISM
Valu
e
-3
-2
-1
0
1
2
3
Average PRISM Value By Alertness Level During Shift
(Selected 'High Risk' Subject)
Neither AlertNor Sleepy
(n=11)
Moderately&VerySleepy
(n=5)
Very&ModeratelyAlert(n=18)
Figure 40: Average PRISM values for different alertness levels (Shift Performance Log) during
Baseline and Post- Implementation. Lower PRISM values indicate lower alertness. Left panel:
Comparison of shifts with self-reported ‘nodding-off/struggling to remain awake’ (red bars) and
without self-reported nodding-off. Right panel: Comparison of shifts with different alertness
ratings (Shift Log) for one selected ‘high risk’ subject with increased occurrence of shifts with
reduced alertness.
Similarly to the Shift Performance Log results shown in Figure 39, the test battery data showed
differences between shifts with low and high PRISM Values (Figure 41). Arousal, Motivation and
Concentration were lower and Physical Fatigue and Karolinska Sleepiness score were higher
during shifts with low (fatigue-indicating) PRISM Values (PRISM zones ‘severe’ and ‘high’; n=26)
than during shift with high PRISM Values (PRISM zones ‘low’ and ‘guarded’; n=138). As with the
related analysis of the Shift Performance Logs, the data from all day and night shifts from both
Baseline and Post-Implementation were combined.
36
Visual Analog Scales for High and Low PRISM Values
VA
S
Scale
0
20
40
60
80
100
PRISM Value: '+2' and Higher (High)
PRISM Value: '-2' and Lower (Low)
Arousal Mood Concen-tration
Motivation Physic.Fatigue(scale inverted)
n.s.* * ** ***
Karolinska Sleepiness Scale for High and Low PRISM Values
KS
S S
core
1
2
3
4
5
6
7
8
9
PRISM Value: '+2' and Higher (High)
PRISM Value: '-2' and Lower (Low)
***
Figure 41: Ratings of arousal, mood, motivation, concentration and physical fatigue (Visual Analog
Scales ranging from 0 to 100; inverted scale for Physical Fatigue) (left) and sleepiness ratings (9-
point Karolinska Sleepiness Scale). Comparison between shifts with high PRISM Values (zones
‘low’ and ‘guarded’ – black bars) and low PRISM Values (zones ‘severe’/‘high’ - red bars).
To further investigate the relationship between PRISM Values and test parameters, within-
subject correlations were performed for subjects who provided a reasonably large number of
data pairs of PRISM data and experimental data (Shift Performance Log data for alertness,
concentration/focus, performance, ‘nodding-off/struggling to remain awake’; VAS data for
Arousal, Mood, Motivation, Concentration and Physical Fatigue and Karolinska Sleepiness score).
Correlation between Alertness (Shift Log) and PRISM Value
Alertness Rating
PR
ISM
Va
lue
-3
-2
-1
0
1
2
3
correlation coefficient-0.351
verysleepy
veryalert
neither alertnor sleepy
example participant #1 (post-implementation data)
moderatelysleepy
moderatelyalert
Correlation between KSS Score (Sleepiness) and PRISM Value
KSS Score
123456789
PR
ISM
Va
lue
-3
-2
-1
0
1
2
3
correlation coefficient-0.641
verysleepy
veryalert
neither alertnor sleepy
example participant #1 (post-implementation data)
Figure 42: Intra-individual correlations between test measures and PRISM values for one selected
individual (Post-Implementations data;). Left: Alertness ratings (Shift Log); n=29. Right: Sleepiness
scores (9-point Karolinska Sleepiness Scale); n=28. Regression lines are shown in blue.
37
These intra-individual correlations were generally relatively low. The low intra-individual
correlations are not very surprising given the nature of the data sets with only few data points
indicating more extreme fatigue. For illustration, Figures 42 and 43 show the correlations for two
selected subjects who had higher correlations than most other participants.
Correlation between Arousal (VAS) and PRISM Value
Arousal Level
0 20 40 60 80 100
PR
ISM
Valu
e
-3
-2
-1
0
1
2
3
4
correlation coefficient0.449
verysleepy
veryalert
example participant #2 (post-implementation data)
Correlation between Mood (VAS) and PRISM Value
Mood Level
0 20 40 60 80 100P
RIS
M V
alu
e
-3
-2
-1
0
1
2
3
4
correlation coefficient0.459
verybad
mood
verygoodmood
example participant #2 (post-implementation data)
Correlation between Motivation (VAS) and PRISM Value
Motivation Level
0 20 40 60 80 100
PR
ISM
Va
lue
-3
-2
-1
0
1
2
3
4
correlation coefficient0.585
notmotivated
at all
verymotivated
example participant #2 (post-implementation data)
Figure 43: Intra-individual correlations between Visual Analog Scale ratings and PRISM values for
one selected individual (Post-Implementations data, n=24). Top left: Arousal. Top right: Mood.
Bottom: Motivation. Regression lines are shown in blue.
38
Overall, the results indicate that PRISM Values and alertness test data show comparable trends
on a group level and demonstrate PRISM’s relevance for detecting impairment. On an individual
level, correlations between PRISM and experimental alertness data were relatively low.
Refinements of the PRISM fatigue management system (e.g., inclusion of prior day’s sleep as
input parameter for risk calculation and added option for cognitive impairment testing) are
aiming to improve PRISM’s sensitivity to impaired alertness, in particular on the individual level.
For example, a new PRISM module was recently incorporated into the PRISM system which takes
into account workers’ prior sleep (entered into the PRISM interface at log-in) in addition to work
hours when computing PRISM values.
4.2. PRISM System Effectiveness
4.2.1. Comparison of Data from Baseline and Post-Implementation
The results from the worker and supervisor surveys, reported in Chapter 3, illustrated several
benefits of PRISM, such as increased awareness of job safety and performance and increased
ability to manage fatigue levels at work. We also compared the data from Baseline and Post-
Implementation testing (Shift Logs, Visual Analog Scales, Karolinska Sleepiness Scale).
Arousal, Mood, Concentration and Physical Fatigue (from the Visual Analog Scales) showed
statistically significant improvements on Post-Implementation night shifts as compared to
Baseline night shifts (Figure 44, left panel). On day shifts (Figure 44, right panel), Mood improved
and Motivation showed a trend towards improvement (significance level just under 0.05,
indicated by ‘(*)’). Sleepiness rated on the Karolinska Sleepiness Scale improved significantly for
both night shifts and day shifts (Figure 45). It should be noted that the improvements were seen
even though participants generally rated their baseline alertness levels - even on night shifts - in
the mid-range of the scales or slightly better. As with the day-night comparisons, the comparison
of Baseline and Post-Implementation was also run using paired comparisons of the subjects who
participated in both test phases (based on within-subject averages across all shifts of a given
subject and test condition). These paired tests, however, did not result in significant differences
between the two test phases, and the relatively low sample size did not grant sufficient statistical
power.
A non-significant trend towards improvement during Post-Implementation night shifts was also
seen in the Shift Performance Log reports of ‘nodding-off/struggling to remain awake’ for night
shifts (Figure 46), with the percentage of night shifts with self-reported ‘nodding-off/struggling to
remain awake’ decreasing from 28% to 21%. Reports of errors and mistakes on the Shift
Performance Log for day shifts also decreased, although this should be interpreted cautiously as
errors/mistakes were disproportionally high during Baseline day shifts. Means of the other Shift
Performance Log measures did not demonstrate differences between Baseline and Post-
Implementation, due to relatively high data variability, fewer response choice options than on the
39
test battery scales (e.g., 5-point alertness scales on Shift Log vs. 9-point scale on Karolinska
Sleepiness Scale) and relatively small differences in the group averages (differences between
Baseline and Post-Implementation were smaller than the differences between night and day
shifts).
PRISM Values were not expected to be significantly different in Baseline and Post-
Implementation (see Figure 47) as the PRISM algorithms used for the study were based on
schedule characteristics only (algorithms did not take into account sleep times or potential
alertness improvements resulting from fatigue countermeasure use).
Visual Analog Scales (Night Shifts)
VA
S
Scale
0
20
40
60
80
100
Baseline
Post-Implementation
** ***n.s.*
Arousal Mood Concen-tration
Motivation Physic.Fatigue(scale inverted)
Visual Analog Scales (Day Shifts)
VA
S S
cale
0
20
40
60
80
100
Baseline
Post-Implementation
Arousal Mood Concen-tration
Motivation Physic.Fatigue(scale inverted)
n.s.n.s.
*n.s. (*)
Figure 44: Ratings of arousal, mood, motivation, concentration and physical fatigue (Visual Analog
Scales ranging from 0 to 100; inverted scale for Physical Fatigue) for Baseline and Post-
Implementation, shown separately for night shifts (left) and day shifts (right).
Karolinska Sleepiness Scale
KS
S
Score
1
2
3
4
5
6
7
8
9
Baseline
Post-Implementation
*** ***
Night Shift Day Shift
veryalert
verysleepy
Figure 45: : Sleepiness scores (9-point Karolinska Sleepiness Scale for Baseline and Post-
Implementation, shown separately for night shifts (left) and day shifts (right).
40
Prevalence of Impaired Alertness:
Self-Reported Nodding Off
Pe
rce
nta
ge
of S
hifts
0
10
20
30
40
50
Baseline
Post-Implementation
Night Shifts Day Shifts
n.s.
n.s.
Figure 46: Frequency of shifts with impaired alertness (self-reported ‘nodding-off/struggling to
remain awake’ - Shift Performance Log) during Baseline and Post-Implementation, for night shifts
and day shifts.
PRISM Value (Average)
PR
ISM
Va
lue
-1
0
1
2
3
Baseline
Post-Implementation
Night Shifts Day Shifts
n.s.
n.s.
Figure 47: Average PRISM values during Baseline and Post-Implementation, for night shifts and
day shifts. Includes all PRISM records for both study participants and non-participants during
Baseline and Post-Implementation.
4.2.2. Countermeasure Usage and Compliance
Countermeasure compliance was assessed by worker surveys (see Chapter 3) and on Shift
Performance Logs. Energy drinks was the most frequently used fatigue countermeasure and
surveys showed that this fatigue countermeasure had the highest compliance. The Shift
performance Logs showed that energy drinks were used during about 30% of the night shifts
and close to 10% of the day shifts (see Figure 48, left panel). In addition to energy drinks,
workers consumed other caffeinated beverages that were not related to the PRISM
countermeasure (see Figure 48, right panel). Similarly, this occurred more during night shifts
41
(100% of Post-Implementation night shifts) than during day shifts (about 50% of Post-
Implementation day shifts).
Interestingly, the consumption of energy drinks was not always related to actual
recommendations by the PRISM system (see Table 3). In many cases, workers used energy
drinks even though PRISM did not indicate low enough values to trigger countermeasure
assignment, and in other cases energy drinks were assigned, but not consumed, partly due to
temporary unavailability of the drinks. As other fatigue countermeasures (napping, light
station) were rarely assigned and used, the Shift Performance Logs did not provide enough
information on compliance for these fatigue countermeasures.
Overall, the data on frequency of countermeasure assignment/use suggest that the potential
for improvements was somewhat limited, despite which we were able to detect significant
positive effects of The PRISM fatigue management system during Post-Implementation.
Energy Drink Consumption
Pe
rce
nta
ge
of
Shifts
0
10
20
30
40
50
Night Shifts
Day Shifts
Caffeine Consumption (Post-Implemenation)
Perc
enta
ge
of
Shifts
0
20
40
60
80
Night Shifts
Day Shifts
0 21 3 4 5+
Number of Caffeinated Beverages
0
Figure 48: Consumption of energy drinks (left panel) and other caffeinated beverages on night
shifts and day shifts during Post-Implementation.
Energy Drink Consumption (Number of Night Shifts)
Yes No Energy Drink Consumption (Number of Day Shifts)
Yes No
Assigned by PRISM
59
73
Assigned by PRISM
11
65
Not assigned by PRISM
12
89
Not assigned by PRISM
2
78
Table 3: Energy drink consumption and PRISM countermeasure assignment
42
5. SUMMARY
PRISM - Predictive Risk Intelligent Safety Module - links human fatigue risk prediction software
and validated alertness technologies to reduce schedule-specific risk. The system is interfaced
with common Time & Attendance systems to predict fatigue risk in real time to provide
practical, schedule-specific fatigue mitigation recommendations.
This study evaluates the PRISM fatigue management system in an around-the-clock operation
at the KIO Kolomela mine in South Africa. PRISM monitored workers’ fatigue levels and sent
automatic SMS notifications to workers and supervisors when PRISM’s Fatigue Risk Index
exceeded certain thresholds. The notifications included one of three sets of fatigue
countermeasure recommendations, depending on the fatigue severity level. The primary
fatigue countermeasures were energy drinks, exercise breaks and napping. All participants
attended a fatigue countermeasure training session.
Data was collected from participating workers for one cycle of the roster schedule of 28 days,
working both sets of multiple 12 hour day shifts and multiple 12 hour night shift blocks, before
and after PRISM implementation (summer and fall 2011, respectively). Two types of surveys
were administered. Fatigue and Health Surveys were administered to workers and also to
supervisors and managers at the end of Baseline and Post-Implementation study phases, as
well as during an early baseline one year before the Post-Implementation study phase. At the
end of the study, PRISM Evaluation Surveys were completed by the workers and
supervisors/managers. On the test days, data were collected on workers’ activity patterns
(hours of sleep, wake, work and commuting to/from work), volunteers completed an alertness
test battery before and after each shift (various subjective alertness scales and a reaction time
test) and a daily Shift Performance Log at the end of each work shift.
Survey results: The survey results demonstrate many positive aspects of the PRISM fatigue
monitoring systems. General benefits, acceptability and operational feasibility/practicability
were rated favorably by the majority of the workers and supervisors/managers. For example:
All of the supervisors/managers and most of the workers (84%) agreed that the system
increases awareness of job safety and performance (with most of the remaining workers
not being sure).
Similarly, all of the supervisors/managers and most of the workers said that PRISM gives
them the ability to manage their employees’ (or their own) fatigue levels at work,
Workers thought that PRISM would help management understand workers better,
improve working conditions and potentially encourage other actions by employers.
Statistical comparisons between the early baseline and post-implementation data
revealed that the percentage of workers being confident in their own and in their
43
managers’ understanding of fatigue levels increased significantly after PRISM
implementation.
Workers also said that they would feel better about their work environment when
knowing that all employees around them were monitored for alertness/fatigue.
While the general acceptance of the PRISM fatigue monitoring system by workers and
supervisors/managers was good, about one third of the workers were somewhat concerned
about privacy.
About two thirds of the workers thought the PRISM system was very or somewhat sensitive to
reduced alertness (with most of the remaining respondents not being sure).
Results of on-shift alertness/performance testing and PRISM: The data from on-shift testing
(Visual Analog Scales on Arousal, Mood, Motivation, Physical Fatigue; Karolinska Sleepiness Scale;
Shift Performance Log) was used to investigate PRISM’s sensitivity to decreased alertness and
impairment and its effectiveness for fatigue mitigation. Overall, the results indicate that PRISM
output and test data show comparable trends on a group level and the data demonstrate PRISM’s
relevance for detecting impairment. On an individual level, correlations between PRISM output
and test data were relatively low. Refinements of the PRISM fatigue management system (e.g.,
inclusion of prior day’s sleep as input parameter for risk calculation and added option for
cognitive impairment testing) are aiming to improve PRISM’s sensitivity to impaired alertness, in
particular on the individual level. Test data demonstrate improved alertness after PRISM
implementation. Some of the results from the statistical analysis include:
Average alertness levels were significantly lower during night shifts as compared to day
shifts (as seen in the data from the Visual Analog Scales, Karolinska sleepiness Scale and
Shift Performance Log), and this day-night difference was also reflected in the PRSIM data,
with significantly lower PRISM values on night shifts.
Focusing on impairment, it was found that low PRISM values (PRISM zones ‘severe’ and
‘high’) were associated with a higher frequency of self-reported ‘nodding off / struggling
to remain awake’ (Shift Log data) and with significant changes in Visual Analog Scale data
(reduced Arousal, Motivation, Concentration and increased Physical Fatigue) and
significantly increased score on the Karolinska Sleepiness Scale as compared to higher
PRISM values. Similarly, shifts on which participants reported ‘nodding off / struggling to
remain awake’ had significantly lower PRISM values than shifts without such reports.
When comparing Post-Implementation and Baseline, most of the test parameters for
night shifts (Arousal, Mood, Motivation, Concentration, Physical Fatigue and Karolinska
Sleepiness score) and some test parameters for day shifts showed significant
improvements after the implementation of the PRISM system.
44
Countermeasure Use: Energy drinks were used most frequently, with about two thirds of the
workers stating that they had (at least) one energy drink on almost every night shift, and about
one third having one during almost every day shift (survey results). The drinks were used during
about 30% of the night shifts and close to 10% during day shifts (Shift Log results). Compliance
with PRISM’s fatigue countermeasure recommendations was reasonable but not optimal, and
energy drinks had the highest compliance. The majority of the survey respondents thought that
the fatigue countermeasures were at least somewhat effective, and energy drinks were rated
as very effective by about one third of the workers who used them. Nearly all of the supervisors
and managers rated energy drinks and exercise breaks as very effective.
The assessment of the napping and the bright light countermeasures are somewhat limited
because of the relatively low number of issued recommendations for napping and bright light
station. A longer implementation time to gain specific experience with more workers taking
naps, and further fine-tuning of the specific napping recommendations (e.g., napping duration,
mitigation of sleep inertia, considerations for other locations) are recommended to better
assess practicality and effectiveness of napping. The data on frequency of countermeasure
assignment versus actual countermeasure use suggest that there is a potential for improving
compliance. However, the evaluation results showed significant positive effects of the PRISM
fatigue management during Post-Implementation.
The success of fatigue monitoring depends on the readiness of workers, supervisors and
managers to change safety culture and a willingness to accept this concept and new
technologies. Participants in this implementation trial thought a fatigue monitoring system
would clearly help increase fatigue awareness and understanding, and specifically, the PRISM
fatigue management system was beneficial, practical and acceptable in managing fatigue in
this work environment. Test data illustrated that PRISM does correlate with alertness on a
group level, and data showed a clear trend in reducing overall fatigue in participating
workers after PRISM implementation. Refinements of the PRISM fatigue management system
(e.g., inclusion of prior day’s sleep as input parameter for risk calculation and added option
for cognitive impairment testing) are expected to improve its sensitivity on an individual level.
Improvements in execution to achieve higher countermeasure compliance should further
enhance its positive effects for fatigue mitigation.
45
APPENDIX
Worker Fatigue and Health Survey
Post-Implementation Results
(Response counts for all survey questions)
46
SECTION I: CONFIDENTIAL GENERAL INFORMATION
1. How old are you?
Under 20 years old 20 – 29 years old 30 – 39 years 40 – 49 years 50 years or older
0 12 9 4 0
2. What is your gender?
Female Male
0 25
SECTION II: FATIGUE AND HEALTH ISSUES
3. Do you do exercises regularly (i.e., such as brisk walking, jogging, biking, play soccer)?
Do not have a regular exercise
schedule
Exercise, but not on a schedule
(once per week or less)
Exercise regularly
(2-3 times per week)
10 11 4
4. I am told I snore loudly or I awake suddenly gasping for breath while I am sleeping
Never Rarely Sometimes Usually Always
10 2 9 3 1
5. Are you currently under doctor’s care for any medical problem or on any
medication program under doctor’s care?
Yes No
2 23
How often did you experience these problems in the past 6 months:
Almost Never
Quite Seldom
Quite Often
Almost Always
6. Heartburn
7. Digestion problems
8. Irregular heartbeat
13
16
19
5
7
6
7
2
0
0
0
0
47
9. Do you think that eating the proper foods at correct times can help you feel better during
shiftwork?
I don’t know how proper foods and
timing can impact me during
shiftwork
I know about proper foods and timing
but I don’t apply during for my
shiftwork
I know about proper foods and
timing and I apply them during for
my shiftwork
5 16 4
10. Do you think that understanding your fatigue level can help you improve your overall
health?
Yes No Uncertain
22 1 2
11. How many hours of overtime do you typically work in a given week?
How much you have experienced any of the following problems:
Chronic
Problem
Frequently a
Problem
Sometimes a
Problem
Rarely a
Problem
Never a
Problem
12. High cholesterol
13. Diabetes
14. Headaches
15. Trouble Sleeping
16. Eye Soreness
17. Fatigue
18. Stiffness, aches or pain
3
3
3
2
2
3
2
1
0
2
1
5
3
4
2
1
13
7
4
18
11
0
2
4
5
4
0
5
19
19
3
10
10
1
3
0 hours 1-4 hours
5-8 hours
9-10 hours
More than 10 hours
17 5 1 1 0
48
SECTION III: SLEEP, ALERTNESS, AND SAFETY ISSUES
Which sleep length best describes your current situation (after implementation of the PRISM
system):
Does not
Apply
5 Hours 6 Hours
7 Hours
8 Hours or
More
19. How many hours of sleep per
day (usually) do you feel you need
to be alert and well rested?
20. How many hours of sleep per
day are you actually getting,
usually, when you work the day
shift?
21. How many hours of sleep per
day are you actually getting,
usually, when you work the night
shift?
22. How many hours of sleep per
day are you actually getting,
usually, during your days off?
2
4
6
2
2
8
11
2
4
4
5
2
8
6
2
7
9
3
0
12
How would you rate the quality of sleep that you are getting:
Excellent Good
Average
Below Average Poor
23. on holiday or
days off?
24.during nighttime
when working the
day shift?
25. during daytime
when working the
night shift?
10
2
0
11
12
7
3
7
8
0
2
6
0
1
4
26. When you are on holiday or on the long weekend break, do you naturally rise early and feel
best (i.e. alert, energetic) in the morning, or do you like to sleep in late?
Rise early
(i.e., 5-7 am)
Rise late
(9-11 am)
Rise somewhere in-between (after 7
am, but before 9 am)
9 9 7
49
How often did each of the following events occur?:
Several
Times per
Shift
Several
Times per
Week
Several
Times per
Month
Several
Times per
Year
Seldom, if
Ever
27. How often do you find
yourself fighting sleep or briefly
nodding-off while working?
28. How often do you find
yourself fighting sleep or briefly
nodding off during breaks?
29. How often do you find
yourself fighting sleep or briefly
nodding-off while driving to and
from work?
30. How often do you make
mistakes or errors due to not
paying attention while working
on your current shift?
31. How often do you experience
muscle pain or discomfort while working on your current
schedule?
32. How often do you feel
fatigued, drowsy or sluggish while working on your current
schedule?
33. How often do you feel your
alertness is too low where you
are not effective while working
shifts?
3
2
4
0
1
2
2
9
8
6
2
3
4
3
8
7
5
5
5
9
16
1
4
1
6
7
7
6
4
4
9
11
9
3
1
34. During your last shift cycle, what was the longest number of consecutive hours you went
without sleep?
1-17 18-20 21-23 24-26 27 or more
14 7 1 2 1
35. How often do you take naps during your waking hours off-the-job?
Every Day Several Times per
Week
Once per Week
Seldom take naps
off-the-job
Never take Naps
5 6 4 7 3
50
36. How often do you take unscheduled naps or fall asleep while on the job?
(unscheduled naps do not include any naps recommended by the PRISM system)
Every Day Several Times per
Week
Once per Week
Seldom take naps
off-the-job
Never take Naps
1 3 3 8 9
37. How well adjusted are you to your current shift schedule?
Poorly adjusted
(having lots of problems)
Getting by Slightly adjusted
Well adjusted Very well adjusted
(having no problems)
1 1 6 11 6
38. How do you typically prepare for the first night shift?
Stay up the night
before and sleep
most of the day
Rise at normal time
that day, stay up all
day, then come to work
Stay up all day, but
take a nap prior to
coming to work
Stay up late and
sleep in late for the
previous two days
Other, please
explain in the
comments section
3 6 14 2 0
39. Do you feel that some type of training on how to better adjust to a shiftwork lifestyle
would make it easier to cope with the special challenges of shiftwork schedules?
Yes No
21 4
How many times has each of the following situations occurred:
0 1-2 3-4
5-6
7 or more
40. Have you had any car accidents or
near accidents in the past 3 months?
41. How many accidents or injuries
have you had on the job in the past 3
months?
42. How many near accidents or
injuries have you had on the job in the
past 3 months?
43. Altogether, how many lost days
have you had in the past 6 months as
a result of any accidents or injuries?
20
21
20
24
4
4
4
0
0
0
0
0
1
0
1
1
0
0
0
0
51
44. Where any of these accidents, injuries, or near misses due to fatigue or lack of
alertness?
Yes No Not Sure Does not Apply
3 7 2 13
SECTION IV: FATIGUE MANAGEMENT STATUS
How do you feel about your job:
Very High High Moderate
Low
Very Low
45. How mentally demanding is your job?
46. How physically demanding is your job?
47. How monotonous or boring is your job?
48. How fatiguing is your job?
9
11
1
3
12
6
0
11
3
6
4
4
1
2
7
6
0
0
11
1
49. Do you think you have understanding of your fatigue levels and have the ability to
manage it properly while on your shift?
Yes No Not Sure
20 0 5
50. Do you think that your manager has an understanding of fatigue levels of your job and
provides the ability to manage it properly while on your shift?
Yes No Not Sure
19 1 5
51. Do you think that a fatigue monitoring system increases awareness of job safety and
performance?
Yes No Not Sure
20 0 5
52. Do you think that a fatigue monitoring system gives you the ability to manage your
fatigue level while on your shift?
Yes No Not Sure
22 0 3