IEEE/HPRCT 2007

Assumptions – and Their Influence on Errors and Events

John C. SummersFleet Human Performance Manager

2

An Example of How a SimpleAssumption Leads to an Error

3

How Individual Performance can be Affected by Assumptions:

•Skill Based•Rule Based•Knowledge Based

How Organizational Behaviors Influence Assumptions

•Success (Production Behaviors)•Avoiding Failure (Prevention Behaviors)

Key Points

4

Individual-Oriented

Generic Error Modeling System(GEMS) Three Modes• Skill-Based• Rule-Based• Knowledge-Based

Human Error

5

Generic Error Modeling System(GEMS)

Familiarity (w/ task)Low High

High

Low

Atte

ntio

n(to

task

)

SB

KB

RBIf - Then

Auto

Patterns

Inattention

Misinterpretation

InaccurateMental Picture

Δ

?

Familiarity (w/ task)Low High

High

Low

Atte

ntio

n(to

task

)

SB

KB

RBIf - Then

Auto

Patterns

Inattention

Misinterpretation

InaccurateMental Picture

Familiarity (w/ task)Low High

High

Low

Atte

ntio

n(to

task

)

SB

KB

RBIf - Then

Auto

Patterns

Inattention

Misinterpretation

InaccurateMental Picture

Familiarity (w/ task)Low High

High

Low

Atte

ntio

n(to

task

)

Familiarity (w/ task)Low HighFamiliarity (w/ task)Low HighLow High

High

Low

Atte

ntio

n(to

task

)

High

Low

High

Low

Atte

ntio

n(to

task

)

SB

KB

RBIf - Then

Auto

Patterns

Inattention

Misinterpretation

InaccurateMental Picture

SB

KB

RBIf - Then

Auto

Patterns

SB

KB

RBIf - Then

Auto

Patterns

Inattention

Misinterpretation

InaccurateMental Picture

6

Skill-based BehaviorBehavior is associated with highly practiced actions in a familiar situation usually executed from memory without significant conscious thoughtSkill-based activities examples-driving a vehicle, sports (golfing, biking,skiing, bowling), typing, opening or closing your front door.

Error mode for skill-based performance is inattention involving slips and lapses in attention or concentration

•The chance for error is typically less than 1 in 10,000.

•25 percent of all errors are attributable to skill-based errors

•The greater the familiarity the less likely perceived risk will match actual risk.

Roughly 90 percent of a person's daily activities are spent in the skill-based performance mode

7

Skill-Based error mode:

“Expert Knowledge of Task”

Errors committed while performing common activities in routine situations. “Repetitive behavior performed many times”“Lights are on, but nobody is home!”

Human Error

Tools: Self Check, Peer Check

8

Done thousands of times….

9

Rule-based BehaviorBehavior based on selection of stored rules in an IF (symptom X) THEN (Situation Y) logic recognitionExamples include deciding whether to change your car’s oil, responding to a smoke alarm, stopping for gasoline, choosing answers for Dr. Anderson’s exams, doing homework.Prevalent error mode is misinterpretation. Errors examples: deviating from an approved method, applying the wrong response to a work situation, or applying the correct action but to the wrong situation.The chance for error increases when people make choices or decisionsRule-based and knowledge-based performance modes involve making choices. The chance for error increases to roughly 1 in 1,000. In terms of reliability, this is still very good (99.9 percent). 60 % error contribution

10

Rule-Based error mode“Instructions for Task”

Errors committed due to misapplication of stored rules accumulated through experience and training. “Intentional or non-intentional misapplication of a rule”“If this is true, then I should do this.”

Human Error

Tool: Procedure Use and Adherence

11

Watch the lady with the small dog….

12

Knowledge-based behaviorBehavior in response to a totally unfamiliar situation (no skill, rule or pattern recognizable to the individual)Examples-taking a pop quiz, Being downtown Cleveland at night in a part of town you have not visited before, first day on campus to attend class, performing a Boolean logic search using the “Google” search engine.

Error mode for knowledge-based activities require diagnosis and problem-solving. Most decisions are made with limited information and assumptions. Consequently, the prevalent error mode is an inaccurate mental model of the system, process, or status. The mental picture people use to solve a problem may be based on insufficient information. Under such circumstances, the chance for error is particularly high, approximately one in two (50 percent).

In the nuclear industry, studies have shown that roughly 15 percent of all errors are knowledge-based

13

Errors committed due to misapplication of stored knowledge (no skill, rule, or pattern recognizable to the individual). “You don’t know what you don’t know”

Knowledge-Based error mode:

“Knowledge of System”

Human Error

This person would have done this, Even if their life depended on it.

Tool: Questioning Attitude

14

Does he have the Knowledge?

15

Besides assumptions, why worry?

ν First – Understanding which mode helps to verify adequate defenses are in place (i.e. use self check/peer check for Skill-based error potential)

ν Second – much more important for reducing the potential for error and events – people change modes, often not realizing it. More specifically, unconsciously changing from rule-based (procedures) to knowledge-based work practices. How to identify?

IEEE/HPRCT 2007

I’ve done th

is hundreds o

f times…

.

I know this is in the turbine building but...

I was doing this while you were still in grade

school...

“Faint Signals”

Most mechanics could do this in their sleep..

I know I’m qualified but...

Just give me the work package and I’ll let you know when I’m done...

I have not done this in several years but..

How long will this briefing take, I have lunch to worry about!

17

GetFacts!

18

Assumptions tend to occur more often, when people experience uncertainty. Assumptions can also be an outfall of unsafe attitudes and inaccurate mental models. Since assumptions are often treated as fact, challenging assumptions is important in improving mental models, solving problems, and optimizing team performance.

Add to this the following:People judge risk poorly, typically underestimating it.People are less afraid of risks or situations when…What feels safe may, in fact, be dangerous.

Expect success but anticipate failure!How? - People should maintain a “healthy uneasiness”during any activity

IEEE/HPRCT 2007

Organizational Influences

20

A Challenging Situation?

21

How Organizational Behaviors Influence Assumptions

ν Achieving Success (Production Behaviors)

ν Avoiding Failure (Prevention Behaviors)

22

Prevention

Production

Perfect Organization

J. Reason. Human Error. 1990

23

A Couple Quotes…..

ν We have all the time we need to do the task correctly…….

We just don’t have any time to WASTE!Mark Bezilla VP FENOC

ν A Showman takes something easy and makes it look hard…..

A Professional takes something hard and makes it look easy.

H. Ray Caldwell, Ohio Edison

24

Real World – Doing Well

Prevention

Production

25

Recognition & its Influence on Production

Prevention

Production

“Firefighters”&

“Heroes”(Showmen?)

26

Recognition & its Influence on Prevention

Prevention

Production

“Professionals”

IEEE/HPRCT 2007

Uncertain/

Unsure

Healthy Uneasiness/WarinessToo Certain/

Too Sure

Questioning Attitude Meter

28

One of six tools in use atFirst Energy Nuclear Operating CompanyQuestioning Attitude - Stop if unsureThis is at best a partial tool

unless acted upon!

29

Common Assumption-related Attributes From Major Events

(TMI-2, Chernobyl, Challenger, and Bhopal)

•Mindset•Safety Analysis and responses•Lessons learned disregarded•Design and operating features

Petersen, Human Error Reduction and Safety Management (1996)

30

Organizationally, DonOrganizationally, Don’’t Proceed if t Proceed if Uncertain!Uncertain!For example lifting and rigging lessons learnedFor example lifting and rigging lessons learned……..

31

GetFacts!

32

When the operator inputted a reactor scram signal, all the rods started to drop, and the graphite tips of the control rods contributed to a power pulse (approximately 100 times6 normal full power) in an instant. This pulse vaporized and hurtled airborne approximately 50 tons of fuel, carried away the reinforced roof, a 50-ton crane and the 250-ton refueling machine. It also blew off the 500-ton reactor cover and destroyed most of the building

33

Let’s Close with a PRA Approach…•Assume a technician performing a calibration makes an error 1 out of every 1000 times he performs the task. This is a 1 X 10 -3 probability of error.

•Since the calibration has a nuclear safety defense-in-depth aspect, we call for supervisory review/approval of the data. The supervisor makes an error 1 out every thousand times he reviews documents. This also has a 1 X 10 -3 probability of error. In fault tree analysis this gives us a 1 X 10 -6probability of error. (one in a million)

•The supervisor and technician have worked together for several years, and this is the best tech in the shop. As such, the supervisor doesn’t give much of a review, because time pressure has him working on 7 other items that day and he assumes the level of work is error-free.

•PRA conclusion of our risk?1 X 10 -3 probability of error-Acceptable?

34