Blinking on the Bench: How Judges Make Decisions

Jeffrey J. Rachlinski

Cornell Law School

Business Litigation Section,

Dallas Bar Association

April 12, 2011

My Thesis

• Judges rely on simple rules of thumb that produce intuitions about how to decide cases

• These intuitions can be accurate, but often lead to error

• Accurate judging requires that judges engage in a deliberative assessment of the cases before them

Compare: Psychology of Judgment and Choice

• System 1 -- intuitive, associative, affective, rapid judgment– Legal Realism?

• System 2 -- deliberative, rule-based, calculating, mathematical, deductive, slow judgment– Legal Formalism?

Judging: Generally and in the Courtroom

• In ordinary life, people must use both System 1 and System 2– But System 1 is faster, and a bit less conscious– Knowing when to suppress intuition is essential

to sound judgment

• Why would judgments made in the courtroom be any different?

Our Research Program

• Over 3,000 trial judges & 400 lawyers

• Participating in judicial education conferences

• Hypothetical questions

Are Judges Different? The Cognitive-Reflection Test

• Frederick & Kahnman (2002)

• 3 questions

• Participants asked to answer them “as best as you can”

CRT: Question #1

• A bat and a ball together cost $1.10. The bat costs $1.00 more than the ball. How much does the ball cost?

Intuition: 10 ¢

Answer: 5 ¢

CRT: Question #2

• If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?

Intuition: 100 minutes

Answer: 5 minutes

CRT: Question #3

• In a lake, there is a patch of lily pads. Every day the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake?

Intuition: 24 days

Answer: 47 days

CRT: Three Basic Findings

1. Even though the questions are not difficult, most people get most of them wrong.

2. The wrong answer most commonly chosen tends to be the intuitive one

3. Those who get a problem wrong tend to think it an easier problem than those who get it right

CRT & Judges: Would they Perform Differently?

• Maybe judges are, by nature, System 2 people– Such as engineering students

• 300 trial judges in Florida

CRT Results in Judges

1. Judges got most of the questions wrong:-Average score 1.23 out of 3.00

2. The most common wrong answers were the intuitive ones

-chosen by 97%, 60% & 58% of the judges

3. Judges who chose the intuitive answer thought the problem was easier

-e.g., 90% v 66% on question #1.

Judges Versus Others on CRT

MIT students 2.18

Carnegie Mellon students 1.51

Harvard students 1.43

Florida trial judges 1.23

Web-based participants 1.10

---- State appellate judges 0.95

M----- State students 0.79

The (Unfortunate) Headline:

“Judges Flunk Math Test”-American Bar Association Journal

Judges’ Responses

• “SO WHO CARES about widgets?? What does this have to do with “who gets custody of a child”

• “These questions have nothing to do with making decisions.”

Can Intuition Lead Judges Astray in Legal Settings?

1. Exemplar Cuing

2. Anchoring

3. Consistency seeking

4. Confirmation Bias

1. Exemplar Cuing

Context highlights different aspects of a fact pattern

“Jellybean Phenomenon”

• Suppose I’ll give you $20 if you draw a red jellybean from an urn filled with red and white jellybeans

Choice of Two Urns

Urn #1: 1 red 9 white Urn #2: 10 red 90 white

Civil Commitment & Judges-Monahan & Silver 2003

• Standard: danger to self or others• Description of a mentally disturbed patient

• What threshold risk of violence would justify a civil commitment?– Using two different (but identical) scales:

• % chance of a violent act• n/100 people like this commit a violent act

Civil Commitment: Format Variations

Risk of violence (in %) :

1% 8% 26% 56% 76%

Risk of violence (n/100):

1 8 26 56 76

Sentencing Problem

• 136 Trial Judges at National Conference in US

• Criminal sentencing decision

• Defendant stabbed victim to death after victim taunted him about fiance

• Defendant is guilty of voluntary manslaughter

• Without regard to your jurisdiction, what is the appropriate sentence?

Sentencing Format Variation

______/Years

______/Months

Average Sentencing by Format

10 9 8 7 6 5 4 3 2 1 0

12010896847260483624120

Years Months

2. Anchoring

The tendency to base numeric estimates on numeric reference points, even when the reference points are random

Anchoring: An Example

• What is the average price of a textbook in the campus bookstore?

• Before you answer, is it more or less than $103,471

Anchoring in Judges

• Civil rights violation • Defendant is a public-sector employee• Plaintiff is a secretary• Supervisor calls her racial epithets and ridicules

her ancestry in front of co-workers and her daughter

• Plaintiff finds other position, but City human rights commission brings case on her behalf

• Only damages are for “mental anguish”

The Anchor

• No Anchor—Plaintiff asserts that she recently saw a case similar to hers on a “court television show where the plaintiff received a compensatory damage award for mental anguish.”

• Anchor—Plaintiff asserts that she recently saw a case similar to hers on a “court television show where the plaintiff received a compensatory damage award of $415,300 for mental anguish.”

Irrelevant Anchor: Results(Median Award, in $ thousand)

0

10

20

30

40

50

60

70

No Anchor

Anchor

Irrelevant Anchor #2:Personal Injury Suit (large)

(Median Award, in $ thousand)

0

400

500

600

700

800

900

1,000

No Anchor

Anchor ($75k)

Irrelevant Anchor #3: Personal Injury Suit (small)

(Median Award, in $ thousand)

0

40

50

60

70

80

90

100

No Anchor

Anchor ($332k)

Irrelevant Anchor #4:Sentencing Order

• Military trial judges sentencing 2 defendants– Threatening with an unloaded weapon (3 years)– Voluntary manslaughter (15 years)

• Order of sentencing varied– Half sentenced “threat” then “manslaughter”– Half sentenced “manslaughter” then “threat”

• 75% of appellate judges stated order has no effect

Sentencing (Threat): Results(average sentence in years)

Sentencing (Manslaughter): Results(average sentence in years)

3. Consistency Seeking

• The tendency to see the facts and law in ways that are consistent (even when they need not be consistent)

Immigration Case

• Defendant, a Peruvian citizen, pled guilty to using a false identification document, a misdemeanor- Forged U.S. entry visa was pasted into genuine

Peruvian passport• Two different reasons for attempting to enter United States

illegally, either- Father: find a job to pay for daughter’s liver transplant- Killer: track down a rogue member of a cartel who

had stolen drug proceeds

Immigration Case

• Legal issue of first impression: does pasting a forged U.S. entry visa into a genuine foreign passport constitute “forgery of an identification card”

• If so, 6 months in prison (deportation follows either way)

Consistency Seeking: Results% finding against defendant?

Consistency Seeking in New Judges% finding against defendant?

4. Confirmation Bias

• Tendency to seek information that is consistent with a hypothesis—even when that information is logically irrelevant to that hypothesis

Card-Selection Problem

• “If there is a vowel on one side of the card, then there is an even number on the other side.”

E P 4 7

•Which one(s) must you turn over to test the hypothesis?

X X

Logical Problem• “If there is a vowel on one side of the card, then there is an even

number on the other side.” • 4 cards:

– 1: the letter “E” on one side and the number ___ on the other– 2: the letter “P” on one side and the number ___ on the other– 3: the number “4” on one side and the letter ___ on the other– 4: the number “7” on one side and the letter ___ on the other

• Which one(s) must you turn over?

Gender Discrimination Problem• “male supervisors only promote male employees”• 4 people:

– 1: a male supervisor who recently promoted a ___ employee

– 2: a female supervisor who recently promoted a ___ employee

– 3: a male employee who was recently promoted by a ___ supervisor

– 4: a female employee who was recently promoted by a ___ supervisor

• Which one(s) must you identify?

Logical/Discrimination Problem: Previous Results

(% Correct)

• Logical Problem: 9% correct

• Discrimination Problem: 14% correct

Addendum: What About Lawyers?

• “Decision makers often pursue non-instrumental information—information that appears relevant but, if simply available, would have no impact on choice. Once they pursue such information, people then use it to make their decision.”– Bastardi & Shafir, 1998

Useless Information: Materials

• Plaintiff is a worker in auto-assembly plant• Injured by defendant’s robotic welding unit• Serious facial scaring• Defendant offers to settle for $400,000• Plaintiff feels it is low, but reluctant to take risks• Government safety report on cause of the accident

The “Useless” Information

• Simple version– Government report blames the manufacturer,

rather than the worker– Settle for $400,000?

• 2-Stage version– Report is not out yet—should you wait?– If wait, told that it blames manufacturer—

should you settle?

Useless Information: Results

• Simple version: 62% settle– The report is irrelevant to this group

• 2-Stage version: 55% wait for report

• Once report is out: 23% settle

Conclusions

• Intuitive thinking can lead judges astray, even in legal contexts in which they are expert

• Mechanisms to engage the slower, deliberative system are essential to quality judicial decision making

“When all is said and done, we must face the fact that judges are human.”

-Jerome Frank, 1949