The New Paradigm in Forensic Science

Geoffrey Stewart Morrison

Quotations

�

�

D. V. Lindley:

“Numeracy is not favoured by British justice.”

R. A. Carr-Hill:

“I believe Lindley’s suggestion is not only mad, it is extremely

dangerous.”

Lindley, D. V. (1977). Probability and statistics. , 27(3), 203–220.The Statistician

Imagine you are driving to the airport...

Imagine you are driving to the airport...

� This is Bayesian reasoning

– It is about logic

– It is not about mathematical formulae

– There is nothing complicated or unnatural about it

– It is the logically correct way to think about many problems

Imagine you work at a shoe recycling depot...

� You pick up two shoes of the same size

– Does the fact that they are of the same size mean they were

worn by the same person?

– Does the fact that they are of the same size mean that it is

highly probable that they were worn by the same person?

Imagine you work at a shoe recycling depot...

�

�

You pick up two shoes of the same size

– Does the fact that they are of the same size mean they were

worn by the same person?

– Does the fact that they are of the same size mean that it is

highly probable that they were worn by the same person?

Both and mattersimilarity typicality

Imagine you are a forensic shoe comparison expert...

�

�

�

The footprint at the crime scene is size 10

The suspect’s shoe is size 10

– What is the probability of the footprint at the crime scene

being size 10 if it had been made by the suspect’s shoe?

(similarity)

Half the shoes at the recycling depot are size 10

– What is the probability of the footprint at the crime scene

being size 10 if it had been made by the someone else’s shoe?

(typicality)

Imagine you are a forensic shoe comparison expert...

�

�

�

The footprint at the crime scene is size 14

The suspect’s shoe is size 14

– What is the probability of the footprint at the crime scene

being size 14 if it had been made by the suspect’s shoe?

(similarity)

1% of the shoes at the recycling depot are size 14

– What is the probability of the footprint at the crime scene

being size 14 if it had been made by the someone else’s shoe?

(typicality)

Imagine you are a forensic shoe comparison expert...

�

�

�

The footprint at the crime science is size 10

similarity / typicality = 1 / 0.5 = 2

The footprint at the crime science is size 14

similarity / typicality = 1 / 0.01 = 100

If you didn’t have a database, could you have made subjective

guesses at relative proportions of different shoe sizes in the

population and applied the same logic to arrive at a

conceptually similar answer?

similarity / typicality = likelihood ratio

The New Paradigm for Forensic-Comparison Science

�

�

�

Use of the likelihood-ratio framework for the evaluation of evidence

– logically correct

– adopted for DNA in the mid 1990s

Use of objective measurements, databases representative of the

relevant population, and statistical models

– transparent and replicable

Empirical testing of validity and reliability under conditions reflecting

those of the case at trial

The New Paradigm for Forensic-Comparison Science

�

�

�

Morrison, G. S. (2009). .

, 49, 298–308.

Morrison, G. S. (2010). . In I. Freckelton, & H.

Selby (Eds.), (Ch. 99). Sydney, Australia: Thomson

Reuters.

Morrison, G. S. (submitted).

. Manuscript submitted for publication, minor

revisions requested.

Forensic voice comparison and the paradigm shift

Forensic voice comparison

Measuring the validity and reliability of

forensic likelihood ratios

Science & Justice

Expert Evidence

The Likelihood-Ratio Frameworkfor the Evaluation of Evidence

Given that it is a cow, what is the probability of it having four legs?

p( 4 legs | cow ) = ?

Given that it has four legs, what is the probability that it is a cow?

p( cow | 4 legs ) = ?

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

0.05

Given two voice samples with acoustic properties and ,

what is the probability that they were produced by the same speaker?

x x1 2

p( same speaker | acoustic properties , ) = ?x1 x2

p( same speaker | ) = ?acoustic properties ,x x1 2

p( cow | legs ) = ?x

posterior odds

likelihood ratio prior odds

p( same speaker | )

=

acoustic properties ,

p( different speaker | acoustic properties , )

p( acoustic properties , | same speaker ) p( same speaker )

p( acoustic properties , | different speaker ) p( different speaker )

x x

x x

x x

x x

1 2

1 2

1 2

1 2

×

Bayes’ Theorem:

¡¡¡ However !!!

The forensic scientist acting as an expert witness

can give the posterior probability. They can

give the probability that two speech samples were

produced by the same speaker.

NOT NOT

Why not?

�

�

�

The forensic scientist does not know the priors.

Determining the probability of guilt (same speaker) is the task of

the trier of fact (judge, panel of judges, or jury), not the

forensic scientist.

The task of the forensic scientist is to present the

which can be extracted from the speech samples.

strength of

evidence

posterior odds

likelihood ratio prior odds

p( same speaker | )

=

acoustic properties ,

p( different speaker | acoustic properties , )

p( acoustic properties , | same speaker ) p( same speaker )

p( acoustic properties , | different speaker ) p( different speaker )

x x

x x

x x

x x

1 2

1 2

1 2

1 2

×

Example

�

�

The likelihood ratio is 100

Whatever the trier of fact’s belief as to the relative probabilities of

the same-speaker versus the different-speaker hypotheses

before being presented with the likelihood ratio, after

they should be 100 times

more likely than before to believe that the voices on the two

recordings belongs to the same speaker rather than to different

speakers.

being

presented with the likelihood ratio

Calculating forensic likelihood ratiosusing objective measurements,databases representative of the

relevant population,and statistical models

Likelihood Ratio:

p( acoustic properties , | same speaker )

p( acoustic properties , | different speaker )

x x

x x1 2

1 2

p( legs | cow )xp( legs | not a cow )x

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8

cows

not cows

legs

pro

port

ion

For continuous data rather than histograms, probability density

functions (PDFs) must be used.

0

0.002

0.004

0.006

0.008

0.010

0.012

0.014

0 20 40 60 80 100 120 140 160 180 200

rectangle width: 10

0 20 40 60 80 100 120 140 160 180 200

rectangle width: 5

(a) (b)

0 20 40 60 80 100 120 140 160 180 200

rectangle width: 2.5

(c)

0 20 40 60 80 100 120 140 160 180 200

rectangle width: 0.1

(d)

0

0.002

0.004

0.006

0.008

0.010

0.012

0.014

20 40 60 80 100 120 140 160 180 2000

0.005

0.010

0.015

0.020

0.025

LR = 11.35

suspect modelbackground modeloffender value

Empirically Testing the Validity of aForensic-Comparison System

Measuring Validity

�

�

�

�

Test set consisting of a large number of pairs known to be sameorigin and a large number of pairs known to be different origin

Test set must represent the relevant population and reflect theconditions of the case at trial

Use forensic-comparison system to calculate LR for each pair

Compare output with knowledge about input

Measuring Validity

�

�

Goodness is to which LRs from same-origin pairs > 1, anddifferent-origin pairs < 1

extentLRs from

Goodness is to which log(LR)s from same-origin pairs > 0,and log(LR)s from different-origin pairs < 0

extent

1/1000 1/100 1/10 1 10 100 1000

-3 -2 -1 0 +1 +2 +3

LR

log (LR)10

� �CN LR N

LRllr

ss i

N

ss ds j

N

ds

ss

i

ds

j� �

�

���

�� � �

�

���

��

� �� �

1

2

11

1 112

1

2

1

log log

� A metric which captures the gradient goodness of a set of likelihoodratios derived from test data is the log-likelihood-ratio cost, Cllr

Log Likelihood Ratio10

Cllr

-3 -2 -1 0 1 2

1

2

3

4

5

6

7

8

9

3

Regina versus T

[2010] EWCA Crim 2439

R v T

�

�

�

“32. It is clear that likelihood ratios have been used in other areas of

expertise by forensic experts when expressing their

conclusions. We are solely concerned in this appeal with the

use in relation to footwear mark evidence.”

“61. [The Forensic Science Regulator] suggested that it was not

logical to adopt the position that the Bayesian or likelihood

ratio approach could be used in some areas, but not in others...”

“76. ...We do not agree with the observations of the Regulator that a

similar approach is justified in all areas of forensic expertise.

Each area requires a separate analysis because of the

differences that there are in the nature of the underlying data.”

R v T

�

�

“79. The paper by Jackson, Champod and Evett [2001] rejected the

suggestion that hard data were needed to evaluate a likelihood

ratio...”

“80. We cannot agree with this in so far as it suggests that a

mathematical formula can be used. An approach based on

mathematical calculations is only as good as the reliability of

the data used...”

R v T

�

�

“83. ... the data on footwear distribution and use is quite unlike

DNA. A person’s DNA does not change and a solid statistical

base has been developed which enable accurate figures to be

produced...”

“84. Use of the FSS’s own database could not have produced

reliable figures as it had only 8,122 shoes whereas some 42

million are sold every year...”

R v T

� “87. It is of course regrettable that there are, at present, insufficient

data for a more certain and objective basis for expert opinion

on footwear marks, but it cannot be right to seek to achieve

objectivity by reliance on data which does not enable this to be

done. We entirely understand the desire of the experts to try

and achieve the objectivity in relation to evidence of footwear

marks, but the work done has never before, as we understand

it, been subject to open scrutiny by a court.”

Further reading

�

�

�

�

�

�

R v George [2007] EWCA Crim 2722

R v GK [2001] NSWCCA 504

Morrison, G. S. (2009). Comments on Coulthard & Johnson’s (2007) portrayal of the

likelihood-ratio framework. , 41,

155–161.

Rose, P., & Morrison, G. S. (2009). A response to the UK position statement on

forensic speaker comparison.

, 16, 139–163.

Balding D. J. (2005). . Chichester, UK:

Wiley.

Robertson, B., & Vignaux, G. A. (1995). .

Australian Journal of Forensic Sciences

International Journal of Speech, Language and the

Law

Weight-of-evidence for forensic DNA profiles

Interpreting evidence Chichester, UK: Wiley.

Thank You

http://geoff-morrison.net

http://forensic-voice-comparison.net

http://forensic.unsw.edu.au