EIANZ October 2008EIANZ October 2008
U i t j d t dUsing expert judgement and uncertainty analysis for y y
biodiversity management
Mark Burgman
OutlineOutline
A taxonomy of uncertaintyMaking decisions under uncertainty: practicalMaking decisions under uncertainty: practical
applications of uncertainty analysis‘T ki ’ Si dh Ib• ‘Taking’ Sindh Ibex
• Habitat models • How many samples?• IUCN conservation assessmentsIUCN conservation assessments
‘Classical’ taxonomy of uncertaintyuncertainty
Variability is naturally occurring, unpredictable change, differences in parameters attributable to ‘true’ or ‘inherent’ variation. i.e. natural variation
Incertitude is lack of knowledge about parameters or models. i.e, measurement error, systematic error, model uncertainty, subjective judgement
Quantitative methodsQuantitative methods
Natural variation
• depends on unjustifiable assumptions• can’t deal with non-statistical uncertainty
RiskLack of knowledge• deals poorly with dependencies• creates an illusion of false precision
t t+1Time
Probability arithmetic, ‘classical’ decision theory, Monte Carlo
Linguistic uncertaintyg y
• Ambiguity – words have two or more meanings, and it is not clear which is meant.
V b d li• Vagueness – borderline cases.
• Underspecificity unwanted generality(Regan et al 2002)
• Underspecificity – unwanted generality.
• Context dependence a failure to specify• Context dependence – a failure to specify context.
UnderspecificityUnderspecificity
There is a 70% chanceThere is a 70% chance of rain.
Possible interpretations• rain during 70% of the day (time)• rain over 70% of the area (area) • on 70% of days such as this one, there will be at least some rain at a particular point (the weather station) (days such as this one)
UnderspecificityUnderspecificity
There’s a 70% chance of rain
Gi H t i d B k F l & K t ik l Ri kGigerenzer, Hertwig, van den Broek, Fasolo, & Katsikopoulos, Risk Analysis (in press)
Sindh IbexSindh Ibex
“We don’t know enough gto build a model”
How many can we take?
1
How many can we take?
0.75
0.5ob
abili
ty
0.25
Pro
00 2500 5000 7500 10000 12500
Population size
GAM
Model and parameter uncertainty
GLM
GAM
GARP
parameter uncertainty
GLMGARP
Florida scrub jayFlorida scrub jay
How do you ydecide that a species is absent from an area?
Overconfidence
Settlement depth Height to failure
0 8
1of a clay layer of an earth embankment
0.6
0.8
udgm
ents
0.2
0.4Ju
01 2 3 4 5 6 7 1 2 3 4 5 6 7
ExpertsExperts
(Hynes and Vanmarche 1977)
Insensitivity to sample sizeInsensitivity to sample size
Tversky and Kahneman (1974)
‘the law of small numbers’: most people (including experienced scientists) expect small samples to represent the pop lation from hich the ererepresent the population from which they were drawn to a degree that can only be assumed with much larger samplesmuch larger samples.
Challenge problemg p
H 0 1 h d d d b• How many 0.1 ha quadrats do you need to be 90% certain of encountering 95% of the
d l i 40 h ?woody plants in a 40 ha area?
(Baran, 2000)
Testing Risk Classificationsg
Mountain pygmy possum(B )
Desert tortoise(G h i ii)(Burramys parvus)
IUCN: Endangered Florida scrub jay(Aphelocoma
(Gopherus agassizii)Heritage: Apparently Secure (G4)
coerulescens coerulescen)Millsap: Threatened(Regan et al 2005)
IUCN Critically endangeredIUCN Critically endangered
IF Decline of >= 80% in 10 years or 3 generationsOR Range <100 km2 or occupied habitat <10 km2
ANDat least 2 of the following 3 conditions are met:1) severely fragmented or in 1 subpopulation1) severely fragmented or in 1 subpopulation2) continuing to decline3) fluctuations > 1 order of magnitude
OR b f t i di id l < 250OR number of mature individuals < 250AND
at least 1 of the following 2 conditions are met:1) >=25% decline in 3 years / 1 generation1) 25% decline in 3 years / 1 generation2) continuing decline and 1 subpopulation or <=50 per subpopulation
OR < 50 mature individualsOR >= 50% risk of extinction in 10 years / 3
generations.
IUCN
1
IUCN
12345678ec
ies
89
1011
Spe
121314
LR VU EN CRIUCN 2000
LR VU EN CR
(Regan et al 2005)
IUCN
1
IUCN
12345678ec
ies
89
1011
Spe
121314
LR VU EN CRIUCN 2000
LR VU EN CR
(Regan et al 2005)
IUCN
1
IUCN
Why?1234
• Unwillingness to make inferences
5678ec
ies • Parameter estimates spanning thresholds
V i i i li i j d (l )89
1011
Spe • Variation in qualitative judgements (language)
• Inconsistent logic and data entry mistakes121314
LR VU EN CR
Inconsistent logic and data entry mistakes (relatively rare)
IUCN 2000LR VU EN CR
(Regan et al 2005)
Expert judgement?Expert judgement?
A specialist knows... ‘a lot of things, … details … not covered in the general texts...’
‘I t thi ki f l t f li b t k l d d‘I am not thinking of general gut feeling but knowledge and experiences … in the light of which [the expert] can judge’
Appeals to authority
Irrefutable, unspecified wisdom and unassailable expert status generates a culture of technical controlculture of technical control
‘There is a naked assertion that the identity of the expert warrants acceptance of theof the expert warrants acceptance of the proposal’ (Walton).
Appeal to authority is legitimate only ifAppeal to authority is legitimate only if it can be challenged.