Expert judgment-based fragility functions to better characterize physical vulnerability worldwide

Kishor Jaiswal, Ph.D., P.E.USGS/Synergetics Inc. 2014 EERI Housner FellowGolden CO

GEM Vulnerability Estimation Methods (GEMVEM)

Empirical Analytical Expert-Judgment

Empirical-National

USGS Role: Contributing Empirical-National Vulnerability Models: Fatality & Economic loss models Developing protocols and guidelines on Expert Judgment-based Vulnerability Estimation Methods Collaborating with GEMGVC Team in developing Seismic Vulnerability Database & Mapping Schemes

Background

To understand and quantify the physical vulnerability of built infrastructure, we need a suite of vulnerability/fragility models

‣ “There is a rich literature of seismic vulnerability information, only exceeded by the vast need for more” (Porter et al., 2010).

‣ Expert judgments are essential for assessing vulnerability of building types for which the empirical data are lacking and the physical models have limited predictive capabilities.

‣ Expert judgments have been used in the past for assessing seismic vulnerability (ATC 13, 1985 survey used the modified Delphi process).

‣ We plan to use structured the elicitation procedure proposed by Cooke (1981) for eliciting judgments on seismic collapse fragility of global building types.

Cooke’s Approach (1991)

‣ It is a structured elicitation process

‣ Experts are treated as a statistical hypothesis

‣ The approach is subject to:‒ Scrutiny/accountability

[data/process open]‒ Empirical control [quantitative,

subject to empirical quality control]

‒ Neutrality [encourage experts to provide their true beliefs, and help avoid bias]

‒ Fairness [expertise is not pre-judged]

Credit: Aspinall 2008

Cooke’s approach uses ‘performance-based’ scoring system instead of the traditional ‘self-rating’ and ‘consensus building’ approaches used in the Delphi procedure.

Calibration Score:C(e) = 1 - χB

2 [ Σ 2 * nb * I(sb , pb) ]Average Response Entropy: (Inverse of entropy is called Information Score)

He(n) = [Σ nb * H(pb) ] / N where H(p)=-Σp *ln(p)

UnNormalized Weight:w’eα = Ια [C(e)] * C(e) / He(n)

Let us assume that we have expert ‘e’ who assesses the probability of each uncertain event by assigning a corresponding indicator function I to one of Bprobability bins. Here each probability bin is associated with a distribution, and the term Pb indicates the occurrence probability P of bin b.

Let nb denote the number of variables assigned to bin b, and sb denote thesample distribution of variables in bin b, where b=1,…B bins.

The term I(s,p) provides relative information of s with respect to p is given as

I(s, p) = Σ s * ln(s / p)

Performance-based Scoring Metric…

Key Aspects:

Seed Questions

‣ These are the questions for which the answers are already known to the facilitator

‣ Experts’ answers to seed questions are used to estimate the calibration and information scores

‣ Performance measured using both score is then combined to estimate the weights

Target Questions

‣ The answers to target questions are unknown, and thus the participating experts help to derive them collectively

‣ Each expert’s answers are weighted according to the weights he/she received

‣ Multiple judgments are combined to provide estimate for an unknown quantity and to quantify associated uncertainties

Scoring approach:

Calibration Score

‣ Impetus is given on measuring the ‘statistical accuracy’ associated with elicited quantiles

‣ Penalizes heavily for larger deviations/mismatch

Entropy Score

‣ Measures the degree with which the elicited distribution is concentrated with respect to some background distributions

‣ Penalizes for presenting estimates with relatively poorer information

Summary: Deriving Generic Building Collapse Fragility Models

• Supported by GEM Foundation• Focus on generic global construction

types that are common in earthquake prone countries

• Thirteen experts from eight countries participated in the exercise

• Construction types included: Unreinforced masonry – 6 Types Concrete frame– 6 Types

• Supported by USGS, GEM and EERI• Focus on selected U.S. construction types

that are deemed to be vulnerable• Twelve experts from U.S. and one from

Canada participated in the exercise• Construction types included:

Weak first story wood frame – 2 Types Unreinforced masonry (retrofitted and

unretrofitted) – 3 Types Concrete types– 20 Types

Lisbon Workshop [Sept 23, 2012] Oakland Workshop [May 8, 2013]

Sample Responses…

Typical Response to One of the Seed Question:

Typical Response to One of the Target Question:

Generation of a Collapse Fragility Curve…

Collapse fragility estimate of seismically retrofitted unreinforced masonry bearing wall construction

Summary

‣ Expert elicitation requires careful deliberation (e.g., recruiting experts, planning, preparation notes, survey instruments)

‣ Cooke’s method rewarded heavily to the experts who performed well in calibration exercise, but then it penalized significantly to the experts who failed to do so.

‣ The exercise resulted in generating dozens of generic seismic collapse fragility functions to be used within the Global Earthquake Model (GEM)’s OpenQuake engine as well as within the PAGER’s semi-empirical model

‣ Further research is necessary in harmonizing the influence that ultimately yielded in estimating the final score, i.e., scoring based on being statistically accurate vs. being a real expert who is not so good in quantifying answers in probabilistic sense.

‣ Collapse fragility results will be made available through the GEM Nexus and the USGS PAGER website.

Acknowledgements:

‣ Participating Experts:Lisbon Workshop: Daniel Abrams, Edmund Booth, Michele Calvi, Dina D’Ayala, Ken Elwood, Polat Gulkan, Michael Griffith, Jason Ingham, Sergio Lagomarsinho, Paulo Lourenco, Guido Magenes, David Mar, and Ahmed YakutOakland Workshop: David Bonowitz, Kelly Cobeen, Craig Comartin, Greg Deirlein, Jim Harris, John Hooper, Charlie Kircher, Greg Kingsley, Bret Lizundia, Jay Love, Joe Maffei, Brian McDonald, John Sherstobitoff

‣ GEM Foundation:‒ Rui Pinho and Helen Crowley

‣ Earthquake Engineering Research Institute:‒ Jay Berger, Marjorie Greene, Maggie Ortiz, Jonathon Tai, Christopher Lee, Patrick

Bassal‣ University of Bristol, UK:

‒ Willy P. Aspinall‣ U.S. Geological Survey:

‒ David Perkins, David Wald, and Nico Luco‣ Designing of the Survey Questions:

Willy Aspinall, Dimitrios Vamvatsikos, Keith Porter, Bill Holmes, Nico Luco, Abbie Liel, David Perkins, Helen Crowley, and Siamak Sattar

Except where otherwise noted, this work is licensed under: creativecommons.org/licenses/by-nc-nd/4.0/

Please attribute to the GEM Foundation with a link to -www.globalearthquakemodel.org