Earthquake Early Warnings as Prototype External Forecasts and Predictions (EFPs)
description
Transcript of Earthquake Early Warnings as Prototype External Forecasts and Predictions (EFPs)
SCEC: An NSF + USGS Research Center
Earthquake Early Warnings as Prototype External Forecasts and Predictions (EFPs)
Philip MaechlingInformation Technology ArchitectSouthern California Earthquake Center (SCEC)7 May 2013
Premise: EEW In California Is Imminent
EEW Systems may forecast (1) event-specific parameters (mag, location) and site-specific earthquake parameters (peak ground motion).
– Forecast Final Event Parameters from Preliminary Data• Final Magnitude• Final Location
– Site-specific Parameters:• Site-specific peak ground motion intensity based on earthquake location, earthquake magnitude, and
attenuation relation to forecast peak ground motions at sites of interest.
3
Earthquake Parameters Forecast by EEW Systems
Warning time depends on your location’s distance from where the earthquake begins. The slanted red line shows how warning time increases with distance from the epicenter. In this case, warning time increases beyond the 21 mile-radius blind zone with, for instance, approximately 10 seconds warning at 40 miles distance.
USER Module- Single site warning- Map view
CISN EEW Testing Center Test users
• predicted and observed ground motions
• available warning time• probability of false alarm•…
Decision Module(Bayesian)
CISN ShakeCISN Shake AlertAlertτc-Pd
On-site Algorithm
Virtual Seismologist
(VS)ElarmS
Single sensor Sensor network Sensor network
ShakeAlert Forecast Evaluation Problems:– Scientific publications provide insufficient information for independent evaluation
– Data to evaluate forecast experiments (e.g. catalog used to evaluate forecasts) are often incompletely defined
– Active researchers are frequently tweaking their codes and procedures, which become moving targets
– Difficult to find resources to conduct and evaluate long term forecasts
– Standards are lacking for testing forecasts against reference observations
8
Issues Assessing EEW Forecasts
Different algorithms produce different forecast parameters
• Some (e.g. On-site) produce site-specific information (PGA), event magnitude, but no origin time or distance to event
• Some (e.g. Vs) produces full event parametric information.• Some (e.g. ElarmS) produce site specific ground motion estimates on a regular grid.• Some produce single forecast values (On-site)• Some produce time-series with updates (Vs,Elarms)
9
EEW Algorithm Differences
Applying the CSEP Testing Approach to Earthquake Early Warning Forecasts
CISN and SCEC received funding from USGS to develop and evaluate prototype network-based EEW:
SCEC has implemented the CISN Testing Center (CTC) to evaluate the system and seismological performance of the CISN and USGS ShakeAlert earthquake early warning prototype system.
CISN Testing Center Design Goals and Constraints:
– Establish scientific framework for ShakeAlert Testing– Simple and inexpensive to develop and operate– Provide value to USGS and ShakeAlert developers– Communicate value of EEW testing to SCEC community and
CISN
15
Scale of SCEC CTC Activity
• The CISN Testing Center (CTC) generates ShakeAlert (1) cumulative performance summaries over specific periods of time, and (2) event-oriented performance summaries for each significant California event.
• Many of the CTC ShakeAlert performance summaries compare final observed parameters in the ANSS earthquake catalog against ShakeAlert forecast parameters, such as location and magnitude.
• SCEC worked with CISN EEW Developers (Caltech, UCB, ETH) to define Testing Experiment, then used CSEP Software to test ShakeAlert as external forecasts.
16
Overview of CTC ShakeAlert Testing Approach
Many CSEP testing principles are applicable to CISN EEW Testing. The following definitions need to be made to evaluate forecasts:– Exact definition of testing area– Exact definition of a forecast– Exact definition of input data used in forecasts– Exact definition of reference observation data– Measures of success for forecasts
17
Design of an Experiment
CISN Testing Document Defines the ShakeAlert Testing Experiment
Decide if the 3 CSEP regions valid for EEW
• Region Under Test• Catalog Event Region• Buffer to avoid catalog issues
19
Selection of a Testing Region
Summary 1.1: Magnitude X-Y Diagram
Measure of Goodness: Data points fall on diagonal line
Relevant: T2,T3,T4
Drawbacks: Timeliness element not represented
Which in series of magnitude estimates should be used in plot.
20
Experiment Design
Summary 2.2: Magnitude and Location error by time after origin
Measure of Goodness: Data points fall on horizontal zero line
Relevant: T3, T4
Drawbacks: Event-specific not cumulative
21
Experiment Design
CISN Testing Document
Earthquake Catalog
Retrieve Data
FilterCatalog
Filtered Earthquake
Catalog
CISN EEW Performance Summary Processing
CISN EEW Testing Center and Web Site
ANSS Earthquake
Catalog
UCB/ElarmSNIEEW Data Source
CIT/OnSite EEW Data Source
Load Reports
EEW Trigger
Reports
EEW Trigger
Reports
Observed ANSS Data
CISN EEW Trigger Data
Produce Web
Summaries
Why Use CSEP Software for EEW Testing
CISN Testing Document
ShakeAlert Performance Testing System Overview
CISN Testing Center (top right) retrieves a daily earthquake catalog from ANSS Data Center (bottom right) and ShakeAlert performance logs from U.C. Berkeley (left). It then matches new ANSS events to Algorithm Alerts and Decision Module Alerts and plots (1) Event Performance summaries, and (2) Cumulative Performance Summaries
at USC
Types of ShakeAlert Performance Summaries Currently Available
Summaries posted online at: http://scec.usc.edu/scecpedia/CTC_Results
Cumulative ShakeAlert Performance Results for all ANSS catalog events M3.5+ with Network
Codes CI and NC between 4 March 2012 and 31 March 2013.
M4.5 at 2.4 km (1.5 mi) NE of The Geysers, CA 38.8123, -122.786, 2km Mar 14 2013 09:09:23 UTC 71954065
4.0 at 21.3 km (13.3 mi) SE of Markleeville, CA 38.5598, -119.616, 7km Jan 24 2013 23:25:51 UTC 71928201
EEW Scientific Testing Lessons Learned as External Forecasts and Predictions (EFPs)
– Difficult to determine (real-time) information used in forecast, and especially when Bayesian approach is fully implemented
– Forecasts are not evaluated before earthquake occurs and final results are known. Tests based on forecast logs are treated as prospective testing.
– Performance on an event by event basis are higher priority, catalog delays have significant impact.– Performance of systems changes as ANSS catalog changes. Single measurement less useful than
ongoing evaluations over time.
36
End
Forecast Testing Should Increase Along with Forecast Impact
Public and Governmental Forecasts
Engineering and Interdisciplinary
Research
Collaborative Research Project
Individual Research Project
Computational codes, structural models, and simulation results versioned with associated tests.
Development of new computational, data, and physical models.
Automated retrospective testing of forecast models using community defined validation problems.
Automated prospective testing of forecast models over time within collaborative forecast testing center.
SCEC Computational Forecast Users
Scientific and Engineering Requirements for Forecast Modeling Systems
Testing Center System Requirements
The goals of both an EEW and Earthquake Forecast Testing Center Goals (as outlined by Schorlemmer and Gerstenberger (2007)) describe what is needed to build trust in results:
Controlled EnvironmentTransparencyComparabilityReproducibility
Summary Reports for each M ≥ M-min: Key documents is 3 March 2008 document which specifies six types of tests.
– Summary 1: Magnitude
– Summary 2: Location
– Summary 3: Ground Motion
– Summary 4: System Performance
– Summary 5: False Triggers
– Summary 6: Missed Triggers
40
Proposed Performance Measures
Testing Center Conclusions
1. Broad impact of seismological technologies like EEW are great enough to warrant significant effort for evaluation.
2. Independent evaluation for EEW provides valuable service to agencies including CISN, USGS, CPEC, NEPC, and others.
3. Prospective must be done to before techniques will be accepted. 4. Similarities between problems lead to similar scientific techniques.5. Similarities between problems lead to similar technology approach and
potentially common infrastructure.6. “Neutral” third party testing has significant benefits to the science groups
involved in forecasting.7. CSEP infrastructure can be adapted for use in CISN EEW Testing
Centers.
Useful Information to Help Interpretation of CTC Performance Summary Results
• CTC performance summaries are focused on ShakeAlert speed of operation and earthquake event parameter forecasts. No site specific ground motion performance summaries have been developed.
• CTC ShakeAlert performance summaries compare ANSS event parameters to ShakeAlert alert event parameter forecasts. No searching for “false alerts” is currently done.
• Because ANSS Catalog is considered the correct observation, the CTC waits 48 hours after events to let the ANSS catalog settle down before generating CTC performance summaries.
• Cumulative summaries depend on the catalog filter criteria (date range, magnitude, and region) used to select ANSS catalog events. Currently, ANSS events are selected for Mag >= 3.5, Network ID NC or CI. And, we currently run for two time frames Sept 2011 through present and March 3, 2012 through present (considered the Elarms2 era).
• One pager event summaries are produced for new all ANSS events with Network ID NC or CI and Mag >= 3.0
EEW As An Earthquake Forecast
Earthquake Earthquake Warning systems predict final earthquake magnitude before it is known, possibly before earthquake rupture is completed.
-Most rigorous type of forecast testing in prospective testing.-We consider prospective EEW testing if the EEW algorithms log their forecasts in real-time, before final forecasts parameters (e.g. final magnitude) is known.