Phillip E. Shafer Henry E. Fuelberg Florida State University April 4, 2007

Dept. of Meteorology

Developing Gridded Forecast Guidance for Warm Season

Lightning over Florida Using the Perfect Prognosis Method and

Mesoscale Model Output

Developing Gridded Forecast Guidance for Warm Season

Lightning over Florida Using the Perfect Prognosis Method and

Mesoscale Model Output

Phillip E. ShaferHenry E. Fuelberg

Florida State University

April 4, 2007

Phillip E. ShaferHenry E. Fuelberg

Florida State University

April 4, 2007


Accurate Lightning Forecasts Are Important to FPL

Accurate Lightning Forecasts Are Important to FPL

Lightning leads to outages

FPL crews should be ready

to respond

Don’t want un-needed crews

Lightning leads to outages

FPL crews should be ready

to respond

Don’t want un-needed crews


• Phase I: 1 August 2002 – 31 July 2003

• Phase II: 1 June 2003 – 31 May 2004

• Phase III: 1 June 2004 – 31 May 2005

• Phase IV: 1 June 2005 – present

• Phase I: 1 August 2002 – 31 July 2003

• Phase II: 1 June 2003 – 31 May 2004

• Phase III: 1 June 2004 – 31 May 2005

• Phase IV: 1 June 2005 – present

Project TimelineProject Timeline


Phase I—Develop Lightning Climatologies

Phase I—Develop Lightning Climatologies

Southeast Flow Southwest Flow

Flashes/km^2/Regime Day


Detailed ClimatologiesFor Dispatch CentersDetailed ClimatologiesFor Dispatch Centers


Phases II & IIIPhases II & III• Equations derived for 11

FP&L service areas.• Morning radiosonde

parameters used as predictors for afternoon lightning in each area.

• Miami, Tampa, Jacksonville, Cape Canaveral

• Generally, the sounding closest to each forecast area was used.

• Equations derived for 11 FP&L service areas.

• Morning radiosonde parameters used as predictors for afternoon lightning in each area.

• Miami, Tampa, Jacksonville, Cape Canaveral

• Generally, the sounding closest to each forecast area was used.


Phase 3 Sample ForecastPhase 3 Sample Forecast


Forecasts Useful to FP&L Dispatchers:Forecasts Useful to FP&L Dispatchers:

“I can state, unequivocally, that SSD uses the forecasts daily as an integral part of the resource and switching decision making process.”

“On many occasions, we may have not held resources based on the projected weather levels only but the lightning forecasts were solid enough to override that decision- to our advantage I might add.”

“I can state, unequivocally, that SSD uses the forecasts daily as an integral part of the resource and switching decision making process.”

“On many occasions, we may have not held resources based on the projected weather levels only but the lightning forecasts were solid enough to override that decision- to our advantage I might add.”


Phase IV—Space and Time Varying Guidance for all of Florida

Phase IV—Space and Time Varying Guidance for all of Florida


Presentation OutlinePresentation Outline

1. Motivation and Objectives2. Background3. Data4. Model Development5. Model Parameters6. Results for Dependent Data7. Results for Independent Test8. Summary & Conclusions

1. Motivation and Objectives2. Background3. Data4. Model Development5. Model Parameters6. Results for Dependent Data7. Results for Independent Test8. Summary & Conclusions


1. Motivation and Objectives1. Motivation and Objectives


MotivationMotivation

Flashes km-2 warm season-1

1989-2006 (May-September)Flashes km-2 warm season-1

1989-2006 (May-September)

• Lightning is one of the leading causes of weather related fatalities in the U.S.

• Lightning can cause damage to trees and utility lines, leading to disruptions in power and communications.

• Florida is the lightning capital of the U.S.

• Many heavily populated areas are vulnerable.

• Skillful probabilistic guidance in the 3-12 h time frame would have many potential societal benefits.

• Lightning is one of the leading causes of weather related fatalities in the U.S.

• Lightning can cause damage to trees and utility lines, leading to disruptions in power and communications.

• Florida is the lightning capital of the U.S.

• Many heavily populated areas are vulnerable.

• Skillful probabilistic guidance in the 3-12 h time frame would have many potential societal benefits.


ObjectivesObjectives

1. Use the perfect prognosis (PP) method to develop a high-resolution gridded forecast guidance product for warm season cloud-to-ground (CG) lightning for all of Florida:

-- Equations to produce spatial probability forecasts for one or more CG flashes, and the probability of exceeding

various flash count percentile thresholds.

-- 10 x 10 km grid, 3-h intervals

2. Evaluate the utility and skill of the PP scheme when applied to forecast output from several mesoscale models during an independent test period (2006 warm season).

1. Use the perfect prognosis (PP) method to develop a high-resolution gridded forecast guidance product for warm season cloud-to-ground (CG) lightning for all of Florida:

-- Equations to produce spatial probability forecasts for one or more CG flashes, and the probability of exceeding

various flash count percentile thresholds.

-- 10 x 10 km grid, 3-h intervals

2. Evaluate the utility and skill of the PP scheme when applied to forecast output from several mesoscale models during an independent test period (2006 warm season).


2. Background2. Background


Myriad FactorsMyriad Factors• Sea breeze usually the dominant forcing mechanism over

Florida during the warm season.

• Interactions between the sea breeze, the prevailing wind, and coastline curvature have been shown to influence lightning patterns (e.g., Lopez and Holle 1987; Hodanish et al. 1997; Camp et al. 1998; Lericos et al. 2002).

• Other myriad factors influence timing and location of convection and lightning:

-- Local thermal circulations (e.g., water conservation areas, lakes, rivers, etc.)

-- Urban effects (e.g., Westcott 1995; Steiger et al. 2002)

-- Thunderstorm outflows

• Sea breeze usually the dominant forcing mechanism over Florida during the warm season.

• Interactions between the sea breeze, the prevailing wind, and coastline curvature have been shown to influence lightning patterns (e.g., Lopez and Holle 1987; Hodanish et al. 1997; Camp et al. 1998; Lericos et al. 2002).

• Other myriad factors influence timing and location of convection and lightning:

-- Local thermal circulations (e.g., water conservation areas, lakes, rivers, etc.)

-- Urban effects (e.g., Westcott 1995; Steiger et al. 2002)

-- Thunderstorm outflows


Cloud MicrophysicsCloud Microphysics

• Lightning ultimately is governed by cloud microphysical processes that are poorly resolved by NWP models.

• Factors influencing cloud electrification are poorly understood.

• Several hypotheses have been proposed:

-- Precipitation hypothesis (Reynolds et al. 1957)

-- Convection hypothesis (Vonnegut 1963)

-- Non-inductive ice-ice collision mechanism (Williams 1985)

• Hypotheses depend on a vigorous updraft and robust ice phase for charge generation (Price and Rind 1992, 1993).

• But, advances have been made in our understanding of the factors influencing lightning production.

• Lightning ultimately is governed by cloud microphysical processes that are poorly resolved by NWP models.

• Factors influencing cloud electrification are poorly understood.

• Several hypotheses have been proposed:

-- Precipitation hypothesis (Reynolds et al. 1957)

-- Convection hypothesis (Vonnegut 1963)

-- Non-inductive ice-ice collision mechanism (Williams 1985)

• Hypotheses depend on a vigorous updraft and robust ice phase for charge generation (Price and Rind 1992, 1993).

• But, advances have been made in our understanding of the factors influencing lightning production.


Statistical StudiesStatistical Studies

• A variety of statistical techniques has been used to develop forecast models for thunderstorms and lightning:

-- Multiple linear regression often employed in earlier studies (less computationally demanding).

-- Binary logistic regression more appropriate when predictand is “yes” or “no” (e.g., Mazany et al. 2002; Bothwell 2002; Lambert et al. 2005; Shafer and Fuelberg 2006).

-- Classification and regression trees (e.g., Burrows et al. 2004).

• Many studies have used data from morning soundings to forecast afternoon lightning.

• Data from NWP models is more location and time specific.

• A variety of statistical techniques has been used to develop forecast models for thunderstorms and lightning:

-- Multiple linear regression often employed in earlier studies (less computationally demanding).

-- Binary logistic regression more appropriate when predictand is “yes” or “no” (e.g., Mazany et al. 2002; Bothwell 2002; Lambert et al. 2005; Shafer and Fuelberg 2006).

-- Classification and regression trees (e.g., Burrows et al. 2004).

• Many studies have used data from morning soundings to forecast afternoon lightning.

• Data from NWP models is more location and time specific.


Model Output StatisticsModel Output Statistics

• Model Output Statistics (MOS): Objective forecasting technique in which statistical relationships are determined between a predictand and variables forecast by an NWP model.

• Advantage: Model biases and local climatology are automatically built into the equations. Usually the method of choice when practical.

• Drawback: NWP models are constantly changing. Any modifications to the NWP model that change systematic model errors require redevelopment of the MOS equations.

• Model Output Statistics (MOS): Objective forecasting technique in which statistical relationships are determined between a predictand and variables forecast by an NWP model.

• Advantage: Model biases and local climatology are automatically built into the equations. Usually the method of choice when practical.

• Drawback: NWP models are constantly changing. Any modifications to the NWP model that change systematic model errors require redevelopment of the MOS equations.


Perfect PrognosisPerfect Prognosis

• Perfect prognosis (PP): Statistical relationships are determined between observations of the predictand and observed atmospheric predictors.

• Advantages:

-- Equations are developed without NWP forecasts (i.e., they are model independent).

-- Equations can be used with any NWP model and forecast projection, even as the models change.

• Drawback: Assumes a “perfect” forecast of the predictors by the NWP model and thus, does not account for model biases.

• Bothwell (2002): Used PP method to develop forecast equations for CG lightning over the western U.S.

• Perfect prognosis (PP): Statistical relationships are determined between observations of the predictand and observed atmospheric predictors.

• Advantages:

-- Equations are developed without NWP forecasts (i.e., they are model independent).

-- Equations can be used with any NWP model and forecast projection, even as the models change.

• Drawback: Assumes a “perfect” forecast of the predictors by the NWP model and thus, does not account for model biases.

• Bothwell (2002): Used PP method to develop forecast equations for CG lightning over the western U.S.


3. Data3. Data


Study DomainStudy Domain

Grid spacing = 10 kmOnly land grid points used in model development

Grid spacing = 10 kmOnly land grid points used in model development


Lightning DataLightning Data

• The dependent variable• National Lightning Detection Network• System wide upgrades in 1995 &

2002• 1995-2005 warm seasons used to

develop climatological predictors.• 2002-2005 warm seasons used in

equation development.• Data quality controlled for duplicate

flashes and non-CG discharges.• Flashes summed within a 10-km

radius of each grid point during each 3-h period (e.g., 0000-0259 UTC, …, 2100-2359 UTC).

• The dependent variable• National Lightning Detection Network• System wide upgrades in 1995 &

2002• 1995-2005 warm seasons used to

develop climatological predictors.• 2002-2005 warm seasons used in

equation development.• Data quality controlled for duplicate

flashes and non-CG discharges.• Flashes summed within a 10-km

radius of each grid point during each 3-h period (e.g., 0000-0259 UTC, …, 2100-2359 UTC).


Lightning PredictandsLightning Predictands

• 3-h flash totals transformed into binary variables.

• “1” if one or more flashes or “0” if no lightning

• Binary variables also assigned based on whether the flash total exceeds the 50th, 75th, 90th, and 95th percentiles for a given 3-h period:

• 3-h flash totals transformed into binary variables.

• “1” if one or more flashes or “0” if no lightning

• Binary variables also assigned based on whether the flash total exceeds the 50th, 75th, 90th, and 95th percentiles for a given 3-h period:


Rapid Update Cycle (RUC)Rapid Update Cycle (RUC)• “Observed” atmospheric predictors derived from RUC analyses

during 2002-2005 warm seasons (May-Sept).

• RUC data sources:

1. Atmospheric Radiation Measurement (ARM) Program (http://www.arm.gov/xds/static/ruc.stm)

2. National Climatic Data Center (http://nomads.ncdc.noaa.gov)

• 20-km, 50 level, hourly version (RUC20) implemented at NCEP during April 2002, with improvements in the analysis/physics.

• 13-km version (RUC13) implemented at NCEP on 28 June 2005 with further improvements in the analysis/physics.

• ~ 1.2 TB of RUC grib data was acquired and processed!

• “Observed” atmospheric predictors derived from RUC analyses during 2002-2005 warm seasons (May-Sept).

• RUC data sources:

1. Atmospheric Radiation Measurement (ARM) Program (http://www.arm.gov/xds/static/ruc.stm)

2. National Climatic Data Center (http://nomads.ncdc.noaa.gov)

• 20-km, 50 level, hourly version (RUC20) implemented at NCEP during April 2002, with improvements in the analysis/physics.

• 13-km version (RUC13) implemented at NCEP on 28 June 2005 with further improvements in the analysis/physics.

• ~ 1.2 TB of RUC grib data was acquired and processed!


Model Analyzed PredictorsModel Analyzed Predictors• Plethora of RUC-analyzed predictors investigated for possible

inclusion in candidate predictor pool.

• Parameters found useful in previous studies were examined:

-- Temperature (layer thickness, temperature advection, cold cloud thickness, etc.)

-- Moisture (moisture flux convergence, theta-e advection, PW, layer mean RH, etc.)

-- Stability (most unstable CAPE in various layers, CIN, best lifted index, Showalter Stability Index, TT, KI, temperature and

theta-e lapse rates, etc.)

-- Wind (wind divergence, vorticity, vorticity advection, layer mean U and V components, layer mean speed, layer

shear).

• Plethora of RUC-analyzed predictors investigated for possible inclusion in candidate predictor pool.

• Parameters found useful in previous studies were examined:

-- Temperature (layer thickness, temperature advection, cold cloud thickness, etc.)

-- Moisture (moisture flux convergence, theta-e advection, PW, layer mean RH, etc.)

-- Stability (most unstable CAPE in various layers, CIN, best lifted index, Showalter Stability Index, TT, KI, temperature and

theta-e lapse rates, etc.)

-- Wind (wind divergence, vorticity, vorticity advection, layer mean U and V components, layer mean speed, layer

shear).


Model Analyzed PredictorsModel Analyzed Predictors

• All parameters calculated from the RUC 0-h temperature, dew point, wind, height, and surface pressure fields valid every three hours (e.g., 0000 UTC, 0300 UTC, …, 2100 UTC).

• Fields interpolated to array of 10 km grid points and transformed into a vertical sounding.

• RUC cloud hydrometeor profiles found to be unusable.

• Assumption: The model analyses give the best estimate of the state of the atmosphere at the analysis time, and thus, can be treated as “observations” for purposes of developing the PP equations.

• We focused mainly on parameters that are well handled by today’s NWP models.

• All parameters calculated from the RUC 0-h temperature, dew point, wind, height, and surface pressure fields valid every three hours (e.g., 0000 UTC, 0300 UTC, …, 2100 UTC).

• Fields interpolated to array of 10 km grid points and transformed into a vertical sounding.

• RUC cloud hydrometeor profiles found to be unusable.

• Assumption: The model analyses give the best estimate of the state of the atmosphere at the analysis time, and thus, can be treated as “observations” for purposes of developing the PP equations.

• We focused mainly on parameters that are well handled by today’s NWP models.


Statistics SoftwareStatistics Software

• S-PLUS version 6.1 for Windows.

• Statistical Package for the Social Sciences (SPSS) version 11.5 for Windows.

• Both are state-of-the-art software packages with a wide range of analysis and modeling capabilities.

• S-PLUS version 6.1 for Windows.

• Statistical Package for the Social Sciences (SPSS) version 11.5 for Windows.

• Both are state-of-the-art software packages with a wide range of analysis and modeling capabilities.


4. Model Development4. Model Development


Map Type PredictorsMap Type Predictors

• Pattern type lightning frequencies developed and used as candidate predictors.

• Capture local enhancements due to interactions between the low-level wind, thermal circulations, and coastline topography, which are not well resolved by NWP models.

• 3-hourly observed sea-level pressure fields used for pattern classification- implies direction and speed of low-level flow.

• SLP fields obtained from RUC analyses spanning the 1998-2005 warm seasons (~1224 days).

• Simple correlation technique used to develop the map types (e.g., Lund 1963, Reap 1994).

• Pattern type lightning frequencies developed and used as candidate predictors.

• Capture local enhancements due to interactions between the low-level wind, thermal circulations, and coastline topography, which are not well resolved by NWP models.

• 3-hourly observed sea-level pressure fields used for pattern classification- implies direction and speed of low-level flow.

• SLP fields obtained from RUC analyses spanning the 1998-2005 warm seasons (~1224 days).

• Simple correlation technique used to develop the map types (e.g., Lund 1963, Reap 1994).



SLP fields interpolated to array of 100 km grid points.SLP fields interpolated to array of 100 km grid points.



• Relative lightning frequencies and the unconditional mean number of flashes calculated for each map type and 3-h period.

• When developing equations, all unclassified maps were assigned the type with which they were most correlated.

• Relative lightning frequencies and the unconditional mean number of flashes calculated for each map type and 3-h period.

• When developing equations, all unclassified maps were assigned the type with which they were most correlated.

• 5 map types developed using correlation threshold of 0.70.

• 2 dominant types (A and B) comprise ~44% of the sample.

• ~22% unclassified at a threshold of 0.70.

• 5 map types developed using correlation threshold of 0.70.

• 2 dominant types (A and B) comprise ~44% of the sample.

• ~22% unclassified at a threshold of 0.70.


Map Type PredictorsMap Type PredictorsType A compositeType A composite Mean no. flashes: 1800-2059 UTCMean no. flashes: 1800-2059 UTC

• High northeast of Florida• Prevailing E-SE flow

• High northeast of Florida• Prevailing E-SE flow

Most lightning confined to West Coast and east of Lake Okeechobee.

Most lightning confined to West Coast and east of Lake Okeechobee.



Mean no. flashes: 1800-2059 UTCMean no. flashes: 1800-2059 UTC

• Ridge over South Florida• SW flow across the state

• Ridge over South Florida• SW flow across the state

Lightning focused along East Coast and Big Bend region.Lightning focused along East Coast and Big Bend region.

Type B compositeType B composite




• Transition between A and B• SE flow over South Florida,

S-SW flow across the north.

• Transition between A and B• SE flow over South Florida,

S-SW flow across the north.

Lightning maxima evident along both coasts.Lightning maxima evident along both coasts.

Type C compositeType C composite




• High north of Florida, lower pressure to the SE.

• Most common after cold frontal passage.

• High north of Florida, lower pressure to the SE.

• Most common after cold frontal passage.

Dry NE flow confines most lightning to South Florida.Dry NE flow confines most lightning to South Florida.

Type D compositeType D composite




• Variation of type B- lobe of high pressure over Gulf.

• W-NW flow across the state.

• Variation of type B- lobe of high pressure over Gulf.

• W-NW flow across the state.

Lightning confined to East Coast and Big Bend, with less coverage than type B.

Lightning confined to East Coast and Big Bend, with less coverage than type B.

Type E compositeType E composite


Generalized Linear Models (GLMs)Generalized Linear Models (GLMs)

• MLR assumptions of constant variance and Gaussian residuals rarely are met with count data- can lead to undesirable and sometimes nonsensical results.

• We considered several regression methods:

-- Forecasting one or more flashes: Binary Logistic Regression

-- Forecasting the amount of lightning: Poisson and Negative Binomial Regression

• GLMs can be used for response variables that follow any probability distribution in the exponential family (e.g., Normal, Binomial, Poisson, Negative Binomial, etc.).

• GLMs accommodate non-Gaussian distributions of residuals and non-constant variance.

• MLR assumptions of constant variance and Gaussian residuals rarely are met with count data- can lead to undesirable and sometimes nonsensical results.

• We considered several regression methods:

-- Forecasting one or more flashes: Binary Logistic Regression

-- Forecasting the amount of lightning: Poisson and Negative Binomial Regression

• GLMs can be used for response variables that follow any probability distribution in the exponential family (e.g., Normal, Binomial, Poisson, Negative Binomial, etc.).

• GLMs accommodate non-Gaussian distributions of residuals and non-constant variance.


Binary Logistic RegressionBinary Logistic Regression

• Most appropriate when predictand is “yes” or “no”• Log link function relates odds ratio to linear

combination of predictors.• Probabilities bounded on the interval [0,1]• Accommodates Bernoulli distribution of residuals.

• Most appropriate when predictand is “yes” or “no”• Log link function relates odds ratio to linear

combination of predictors.• Probabilities bounded on the interval [0,1]• Accommodates Bernoulli distribution of residuals.

KKi

i xbxbbp

p

...1

ln 110

)...exp(1

)...exp(

110

110

KK

KKi xbxbb

xbxbbp


Poisson RegressionPoisson Regression

• More appropriate model for count data• Log link function linearizes the expected value () of

the dependent variable (y)• Poisson probability model assumes that events occur

randomly and at a constant average rate () with

Var(y) = , where is a dispersion parameter.• Poisson model assumes = 1

• More appropriate model for count data• Log link function linearizes the expected value () of

the dependent variable (y)• Poisson probability model assumes that events occur

randomly and at a constant average rate () with

Var(y) = , where is a dispersion parameter.• Poisson model assumes = 1

KKi xbxbbx ...])[ln( 110

)...exp(][ 110 KKi xbxbbx

!

)exp()|Pr(

yy

y


Count DistributionCount Distribution• Strongly skewed distribution• Many cases with 10 or fewer

flashes, few with 100 or more.• Very large variance (~80 times

greater than the mean)• Data significantly over-

dispersed with respect to Poisson model ( >> 1)

• Likely cause: Counts generated by an inhomogeneous Poisson process- count rates vary in space and time.

• Strongly skewed distribution• Many cases with 10 or fewer

flashes, few with 100 or more.• Very large variance (~80 times

greater than the mean)• Data significantly over-

dispersed with respect to Poisson model ( >> 1)

• Likely cause: Counts generated by an inhomogeneous Poisson process- count rates vary in space and time.

1800-2059 UTC periodCases with one or more flashes

1800-2059 UTC periodCases with one or more flashes


Negative Binomial RegressionNegative Binomial Regression

• Alternative probability model with shape parameter • Var(y) is a quadratic function of :

Var(yi | [xi] ) = ( [xi] + -1[xi] 2 )

• More accurately characterizes the uncertainty in the predicted count than does the Poisson model.

• Alternative probability model with shape parameter • Var(y) is a quadratic function of :

Var(yi | [xi] ) = ( [xi] + -1[xi] 2 )

• More accurately characterizes the uncertainty in the predicted count than does the Poisson model.

KKi xbxbbx ...])[ln( 110

)...exp(][ 110 KKi xbxbbx y

i

i

ii x

x

xy

yxy

][

][

][)(!

)()],[|Pr(


Poisson vs. Negative BinomialPoisson vs. Negative Binomial

• Poisson model poorly represents count distribution• Negative binomial captures large number of cases

with 10 or fewer flashes.

• Poisson model poorly represents count distribution• Negative binomial captures large number of cases

with 10 or fewer flashes.

Flash Count Probability Distribution Implied fromPoisson and Negative Binomial Models

1800-2059 UTC period

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.18

0.20

0.22

0.24

0.26

0 10 20 30 40 50 60

(Flash Count - 1)

Pro

bab

ility

Neg Bin (Null M odel)

Poisson (Null M odel)

Observed Frequency

Mean = 23.15Theta = 0 .3420


Negative Binomial RegressionNegative Binomial Regression• NB distribution has been used in previous studies to model

thunderstorms at KSC.• No known study has used the NB as the probability model for

lightning counts.• Since the count distribution is left-truncated at y=1, we can treat

y-1 as having a NB distribution.• Probabilities for each y-1 must sum to 1. • Probability of exceeding any count threshold T :

• NB distribution has been used in previous studies to model thunderstorms at KSC.

• No known study has used the NB as the probability model for lightning counts.

• Since the count distribution is left-truncated at y=1, we can treat y-1 as having a NB distribution.

• Probabilities for each y-1 must sum to 1. • Probability of exceeding any count threshold T :

Ty

T

y

yyTy1

1

)Pr(1)Pr()Pr(


Regionalized ApproachRegionalized Approach• Domain first divided into 9

areas- separate models developed for each.

• Best results achieved by consolidating 9 areas into 4 larger regions:

-- East Coast

-- West Coast

-- Panhandle

-- Alabama & Georgia• Regions overlap to

minimize problems at regional boundaries.

• Domain first divided into 9 areas- separate models developed for each.

• Best results achieved by consolidating 9 areas into 4 larger regions:

-- East Coast

-- West Coast

-- Panhandle

-- Alabama & Georgia• Regions overlap to

minimize problems at regional boundaries.


Final Candidate PredictorsFinal Candidate Predictors• Long list of candidate predictors

contains redundant information.

• Principal component analysis used to select subset of predictors with less mutual correlation.

• Correlations with lightning predictands are low- no single observed predictor is good indicator of lightning.

• Power terms and cross products (interactions) also calculated and included in final predictor pool.

• 3-h change in each parameter also included (trend indicators).

• Map type predictors

• Climatological predictors

• Long list of candidate predictors contains redundant information.

• Principal component analysis used to select subset of predictors with less mutual correlation.

• Correlations with lightning predictands are low- no single observed predictor is good indicator of lightning.

• Power terms and cross products (interactions) also calculated and included in final predictor pool.

• 3-h change in each parameter also included (trend indicators).

• Map type predictors

• Climatological predictorsCorrelations for East Coast region1800-2059 UTC period

Correlations for East Coast region1800-2059 UTC period


Equation DevelopmentEquation Development

• Combination of forward stepwise selection and cross-validation used to develop BLR and NB models.

• Even years (2002 and 2004) used as “learning” sample.

• Odd years (2003 and 2005) used as “evaluation” sample.

• Procedure identifies best combination of predictors that is most likely to generalize to independent data, and not over-fit the dependent sample.

• Models containing only climatology and persistence (L-CLIPER) also developed as a benchmark for assessing forecast skill.

• Combination of forward stepwise selection and cross-validation used to develop BLR and NB models.

• Even years (2002 and 2004) used as “learning” sample.

• Odd years (2003 and 2005) used as “evaluation” sample.

• Procedure identifies best combination of predictors that is most likely to generalize to independent data, and not over-fit the dependent sample.

• Models containing only climatology and persistence (L-CLIPER) also developed as a benchmark for assessing forecast skill.


5. Model Parameters5. Model Parameters


Model ParametersModel Parameters

BLR models for one or more CG flashesBLR models for one or more CG flashes

NB models for the amount of lightningNB models for the amount of lightning

Most important parameters for 1800-2059 UTCMost important parameters for 1800-2059 UTC






MoistureMoisture


MoistureMoisture

• Deep layer moisture most favorable large-scale environment for Florida thunderstorms.

• Peak in FREQ1 for PRECPW ~ 5.5 cm.• Largest PRECPW usually associated with

tropical influences.

• Deep layer moisture most favorable large-scale environment for Florida thunderstorms.

• Peak in FREQ1 for PRECPW ~ 5.5 cm.• Largest PRECPW usually associated with

tropical influences.

Frequency of One or More Flashes vs. Precipitable Water1800-2059 UTC period

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

1.00 2.00 3.00 4.00 5.00 6.00 7.00

Precipitable Water (cm)

Fre

qu

en

cy

of

On

e o

r M

ore

Fla

sh

es

Unconditional Mean Number of Flashes vs. K-index1800-2059 UTC period

0

1

2

3

4

5

6

7

8

9

10

-10.0 0.0 10.0 20.0 30.0 40.0

K-index (deg C)

Un

co

nd

itio

na

l M

ea

n N

o.

of

Fla

sh

es






InstabilityInstability


InstabilityInstability

• Likelihood of one or more flashes and the amount of lightning increases with decreasing BESTLI.

• Sufficient instability leading to a strong updraft is necessary for charge generation.

• Layer CAPE and SSI selected for other periods.

• Likelihood of one or more flashes and the amount of lightning increases with decreasing BESTLI.

• Sufficient instability leading to a strong updraft is necessary for charge generation.

• Layer CAPE and SSI selected for other periods.

Frequency of One or More Flashes vs. Best Lifted Index1800-2059 UTC period

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

-12.0 -10.0 -8.0 -6.0 -4.0 -2.0 0.0 2.0 4.0 6.0

Best Lifted Index (deg C)

Fre

qu

en

cy

of

On

e o

r M

ore

Fla

sh

es

Unconditional Mean Number of Flashes vs. Best Lifted Index1800-2059 UTC period

0

1

2

3

4

5

6

7

8

9

10

-12.0 -10.0 -8.0 -6.0 -4.0 -2.0 0.0 2.0 4.0 6.0

Best Lifted Index (deg C)

Un

co

nd

itio

na

l M

ea

n N

o.

of

Fla

sh

es






Near surface (1000 hPa) forcingNear surface (1000 hPa) forcing


Near Surface ForcingNear Surface Forcing

• Low-level moisture flux convergence due to the sea breeze, lake/river breezes, outflows, etc.

• Weak non-linear effect--lightning that occurs in divergent stratiform regions.

• Low-level moisture flux convergence due to the sea breeze, lake/river breezes, outflows, etc.

• Weak non-linear effect--lightning that occurs in divergent stratiform regions.

Frequency of One or More Flashes vs. 1000 hPa Moisture Flux Convergence


0.10

0.15

0.20

0.25

0.30

0.35

-10.00 -5.00 0.00 5.00 10.00 15.00

1000 hPa Moisture Flux Convergence (x 10-7 kg kg-1s-1)

Fre

qu

en

cy

of

On

e o

r M

ore

Fla

sh

es

Unconditional Mean Number of Flashes vs. 1000 hPa Moisture Flux Convergence


2

3

4

5

6

7

8

9

-10.00 -5.00 0.00 5.00 10.00 15.00

1000 hPa Moisture Flux Convergence (x 10 -7 kg kg-1s-1)

Un

co

nd

itio

na

l M

ea

n N

o.

of

Fla

sh

es






Prevailing low-level windPrevailing low-level wind


Prevailing Low-Level WindPrevailing Low-Level Wind

• Highly non-linear relationship. • Peak lightning for offshore speeds between 2 - 4 m s-1

• Offshore flow produces better developed sea breeze and greater convergence.

• Highly non-linear relationship. • Peak lightning for offshore speeds between 2 - 4 m s-1

• Offshore flow produces better developed sea breeze and greater convergence.

Frequency of One or More Flashes vs. 1000-700 hPa Mean U-wind Component

1800-2059 UTC: East Coast Region

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

-12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12

1000-700 hPa Mean U-wind Component (m s-1)

Fre

qu

ency o

f O

ne o

r M

ore

Fla

sh

es

Unconditional Mean Number of Flashes vs. 1000-700 hPa Mean U-wind Component

1800-2059 UTC: East Coast Region

0

2

4

6

8

10

12

-12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12

1000-700 hPa Mean U-wind Component (m s-1)

Un

co

nd

itio

nal M

ean

No

. o

f F

lash

es


6. Results for Dependent Data6. Results for Dependent Data


ReliabilityReliability

• Reliability is a measure of the quality of probabilistic forecasts.

• Forecast probabilities correspond well with observed frequencies.

• The forecasts “mean what they say.”

• Reliability for other time periods also is very good.

• Reliability is a measure of the quality of probabilistic forecasts.

• Forecast probabilities correspond well with observed frequencies.

• The forecasts “mean what they say.”

• Reliability for other time periods also is very good.

Forecasting one or more flashesAll regions combined

Forecasting one or more flashesAll regions combined



Forecasting ≥ 50th percentileAll regions combined





Verification ScoresVerification Scores• Probabilistic forecasts converted to deterministic for verification.

• Optimum probability thresholds determined for each model and time period based on several verification scores from 2 x 2 contingency tables:

• Probabilistic forecasts converted to deterministic for verification.

• Optimum probability thresholds determined for each model and time period based on several verification scores from 2 x 2 contingency tables:

• Critical Success Index- hit rate after removing correct “no” forecasts from consideration.

• We chose to maximize PSS- contribution to PSS by a correct forecast increases as the event becomes less likely.

• Critical Success Index- hit rate after removing correct “no” forecasts from consideration.

• We chose to maximize PSS- contribution to PSS by a correct forecast increases as the event becomes less likely.


Verification ScoresVerification Scores

• Scores for forecasting one or more flashes are very good--81% of events correctly forecast, with reasonable FAR and BR.

• Forecasting the amount of lightning is more difficult, with low predictability for 95th percentile events.

• Model CSI and PSS scores for all time periods are an improvement over L-CLIPER and persistence alone.

• Percent improvement in CSI greatest during most active periods.

• Scores for forecasting one or more flashes are very good--81% of events correctly forecast, with reasonable FAR and BR.

• Forecasting the amount of lightning is more difficult, with low predictability for 95th percentile events.

• Model CSI and PSS scores for all time periods are an improvement over L-CLIPER and persistence alone.

• Percent improvement in CSI greatest during most active periods.

Summary for Dependent DataSummary for Dependent Data


Example Probability ForecastExample Probability Forecast

• Forecasts based on RUC analyses.• Maps show expected diurnal trend in lightning.• Good agreement between forecast and verification.


Prob(≥ 1 flash)Prob(≥ 1 flash) Lightning Strike VerificationLightning Strike Verification

1200-1459 UTC 4 June 20041200-1459 UTC 4 June 2004 1200-1459 UTC 4 June 20041200-1459 UTC 4 June 2004


7. Results for Independent Data7. Results for Independent Data


Independent TestIndependent Test• Equations were applied to forecast output from several

mesoscale models during the 2006 warm season:

1. 1500 UTC NCEP 13-km RUC (RUC13):

-- Runs at the highest frequency of any NCEP model

2. 1200 UTC NCEP 12-km NAM-WRF:

-- Transitioned from Eta to WRF-NMM in June 2006

3. 1500 UTC High Resolution (4 km) WRF:

-- South Florida domain (NWS Miami)

-- Initialized with NCEP 1/12th degree SSTs and data from the Local Analysis and Prediction System (LAPS)

• Results are for 1 August - 30 September 2006.

• Equations were applied to forecast output from several mesoscale models during the 2006 warm season:

1. 1500 UTC NCEP 13-km RUC (RUC13):

-- Runs at the highest frequency of any NCEP model

2. 1200 UTC NCEP 12-km NAM-WRF:

-- Transitioned from Eta to WRF-NMM in June 2006

3. 1500 UTC High Resolution (4 km) WRF:

-- South Florida domain (NWS Miami)

-- Initialized with NCEP 1/12th degree SSTs and data from the Local Analysis and Prediction System (LAPS)

• Results are for 1 August - 30 September 2006.


What is LAPS?What is LAPS?• Diagnostic tool in AWIPS--produces a high resolution 3D analysis

of the atmosphere.

• Combines a background field (1-h forecast from AWIPS RUC40) with local data from a variety of observing systems:

-- Surface observations (e.g., ASOS, local mesonetworks)

-- Doppler radar reflectivity (Miami and Key West radars)

-- Satellites (for cloud hydrometeors)

-- Wind and temperature profilers

-- Aircraft

• Produces 3D diabatic analysis grids for WRF initialization.

• WRF initialized with LAPS produces better forecasts of the sea breeze and surface parameters (Bogenschutz 2004; Etherton and Santos 2006).

• Diagnostic tool in AWIPS--produces a high resolution 3D analysis of the atmosphere.

• Combines a background field (1-h forecast from AWIPS RUC40) with local data from a variety of observing systems:

-- Surface observations (e.g., ASOS, local mesonetworks)

-- Doppler radar reflectivity (Miami and Key West radars)

-- Satellites (for cloud hydrometeors)

-- Wind and temperature profilers

-- Aircraft

• Produces 3D diabatic analysis grids for WRF initialization.

• WRF initialized with LAPS produces better forecasts of the sea breeze and surface parameters (Bogenschutz 2004; Etherton and Santos 2006).


WRF-LAPSWRF-LAPS• WRF Environmental Modeling

System (“Workstation-WRF”)

• Runs for 19-30 September provided by Dr. Pablo Santos

• Runs for 1 Aug - 18 Sept produced at FSU using WRF-EMS package

• Same model configuration as used at NWS Miami.

• We did not compare results using different physics options or cumulus schemes (future research???)

• WRF Environmental Modeling System (“Workstation-WRF”)

• Runs for 19-30 September provided by Dr. Pablo Santos

• Runs for 1 Aug - 18 Sept produced at FSU using WRF-EMS package

• Same model configuration as used at NWS Miami.

• We did not compare results using different physics options or cumulus schemes (future research???)

• 4-km resolution• 1500 UTC initialization• Forecasts out to 12 h

• 4-km resolution• 1500 UTC initialization• Forecasts out to 12 h


Skill ScoresSkill Scores

• Scores are very encouraging!

• All models show positive skill through 2100-2359 UTC.

• WRF-LAPS generally outperforms RUC13 and NAM-WRF during most active period (1800-2059 UTC).

• Expected degradation in skill at longer forecast projections.





Forecasting One or More FlashesCritical Success Index

Forecasting One or More FlashesCritical Success Index











Forecasting One or More FlashesPeirce Skill Statistic

Forecasting One or More FlashesPeirce Skill Statistic



• RUC13 is best performer through 6 h for forecasting 95th percentile events.

• All models show positive skill relative to persistence alone through 2100-2359 UTC.

• 1800 UTC cycles (not examined) likely would show positive skill for the 0000-0259 UTC period.

• RUC13 is best performer through 6 h for forecasting 95th percentile events.

• All models show positive skill relative to persistence alone through 2100-2359 UTC.

• 1800 UTC cycles (not examined) likely would show positive skill for the 0000-0259 UTC period.

Forecasting ≥ 95th percentilePeirce Skill Statistic

Forecasting ≥ 95th percentilePeirce Skill Statistic



• Forecasts based on 1500 UTC WRF-LAPS.• Timing and placement of features is not perfect, but…• Generally good agreement between the forecasts and

the verification.

• Forecasts based on 1500 UTC WRF-LAPS.• Timing and placement of features is not perfect, but…• Generally good agreement between the forecasts and

the verification.

Prob(≥ 1 flash)Prob(≥ 1 flash) VerificationVerification

0-h WRF-LAPS forecast: 1500-1759 UTC 16 August 20060-h WRF-LAPS forecast: 1500-1759 UTC 16 August 2006

Prob(≥ 90th percentile)Prob(≥ 90th percentile)






• Forecasts based on 1500 UTC WRF-LAPS.• Timing and placement of features is not perfect, but...• Generally good agreement between the forecasts and

the verification.


the verification.







the verification.


the verification.


8. Summary & Conclusions8. Summary & Conclusions


Summary & ConclusionsSummary & Conclusions

• Four warm seasons of RUC analyses and NLDN data were used to develop a perfect prognosis scheme for forecasting CG lightning over Florida:

-- Binary logistic regression was used to develop equations giving the probability of one or more CG flashes.

-- Negative binomial models were used to forecast the amount of lightning, conditional on one or more flashes occurring.

• Pattern type predictors were developed to capture enhancements due to local forcing.

• Deep layer moisture, instability, near-surface forcing, and the prevailing low level wind were found to have the greatest influence on the likelihood of one or more flashes and amount of lightning.

• Four warm seasons of RUC analyses and NLDN data were used to develop a perfect prognosis scheme for forecasting CG lightning over Florida:

-- Binary logistic regression was used to develop equations giving the probability of one or more CG flashes.

-- Negative binomial models were used to forecast the amount of lightning, conditional on one or more flashes occurring.

• Pattern type predictors were developed to capture enhancements due to local forcing.

• Deep layer moisture, instability, near-surface forcing, and the prevailing low level wind were found to have the greatest influence on the likelihood of one or more flashes and amount of lightning.


Summary & ConclusionsSummary & Conclusions

• PP equations show forecast skill over L-CLIPER and persistence alone when applied to the dependent sample of RUC analyses, and during an independent test period using model forecasts.

• Results demonstrate that the scheme is model independent!

• WRF-LAPS generally performs the best during the most active lightning period (1800-2059 UTC).

• Exact timing and placement of lightning maxima is not perfect, but there generally is good agreement between the forecasts and the verification.

• Methodology is an enhancement to schemes already in use.

• Scheme will be used operationally by FP&L beginning in May.

• PP equations show forecast skill over L-CLIPER and persistence alone when applied to the dependent sample of RUC analyses, and during an independent test period using model forecasts.

• Results demonstrate that the scheme is model independent!

• WRF-LAPS generally performs the best during the most active lightning period (1800-2059 UTC).

• Exact timing and placement of lightning maxima is not perfect, but there generally is good agreement between the forecasts and the verification.

• Methodology is an enhancement to schemes already in use.

• Scheme will be used operationally by FP&L beginning in May.


Future WorkFuture Work• Temporal resolution can be increased by developing separate

equations for each hour--can be applied to hourly RUC forecasts.

• Plans are in place to incorporate the guidance into the Interactive Forecast Preparation System (IFPS) Graphical Forecast Editor (GFE) for use by forecasters at all Florida WFOs:

-- A forecaster can use output from one NWP model or a blend of two or more models to generate the lightning forecasts.

-- Forecasts can be accessed by the public through NWS web sites.

• Scheme may be expanded to other parts of the country.

• Bayesian framework may produce better results?

• Availability of higher resolution analyses and larger developmental sample should result in more robust parameter estimates.

• Temporal resolution can be increased by developing separate equations for each hour--can be applied to hourly RUC forecasts.

• Plans are in place to incorporate the guidance into the Interactive Forecast Preparation System (IFPS) Graphical Forecast Editor (GFE) for use by forecasters at all Florida WFOs:

-- A forecaster can use output from one NWP model or a blend of two or more models to generate the lightning forecasts.

-- Forecasts can be accessed by the public through NWS web sites.

• Scheme may be expanded to other parts of the country.

• Bayesian framework may produce better results?

• Availability of higher resolution analyses and larger developmental sample should result in more robust parameter estimates.


Questions?Questions?

Phillip E. Shafer Henry E. Fuelberg Florida State University April 4, 2007

Documents

Transcript of Phillip E. Shafer Henry E. Fuelberg Florida State University April 4, 2007