2012 Pre-Season Forecasts for the Stillaguamish River Chinook

EMPAR(Environmental Model Predicting Adult Returns)

January 14, 2012

Developed By: Jason Hall1 and Dr. Correigh Greene2

1HALL AND ASSOCIATES CONSULTING, INC.Jason@haaci.com

2NOAA NORTHWEST FISHERIES SCIENCE CENTERCorreigh.Greene@noaa.gov

2

Background: EMPAR Concept

• Return rates are driven by survival across multiple life stages– Unique environmental conditions are experienced during each life stage– Life stage specific environmental conditions can influence survival

• Forecasts that consider life-stage specific environmental conditions may provide better forecasts– EMPAR developed to provide an accurate and robust forecast model

that incorporates life-stage specific environmental conditions– Approach adapted from Greene et al. (2005)*– EMPAR development started with 2009 return year forecast

*Greene, C.M., D.W. Jensen, G.R. Pess, and E.A. Steel. 2005. Effects of environmental conditions during stream, estuary, and ocean residency on Chinook salmon return rates in the Skagit River, Washington. Transactions of the American Fisheries Society 134:1562-1581.

3Jun-

00

Jan-

01

Jul-0

1

Feb-

02

Sep-

02

Mar

-03

Oct

-03

Apr

-04

Nov

-04

May

-05

Dec

-05

EGGHATCH

PINK/CHUMQMAX

Delta

Near

Ocean1

Ocean2

Ocean3

Ocean4

FW

Age 3 Spawners

Age 4 Spawners

Age 5 Spawners

DOTEMPSAL

SSTPDOSOIUWISL

Age 2 Spawners

Background: Life stage concept

4

Background: Broodyear Model Concept

RY 2000Spawners (S)

2002 2003 2004 2005 2006 2007 2008

RY 2001Spawners (S)

RY 2002Spawners (S)

RY 2003Spawners (S)

Age 2Age 3Age 4Age 5

SPSt = spawners per spawner in year tNt = adult escapement in year tPx,t = proportion of age x in return year t

5

RY 2000Spawners (S)

2002 2003 2004 2005 2006 2007 2008

RY 2001Spawners (S)

RY 2002Spawners (S)

RY 2003Spawners (S)

Age 2Age 3Age 4Age 5

*Return rate calculated for each age class – Age 3 example shown here

Background: Age-Specific Model Concept

6

Background: EMPAR Updates

• Removed some environmental factors from consideration:– Infrequent data update schedule– Forecast years rely on estimated data

• Added 2009 and 2010 return years to model training set: – Increases sample size by almost 10%– Return year 2011 was used as sole test set

• Working with age-specific models only: – Removes errors associated with applying average age structure– Makes more sense from a biological standpoint

7

• Incorporated Principle Components Analysis (PCA):– Common factor analysis technique– Synthesize multiple factors within a life stage into primary components – Longer temporal patterns can be considered – More arbitrary than using actual factors, but is more robust– Allows trends in many factors within a life stage to be considered

Background: EMPAR Updates

8

PCA Approach:• PCA for Freshwater Life Stage (1989-2010)

– EGG, PKCM, HATCH, and QMAX– 62% of variance explained with first two components

• PCA for Delta/Nearshore Life Stage (1989-2010)– DO, TEMP, and SAL– 50% of variance explained with first two components

• PCA for Ocean Life Stage (1949-2010)– SST, UWI, PDO, SOI, and SL – 73% of variance explained with first two components

9

PCA Approach:• Linear regression models:

– Combination of PCA components and selected raw factors– 2 freshwater, 1 delta/near, and 2 ocean life stage factors– PCA components (representing multiple factors) count as 1 factor

• Over-parameterized model?– Significant increase in predictive power for key age groups– Describes complicated life cycle well– Several evaluation techniques indicate that these models are not over-

parameterized

10

Test Set Validation: SNOR Models

1994 1996 1998 2000 2002 2004 2006 2008 20100E+00

1E+05

2E+05

3E+05

4E+05

5E+05

6E+05

Return Year of First True Forecast

Mea

n Sq

uare

Err

or

*MSE decays as test set increases

*Similar patterns observed for factor coefficients

11

Results: PCA EMPAR Model SummariesPopulation Age R2 P-Value F-Statistic DF Factor1* Factor2* Factor3* Factor4* Factor5*

SHOR 2 0.47 0.083 2.474 5,14 (EGG) (QMAX) NEAR_PC1 OCEAN1_PC1 OCEAN1_PC2

SHOR 3 0.65 0.009 4.929 5,13 (EGG) (HATCH) NEAR_PC1 (OCEAN1_PC1) OCEAN1_PC2

SHOR 4 0.46 0.138 2.084 5,12 (EGG) QMAX NEAR_PC1 OCEAN1_PC1 OCEAN1_PC2

SHOR 5 0.39 0.302 1.388 5,11 (EGG) (QMAX) DELTA_DO OCEAN1_PC1 OCEAN2_PC1

SHOR Total** r=0.77                

SNOR 2 0.49 0.065 2.701 5,14 (EGG) QMAX DELTA_DO OCEAN1_PC1 (OCEAN1_PC2)

SNOR 3 0.65 0.010 4.916 5,13 (EGG) (QMAX) DELTA_DO (OCEAN1_PC1) OCEAN2_PC1

SNOR 4 0.45 0.163 1.930 5,12 (EGG) (QMAX) NEAR_PC1 OCEAN1_PC1 OCEAN2_PC1

SNOR 5 0.48 0.157 2.000 5,11 (EGG) (QMAX) (NEAR_DO) OCEAN2_PC1 (OCEAN3_PC1)

SNOR Total** r=0.78                

FNOR 2 0.43 0.126 2.099 5,14 (EGG) QMAX NEAR_PC1 (NEAR_PC2) OCEAN1_PC1

FNOR 3 0.59 0.025 3.772 5,13 (EGG) (QMAX) (NEAR_PC1) (OCEAN1_PC1) OCEAN2_PC1

FNOR 4 0.39 0.257 1.515 5,12 (EGG) (QMAX) (NEAR_DO) OCEAN1_PC1 OCEAN1_PC2

FNOR 5 0.63 0.031 3.780 5,11 (EGG) (QMAX) DELTA_DO OCEAN2_PC1 (OCEAN3_PC1)

FNOR Total** r=0.61                

* Factors in parentheses have negative coefficients** Pearson’s correlation calculated based on sum of predicted returns of each age class by return year and observed escapement

12

Results: SHOR Model Output

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20120

500

1000

1500

2000

2500Ob-servedupr 95EMPARlwr 95

Return Year

Esca

pem

ent

13

Results: SNOR Model Output

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20120

200

400

600

800

1000

1200

1400

1600

1800Ob-servedupr 95EMPAR

Return Year

Esc

apem

ent

14

Results: FNOR Model Output

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 20120

100

200

300

400

500

600

700

800

900Observedupr 95EMPARlwr 95

Return Year

Esca

pem

ent

15

EMPAR Performance: Previous Models

Model SNOR 2009

SNOR 2010

SNOR 2011

SHOR 2009

SHOR 2010

SHOR 2011

FNOR 2009

FNOR 2010

FNOR 2011

AIC V1 464 530 509 361 517 502 131 180 148

AIC V2 632 714 899 635 1009 1128 250 305 341

PCA 395 473 428 547 518 693 140 97 120

Observed 388 352 276 570 412 738 44 20 100

= Best

= Middle

= Worst

• With return years 2009 – 2011 as test sets:– Derived forecast from all three selected EMPAR models– PCA model shows best track record when compared on

equal terms

16

• Use EMPAR models that incorporate Principle Components Analysis (PCA):– Forecast trends track well with observed trends– Factor sensitivity does not appear to be an issue as compared to

the full permutation AIC based approaches– Forecast performance comparisons indicate that the PCA model has

better predictive accuracy– PCA model does not appear to be over-parameterized and training set

appears valid– More arbitrary than using actual factors, but is a more statistically

robust procedure– Allows consideration of trends in multiple factors within a life stage

Recommendations:

17

Results: 2012 Forecast

Age 2 Age 3 Age 4 Age 5 Total 

SNOR 28 126 180 5 338

FNOR 1 6 77 1 86

SHOR 79 136 325 41 580

Age 2 Age 3 Age 4 Age 5 Total 

SNOR 40 179 256 7 481

FNOR 2 9 110 2 122

SHOR 112 193 463 58 827

Escapement with Fishing

Escapement without Fishing (assumes average exploitation rates)

18

Results: 2012 Forecast FRAM Conversion

Age 2 Age 3 Age 4 Age 5 Total 

SNOR 738 592 198 5 1534

FNOR 26 28 85 1 140

SHOR 2083 639 357 41 3121

Age 2 Age 3 Age 4 Age 5 Total 

SNOR 1230 846 247 6 2330

FNOR 44 40 106 1 191

SHOR 3472 913 447 46 4878

FRAM Input MM Run

FRAM Recruits

19

EMPAR Supporting Information:

The following slides are supplemental information to support the presentation and detailed questions…

20

PCA Example: Delta/Nearshore Life Stage PCA

Component Variance Explained

Cumulative Variance

Delta DO

Delta SAL

Delta TEMP

Near DO

Near SAL

Near TEMP

1 0.27 0.27 - + - - +

2 0.23 0.50 - - -

3 0.20 0.70 + - - + - -

4 0.14 0.84 + - - + -

5 0.08 0.92 - - + +

6 0.08 1.00 - - + + -

21

PCA Example: Ocean Life Stage PCA

Component Variance Explained

Cumulative Variance SOI SL SST UWI PDO

1 0.49 0.49 + - - + -

2 0.24 0.73 - + - - -

3 0.15 0.88 - + +

4 0.07 0.96 - - - - +

5 0.04 1.00 + + - + +

0 100 200 300 400 500 6000

200

400

600

800

1000

1200

f(x) = 1.08714300474013 x + 65.2866563127376R² = 0.370966047733079

f(x) = 1.3834344925073 x + 92.0374287903067R² = 0.599812688008294

f(x) = 1.28117172740322 x + 91.7906305117302R² = 0.608144549939346

SNOR

Linear (SNOR)

SHOR

Linear (SHOR)

FNOR

Linear (FNOR)

Predicted Escapement

Obs

erve

d E

scap

emen

t

22

EMPAR Validation: Forecast Compensation

1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3Interceptlog(FW_EGG)log(FW_QMAX)log(Delta_DO)Ocean1_PC1Ocean1_PC2

Return Year of First True Forecast Year

Prop

ortio

nal C

hang

e in

Coe

ffic

ient

s

23

Test Set Validation: SNOR Age 3 Model

*Coefficient change decays as test set increases

24

Background: Model Structures

• Broodyear Model:– Simplest model structure – Calculate return rates for each broodyear– One model for all spawners produced from each broodyear– Separate model for SNOR, SHOR, and FNOR– Allocate predicted returns by average age structure

• Age-Specific Model:– More complicated model structure– Calculate return rates for each age class by broodyear– Separate model for each age class– Separate model for SNOR, SHOR, and FNOR

25

Background: Life stage factors • Freshwater (Aug – Feb)

– Egg deposition– Pink and Chum escapement – Hatchery Releases– Max incubation flow– Min spawning flow

• Delta (Feb – Jun)– Surface DO– Surface Temp– Surface Salinity– Sea Level

• Nearshore (Jun – Oct)– Surface DO– Surface Temp– Sea Level– Upwelling Index

• Ocean Year 1 – 4 (Oct – Sep)– Sea Surface Temperature– Upwelling Index – Pacific Decadal Oscillation– Southern Oscillation Index– Sea Level– Aleutian Low Pressure Index*– SVI boreal copepod*– SVI southern copepod*

*Removed from candidate list

26

Background: Life stage factors

27

Background: EMPAR Approaches

• Several model selection and model development approaches have been considered during the development of EMPAR: – Full permutation models with Akaike's Information

Criterion score (AICc) model selection– Stepwise regression techniques– Principle Components Analysis (PCA) based approach

28

EMPAR Approaches: AIC Models

• Full permutation models with Akaike's Information Criterion score (AICc) model selection – Multiple models provide information about dependent variables– The best models are those that have strong predictive power

but use fewer independent variables– AIC scores models based on their ability to reduce uncertainty

but penalizes by the number of variables in the model– Not sensitive to the order variables enter as in stepwise regressions

• Model structure caveats – Large test model sets increases risk of selecting randomly

correlated models– Sensitivity to collinearities were initially a problem, but were

subsequently resolved in later models– Forecast outputs show sensitivity to variations in strong factors,

but were more accurate than stepwise regression models

29

EMPAR Approaches: Stepwise Regression

• Stepwise regression model selection– Common and well established approach– An aggressive fitting technique that can be overly greedy

• Model structure caveats – Sensitive to factor order– Favors models with fewer factors, and therefore does not

consider all life stages – Stepwise regression approaches appear to produce less accurate

forecasts despite the caveats associated with the full permutation AIC approach

30

EMPAR Approaches: PCA

• Principle Components Analysis– Common factor analysis technique– Reduces the number of variables and detects structure within

a set of factors– Can be used to synthesize multiple factors within a life stage

into primary components – Longer temporal patterns can be considered since components

can be derived independently• Model Structure Caveats

– Models using PCA components can be more conservative– Interpretation of the influence of factors within components

is not as direct as in AIC or stepwise regression techniques

31

EMPAR: Previous Model Forecasts

Population Forecast Year

Observed Escapement

Forecasted Escapement as Presented

SNOR 2009 388 697

SHOR 2009 570 202

FNOR 2009 44 131

SNOR 2010 352 701

SHOR 2010 412 551

FNOR 2010 20 116

SNOR 2011 276 534

SHOR 2011 738 799

FNOR 2011 100 26