A Statistical and Programmatic Review of the Idaho Supplementation Studies

1 Jeffrey Lutch, Sr. Fishery Research Biologist, Idaho Dept. of Fish and Game, 1414 East Locust Lane,Nampa, ID 83709.

J Lockhart, Fishery Biologist, Nez Perce Tribe. PO Box 1942, McCall, ID 83638

C. Beasly FishPro, a Division of HDR, Inc., 3780 S.E. Mile Hill Drive, Port Orchard, WA 98366.

K. Steinhorst, University of Idaho, Statistical Consulting Center, Moscow ID, 83844.

Idaho Dept of Fish and GameU.S. Fish and Wildlife ServiceNez Perce TribeShoshone-Bannock Tribes

Cooperating agencies

Bonneville Power Administration

Evaluate the benefits and risks of supplementing spring and summer Chinook salmon stocks returning to Idaho

Project Goal

Supplementation effects on natural production

Changes in natural productivity

Identify effective supplementation strategies

Recommendations

Research Objectives

Changes in genetic composition

M&E

Evaluate the benefits and risks of supplementing spring and summer Chinook salmon stocks returning to Idaho

Project Goal

Supplementation effects on natural production

Changes in natural productivity

Identify effective supplementation strategies

Recommendations

Research Objectives

Changes in genetic composition

M&E

Experimental comparison of production and productivity between treated and control streams

Experimental Design (Bowles and Leitzinger, 1991)

Evaluation Levels

• Stream specific

• Broad scale

• Small-scale

Phase I – Baseline data, broodstock development, M&E

Phase II – Treatment, M&E

Phase III – Evaluation, M&E

Proposed Study Phases

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review2002 - 2004

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review2002 - 2004

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses applied through Phase III

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review2002 - 2004

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review2002 - 2004

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review2002 - 2004

Determined geographic/temporal coverageDocumented inconsistencies

Compiled cooperative data

• Weir counts • Redd counts• Carcasses• Juvenile abundance• Juvenile survival

Data Set Review

Evaluated treatment levels

Determined geographic/temporal coverageDocumented inconsistencies

Data Set Review

Evaluated treatment levels

Compiled cooperative data

• Weir counts • Redd counts• Carcasses• Juvenile abundance• Juvenile survival

Evaluated treatment levels

Data Set Review

Staggered / Incomplete Treatment - Low adult escapement in Phase I

Determined geographic/temporal coverageDocumented inconsistencies

Compiled cooperative data

• Weir counts • Redd counts• Carcasses• Juvenile abundance• Juvenile survival

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review

Objective 1. Define the extent of straying

Carcass data to predict straying rates• Naturally produced (unmarked)• Hatchery supplementation (uniquely marked)• Hatchery general production (uniquely marked)

Treatments streams: General production

Control streams: General productionISS production

Straying was widespread and often substantial • Detected in all but one study stream • Exceeded 50% in some treated streams

Objective 1. Define the extent of straying

Carcass data to predict straying rates• Naturally produced (unmarked)• Hatchery supplementation (uniquely marked)• Hatchery general production (uniquely marked)

Treatments streams: General production

Control streams: General productionISS production

Objective 2. Determine stray / production relationship

• Calculated redd production attributable to strayingStray proportions x redds / kilometer

• Selected streams with the best carcass data 11 streams (6 control and 5 treated)

• Plotted redd densities as function of strayingStrays per kilometer vs redd densities

Objective 2. Determine stray / production relationship

• Calculated redd production attributable to strayingStray proportions x redds / kilometer

• Selected streams with the best carcass data 11 streams (6 control and 5 treated)

• Plotted redd densities as function of strayingStrays per kilometer vs redd densities

Objective 2. Determine stray / production relationship

• Calculated redd production attributable to strayingStray proportions x redds / kilometer

• Selected streams with the best carcass data 11 streams (6 control and 5 treated)

• Plotted redd densities as function of strayingStrays per kilometer vs redd densities

Objective 2. Determine stray / production relationship

• Calculated redd production attributable to strayingStray proportions x redds / kilometer

• Selected streams with the best carcass data 11 streams (6 control and 5 treated)

• Plotted redd densities as function of strayingStrays per kilometer vs redd densities

00.5

11.5

22.5

33.5

0 0.2 0.4 0.6 0.8 1

Straying

Red

d D

ensi

ty Relationship • Contributing to redd production

Objective 3. Determine the statistical effect of straying

Repeated measures statistical design (ANCOVA)5 treated 6 control streamsDependent variable = Redd densitiesCovariate = Stray reddsTreatment x Time interaction

Objective 3. Determine the statistical effect of straying

Repeated measures statistical design (ANCOVA)5 treated 6 control streamsDependent variable = Redd densitiesCovariate = Stray reddsTreatment x Time interaction

• Straying insignificant• Intensify monitoring for ISS

ANCOVA = Non-significantANOVA = Non-significant

• Straying effects (ANCOVA)

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review

Statistical Adaptations

• Grouped streams on geographic/habitat similarities

• Re-categorized stream designations

• Created partial treatment category• Streams receiving <50% treatment

• Re-coded time variable• Compensated for staggered treatments and responses

Study deviations

Prescribed treatments not achieved

Treatments staggered

Mixed Model Analysis (ANOVA)

Statistical Adaptations

• Grouped streams on geographic/habitat similarities

• Re-categorized stream designations

• Created partial treatment category• Streams receiving <50% treatment

• Re-coded time variable• Compensated for staggered treatments and responses

Study deviations

Prescribed treatments not achieved

Treatments staggered

Mixed Model Analysis

Statistical Adaptations

• Grouped streams on geographic/habitat similarities

• Re-categorized stream designations

• Created partial treatment category• Streams receiving <50% treatment

• Re-coded time variable• Compensated for staggered treatments and responses

Study deviations

Prescribed treatments not achieved

Treatments staggered

Mixed Model Analysis

Statistical Adaptations

• Grouped streams on geographic/habitat similarities

• Re-categorized stream designations

• Created partial treatment category• Streams receiving <50% treatment

• Re-coded time variable• Compensated for staggered treatments and responses

Study deviations

Prescribed treatments not achieved

Treatments staggered

Mixed Model Analysis

Statistical Adaptations

• Grouped streams on geographic/habitat similarities

• Re-categorized stream designations

• Created partial treatment category• Streams receiving <50% treatment

• Re-coded time variable• Compensated for staggered treatments and responses

Study deviations

Prescribed treatments not achieved

Treatments staggered

Mixed Model Analysis

Re-coding Time for Evaluating ISS

1994 1995 1996 1997 1998 1999 2000

Treatments Calendar Year

1994 1995 1996 1997 1998 1999 2000

- - - X X X XStream X

(Treated BY 1993)

Treatments Calendar Year

Re-coding Time for Evaluating ISS

1994 1995 1996 1997 1998 1999 2000

- - - X X X X

- - - - - X X

Stream X(Treated BY 1993)

Stream Y(Treated BY 1996)

Treatments Calendar Year

Re-coding Time for Evaluating ISS

- - - X X X X

- - - - - X X

Stream X(Treated BY 1993)

Stream Y(Treated BY 1996)

- 2 - 1 0 1 2 3 4

- 4 - 3 - 2 - 1 0 1 2

Stream X(Treated BY 1993)

Stream Y(Treated BY 1996)

Treatments Response Year

Re-coding Time for Evaluating ISS

- - - X X X X

- - - - - X X

Stream X(Treated BY 1993)

Stream Y(Treated BY 1996)

- 2 - 1 0 1 2 3

- 2 - 1 0 1 2 3

- 2 - 1 0 1 2 3 4

- 4 - 3 - 2 - 1 0 1 2

Stream X(Treated BY 1993)

Stream Y(Treated BY 1996)

Stream X(Treated BY 1993)

Stream Y(Treated BY 1996)

Treatments Response Year

Re-coding Time for Evaluating ISS

Statistical Prototype Model

Class Levels Values

Blocks 9 Major sub-basins

Treatment 3 Treated, Partial, Control

Stream 30 5 Treatment, 11 Partial, 14 control

Time 2 1, 2

Year 16 ….. -3, -2, -1, 0, 1, 2, 3…….

Dependent Variable = Log transformed redds/km

Mixed Model Results

• Power to detect a response in Phase II• Prototype analysis compensates for study deviations• Experimental analysis feasible through Phase III

Effect Numerator DF Denominator DF F-Value Pr>F

Treatment 2 19 0.82 0.4572

Time 1 14 4.15 0.0610

Treatment x Time 2 240 18.11 <0.0001 **

00.1

0.20.3

0.40.5

0.6

0 1 2 3

Time

Me

an

re

dd

de

ns

ity Control

Partial

Treatment

Regression Model

• Grouped streams on geographic/habitat similarities

• Quantified treatment (levels)

• Applied stray percentage

Regression Analysis

Rationale

Model the effects of strays/treatment

Straying widespread (all streams treated at some level)

Regression Model

• Grouped streams on geographic/habitat similarities

• Quantified treatment (levels)

• Applied stray percentage

Regression Analysis

Rationale

Model the effects of strays/treatment

Straying widespread (all streams treated at some level)

Regression Model

• Grouped streams on geographic/habitat similarities

• Quantified treatment (levels)

• Applied stray percentage

Regression Analysis

Rationale

Model the effects of strays/treatment

Straying widespread (all streams treated at some level)

Regression Analysis

Rationale

Model the effects of strays/treatment

Straying widespread (all streams treated at some level)

Regression Model

• Grouped streams on geographic/habitat similarities

• Quantified treatment (levels)

• Applied stray percentage

Regression Analysis

Rationale

Model the effects of strays/treatment

Straying widespread (all streams treated at some level)

Regression Model

• Grouped streams on geographic/habitat similarities

• Quantified treatment (levels)

• Applied stray percentage

Dependent Variables = Redds/km, Juvenile abundance/survival

Regression Analysis

Variable DF Estimate Error t-Value Pr>F

Intercept 1 0.54860 0.06759 8.12 0.0001

Treatment % 1 0.61024 0.15702 3.89 0.0001

Straying 1 0.91322 0.18545 4.92 0.0001

G8 1 0.95534 0.15145 6.31 <.0001

• Production increases with treatment/straying• Regression approach feasible through Phase III

Regression Analysis

Variable DF Estimate Error t-Value Pr>F

Intercept 1 0.54860 0.06759 8.12 0.0001

Treatment % 1 0.61024 0.15702 3.89 0.0001

Straying 1 0.91322 0.18545 4.92 0.0001

G8 1 0.95534 0.15145 6.31 <.0001

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analyses

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review

Results

Statistical power maintained when 25% of streams removed

Study Deviations

Research / management conflicts

Numerical power analysis

25% of streams removed in four test simulationsPrototype analysis appliedResults compared to baseline prototype

Loss of streams may not compromise statewide evaluations but effects on other comparisons need to be examined

Results

Statistical power maintained when 25% of streams removed

Study Deviations

Research / management conflicts

Numerical power analysis

25% of streams removed in four test simulationsPrototype analysis appliedResults compared to baseline prototype

I. Data set and study stream reviewTreatment level

2. StrayingEffect on ISS evaluation

3. Statistical evaluation Prototype analysis

4. Statistical power of ISSEffect of losing study streams

5. Programmatic recommendationsTreatment, M&E, statistical analysis

Programmatic and Statistical Review

Statistical Analysis• Prototype for statewide evaluation• Model other effects

Monitoring and Evaluation• Intensify carcass surveys • Increased precision

Redefine Phase II and III timeline

ISS Programmatic Adaptations

DNA Microsatellite Analysis• Separate stray production

Project Coordination• Project website• Juvenile / adult database• Work statements• Reporting

Redefined study design• Data types meet objectives

ISS Original Study Timeline

1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

Baseline data/genetics collection

Brood stock development

F1 Juvenile releases

JuvenileRelease

Adult return, supplementation

Adult return, supplementation, F2 brood

Control &Treatment

StreamMonitoring

Adapted Timeline

1990

1992

1994

1996

1998

2000

2002

2004

2006

2008

Baseline data/genetics collection

Brood stock development

F1 Juvenile releases

JuvenileRelease

Adult return, supplementation

Adult return, supplementation, F2 brood

Control &Treatment

StreamMonitoring

2007

2012

2004

Statistical Analysis• Prototype for statewide evaluation• Model other effects

Monitoring and Evaluation• Intensify carcass surveys • Increased precision

Redefine Phase II and III timeline

ISS Programmatic Adaptations

DNA Microsatellite Analysis• Separate stray production

Project Coordination• Project website• Juvenile / adult database• Work statements• Reporting

Redefined study design• Data types meet objectives

ISS Programmatic Adaptations

Statistical Analysis• Prototype for statewide evaluation• Model other effects

Monitoring and Evaluation• Intensify carcass surveys • Increased precision

Redefine Phase II and III timeline

DNA Microsatellite Analysis• Separate stray production

Project Coordination• Project website• Juvenile / adult database• Work statements• Reporting

Redefined study design• Data types meet objectives

ISS Programmatic Adaptations

Statistical Analysis• Prototype for statewide evaluation• Model other effects

Monitoring and Evaluation• Intensify carcass surveys • Increased precision

Redefine Phase II and III timeline

DNA Microsatellite Analysis• Separate stray production

Project Coordination• Project website• Juvenile / adult database• Work statements• Reporting

Redefined study design• Data types meet objectives

ISS Programmatic Adaptations

Statistical Analysis• Prototype for statewide evaluation• Model other effects

Monitoring and Evaluation• Intensify carcass surveys • Increased precision

Redefine Phase II and III timeline

DNA Microsatellite Analysis• Separate stray production

Project Coordination• Project website• Juvenile / adult database• Work statements• Reporting

Redefined study design• Data types meet objectives

ISS Programmatic Adaptations

Statistical Analysis• Prototype for statewide evaluation• Model other effects

Monitoring and Evaluation• Intensify carcass surveys • Increased precision

Redefine Phase II and III timeline

DNA Microsatellite Analysis• Separate stray production

Project Coordination• Project website• Juvenile / adult database• Work statements• Reporting

Redefined study design• Data types meet objectives

Questions ??