Post on 02-Jan-2016
description
Natural Resource Consultants Since 1921
Model ComparisonModel Comparisonfor short-term growth for short-term growth
projections on a Western projections on a Western Oregon propertyOregon property
Model ComparisonModel Comparisonfor short-term growth for short-term growth
projections on a Western projections on a Western Oregon propertyOregon property
GMUG MeetingGMUG Meeting
June 2, 2010June 2, 2010
Natural Resource Consultants Since 1921
G&Y Models G&Y Models SurveySurvey
* PACIFIC NORTHWEST TIMBERLAND INVESTMENT SURVEY RESULTS, MARCH 2010, SIZEMORE & SIZEMORE, INC.
• 15 Respondents (Industry + TIMOs + Consultants), multiple responses allowed
G&Y Model Used Percent
SPS/FPS 8 28%
FVS 7 24%
ORGANON 5 17%
In-House 5 17%
Other 2 7%
Excel 1 3%
DFSim 1 3%
*
Natural Resource Consultants Since 1921
Data - LocationData - Location
• Western Oregon
• South of Portland, North of Eugene
• Stands in Valley & up west slope of Cascades
Natural Resource Consultants Since 1921
Data - RemeasurementData - Remeasurement
Interval (Yrs) No Remeasurement No Thin Thinned* Grand Total
0 16 2 18
3 2 2
4 1 1
5 17 13 30
6 2 2
7 2 2
8 3 3
9 1 1 2
10 52 52
13 1 1
Grand Total 16 81 16 113
Count of Plots:
Natural Resource Consultants Since 1921
Data – Age & SIData – Age & SICount of Plots:
Site Category
Age Class HIGH MED LOW Grand Total
15 - 24 7 9 0 16
25 - 34 6 4 1 11
35 - 44 16 6 0 22
45 - 54 18 13 3 34
55 - 64 2 5 0 7
65 - 74 0 2 0 2
Grand Total 49 39 4 92
Natural Resource Consultants Since 1921
Site ClassesSite Classes
Site Category Site Index Assigned SI*
High Greater than 120 125
Medium Greater than 95 and less than or equal to 120
110
Low Less than or equal to 95 95
* Prior to 1997, SI class assigned based on growth rings
Natural Resource Consultants Since 1921
Stand Statistics (Init)Stand Statistics (Init)
TPA BA QMD Avg Ht Ht40 BFV
Count 92 92 92 92 92 92
Mean 238 164 11.7 74 87 20,490
Min 75 17 5.2 28 29 0
Max 620 289 20.1 121 129 48,129
StdDev 101 61 3.5 24 27 13,682
StdErr 10.6 6.4 0.36 2.5 2.8 1,426
StdErrPct 4.4% 3.9% 3.1% 3.4% 3.3% 7.0%
Natural Resource Consultants Since 1921
Merch SpecsMerch SpecsSpecification Value
Minimum DBH 7 inches
Stump Height 1 foot
Log Length 32 feet
Minimum Top DIB 5 inches
Minimum Log Length 16 feet
Log Rule Long Log
Trees Must Contain 1 Full Log No
Trim Amount 12 inches - fixed
Use Scribner Decimal C No
Hidden Defect 0
Natural Resource Consultants Since 1921
Tree StatisticsTree Statistics
• 4,382 trees across 15 species– DF (3,469), WH (604)
Natural Resource Consultants Since 1921
Growth ModelsGrowth Models
• SIS (SPS)
• FPS (v6.73)
• FVS (12/16/2008)– PN, WC
• ORGANON (v8.2)– NWO, SMC
Natural Resource Consultants Since 1921
Growth Models - OptionsGrowth Models - Options
• SIS (SPS)– Client coefficient file
• FPS – Library 11
• FVS– Max SDI: DF 600, WH 720, RA
300
• ORGANON (v8.2)– Max DF SDI = 600
Natural Resource Consultants Since 1921
Comparison MethodsComparison Methods
• Measured vs. Predicted
Average Growth
• Basic Error Statistics
• Error CDF’s
• Error Significance Tests
• Linear Regressions of Errors
• 50-Year Growth Projections
Natural Resource Consultants Since 1921
Comparison CaveatComparison Caveat
• Period length is not uniform across plots
• The same set of data was used across models
• Relative size of errors important
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
• High level view of growth
• PAI
• Focus on four youngest age classes due to sample size
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
Natural Resource Consultants Since 1921
Measured vs. Predicted Measured vs. Predicted Average GrowthAverage Growth
• Performance?– SIS, PN, FPS, NWO all about
same so far– SMC not as good– WC least accurate so far
Natural Resource Consultants Since 1921
Basic Error Stats - StandsBasic Error Stats - Stands
• Errors: Predicted - Observed– Negative = under-prediction– Positive = over-prediction
• Errors not annualized**
• Stand-level and Tree-level
• FPS – no grown treelist output
Natural Resource Consultants Since 1921
Basic Error Stats - StandsBasic Error Stats - Stands SIS FPS FVSPN FVSWC O-SMC O-NWO
TPA Mean 2.3 8.8 -3.5 6.8 10.6 7.1
StdDev 22.3 19.5 20.7 21.5 19.4 20.3
StdError 2.325 2.035 2.159 2.242 2.026 2.112
BA (ft2/ac)
Mean -1.6 -1.1 3.6 -21.3 -8.5 -4.5
StdDev 17.4 15.3 26.5 14.3 15.3 13.8
StdError 1.809 1.598 2.758 1.491 1.596 1.434
QMD (inches)
Mean -0.1 -0.4 0.3 -0.9 -0.6 -0.4
StdDev 0.71 0.52 0.91 0.57 0.46 0.47
StdError 0.074 0.054 0.095 0.059 0.048 0.049
HT40 (feet)
Mean -1.7 na -2.1 -8.5 -2.8 -2.6
StdDev 4.2 na 3.9 4.8 3.7 3.9
StdError 0.438 na 0.410 0.500 0.381 0.402
BFVOL (Bdft/ac)
Mean -1,426 -3,289 -1,581 -6,489 -2,596 -1,811
StdDev 3,397 3,425 4,233 4,008 2,925 2,557
StdError 354.1 357.1 441.3 417.9 305.0 266.6
Natural Resource Consultants Since 1921
Basic Error Stats - StandsBasic Error Stats - StandsTPA
SIS
FPS
FVSPN
FVSWC
O-SMC
O-NWO
-12
-8
-4
0
4
8
12
BA (sqft/ac)
SIS FPS
FVSPN
FVSWC
O-SMCO-NWO
-30
-20
-10
0
10
20
30
QMD (in.)
SISFPS
FVSPN
FVSWC
O-SMC
O-NWO
-1.0
-0.5
0.0
0.5
1.0
BFVol (bdft/ac)
SISFPS
FVSPN
FVSWC
O-SMCO-NWO
-8000
-6000
-4000
-2000
0
2000
4000
6000
8000
HT40 (ft.)
SIS FVSPN
FVSWC
FPS
- not
ava
ilabl
e
O-SMC O-NWO
-10
-5
0
5
10
FPS
- not
ava
ilabl
e
-10
-5
0
5
10
Natural Resource Consultants Since 1921
Basic Error Stats - TreesBasic Error Stats - Trees
SIS FPS FVSPN FVSWC O-SMC O-NWO
DBH
Mean 0.1 NA 0.7 -0.5 -0.4 -0.2
SD 0.9 NA 1.0 0.9 0.8 0.7
Std Err 0.014 NA 0.017 0.015 0.014 0.012
HEIGHT
Mean -0.6 NA -1.2 -6.5 -2.4 -2.5
SD 6.6 NA 7.5 8.0 6.6 6.5
Std Err 0.107 NA 0.123 0.130 0.107 0.106
BFVOL
Mean -8.5 NA 2.6 -32.3 -18.6 -12.1
SD 38.8 NA 36.0 48.9 37.3 32.4
Std Err 0.675 NA 0.626 0.849 0.649 0.562
Natural Resource Consultants Since 1921
Basic Error Stats - TreesBasic Error Stats - TreesDBH Errors
SIS
FVSPN
O-SMC
O-NWO
FVSWC
-1.0
-0.5
0.0
0.5
1.0
Height Errors
SIS FVSPN
FVSWC
O-SMC O-NWO
-10
-5
0
5
10
BFVol Errors
SIS
FVSPN
FVSWC
O-SMCO-NWO
-40
-20
0
20
40
Natural Resource Consultants Since 1921
Basic Error Stats - TreesBasic Error Stats - TreesDF: DBH Errors
SIS
FVSWC
O-NWO
FVSPN
O-SMC-1.0
-0.5
0.0
0.5
1.0
DF: Height Errors
SIS FVSPN
FVSWC
O-SMC O-NWO
-10
-5
0
5
10
DF: BFVol Errors
SIS
FVSPN
FVSWC
O-SMCO-NWO
-40
-20
0
20
40
WH: DBH Errors
SISFVSPN
FVSWC
O-SMC
O-NWO
-1.0
-0.5
0.0
0.5
1.0
WH: Height Errors
SISFVSWC
O-SMC
FVSPN
O-NWO
-10
-5
0
5
10
WH: BFVol Errors
SISFVSPN
FVSWCO-NWO
O-SMC
-40
-20
0
20
40
Natural Resource Consultants Since 1921
Basic Error StatsBasic Error Stats
• Performance?– SIS smallest mean errors for all
but BA (2nd smallest) and average variability
– PN probably second best choice– NWO was least variable– SMC and WC falling out of the
race
Natural Resource Consultants Since 1921
Error CDF’s - StandError CDF’s - Stand
• Cumulative Distribution Functions
• Picture of error distribution by error size
• More information than a mean bias
Natural Resource Consultants Since 1921
Error CDF’s - StandError CDF’s - Stand
Natural Resource Consultants Since 1921
Error CDF’s - StandError CDF’s - Stand
Natural Resource Consultants Since 1921
Error CDF’s - StandError CDF’s - Stand
Natural Resource Consultants Since 1921
Error CDF’s - StandError CDF’s - Stand
Natural Resource Consultants Since 1921
Error CDF’s - StandError CDF’s - Stand
Cumulative Distribution of Prediction Errors: BFVol
0%
20%
40%
60%
80%
100%
<= 5% <=10% <= 15% <= 20% <= 25% <= 100%
% Error in BdVol Prediction
% o
f Plo
ts
SIS
FPS
FVSPN
FVSWC
O-SMC
O-NWO
Natural Resource Consultants Since 1921
Error CDF’s - TreeError CDF’s - Tree
Natural Resource Consultants Since 1921
Error CDF’s - TreeError CDF’s - Tree
Natural Resource Consultants Since 1921
Error CDF’s - TreeError CDF’s - Tree
Cumulative Distribution of Prediction Errors: Tree BFVOL
0%
20%
40%
60%
80%
100%
<= 5% <=10% <= 15% <= 20% <= 25% <= 100%
% Error in Tree BFVol Prediction
% o
f Tre
es
SIS
FPS
FVSPN
FVSWC
O-SMC
O-NWO
Natural Resource Consultants Since 1921
Error CDF’s - TreeError CDF’s - Tree
• Performance?– NWO and SIS had largest % of
small errors– FPS and SMC next best, with
PN close behind– WC doesn’t seem to be a good
fit
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
• Paired t-test vs. Wilcoxon signed rank test– All data– 5-7 yr vs. 8-10 yr Projections– Initial Vol <= 25 MBF vs. Initial
Vol > 25 MBF– Unthinned vs. Thinned
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
p-values SIS FPS FVSPN FVSWC O-SMC O-NWO
TPA
All 0.2459 0.9503 0.0278 0.1497 0.6656 0.2015
5-7 yr Projections 0.8441 0.0529 0.0077 0.3755 0.5902 0.3088
8-10 yr Projections 0.0000 0.0001 0.0000 0.0000 0.0000 0.0000
Initial Vol < 25 MBF 0.0000 0.0506 0.0188 0.0050 0.0479 0.0188
Initial Vol > 25 MBF 0.0187 0.0061 0.4675 0.3694 0.0293 0.3786
UnThinned 0.5254 0.5550 0.1208 0.2195 0.8675 0.2408
Thinned 0.1981 0.2209 0.0480 0.5509 0.1981 0.5936
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
p-values SIS FPS FVSPN FVSWC O-SMC O-NWO
BA
All 0.1610 0.0398 0.0273 0.0000 0.0000 0.0023
5-7 yr Projections 0.0266 0.1347 0.0023 0.0000 0.0049 0.0686
8-10 yr Projections 0.0000 0.0000 0.0000 0.0000 0.0000 0.0557
Initial Vol < 25 MBF 0.5641 0.4096 0.3630 0.0000 0.0718 0.0067
Initial Vol > 25 MBF 0.1099 0.0673 0.0523 0.0001 0.0053 0.1687
UnThinned 0.6885 0.4298 0.1894 0.0000 0.3277 0.0105
Thinned 0.0157 0.0029 0.0008 0.0000 0.0157 0.6832
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
p-values SIS FPS FVSPN FVSWC O-SMC O-NWO
QMD
All 0.0525 0.0001 0.5513 0.0000 0.0000 0.0000
5-7 yr Projections 0.5668 0.2485 0.8441 0.0000 0.0000 0.0006
8-10 yr Projections 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000
Initial Vol < 25 MBF 0.8861 0.0107 0.1361 0.0000 0.0000 0.0073
Initial Vol > 25 MBF 0.0006 0.0001 0.0858 0.0000 0.0000 0.0000
UnThinned 0.2140 0.0013 0.3793 0.0000 0.0000 0.0000
Thinned 0.0843 0.0157 0.3967 0.0010 0.0010 0.0029
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
p-values SIS FPS FVSPN FVSWC O-SMC O-NWO
Height
All 0.0002 NA 0.0000 0.0000 0.0000 0.0000
5-7 yr Projections 0.9659 NA 0.4780 0.0000 0.2089 0.2701
8-10 yr Projections 0.0029 NA 0.0001 0.0000 0.0000 0.0000
Init Vol < 25 MBF 0.0049 NA 0.0002 0.0000 0.0000 0.0105
Init Vol > 25 MBF 0.4780 NA 0.1486 0.0000 0.0010 0.0008
UnThinned 0.0008 NA 0.0000 0.0000 0.0000 0.0000
Thinned 0.6378 NA 0.7299 0.0003 0.7299 0.8261
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
p-values SIS FPS FVSPN FVSWC O-SMC O-NWO
BD FT VOL
All 0.0987 0.0000 0.0005 0.0000 0.0000 0.0000
5-7 yr Projections 0.5214 0.0005 0.0019 0.0000 0.0000 0.0000
8-10 yr Projections 0.0154 0.0000 0.0200 0.0000 0.0000 0.0000
Initial Vol < 25 MBF 0.9414 0.0000 0.7521 0.0000 0.0000 0.0184
Initial Vol > 25 MBF 0.0000 0.0000 0.0000 0.0000 0.0544 0.0004
UnThinned 0.3767 0.0000 0.0065 0.0000 0.0000 0.0000
Thinned 0.0157 0.0010 0.0132 0.0000 0.0132 0.0076
Natural Resource Consultants Since 1921
Significance TestsSignificance Tests
• Performance so far?– Shorter projections generally
better (higher p-values)– Less significance in higher
volume stands than lower– Thinned vs. unthinned unclear– SIS and PN least significant
differences, followed by NWO
Natural Resource Consultants Since 1921
Simple Linear RegressionSimple Linear Regression
• Oi = β0 + β1*Pi + εi
• Nature of the bias if it is present, telling us whether it is constant or changing and to what degree.
Natural Resource Consultants Since 1921
SLR ResultsSLR ResultsTPA BA QMD HT BFVol
Model Parameter Value Sig Value Sig Value Sig Value Sig Value Sig
SISIntercept 7.45 No 21.98 Yes 1.53 Yes 0.94 No 1999.39 Yes
Slope 0.9763 No 0.8852 Yes 0.8779 Yes 0.9745 No 0.8902 Yes
FPSIntercept 12.18 Yes 5.08 No 0.46 Yes ------ --- 926.49 No
Slope 0.9844 No 0.9698 No 0.9404 Yes ------ --- 0.8649 Yes
FVS-PNIntercept 6.39 No 79.12 Yes 2.74 Yes 1.56 No 4830.75 Yes
Slope 0.9545 No 0.6326 Yes 0.8221 Yes 0.9646 Yes 0.7945 Yes
FVS-WCIntercept 6.87 No 7.32 No 0.22 No -11.43 Yes -571.86 No
Slope 0.9995 No 0.8607 Yes 0.9164 Yes 1.0281 No 0.8103 Yes
O-SMCIntercept 13.86 Yes 3.87 No 0.18 No -1.99 No -348.21 No
Slope 0.9849 No 0.9397 Yes 0.9401 Yes 0.9923 No 0.9280 Yes
O-NWOIntercept 11.02 Yes -0.42 No -0.01 No -0.96 No -395.82 No
Slope 0.9821 No 0.9801 No 0.9703 No 0.9837 No 0.9546 YesSignificance at α = 0.05
Natural Resource Consultants Since 1921
SLR ResultsSLR Results
• Performance so far?– NWO did best, especially when
recall low variability– FPS and SMC the next best– SIS and PN several significant
slopes and intercepts
Natural Resource Consultants Since 1921
Long-Term ProjectionsLong-Term Projections
• Long-term remeasurement data uncommon
• Grow young stands and plot against older inventory data
• Less for identifying a ‘best’ so much as pointing out models to avoid
Natural Resource Consultants Since 1921
Long-Term ProjectionsLong-Term Projections
Natural Resource Consultants Since 1921
Long-Term ProjectionsLong-Term Projections
Natural Resource Consultants Since 1921
Long-Term ProjectionsLong-Term Projections
Natural Resource Consultants Since 1921
Long-Term ProjectionsLong-Term Projections
• Performance so far?– NWO, SMC, FPS all reasonable in
long-term predictions– PN and SMC saw affect of Max
BA limit– SIS main point of concern for
long-term predictions
Natural Resource Consultants Since 1921
Overall PerformanceOverall Performance
• No Clear Winner– Basic stats and significance tests
point to SIS– CDF’s and regressions point to
NWO– Overall, SIS, PN, and NWO
comparable– FPS not far behind– SMC and WC lower performance
Natural Resource Consultants Since 1921
Questions?Questions?
Natural Resource Consultants Since 1921
Remeasurement IssuesRemeasurement Issues
IssueNumberOf Trees Types of Cruiser Comments
No Initial Height 28 unable to get height
No Final DBH 9 heavy leaner, unable to measure
No Final Height 28 top bent, leaner, broken top
Shrinking DBH 17
Shrinking Height 150 broken top
Died 373
Grand Total 605