Lecture 9: Nonparametric modelling€¦ · Lecture 9, page 9 • polynomial • running mean •...
Transcript of Lecture 9: Nonparametric modelling€¦ · Lecture 9, page 9 • polynomial • running mean •...
Lecture 9, page 1
Lecture 9: Nonparametric modelling What is the problem of parametric models? • We have no reason to believe the assumptions (e.g., linearity,
normality, stationarity etc.) • We have no reason to chose a specific model In reality: • We discover insufficiencies of our parametric models Nonparametric models may be used for • Visualisation of data • exploring functional relationships • identify candidate models
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 1 of 1
Lecture 9, page 2
Illustration: Abundance of herring (This first example: not time series model. Illustration of principles)
Herring length in mm
30 40 50 60
010
030
050
0
Abu
ndan
ce o
f her
ring
(10^
9)
-> Obviously not linear trend -> Not clear how to fit data
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 2 of 2
Lecture 9, page 3
Another example (inspired by Howell Tong):
Kilpisjärvi data
Year
Trap
inde
x
1960 1970 1980 1990
05
1015
2025
30
Reversed data
Year
Trap
inde
x
1960 1970 1980 1990
05
1015
2025
30
The tops are not symmetric. The build-up phase is not equal to the crash phase => Non-linearities If we know the biological relationship, we may model this parametrically, i.e., y = α1x + α2x2+ε What if we don't?
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 3 of 3
Lecture 9, page 4
Model formulations
Changin
Classical linear model: [ ] ppp XXXXYE βββ +++= ......| 1101
Generalised Linear Model (GLM) [ ]( ) ppp XXXXYEg βββ +++= ......| 1101
Additive Model: [ ] )(...)(...| 1101 ppp XfXfXXYE +++= β
Generalised Additive Model (GAM): [ ]( ) )(...)(...| 1101 ppp XfXfXXYEg +++= β
g to well known notation…
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 4 of 4
Lecture 9, page 5
-> Diffe(generadistribugamma)
Classical linear model: [ ] ptptpttt XXXXXE −−−− +++= ααα ......| 1101
Generalised Linear Model (GLM) [ ]( )
pptptpttt XXXXXEg −−−− +++= ααα ......| 1101
Additive Model: [ ] )(...)(...| 1101 ptptpttt XfXfXXXE −−−− +++= α
Generalised Additive Model (GAM): [ ]( ) )(...)(...| 1101 ptptpttt XfXfXXXEg −−−− +++= α
rent link functions can be used to make the model additive lise the model); logit, reciprocal, log, etc. This depends on what tion the data have (Normal, binomial, gamma, poisson, inverse
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 5 of 5
Lecture 9, page 6
-> Often it is sufficient to stay at the “GLM”, i.e., we log the data and achieve an additive model
Kilpisjärvi data
1960 1970 1980 1990
01
23
Reversed data
1960 1970 1980 1990
01
23
Example:
)loglog(1
2211 −− −−−= tt NN
tt eNN αα )log( tt Nx =
22111 −−− −−= tttt xxxx αα
2211)1( −− −−= ttt xxx αα
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 6 of 6
Lecture 9, page 7
Nonparametric smoothers The function that transforms the data is called a smoother.
Herring length in mm
30 40 50 60
010
030
050
0
Abu
ndan
ce o
f her
ring
(10^
9)
Our smoother need some properties: It should be smooth (mathematical: Twice differenctiable) It should summarise the data; on the expense of fit: What we do is then to penalise the number of parameters (in figure: If we use all the data points, we can fit the data perfectly). Mathematical expression:
( ) ∑=
=t
jjj ySxs
100
S0 is a smoother (of some complicated form).
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 7 of 7
Lecture 9, page 8
Different smoothers
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 8 of 8
Lecture 9, page 9
• polynomial
• running mean
• running line
• loess
• gaussian kernel
• smoothing spline: Used a lot in biology – picewise linear regressions that are twice differentiable in knots.
• regression spline
• natural spline
30 40 50 60
010
030
050
0
Is this important? Probably not very important. Not big difference between smoothers.
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 9 of 9
Lecture 9, page 10
Possible strategy Preliminarily: • Checking parametric models Is
Xt = f(Xt-1)+εt (some nonlinear shape) significantly better than Xt = α+βXt-1+εt ? (lagged linear regression model) P-values: f1 f2 f3 0.433 0.213 0.77 0.444 0.101 0.723 0.358 0.056 0.656 0.251 0.046 0.55 0.146 0.041 0.375 0.067 0.035 0.17 0.035 0.029 0.081
2.4 2.6 2.8 3.0 3.2
-0.6
-0.2
0.0
0.2
0.4
0.6
2 3 4 5 6 7
-0.6
-0.2
0.0
0.2
0.4
0.6
-4 -2 0 2 4
-0.6
-0.2
0.0
0.2
0.4
0.6
degrees of freedom
p-va
lue
3 4 5 6 7 8 9
0.0
0.1
0.2
0.3
0.4
H0: f1 = linear
degrees of freedom
p-va
lue
3 4 5 6 7 8 9
0.0
0.05
0.10
0.15
0.20
H0: f2 = linear
degrees of freedom
p-va
lue
3 4 5 6 7 8 9
0.0
0.2
0.4
0.6
0.8
H0: f3 = linear
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 10 of 10
Lecture 9, page 11
(Software: function "gam" in Splus + +) We want to model, i.e., Xt = f(Xt-1) Stochastic model: Some variability in Xt for a fixed value of Xt-1. i.) Decide on the link-function -> Decided by theory, shape of data etc. ii.) Decide on smoother -> We have seen that the exact form of the smoother is not very important. iii.) Decide on amount of smoothing iv.) Do model selection
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 11 of 11
Lecture 9, page 12
iii.) Decide on amount of smoothing Idea to penalise "un-smoothness" of the smoother:
{ } { }∫∑ +−=
b
a
t
jij dttfxfy 2
1
2 )('')( λ
Complexity Model fit
This is done by evaluating the second derivative (which measures how often the data turns). From the formula: The size of the λ-parameter decides on the balance between fit and smoothness.
(When using splines in Splus, the amount of smoothing is determined by the degrees of freedom, d.f, with default=4). Can be tested with, e.g., Cross Validation (CV) or AIC: AIC = –2 maximized log likelihood + 2 # parameters -> But we have no parameters, or no likelihood! AIC = –deviance + 2df(λ) dispersion parameter Both AIC and CV are functions of λ, and can be calculated for as many values of λ as wanted. The λ giving lowest CV or AIC value is selected as the most appropriate model.
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 12 of 12
Lecture 9, page 13
iv.) Model selection (One aspect is to "select" the amount of smoothing). Nested models can be tested formally:
[ ]( )[ ]( ) 22112
2111
)4,(...|
)3,()4,(...|
−−−−
−−−−
+==
=+==
ttpttt
ttpttt
XdfXsXXXEgdfXsdfXsXXXEg
β
(The "s()" means smoothing spline) The deviance between the two models is approximately χ2-distributed.
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 13 of 13
Lecture 9, page 14
Example: Blowflies – Finding the non-linear shape (Moe et al. 2001) Regard the relationship between larval density and survival from larvae to pupae. Non-parametric approach: No assumptions about the shape of the relationship.
The alternative hypothesis: A linear relationship, i.e., a straight line
20 50 150 400 1000
-2.0
-1.0
0.0
0.5
1.0
Larval density L
f(L)
Formulate a biologically based (parametric) model of the same relationship:
)exp( 114
11
−−+ = t
bt
t
t cLaLLP
20 50 150 400 1000
0.0
0.2
0.4
0.6
0.8
1.0
Larval density L
exp(
a +
f(L))
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 14 of 14
Lecture 9, page 15
Example: Fish species richness – Common structure? (Lekve et al. 2008) Regard the relationship between species richness at time t and t-1 and the influence of environmental stochasticity. Non-parametric approach: No assumptions about the shape of the relationship. Model of species richness, S, in fjord f at time t : log[(St+1)/(St-1+1)] = g1f(log(St-1+1)) + g2f(temperaturet)
yt,f = g1f(log(St-1+1)) + g2f(temperaturet)
where g1f is the function of richness-dependent regulation, and g2f is the functions of environmental influence on species richness (here: temperature). Testing for common structure in, say, the function g1f between fjords i and j will be to test
H0: g1,i = g1,j versus
H1: g1,i ≠ g1,j using software developed by Ole Chr. Lingjærde:
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 15 of 15
Lecture 9, page 16
1.5 2.0 2.5 3.0 3.5
-1.0
0.0
0.5
1.0
GI
Richness-dependencet
2 3 4 5 6 7
-1.0
0.0
0.5
1.0TJ
GI
Temperaturet
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 16 of 16
Lecture 9, page 17
___________________________________________
C:\Kyrre\studier\drgrad\Kurs\Timeseries\lecture 09 021103.doc, KL, 04.11.02, page 17 of 17
Assessment Nonparametric approach can be valuable in: • Detecting (and testing for) nonlinearities • Testing for common structure • Finding parametric form of models