Looking Backwards to the Future - University of Warwick · 3 Looking backwards to the future –...
Transcript of Looking Backwards to the Future - University of Warwick · 3 Looking backwards to the future –...
Looking Backwards to theFuture
Tony Lawrance
Department of Statistics
University of Warwick
1
2
First of all, sincere thanks for making this such a great day for me -(provisional remark…)
Especially – JohnTheodore
and thanks to the Statistics Department for ‘sponsoring’ the event
3
Looking backwards to the future – what does it mean ?
An excuse to briefly look back on an enjoyable time in statistics with a wish to alsolook forward to some more time in statistics… Will try and pin the talk on somesignificant and not so significant events in my statistics life
Nearly 40 years of statistics before Warwick – so some reminiscing here for the firsttime here may be acceptable…
In Warwick for just less 10 years – but very enjoyable ones
Most of my publications are now on the site ‘researchgate.net’
Diary of LifeMaths undergraduate in Leicester – graduated 1963‘Intimidated’ into statistics by Nageeb Rahman, a Cambridge PhD student of HenryDaniels – in that, I am the two-year elder ‘statistical brother’ of Phil Brown
Nageeb sent me in 1963 to Aberystwyth for an MSc (and then Phil Brown in 1965)because Dennis Lindley from Cambridge had started a Stats Department there in1960, with David Bartholomew, Mervyn Stone and Ann Mitchell (Dennis was inHarvard for half my year, but taught frequentist inference in the second term)
4
Carol? DonaldEast SylviaLutkins DavidBartholomew DennisLindley MervynStone AnnMitchell PeterKing Eileen?
GwynJones MikeSamworth PgslyGwynne GrahamPhipp ^ JeffWood ClivePayne ?Bambegye BasilSpringer ErylBasset RichdMorton
Department of Statistics, Aberystwyth 1963-64
5
The IBM 1620 Electronic Computer, Aberystwyth Stats Dept 1963
Out of bounds to MSc students
6
After MSc -Leicester October 1964 - started as a tutorial assistant 1 year -> assistant lecturer
Frank Downton, d 1986 ?Nageeb Rahman, d 90’s ?Mike Phillips – 1968-…Brian English – 1969-70?
Took 4 ‘summers’ to get a PhD, Stochastic Point Processes’, awarded in 1969.Started by Frank Downton giving me a sheet with a few references … To 7
Lightly supervised by Frank Downton, who almost immediately after my arrival backin Leicester moved to Birmingham, enticed by Henry Daniels
Never-the-less, Frank Downton had big influence encouraging me, researchconfidence building…
Another big influence in supporting my career was my external examiner David Cox
So this seems a good point to get a bit more technical
Diary of Life
7
(back to 6)
PhD and Point Processes…Time series of point events on the line – mainly Poisson and renewal processes at thetime – spatial or dependent interval versions had not been much considered
8
My first issue was what was meant by a ‘typical event’ to start an interval in astationary point process ?
I wrote to David Cox – good question, he said ! “We have avoided it in my justcompleted Methuen monograph with Peter Lewis” on ‘Series of Events’ – 1966
I went for dependent interval versions with stationarity and first studied Cox’s 1954Biometrika paper on ‘superposition of renewal processes’ or ‘pooled processes’
What was the inter-point distribution and dependency of this process ?
So after a while I investigated two ideas…
time
Process 1
Process 2
Superposition
(I hope my memory is correct !)
9
An Average Event – an interval beginning with an ‘average event’ in thestationary PP with intervals
1 2, ,...X X
has distribution 1
1limn
ii
P X x P X xn n
…a bit clunky
An Arbitrary Event - a more elegant approach follows from Khintchine’s (1955)work** on stationary input processes for queues**. This developed from ‘Palmdistributions’, referencing Palm (1943) , who introduced the idea of an intervalbeginning with ‘at least one point’ in a telephone queuing context
( , )N t t Thus, with the counting variable in a stationary point process, thedefinition of the distribution of an interval beginning with an arbitrary event is
lim ( , ) 0 | (0, ) 1)0P X x P N x N
It turned out that this definition mathematically connected the idea of an arbitraryevent with that of an arbitrary time, and involved length-biased sampling andforward and backward recurrence times – previously informal concepts for ageneral stationary point process
My thesis work also contained work on this arbitrary event approach and onparticular point processes…
(To 10)
10
Khintchine (1894-1959). Mathematical Methods of Queuing, 1955, English Eds,1960, 1969, Griffin
From the introduction…
(back to 9)
.
11
My First Seminar was 25 Feb 1970 at UMIST, Manchester, on ‘selectiveinteraction of point processes’, one of my PhD point processes
My Most Recent Seminar reconstructed part of my first seminar at the MauricePriestley memorial meeting, 18 December 2013…
The selective interaction model was introduced by the Dutch neurophysiologists TenHoopen and Reuver (1965, 1967) to explain multi-modal inter-spike distributions fordark firing of lateral geniculate neurons, observed by Bishop et al (1964)
The process can be explained as follows - you can see that I was rather keen ongraphics even in those distant days…
I explored it as an applied probability model. I really wish now that I had followed up onthe statistical aspects, contacting the experimenters, analysing their data, attempting tocollaborate, etc, and doing simulations – but there was little electronic computing andno internet, and Holland was a long way away
Diary of Life
(from my thesis)
The Selective Interaction Neuron Firing Point Process Model
Excitatory
Inhibitory
ObservedResponse
The model was justified empirically by a multi-modal distribution of times betweenthe responses’, in the ‘spike trains’ of observed neuron firings – convolutions ofexcitatory intervals
Poisson excitatory results by very detailed calculation – in my thesis
General results by appealing to the compound distribution structure of the observedresponse count, resulting in
stnryIntervalprocess
stnry stocpnt countprocess
This image cannot currently be displayed.
, ( )i II y
This image cannot currently be displayed.Selective interaction process
12
from Priestley meeting talk
( )
, ,1
( ) ( ) , 1 { ( ) 0} 1 { ( ) 0}, 0IN t
i iR E E I E I E i E i
i
N t N t with prob P N I P N I otherwise
Continued, (J Appl Prob papers 1970-71 &1979)
Excitatory
Inhibitory
Response
It follows
and approximately (?) via compound distribution results
sdevs
Compounding the exciting process intervals using the inhibitory process to get theinter-response distribution is more difficult…but I used arbitrary events
For more detailed results when the excitatory process is Poisson, see my 4 JAPpapers in the 70’s. No model fitting, no simulations – what a pity !
stationaryintervalprocess
stationarystoc ptcountprocess
( )EN t
, ( )i II y
( )RN tSelective interaction process
0{ ( )} Pr{ ( ) 1} ( )R E I E Iy
E N t N y y dy t
,var{ ( )} [ ( ) var( )]R E I I E IN t E t
( )
, ,1
( ) ( ) , 1 { ( ) 0} 1 { ( ) 0}, 0IN t
i iR E E I E I E i E i
i
N t N t with prob P N I P N I otherwise
,E I
14
1970 - Next move - the year 970/71 at the ‘IBM Thomas J Watson ResearchCenter’, New York, invited by Peter Lewis
Extended and consolidated PhD work by investigating branching Poisson processpoint models for computer failures, and co-organizing big point process conference
1972 – Returned to Leicester for 1 year – moved to Birmingham for years
1973-2004 My Birmingham Years
1970 – After PhD exam joined David Cox’s weekly PP journal club at IC from Leicester
met Valery Isham,Anthony Atkinsonat IC
Diary Life
52
Henry DanielsDavid WishartPaul DaviesPhil BertramRoger Holder
Frank DowntonMalcolm Faddy
Alan GirlingJohn CopasChris Jones
Richard AtkinsonFrank CritchleyPrakash Patil
Christmas Meal 1981/82PhilB? FrankD ? Chris Gray AJL AnnieM ChrisJ TriciaC
15
KamilaZ WolfgangB AlanG PrakashP SaidS MalcolmF RichardA
Birmingham Group (when MalcolmF moved back to NZ for second time, 2003)
16
1973 – Farewell Point Processes
Found research opportunities in hydrology (from teaching with Nath Kottegoda inCivil Engineering) after devising a course in hydrological time series for Bham MSc inHydrology
RSS Read Paper on the topic with Nath Kottegoda (Stochastic Modelling of Riverflow Time Series)
Teaching has influenced my ‘choice’ of research areas quite a bit but not thereverse
1973 – Hello Time Series – as it was moving into the nonlinear era
Time series started to move away in several directions from ‘Box-Jenkins’ linearGaussian models to be able to capture more statistically varied and complexbehaviour
Maurice Priestley, with non-stationary processes and spectraHowell Tong, with dynamical-statistical thresholdsRobert Engle, Clive Granger, with volatility, co-integrationPeter Lewis et al, with specified nonGaussian models, including discrete distributionmodels, simulation in operations research
1980-1990 Worked on non-Gaussian time series models with Peter Lewis, by thenat Naval Postgraduate School, Monterey, California (nice summers)
Examined Jane’s PhD on ‘dry’ rivers…
17
Peter Lewis, 1932-2011
18
1978-80 – Work started with nonGaussian solutions to linear time series models,exponential, mixed exponential, gamma
1980-87 - Then ways to formulate autoregression operation with nonGaussianvariables – in ways natural to the particular distribution, e.g. convolution andmultiplication, minimization
1989-90 – Non-reversibility, directionality, in nonGaussian linear time series
An early linear problem – it’s easy to set up …(so I describe here)
The AR(1) Innovation Problem
How to specify the error distribution for an AR(1) process with specified marginaldistribution
Gaver & Lewis made a start with the gamma distribution but could not explicitly obtainthe innovation distribution..…
1t t tX X
19
The AR(1) Innovation Problem – ‘epsilon for given X’
Solution easy in terms of Laplace transforms – Gaver & Lewis, from
Exponential( ) solution clear:0 with proby
( ) with proby1t
tE
Gamma solution –>( )k
X zz
Can you invert this Lapalce transform without serendipity ?
( )k
zz
z
‘Consider a shot noise process in continuous time’, of course…
1
i
NU
t ii
Y
(0,1)iU uniform1( log )N Poisson k
( )iY exponential
A compound Poisson distribution
( ) ( ) ( ), ( ) ( ) ( )X X X Xz z z z z z
1 , 0 1, , ??distbnt t t t tX X X D
20
Diary Life
1985 - RSS ‘read paper’ on nonlinear AR exponential variables, with Peter Lewis
1986 – ISI Tashkent - Very Sick ! (Time series directionality)
1986 - Began teaching inference in Bham - beginning of regression diagnostics
1986 – Seconded RSS vote of thanks at Cook’s 1986 local influence read paper, andshowed how it applied to regression transformation diagnostics
1988 - JASA paper on regression transformation local influence diagnostics
1988 Got chair in Bham (poster of inaugural lecture)
1989 – Papers on regression transformation score statistics
1991 - IMA Minnesota Robustness & Diagnostics workshop (photos Anthony, Frank)
1981-1991 Tim Davis PhD collaboration ‘Survival of Tyres’, Dunlop-Sumitomo-Ford1991 Tim Davis PhD
1995 - Regression diagnostics – Cook’s bivariate & conditional distance
1995 - 98 Engine mapping, with Tim Davis, Tim Holiday- PhD-1996
1992- Statistical aspects of chaos
1998- Chaos-based communications
(To 23, 24)
(To 21, 22)
Gary Brown PhD 1995
Technometrics paper 1998
Biometrika papers 1987, 1989-ACA
took over my research & publication (To 25)
21
(Back to 20)
(Back to 20)
23
A trip across Minnesota and Iowa with Anthony Atkinson and Frank Critchley toSpillville, Iowa, to visit Dvorak connections, 1991, on the workshop rest day…
(To 24)
Spillville, Iowa 1991
Dvorak’s ‘American Quartet’(String Quartet in F Major,op96) composed here in1893, also, String Quintet inE Flat Major,op97(sometimes called the‘Spillville Quintet’), and afterreturning to NY, hisHumoresque, No 7 in G FlatMajor
Back to 20
25
25
1992- 2010 Statistical aspects of chaos, leading to
Chaos-based communications
‘What got me started’… the Uniform Distribution Solution to the AR(1)Process – Bartlett’s last paper, probably (another case of the AR(1) innovation problem)
1
1 1 1, , 1, 2, ...,t t t t
iU U wp i k
k k k
Chaos – instabilities produced by a deterministic rule
CollaboratorsBala BalakrishnaAlexander BaranovskyTohru KhodaGan OhamaRodney WolfTheodore PapamarkouNancy SpencerAtsushi UchidaChibisi Chima-Okereke
26
1
1 1 1, , 1, 2, ...,t t t t
iU U wp i k
k k k
1 ( ) mod(1)t tU kU
The reverse of this model is the following chaotic and deterministic model
And, incidentally, there is a negatively correlated version reversing to
1 { (1 )}mod(1)t tU k U
deterministic rule called achaotic shift map ~ like cntscongruential random numbergenerator
It follows (and more) generally that deterministic chaotic processes have statisticalproperties, i.e., there are statistical properties of chaos
Such ideas prompted some electronic engineers to have the idea of ‘communicatingwith chaos’ – instead of communicating with sinusoidal radio waves
Where is the chaos from this model?
Transmit onebit b=+/- 1
ChaoticSpreading
Signal
1
n
i iX
( )
1,2,...,
ib X
i n
Channel Noise
1
n
i i
Received Signal
Also available incoherent case
( )
1,2,...,
i i iR b X
i n
1
n
i iX
Decoder
bit =b̂
A particular chaos communication system using a chaotic map is
Chaos Shift Keying (CSK) – ‘Coherent’ Case -simplest
Exact theory for bit error rate of such a system, Lawrance & Ohama (IEEE, 2002)
2
( 1)
1( )
( ) ( )
n id
i
x c
xBER N f x dx
1( )i iX X
(estimate b)
28
Performance of CSKAssessed by bit error rate (BER)
Depends on statistical aspects of the system as well as the dynamics, according toprevious formula
Worst; IID Gaussian
Logistic map
Shift map
Best:circular map andtheoretical lower bound
Area has moved on from chaotic-map and electronic circuitry chaos to laser-chaos communication; this is still a research area but with several experimentaldemonstrations and US military applicationsCurrent work with Atsushi Uchida and Chibisi Chima-Okereke
Different types of chaoticspreading, compared toIID Gaussian
Optimum circular mapspreading: Ji Yao, TPapamarkou
-> Police mergers
29
A Brief Diversion - In the Press…
Police Mergers 2006 – the misuse of statistics
Total Score by Force (excluding London)
p is significant to the 0.01 level
R2 = 0.58090.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Force Size (Officer Strength)
To
talS
co
re
Line equates
to an average
score of 3score = 3
63
Government O’Connor Report said : This strongly suggests that forces with over4,000 officers (or 6,000 total staff) tend to meet the standard across the range ofservices measures in that they demonstrate good reactive capability with a clearmeasure of proactive capacity…’.
Charles Clarke.Home Secretary
4000
30
What this plot shows to me is:
Rather rough upward scatter of points
Least squares line is misleading because of extremes
Large variability at each force size – very important
Line at 63 shows most forces ‘fail’ – artifact of scoring and choice of ‘3’
Meaningless statistical elaborations of p-value and R-squared due to automatic use ofsoftware
No justification of 4,000 figure
Total Score by Force (excluding London)
p is significant to the 0.01 level
R2 = 0.58090.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
0 1000 2000 3000 4000 5000 6000 7000 8000 9000
Force Size (Officer Strength)
To
tal
Sco
re
Line equatesto an averagescore of 3score = 3
63
What I said about the O’Connor Report:
31
Another example of rubbish in the O’Connor report
Overall Trend for Protective Services
For c e S i z e (Smallest f rom lef t )
Sco
re
Ser ious &Or ganised
Publ ic Or der
Major Cr ime
Roads Pol icingCivi l Contingencies
Cr i tical Incidents
CT &DE
What I said about this plot ‘This is an almost perfect example of how not to present agraph - no scales on either axis, no data plotted to justify the lines drawn. It is almostimpossible to obtain any critical understanding from it, except that it is intended toprove that score for protective capability increases with force size’
32
What was said in the House of Commons:
MP David Davis: ….Frankly, the best that I can do is to repeat to the House thecoruscating opinion of Professor Lawrance, a professor of statistics at WarwickUniversity…
MP Adrian Baily: …I rather regret the attempt by the University of Warwick torubbish the statistical basis and the credibility of that report. It has a goodpedigree and I shall make my judgement on the balance of professional policeopinion, rather than on the opinion of university professors in Warwick…
Another newspaper appearance…
33
A Publication in ‘The Sun’… - 14th October 2013
34
A Publication in ‘The Sun’… - 14th October 2013
A MATHS professor has told The Sun bills are so complicated even he can’t understand them. Tony Lawrance,right, of Warwick University said : “They’re absurdly over-complicated. Most professors would find them difficultto understand – the public doesn’t stand a chance.’’
35
Bala BalakrishnaGan OhamaRachel HilliamYi YaoTheodore PapamarkouChibisi Chima-OkerekeAtsushi Uchida
Chaos-based Communications 2001 – 2014 - ??
Current work|: laser-chaos-based communications
(laser = light amplification by stimulated emission of radiation)
Key laser features of laser-based communication
Collaborators:
A message is hidden in a segment of the chaotic laser sequence - steganography,rather than cryptography when a message is visible but has to be decoded
75%
With BalaBalakrishna,CochinUniversityKerala
1. Lasers can produce chaotic waveswhich look stochastic – (use semi-conductor laser with optical feedback)
2. Lasers producing chaotic behaviourcan be synchronized by a trigger signal
36
Current Work-1: Laser-based Chaos Communication
Experimental data via collaboration with Atsushi Uchida, Saitama University,Tokyo, and analysis collaboration with Chibisi Chima-Okereke ofActiveAnalytics, Bristol
Each set of data consists of three time series of 10m values
Experiment set up to probe chaos shift-keying system of communication using semi-conductor lasers with optical feedback and transmission though 60m fibre optic cable
andbinarymessage
binarymessage b
37
Experimental setup not quite so simple as it may have seemed…
38
Some Experimental Results
5004003002001000
0.3
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
Time Index - 5m
ad
jDrv
_w
Op
tN
se
_1
Adjusted Received and Synchronized Laser Signals (5,000,001:1,000,500)
0.30.20.10.0-0.1-0.2-0.3-0.4-0.5-0.6-0.7-0.8
12
10
8
6
4
2
0
Intensity
De
nsit
y
Intensity of Drive Laser
0.30.20.10.0-0.1-0.2-0.3-0.4-0.5-0.6-0.7-0.8
16
14
12
10
8
6
4
2
0
Intensity
De
nsit
y
Adjusted Optical Noise
Example of laser synchronization
Is Optical NoiseIndependent ?
Drive laser Optical Noise
Based on post-processing for instrument effects – Noise not Gaussian
39
0.23
5
0.22
5
0.21
5
0.20
5
0.19
5
0.18
5
0.17
5
0.16
5
0.15
5
0.14
5
0.13
5
0.12
5
0.11
5
0.10
5
0.09
5
0.08
5
0.07
5
0.06
5
0.05
5
0.04
5
0.03
5
0.02
5
0.01
5
0.00
5
-0.005
-0.015
-0.025
-0.035
-0.045
-0.055
-0.065
-0.075
-0.085
-0.095
-0.105
-0.115
0.15
0.10
0.05
0.00
-0.05
-0.10
No
ise
Bo
xp
lots
Boxplots of Optical Noise versus Drive Signal Strength
Distribution of Optical Noise Conditional on Driver Signal Strength
N.B. BER v SNR plot under development, but initial work indicatesacceptable values can be obtained using range of SNR controlled byrange of spreading
40
Current Work-2:Volatility Modelling and Exploratory Graphics
Topic comes from teaching financial time series in the Financial Mathematicsmasters program
Financial time series ‘means’ volatility modelling
Volatility is changing conditional variance in a time series
Motivation – volatility models are routinely used without justification of the type ofvolatility structure existing in the data series
But it has not been clear how to reveal volatility structure
Attitude has been ‘fit the model you think will be ok and undertake somegeneral tests of its fit’ - but never obtain the empirical volatility and compareit with the model volatility
My attitude is ‘get an empirical version of the volatility function and choosea model which gives a good volatility fit, i.e. get the volatility right first’ -may be not the purest of likelihood approaches – but surely volatility is themost important aspect of volatility models !
1var( | )t tX X
1 1 , (0,1)( )t tX IIDt t tXX
The General Volatility Model to be used
41
FTSE100 Daily Data
4th Jan 2005 – 10th Feb 2011
Daily Adjusted Closing Values and Daily Returns
01/01/201101/01/201001/01/200901/01/200801/01/200701/01/200601/01/2005
8000
7000
6000
5000
4000
10%
5%
0%
-5%
-10%
Daily Date
Retu
rns
FTSE
Valu
es
42
1 1ˆ ) ( | )t t t tcalculate x smo x
1 2
211 1
2
ˆ1 ( 1) ( ) / ( | )n
t t tt
n x smo x
1 1( ) ( | )t tx smo x
1 1 , (0,1)( )t tX IIDt t tXX
Volatility Graphics
Based on the general volatility model for returns
1 volatility functiontX
Graphics Steps
(unscaled individual volatilities) (smoothed unscaled individual volatilities)
scaling gives standardized innovations
Smoothed & scaled i-volatilities give empirical version of volatility function1( )tx
Journal of the Royal StatisticalSociety, Series C, Applied Statistics(2013) 62, Part 5, pp. 669-686
( nearly constantwith returns)
43
75310-1-3-5-7
12
10
8
6
4
2
1
0
Previous Return
Vo
lati
lity
Scaled Individual Volatilities and Their Smooth
Empirical volatilityfunction
44
75310-1-3-5-7
12
10
8
6
4
2
1
0
Previous Return
Vo
lati
lity
Bootstrapping the Volatility Function
That’s Enough, except…
(see 20013 JRSS’C’ paper for more details)
45
The one nice thing about getting olderis that younger people follow you…
46
47
Many thanks