Interrupted Time Series: What, Why and How Karen Smith An Example From Suicide Research.
-
Upload
arely-sarr -
Category
Documents
-
view
236 -
download
0
Transcript of Interrupted Time Series: What, Why and How Karen Smith An Example From Suicide Research.
Interrupted Time Interrupted Time Series:Series:What, Why and HowWhat, Why and How
Karen Smith
An Example From Suicide Research
AcknowledgementAcknowledgementMotivated by consultancy work
with the Centre for Suicide Research, University of Oxford
All analyses and graphs produced by Helen Bergen, Centre for Suicide Research
Motivating exampleWhat is Interrupted Time Series?Why use it?Design issuesAnalysis issuesGuidelines on use
Motivating ExampleMotivating ExampleBetween 1997 and 1999 the analgesic co-
proxamol was the single drug used most frequently for suicide by self-poisoning in England and Wales, with 766 over the 3 year period
There is a relatively narrow margin between therapeutic and potentially lethal levels
Death occurs largely because of the toxic effects of dextropropoxyphene on respiration and cardiac conduction
MHRA conducted a review of the efficacy/safety profile
Committee on Safety of Medicines advised withdrawal from use in the UK, the final date being 31 December 2007
Patients who find it difficult to move to an alternative drug can still be prescribed co-proxamol
The ProblemThe ProblemHow to evaluate the impact of
the announcement to withdraw co-proxamol on◦Prescribing of analgesics◦Mortality involving co-proxamol◦Mortality involving other analgesics
(substitution of method is of concern)
Available DataAvailable DataQuarterly data on prescriptions of co-proxamol,
cocodamol, codeine, codydramol, dihydrocodeine, NSAIDs, paracetamol and tramadol (from Prescription Statistics department of the Information Centre for Health and Social Care, England, and Prescribing Service Unit, Health Solutions Wales)
Quarterly data on drug poisoning deaths (suicides, open verdicts and accidental poisonings) involving co-proxamol alone, cocodamol, codeine, codydramol, dihydrocodeine, NSAIDs, paracetamol and tramadol, based on death registrations in England and Wales (from ONS) – single drug, with and without alcohol
Quarterly data for overall drug poisoning deaths and for all deaths receiving suicide and undetermined verdicts
Simple AnalysisSimple AnalysisCompare the proportion of deaths
involving co-proxamol prior to the legislation with proportion following legislation
Compare total number of poisoning deaths before and after legislation
Time series plots of prescriptions and deaths
Co-proxamol withdrawal has reduced suicide from drugs in Scotland, E. A. Sandilands & D. N. Bateman, British Journal of Clinical Pharmacology, 2008.
What’s Wrong With This?What’s Wrong With This? Ignores any trends, both before and after
change in legislation (or intervention in a more general setting)
Ignores any possible cyclical effectsDoesn’t pick up on any discontinuityVariances around the means before and after
the intervention may be differentEffects may drift back toward the pre-
intervention level and/or slope over time if the effect wears off
Effects may be immediate or delayedDoesn’t take account of any possible
autocorrelation
A Solution – Interrupted Time A Solution – Interrupted Time SeriesSeriesA special kind of time series in which
we know the specific point in the series at which an intervention occurred
Causal hypothesis is that observations after treatment will have a different level or slope from those before intervention – the interruption
Strong quasi-experimental alternative to randomised design if this is not feasible
Ramsay et al, 2003
The ModelThe ModelUse segmented regression analysis (Wagner et al, 2002):
Ŷt = β0 + β1 x timet + β2 x interventiont + β3 x time_after_interventiont + et
Yt is the outcome time indicates the number of quarters from the start of the series intervention is a dummy variable taking the values 0 in the pre-
intervention segment and 1 in the post-intervention segmenttime_after_intervention is 0 in the pre-intervention segment and
counts the quarters in the post-intervention segment at time t β0 estimates the base level of the outcome at the beginning of
the seriesβ1 estimates the base trend, i.e. the change in outcome per
quarter in the pre-intervention segmentβ2 estimates the change in level in the post-intervention segmentβ3 estimates the change in trend in the post-intervention
segmentet estimates the error
Threats to ValidityThreats to ValidityForces other than the intervention under
investigation influenced the dependent variable◦ Could add a no-treatment time series from a
control group◦ Use qualitative or quantitative means to
examine plausible effect-causing events Instrumentation – how was data
collected/recordedSelection – did the composition of the
experimental group change at the time of intervention?
Poorly specified intervention point; diffusionChoice of outcome – usually have only routinely
collected dataPower, violated test assumptions, unreliability of
measurements, reactivity etc.
Design ConsiderationsDesign ConsiderationsAdd a non-equivalent no-
treatment control groupAdd non-equivalent dependent
variables◦Intervention should not affect but
would respond in the same way as primary variable to validity threat
Remove intervention at a known time
Add multiple replicationsAdd switching replications
ProblemsProblemsInterventions implemented slowly and
diffuseEffects may occur with unpredictable time
delaysMany data series much shorter than the 100
observations recommended for analysisDifficult to locate or retrieve dataTime intervals between each data point in
archive may be longer than neededMissing dataUndocumented definitional shifts
Applied to the Co-Proxamol Applied to the Co-Proxamol DataData28 quarters in the pre-intervention
period and 12 in post-interventionExamined a number of common
analgesics◦ Prescriptions◦ Deaths
Examined overall suicidesSome evidence of autocorrelation
in the data, hence Cochrane-Orcutt autoregression used (Durbin Watson statistic of final models close to 2)
Prescriptions* for analgesics dispensed in England and Wales, 1998-2007
0
1000
2000
3000
4000
5000
6000
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
Year (quarters)
Pre
sc
rip
tio
n it
em
s d
isp
en
se
d p
er
qu
art
er
(th
ou
sa
nd
s)
co-proxamol
NSAIDs
paracetamol
co-codamol
tramadol
co-dydramol
codeine
dihydrocodeine
Co-proxamol w ithdrawal announced
1998 20001999 200620052004200320022001 2007
* excluding liquids, suppositories, granules, powders and effervescent preparations
Mortality in England and Wales from analgesic poisoning (suicide and open verdicts), 1998-2007, for persons aged 10 years and over (substances taken alone, +/- alcohol)
0
10
20
30
40
50
60
70
80
90
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
Year (quarters)
Nu
mb
er
of
de
ath
s
co-proxamol
otheranalgesics
co-proxamolbest fit withoutannouncement
co-proxamolbest fit withannouncement
Co-proxamol withdrawal announced
1998 20001999 20052004200320022001 2006 2007
PrescriptionsPrescriptions
Pre-intervention Post-intervention
Base level, β0 (SE)
p Base trend,β1 (SE)
p Change inlevel, β2 (SE)
p Change intrend, β3
(SE)
p
Co-proxamol 3050.1 (139.9) <0.001 -45.9 (7.7) <0.001 -554.8 (74.9) <0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) <0.001 34.1 (0.8) <0.001 300.5 (53.6) <0.001 30.7 (6.4) <0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) <0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) <0.001 -1.1 (0.3) 0.004 148.7 (35.8) <0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) <0.001 -1.5 (1.4) 0.291 -29.7 (2.4) <0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) <0.001 28.4 (3) <0.001 -622.9 (70.2) <0.001 -66.2 (8.5) <0.001
Paracetamol 1493.4 (56.1) <0.001 42.1 (3.0) <0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) <0.001 -41.9 (6.8) <0.001 16.3 (5.2) 0.004
PrescriptionsPrescriptions
Pre-intervention Post-intervention
Base level, β0 (SE)
p Base trend,β1 (SE)
p Change inlevel, β2 (SE)
p Change intrend, β3
(SE)
p
Co-proxamol 3050.1 (139.9) <0.001 -45.9 (7.7) <0.001 -554.8 (74.9) <0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) <0.001 34.1 (0.8) <0.001 300.5 (53.6) <0.001 30.7 (6.4) <0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) <0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) <0.001 -1.1 (0.3) 0.004 148.7 (35.8) <0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) <0.001 -1.5 (1.4) 0.291 -29.7 (2.4) <0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) <0.001 28.4 (3) <0.001 -622.9 (70.2) <0.001 -66.2 (8.5) <0.001
Paracetamol 1493.4 (56.1) <0.001 42.1 (3.0) <0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) <0.001 -41.9 (6.8) <0.001 16.3 (5.2) 0.004
PrescriptionsPrescriptions
Pre-intervention Post-intervention
Base level, β0 (SE)
p Base trend,β1 (SE)
p Change inlevel, β2 (SE)
p Change intrend, β3
(SE)
p
Co-proxamol 3050.1 (139.9) <0.001 -45.9 (7.7) <0.001 -554.8 (74.9) <0.001 -46.8 (16.8) 0.01
Cocodamol 1349 (12.4) <0.001 34.1 (0.8) <0.001 300.5 (53.6) <0.001 30.7 (6.4) <0.001
Codeine 204.8 (4.2) 0.007 9.6 (0.2) <0.001 20.5 (11.6) 0.089 3.5 (1.2) 0.007
Codrydamol 1055.2 (6.1) <0.001 -1.1 (0.3) 0.004 148.7 (35.8) <0.001 -4.2 (4.1) 0.316
Dihydrocodeine 686.4 (31.1) <0.001 -1.5 (1.4) 0.291 -29.7 (2.4) <0.001 -0.8 (2.4) 0.731
NSAIDs 4652.4 (47.2) <0.001 28.4 (3) <0.001 -622.9 (70.2) <0.001 -66.2 (8.5) <0.001
Paracetamol 1493.4 (56.1) <0.001 42.1 (3.0) <0.001 232 (66.6) 0.001 23 (8.2) 0.01
Tramadol 47.2 (51.4) 0.365 31.4 (2.7) <0.001 -41.9 (6.8) <0.001 16.3 (5.2) 0.004
SuicidesSuicides
Pre-intervention Post-intervention
Base level, β0 (SE)
p Base trend,β1 (SE)
p Change inlevel, β2 (SE)
p Change intrend, β3 (SE)
p
Co-proxamol 81.0 (4.5) <0.001 -1.194 (0.3) <0.001 -28.3 (4.9) <0.001 0.6 (0.6) 0.355
Other analgesics
51.3 (2.8) <0.001 -0.3 (0.2) 0.095 6.4 (6.0) 0.297 -0.3 (0.6) 0.724
All drugs except co-proxamol and other analgesics
221.2 (7.3) <0.001 -0.5 (0.5) 0.299 21.6 (12.8) 0.100 -5.4 (1.4) <0.001
All drugs 353.7 (10.2) <0.001 -2.0 (0.7) 0.008 0.004 (18.1) 1.000 -4.9 (1.7) 0.007
All causes 1319.0 (22.5) <0.001 -4.8 (1.4) 0.002 12.8 (34.8) 0.716 -5.4 (4.1) 0.192
Estimating Absolute EffectEstimating Absolute Effect The model may be used to estimate the absolute effect of
the intervention. This is the difference between the estimated outcome at a certain time after the intervention and the outcome at that time if the intervention not taken place.
For example, to estimate the effect of the intervention at the midpoint of the post-intervention period (when time = 34.5 and time_after_intervention = 6.5), we have
Ŷ34.5 = β0 + β1 x 34.5 without interventionŶ34.5 = β0 + β1 x 34.5 + β2 + β3 x 6.5 with
intervention Thus, the absolute effect of the intervention is
β2 + β3 x 6.5 Standard errors calculated using method of Zhang et al
σ22 + 6.52 x σ3
2 + 2 x 6.5 x σ23 Non-significant terms included due to correlation between
slope and level terms
Results - PrescriptionsResults - PrescriptionsEstimates of absolute effect during 2005 to 2007
Mean quarterly estimated number pre announcement
Mean quarterly number post announcement
Mean quarterly change (95% CI)
Prescriptions (x1000)
Co-proxamol 1465.1 605.7 -859 (-1065 to -653)
Cocodamol 2524.7 3024.6 500 (459 to 540)
Codeine 534.6 578 43 (31 to 55)
Codrydamol 1018.2 1140.0 122 (99 to 145)
Dihydrocodeine 634.6 600.0 -35 (-68 to -2)
NSAIDs 5633.8 4581.0 -1053 (-1186 to -920)
Paracetamol 2947.0 3330.0 382 (268 to 497)
Tramadol 1130.1 1193.9 64 (-5 to 133)
Results - DeathsResults - DeathsEstimates of absolute effect during 2005 to 2007
Mean quarterly estimated number pre announcement
Mean quarterly number post announcement
Mean quarterly change (95% CI)
Suicides, Open
Co-proxamol 39 15 -24 (-37 to -12)
Other analgesics 39 44 5 (-5 to 15)
All drugs except co-proxamol and other analgesics
204 191 -13 (-34 to 8)
All drugs 283 252 -31 (-66 to 3)
All causes 1152 1130 -22 (-89 to 45)
Co-Proxamol PrescriptionsCo-Proxamol PrescriptionsPrescription data for England and Wales
showed a steep reduction in prescribing of co-proxamol in the first two quarters of 2005, with further reductions thereafter.
Regression analyses indicated a significant decrease in both level and slope in prescribing of co-proxamol - the number of prescriptions decreased by an average of 859 (95% confidence interval (CI) = 653 to 1065) thousand per quarter in the post-intervention period.
This equates to an overall decrease of approximately 59% in the three year post-intervention period, 2005 to 2007.
Other Analgesic Other Analgesic PrescriptionsPrescriptionsThere were also significant decreases in
prescribing of NSAIDS of an average of 1053 (95% CI = 920 to 1186) thousand per quarter, equating to an approximate 19% decrease overall for 2005 to 2007; and for dihydrocodeine of an average of 35 (95% CI = 2 to 68) thousand per quarter, or approximately 6% overall for 2005 to 2007.
Prescriptions for the other analgesics increased significantly in the post-intervention period, apart from tramadol. Based on mean quarterly estimates this equated to percentage increases over the 2005 to 2007 period of approximately 20% for cocodamol, 13% for paracetamol, 12% for codydramol, and 8% for codeine.
26
Co-Proxamol DeathsCo-Proxamol Deaths Marked reduction in suicide and open verdicts involving co-
proxamol in the first quarter of 2005, which persisted until the end of 2007.
Prior to 2005 deaths due to co-proxamol alone were 19.5% (95% CI = 16.9 to 22.2) of all drug poisoning suicides, whereas between 2005 and 2007 they constituted just 6.4% (95% CI = 5.2 to 7.5).
Regression analyses indicated a significant decrease in both level and slope for deaths involving co-proxamol which received a suicide or open verdict - decreased by on average 24 (95% CI = 12 to 37) per quarter in the post-intervention period.
This equates to an estimated overall decrease of 295 (95% CI = 251 to 338) deaths, approximately 62%, in the three year post-intervention period 2005 to 2007.
When accidental poisoning deaths involving co-proxamol were included, there was a mean quarterly decrease of 29 (95% CI = 17 to 42) deaths, equating to an overall decrease of 349 (95% CI = 306 to 392) deaths, approximately 61%, in the three year post-intervention period 2005 to 2007.
Other DeathsOther Deaths There were no statistically significant changes in level
or slope in the post-intervention period for deaths involving other analgesics (cocodamol, codeine, codydramol, dihydrocodeine, NSAIDs, paracetamol and tramadol) which received a suicide or open verdict (both including and excluding accidental deaths).
There was a substantial though not statistically significant reduction during the post-intervention period in deaths (suicide and open verdicts) involving all drugs (including co-proxamol and other analgesics), with the mean quarterly change between 2005 and 2007 being -31 (95% CI = -66 to 3) deaths.
The overall suicide rate (including open verdicts) during this period also decreased, though to a lesser extent, and the mean quarterly change of -22 (95% CI = -89 to 45) deaths was not statistically significant.
28
Substitution of Method of Substitution of Method of SuicideSuicidePossible substitution of method must be
considered in estimating the effect of changing availability of a specific method of suicide.◦ Research evidence on failed suicide attempts
suggests that it’s unusual for completely different method to be used.
Withdrawal of co-proxamol was associated with changes in prescribing of other analgesics.◦ Significant increases in prescribing of co-codamol,
paracetamol, and codydramol occurred during 2005-2007.
Analyses of suicide and open verdict deaths involving other analgesics combined indicated little evidence of substitution.
29
NSAIDsNSAIDsAn abrupt reduction in prescribing of
NSAIDs occurred shortly before the announcement of the withdrawal of co-proxamol due to concerns about Cox 2 inhibitors.
However, NSAIDs are rarely a direct acute cause of death, especially by suicide.
30
InterpretationInterpretation
Following the announcement of the withdrawal of co-proxamol in January 2005 there was an immediate large reduction in prescriptions. This was associated with a 62% reduction in suicide deaths (including open verdicts), or an estimated 295 fewer deaths.
Inclusion of accidental deaths, some of which were likely to have been suicides increased the estimated reduction in number of deaths to approximately 349 over 3 years.
Overall suicide and open verdict deaths decreased in England and Wales during 2005 to 2007. Thus underlying downward trends in suicide cannot explain the full extent of the decrease in co-proxamol related deaths following the MHRA announcement to withdraw co-proxamol.
LimitationsLimitations Interrupted time series autoregression controls for
baseline level and trend when estimating expected changes in the number of prescriptions (or deaths) due to the intervention.
The estimates of the overall effect on prescriptions and mortality involved extrapolation, which is inevitably associated with uncertainty.
The regression method assumes linear trends over time, and the co-proxamol prescribing data, in particular, had a poor fit, resulting in large standard errors in the post-intervention period.
Estimates of the standard errors for absolute mean quarterly changes in number of prescriptions or deaths were determined exactly, including the covariance of level and slope terms.
Estimates of percentage changes over the three year post-estimation period are point estimates and were not determined with standard error calculations.
Threats to ValidityThreats to ValidityCo-proxamol prescription only drug, often
given to elderly patients suffering arthritisMost poisoning suicides are in younger people,
so co-proxamol use considered to be opportunistic
Recording of suicide may be coroner dependent
Open verdict may be given when there is lack of suicide note
Prescriptions numbers were reducing as GPs tried to move patients to alternative analgesics prior to withdrawal so some diffusion of intervention
33
Design ConsiderationsDesign ConsiderationsPost-hoc analysisVery difficult to identify suitable
non-equivalent no treatment control group
Inclusion of overall suicide rates goes some way towards examination of validity threat
34
ProblemsProblems Diffusion of intervention but in this case prior to
identified intervention point as demonstrated by decrease in prescriptions prior to announcement
Data series rather shorter than the ideal of 100 observations, but minimum of 12 before and after intervention point considered not unreasonable
Quarterly figures give reasonable time intervals There’s no concrete evidence of a definitional shift in
suicides in this time interval although this cannot be ruled out
Open verdicts were included as a sensitivity check as one way of addressing potential missing data
Almost impossible to consider impact of missing data on agent used in self-poisoning
35
Guidelines on UseGuidelines on UseRamsay et al, 2003
◦ Quality criteria Intervention occurred independently of other changes
over time Intervention was unlikely to affect data collection The primary outcome was assessed blindly or was
measured objectively The primary outcome was reliable or was measured
objectively The composition of the data set at each time point
covered at least 80% of the total number of participants in the study
The shape of the intervention effect was prespecified A rationale for the number and spacing of data points
was described The study was analyzed appropriately using time series
techniques
Findings of the Systematic Findings of the Systematic ReviewsReviews Mass media review of 20 studies, Guideline
dissemination and implementation review of 38 studies◦ Most studies had short time series
Standard errors increased Reduced power Type I error increased Failure to detect autocorrelation or secular trends
◦ Over 65% analysed inappropriately Of the 37 re-analysed, 8 had significant pre-
intervention trends◦ Most were underpowered
Rule of thumb: with 10 pre- and 10 post-intervention time points the study would have at least 80% power to detect a change in level of five standard deviations of the pre-data if the autocorrelation >0.4
Long pre-intervention phase increases power to detect secular trends
ReferencesReferences Shadish, Cook and Campbell, 2002, Experimental and quasi-
experimental designs for generalised causal inference, Houghton Mifflin.
Ramsay CR, Matowe L, Grilli R, Grimshaw JM, Thomas RE. Interrupted time series designs in health technology assessment: Lessons from two systematic reviews of behavior change strategies. Int.J.Technol.Assess.Health Care 2003;19:613-23
Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J.Clin.Pharm.Ther. 2002;27:299-309
Zhang, F, Wagner, A, Soumerai, S. B., and Ross-Degnan, D. Estimating confidence intervals around relative changes in outcomes in segmented regression analyses of time series data. 15th Annual NESUG (NorthEast SAS Users Group Inc) Conference Last update 2002. http://www.nesug.info/Proceedings/nesug02/st/st005.pdf. Accessed 22 October 2008.
Examples of UseExamples of Use Matowe, L, Ramsay, C. R., Grimshaw, J. M., Gilbert F.
J., MacLeod, M.-J. and Needham, G. Effects of mailed dissemination of the Royal College of Radiologists’ Guidelines on general practitioner referrals for radiography: a time series analysis. Clinical Radiology 2002, 57, 575-578
Neustrom, M. W. and Norton, W. M. The impact of drunk driving legislation in Louisiana. Journal of Safety Research, 1993, 24, 107-121
Ansari, F, Gray, K, Nathwani, D, Phillips, G, Ogston, S, Ramsay, C and Davey, P. Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. Journal of Antimicrobial Chemotherapy, 2003, 52, 842-848
Morgan, O. W., Griffiths, C and Majeed, A. Interrupted time-series analysis of regulations to reduce paracetamol (acetaminophen) poisoning. PLoS Medicine, 2007, 4, 0654-0659
K. Hawton, H. Bergen, S. Simkin, A. Brock, C. Griffiths, E. Romeri, K. L. Smith, N. Kapur, D. Gunnell (2009). Effect of withdrawal of co-proxamol on prescribing and deaths from drug poisoning in England and Wales: time series analysis. BMJ, 338:b2270