Authors and Affiliations:

James D Stefaniak BM BCh1, Thomas C H Lam MD2, Naomi E Sim MD3, Rustam Al-Shahi

Salman MA PhD FRCP4, David P Breen MRCP PhD5*

1Department of Neurology, Addenbrooke’s Hospital, Cambridge, England

2Department of Surgery, Kwong Wah Hospital, Yau Ma Tei, Hong Kong

3Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

4Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, Scotland

5Morton and Gloria Shulman Movement Disorders Centre and Edmond J Safra Program in

Parkinson’s disease, Toronto Western Hospital, Toronto, Canada

*Correspondence to: Dr David P Breen, Morton and Gloria Shulman Movement Disorders

Centre and Edmond J Safra Program in Parkinson’s disease, Level 7 McLaughlin Pavillion,

Toronto Western Hospital, 399 Bathurst Street, Toronto, Ontario, M5T 2S8, Canada; E-mail:

dpbreen1@gmail.com; Tel: 001 416 6035800; Fax: 001 416 6035004

Total word count (including title page, references, legends and structured abstract): 2750

DISCONTINUATION AND NON-PUBLICATION OF

NEURODEGENERATIVE DISEASE TRIALS: A CROSS-SECTIONAL

ANALYSIS

1

Running title: Discontinuation and non-publication of neurodegenerative disease trials

Keywords: Alzheimer’s disease, bias, clinical trials, multiple sclerosis, neurodegeneration,

Parkinson’s disease, research waste

2

ABSTRACT

Background and purpose: Trial discontinuation and non-publication represent major

sources of research waste in clinical medicine. No previous studies have investigated non-

dissemination bias in clinical trials of neurodegenerative diseases.

Methods: We searched ClinicalTrials.gov for all randomised, interventional, phase II-IV trials

that were registered with ClinicalTrials.gov between 1st January 2000 and 31st December

2009 and included adults with Alzheimer’s disease, multiple sclerosis, Parkinson’s disease or

motor neurone disease. We identified publications from these trials by extensive online

searching and contact with authors, and performed multiple logistic regression analysis to

identify characteristics associated with trial discontinuation and non-publication.

Results: We identified 362 eligible trials, of which 12% (42/362) were discontinued. 28%

(91/320) of completed trials remained unpublished after five years. Trial discontinuation

was independently associated with number of patients (p=0.015; more likely in trials with

≤100 patients; odds ratio 2.65, 95% confidence interval 1.21-5.78) and phase of trial

(p=0.009; more likely in phase IV than phase III trials; odds ratio 3.90, 95% confidence

interval 1.41-10.83). Trial non-publication was independently associated with blinding status

(p=0.005; more likely in single-blind than double-blind trials; odds ratio 5.63, 95%

confidence interval 1.70-18.71), number of centres (p=0.010; more likely in single-centre

than multi-centre trials; odds ratio 2.49, 95% confidence interval 1.25-4.99), phase of trial

3

(p=0.041; more likely in phase II than phase IV trials; odds ratio 2.88, 95% confidence

interval 1.04-7.93), and sponsor category (p=0.001; more likely in industry-sponsored than

university-sponsored trials; odds ratio 5.05, 95% confidence interval 1.87-13.63).

Conclusions: There is evidence of non-dissemination bias in randomised trials of

interventions for neurodegenerative diseases. Associations with trial discontinuation and

non-publication were similar to findings in other diseases. These biases may distort the

therapeutic information available to inform clinical practice.

4

INTRODUCTION

Trial discontinuation and non-publication represent major sources of research waste in

clinical medicine. Publicly available registers of trials have been established in the hope that

increased transparency surrounding trial registration will lead to fewer being discontinued

and fewer remaining unpublished. Since 2004, journals affiliated with the International

Committee of Medical Journal Editors have required prospective registration of all clinical

trials in a public database [1]. CONSORT guidelines similarly state that registration numbers

must be reported when publishing randomised trials [2]. In the United States, registration

on ClinicalTrials.gov – the online database run by the National Library of Medicine - is

mandatory for trials involving drugs or devices subject to Food and Drug Administration

regulation [3]. Despite these stipulations, up to half of registered trials never result in a

publication [4].

No previous studies have investigated the factors influencing trial discontinuation and non-

publication (non-dissemination bias) in clinical trials of neurodegenerative diseases. This is

important given the increasing prevalence of these disorders within our ageing population,

and the need for symptomatic and disease-modifying therapies that will require testing in

randomised trials.

We investigated the characteristics associated with trial discontinuation and non-publication

in four major neurodegenerative diseases (Alzheimer’s disease [AD], multiple sclerosis [MS],

5

Parkinson’s disease [PD] and motor neurone disease [MND]), by performing a cross-

sectional analysis of the ClinicalTrials.gov database.

METHODS

ClinicalTrials.gov search strategy

TCHL and NES searched ClinicalTrials.gov using its ‘Advanced Search’ facility with the

following key words: ‘Parkinson’; ‘Alzheimer’; ‘motor neuron OR amyotrophic lateral

sclerosis’; and ‘multiple sclerosis’. Two authors (TCHL/NES and JDS) independently reviewed

all trial entries and decided which met inclusion criteria. We specified the following inclusion

criteria a priori: interventional trials; randomised trials with at least one comparator group;

phase II, III or IV trials; trials involving adults and seniors; and trials registered between 1st

January 2000 and 31st December 2009. There were no restrictions in terms of language,

blinding status or methodological quality. We included trials of several conditions if the

neurodegenerative disease of interest was one of the conditions. We excluded purely

observational studies; trials of children only; or trials listed as ‘unknown’/‘enrolling by

invitation’.

We extracted the following variables from each trial’s database entry: disease, trial status,

blinding status, number of centres, number of patients, phase of trial, sponsor category, and

type of intervention. We considered ‘terminated’, ‘withdrawn’ or ‘suspended’ trials to be

6

discontinued. We considered ‘phase II/III’ trials as phase III. We defined ‘industry’ as any

commercial source; ‘healthcare organisation’ as any organisation directly providing clinical

care; and ‘funding organisation’ as any funding body outside of industry that was not a

university or healthcare organisation. We classed ‘non-pharmacological’ interventions as

devices, changes to participants’ behaviour, or procedures. We resolved cases of

disagreement in category selection by consensus with the senior author (DPB).

Publication search strategy

We defined ‘publications’ as peer-reviewed journal articles containing outcome data. If a

link to an included publication was not present on ClinicalTrials.gov, we searched PubMed

using National Clinical Trial (NCT) number, trial name, author names and keywords.

Publications that did not explicitly state the NCT number were discussed with the senior

author before deciding whether they corresponded to the particular ClinicalTrials.gov trial,

using the trial variables listed above to guide such decisions. If multiple publications were

identified, we used the earliest date. If no publication was found for a completed trial, we

emailed the author (defined as the investigator, sponsor or collaborator listed on

ClinicalTrials.gov) to ask whether the trial had been published, and sent a repeat email after

one week if no reply was received. We found email addresses from ClinicalTrials.gov or

online searching.

7

Statistical Analysis

We used SPSS version 23 to perform bivariate analysis for each trial characteristic using Chi-

squared testing or, if less than five trials per category, Fisher’s exact test to investigate

whether these differed in discontinued or unpublished trials. We defined statistical

significance as p<0.05 with Bonferroni correction. We performed post hoc z-test of two

proportions if a statistically significant trial characteristic had more than two categories. We

included variables with the greatest statistical significance on bivariate analysis in multiple

logistic regression analysis, but limited the number of variables to ensure an ‘events per

predictive variable ratio’ of at least 10. We included the following characteristics as

independent variables in multiple logistic regression against trial discontinuation: blinding

status, number of patients and phase of trial. We included the following characteristics as

independent variables in multiple logistic regression against trial non-publication: disease,

blinding status, number of centres, number of patients, phase of trial, sponsor category and

type of intervention. The reference category for multiple logistic regression analysis was

chosen as the category with the lowest rate of trial discontinuation or non-publication on

bivariate analysis. Trials with missing data were not included in the relevant bivariate or

multivariate analyses.

IRB approval, informed consent and trial registration

No ethical approval, patient consent or trial registration required.

8

RESULTS

Of the 2072 trials identified by ClinicalTrials.gov involving any of the neurodegenerative

diseases of interest, we excluded 1710 and included 362 (Figure 1). 12% (42/362) of

included trials were discontinued; most commonly reported reasons were recruitment

difficulties (31%, 13/42) and lack of efficacy (14%, 6/42). After emailing study authors for

clarification, we found 72% (229/320) of completed trials to have been published.

Trial characteristics associated with discontinuation on bivariate analyses (Table 1) were

number of patients (p=0.002; more discontinuation in trials with ≤100 patients [16.8%,

29/173] compared to >100 patients [6.2%, 11/176]) and phase of trial (p=0.001; more

discontinuation in phase IV trials [25.0%, 14/56] compared to phase II [11.6%, 21/181] or

phase III [5.6%, 7/125] trials). We found no other statistically significant associations. 13/42

(31%) discontinued trials did not report a reason for discontinuation, and phase IV trials

(7.1%, 1/14) were less likely than phase III trials (57.1%, 4/7) to cite ‘lack of efficacy’ as a

reason for discontinuation (p=0.005). We used the three most significant variables on

bivariate analyses (phase of trial, number of patients and blinding status) as independent

variables in multiple logistic regression analysis (with trial discontinuation as the dependent

variable). Number of patients (p=0.015; discontinuation more likely in trials with ≤100

patients than >100 patients; odds ratio 2.65, 95% confidence interval 1.21-5.78) and phase

of trial (p=0.009; discontinuation more likely in phase IV than phase III trials; odds ratio 3.90,

9

95% confidence interval 1.41-10.83) were independently associated with discontinuation

(Table 1).

Trial characteristics associated with non-publication on bivariate analyses (Table 1) were

number of centres (p=0.004; more non-publication in single-centre trials [37.4%, 40/107]

compared to multi-centre trials [22.1%, 45/204]), number of patients (p=0.008; more non-

publication in trials with ≤100 patients [36.1%, 52/144] compared to >100 patients [21.8%,

36/165) and phase of trial (p=0.001; more non-publication in phase II trials [37.5%, 60/160]

compared to phase III [20.3%, 24/118] or phase IV [16.7%, 7/42] trials). We found no other

statistically significant associations. Multiple logistic regression included all variables from

bivariate analyses (with trial non-publication as the dependent variable). Blinding status

(p=0.005; non-publication rate higher in single-blind than double-blind trials; odds ratio

5.63, 95% confidence interval 1.70-18.71); number of centres (p=0.010; non-publication rate

higher in single-centre than multi-centre trials; odds ratio 2.49, 95% confidence interval

1.25-4.99); phase of trial (p=0.041; non-publication rate higher in phase II than phase IV

trials; odds ratio 2.88, 95% confidence interval 1.04-7.93); and sponsor category (p=0.001;

non-publication rate higher in industry-sponsored than university-sponsored trials; odds

ratio 5.05, 95% confidence interval 1.87 to 13.63) were independently associated with non-

publication (Table 1).

10

Table 1

Trial characteristic Trial discontinuation Trial non-publicationTotal (number

of trials)Discontinued(number of trials [%])

Bivariate analysis

Multiple logistic regression Total (number of trials)

Not published(number of

trials, %)

Bivariate analysis

Multiple logistic regression

P value Odds ratio (95% CI) P value P value Odds ratio (95% CI) P valueDisease 0.482 na 0.459 0.528

Alzheimer’s disease 54 8 (14.8) na na 46 12 (26.1) 1.51 (0.60-3.79) 0.380Motor neurone

disease30 3 (10.0) na na 27 7 (25.9) 1.51 (0.50-4.58) 0.469

Multiple sclerosis 113 16 (14.2) na na 97 23 (23.7) Reference ReferenceParkinson’s disease 165 15 (9.1) na na 150 49 (32.7) 1.65 (0.84-3.22) 0.144

Blinding status 0.475 0.828 0.137 0.015No blinding 48 6 (12.5) 0.89 (0.33-2.42) 0.821 42 13 (31.0) 1.74 (0.71-4.24) 0.223Single-blind 21 4 (19.0) 1.39 (0.42-4.60) 0.592 17 8 (47.1) 5.63 (1.70-18.71) 0.005

Double-blind 286 32 (11.2) Reference Reference 254 65 (25.6) Reference ReferenceMissing 7 0 na na 7 5 na na

Number of centres 0.822 na 0.004 naSingle-centre 119 12 (10.1) na na 107 40 (37.4) 2.49 (1.25-4.99) 0.010Multi-centre 225 21 (9.3) na na 204 45 (22.1) Reference Reference

Missing 18 9 na na 9 6 na naNumber of patients 0.002 na 0.008 na

<100 173 29 (16.8) 2.65 (1.21-5.78) 0.015 144 52 (36.1) 1.88 (0.97-3.65) 0.063>100 176 11 (6.3) Reference Reference 165 36 (21.8) Reference Reference

Missing 13 2 na na 11 3 na naPhase of trial 0.001

(IV > II/III)0.009 0.001

(II > III/IV)0.078

II 181 21 (11.6) 1.32 (0.51-3.44) 0.566 160 60 (37.5) 2.88 (1.04-7.93) 0.041III 125 7 (5.6) Reference Reference 118 24 (20.3) 1.78 (0.61-5.17) 0.292IV 56 14 (25.0) 3.90 (1.41-10.83) 0.009 42 7 (16.7) Reference Reference

Sponsor category 0.663 na 0.140 0.003Industry 234 26 (11.1) na na 208 65 (31.3) 5.05 (1.87-13.63) 0.001

University 64 10 (15.6) na na 54 10 (18.5) Reference ReferenceFunding organisation 35 4 (11.4) na na 31 11 (35.5) 1.22 (0.35-4.26) 0.759

Healthcare organisation

29 2 (6.9) na na 27 5 (18.5) 1.14 (0.31-4.24) 0.840

Type of intervention 1.000 na 0.938 naPharmacological 327 38 (11.6) na na 289 82 (28.4) Reference Reference

Non-pharmacological 35 4 (11.4) na na 31 9 (29.0) 0.74 (0.25-2.19) 0.591

11

DISCUSSION

Over a ten-year period, 12% (42/362) of clinical trials of neurodegenerative diseases

registered on ClinicalTrials.gov were discontinued prematurely. Moreover, 28% (91/320) of

completed trials remained unpublished after five years. In total, 12,442 patients contributed

to these unpublished trials of neurodegenerative diseases. These figures are comparable to

previously reported rates in other medical specialties [5, 6].

Examining the factors that influence trial discontinuation and non-publication may identify

ways of reducing research waste in the future. We identified two characteristics that

influenced trial discontinuation: number of patients and phase of trial. One possible reason

for phase IV trials being prematurely discontinued would be to avoid ongoing costs for a

treatment lacking efficacy or leading to adverse events. However, this was not supported by

our results when we examined the reasons for trial discontinuation, although almost one-

third of studies did not cite a reason at all.

We identified a number of characteristics that influenced trial non-publication: blinding

status, number of centres, phase of trial and sponsor category. Single-blind and single-

centre trials are generally considered less methodologically robust when assessing

intervention efficacy, and might therefore be less likely to be accepted for publication. In

the same way, a phase II trial might be considered an intermediate step before a late phase

study can be performed, making it less likely to be submitted for publication. A recent

12

systematic review of trials into medical conditions similarly found decreasing number of

centres and decreasing phase of trial were positively associated with trial non-publication

[4]. Our study found that industry-sponsored trials were significantly less likely to be

published, confirming that neurodegenerative disease research is not immune to the

influence of industry sponsorship on publication rates that has been observed in other

medical conditions [4-6]. This might reflect the desire to protect intellectual property of

early stage therapeutics; however, it might be to conceal negative results. Whilst

recognising the important role of industry in funding research into neurodegenerative

diseases, our results reiterate the importance of creating a regulatory system in which

publication of trial results is mandatory.

The strengths of this study are the large number of trials included from the largest trial

registration database available and the significant efforts made to contact authors to ask

about missing publications. Whilst we did not search other publicly available trial

registration databases, it is known that two-thirds of all studies registered between 2005-

2013 were on ClinicalTrials.gov [7], meaning that we are likely to have captured a significant

proportion of trials performed during the specified time period.

Potential limitations of the study include the exclusion of non-randomised trials, which

might have led to an underestimation of non-publication rates. Restricting our search to

ClinicalTrials.gov might also have led to an underestimation of discontinuation or non-

publication rates because some smaller, single centre studies may only be registered in

13

national trial registries. Our study design means that trials completed before 2010 but

published after 2015 might have been missed. Furthermore, registration and publication

patterns might have changed since the studied time period.

In summary, this is the first study to investigate non-dissemination bias in clinical trials of

neurodegenerative diseases. It is imperative that greater efforts are made to improve the

completion and publication of clinical trials of neurodegenerative diseases since ‘hidden

data’ can impede scientific advancement by overestimating benefits and underestimating

harm [8], as well as wasting money and exposing research patients to unnecessary risk.

Ongoing efforts include the ‘REduce research Waste And Reward Diligence (REWARD)’

campaign launched by The Lancet in 2015, which invites organisations involved in

biomedical research to commit to increasing value and reducing waste [9]. Journals have

been established specifically for the purpose of publishing statistically non-significant

findings, making this reason for non-publication unjustifiable. Ultimately, researchers have

an ethical responsibility to disseminate results from all trials, and the research community

has a responsibility to facilitate and study this transfer of information [10]. This should

include regulatory changes if necessary, as well as greater use of open-access publishing and

mechanisms for sharing patient-level data [11].

Acknowledgements: None

14

Disclosure of conflicts of interest: None

Funding: None

Contributorship Statement: JDS collected the data, carried out the statistical analysis,

interpreted the data, and drafted the manuscript. TCHL and NES collected the data and

reviewed the manuscript. RASS revised the manuscript. DPB conceived the study,

supervised the data collection, interpreted the data, and revised the manuscript. All authors

gave final approval for the article to be published.

REFERENCES

[1] International Committee of Medical Journal Editors.

http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/

clinical-trial-registration.html (accessed 03/04/2017).

[2] Moher D, Schulz KF, Altman D, CONSORT Group (Consolidated Standards of Reporting

Trials). The CONSORT statement: revised recommendations for improving the quality of

reports of parallel-group randomized trials. JAMA 2001; 285: 1987-1991.

[3] National Library of Medicine (US). https://clinicaltrials.gov/ct2/about-site/background

(accessed 03/04/2017).

15

[4] Schmucker C, Schell L, Portalupi S, et al. Extent of Non-Publication in Cohorts of Studies

Approved by Research Ethics Committees or Included in Trial Registries. PLoS One 2014;

9: e114023.

[5] Saito H, Gill C. How Frequently Do the Results from Completed US Clinical Trials Enter

the Public Domain? – A Statistical Analysis of the ClinicalTrials.gov Database. PLoS One

2014; 9: e101826.

[6] Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, Platts-Mills TF. Non-publication of

large randomized clinical trials: cross sectional analysis. British Medical Journal 2013;

347: f6104.

[7] Viergever R, Li K. Trends in global clinical trial registration: an analysis of numbers of

registered clinical trials in different parts of the world from 2004 to 2013. British Medical

Journal Open 2013; 5: e008932.

[8] Eyding D, Lelgemann M, Grouven U, et al. Reboxetine for acute treatment of major

depression: systematic review and meta-analysis of published and unpublished placebo

and selective serotonin reuptake inhibitor controlled trials. British Medical Journal 2010;

341: c4737.

[9] The Lancet. http://www.thelancet.com/campaigns/efficiency/statement (accessed

03/04/2017).

[10] Glasziou P, Altman DG, Bossuyt P, et al. Reducing waste from incomplete or unusable

reports of biomedical research. Lancet 2014; 383: 267-276.

[11] Strom BL, Buyse ME, Hughes J, Knoppers BM. Data Sharing – Is the Juice Worth the

Squeeze? New England Journal of Medicine 2016; 375: 1608-1609.

16

FIGURE LEGEND

Figure 1: Flow chart showing numbers of included, completed and published trials

TABLE LEGEND

Table 1: Characteristics associated with trial discontinuation and non-publication. Table

shows the number of trials for each characteristic that were discontinued or not published.

Bivariate analyses within each category were performed using Chi-squared test or, if less

than 5 trial in a category, Fisher’s exact test.

17