BMJ OpenPR1 2HE, UK Corresponding author: ... Naoimh McMahon, Clinical Practice Research Unit,...

For peer review only

Activities to support the implementation of complex interventions as part of routine care: a review of the quality

of reporting in cluster randomised controlled trials

Journal: BMJ Open

Manuscript ID: bmjopen-2015-008251

Article Type: Research

Date Submitted by the Author: 20-Mar-2015

Complete List of Authors: McMahon, Naoimh; University of Central Lancashire, Clinical Practice Research Unit Holland, Emma-Joy; University of Central Lancashire, Clinical Practice

Research Unit Miller, Colette; University of Central Lancashire, Clinical Practice Research Unit Patel, Kulsum; University of Central Lancashire, Clinical Practice Research Unit Connell, Louise; University of Central Lancashire, Clinical Practice Research Unit

<b>Primary Subject Heading</b>:

Health services research

Secondary Subject Heading: Health services research

Keywords: n/a, n/a, n/a

For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml

BMJ Open on S

eptember 17, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2015-008251 on 9 O

ctober 2015. Dow

nloaded from

http://bmjopen.bmj.com/


- 1 -

Activities to support the implementation of complex interventions as part

of routine care: a review of the quality of reporting in cluster randomised

controlled trials

Naoimh E McMahon, research physiotherapist, Emma-Joy Holland, senior research assistant,

Colette Miller, research assistant, Kulsum Patel, senior research assistant, Louise A Connell,

NIHR career development fellow

Clinical Practice Research Unit, School of Health, University of Central Lancashire, Preston,

PR1 2HE, UK

Corresponding author:

Naoimh McMahon,

Clinical Practice Research Unit,

School of Health,

University of Central Lancashire

[email protected]

01772 893654

Page 1 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem

ber 17, 2020 by guest. Protected by copyright.

http://bmjopen.bm

j.com/

BM

J Open: first published as 10.1136/bm

jopen-2015-008251 on 9 October 2015. D

ownloaded from



- 2 -

Abstract

Objective

To review a sample of cluster randomised controlled trials (RCTs) and explore the quality of

reporting of (i) enabling or support activities provided to the staff during the trial, (ii)

strategies used to monitor fidelity throughout the trial and (iii) the actual implementation of

the intervention.

Design

A descriptive review.

Data sources and study selection

We searched MEDLINE for trial reports published between 2008 and 2014 with

combinations of the search terms ‘randomised’, ‘cluster’, ‘trial’ ‘study’, ‘intervention’ and

‘implement*’. We included trials in which healthcare professionals (HCPs) implemented the

intervention being tested as part of routine practice. We excluded trials (i) conducted in non-

health services settings, (ii) where the intervention explicitly aimed to change the behaviours

of the HCPs and (iii) where the trials were ongoing or for which only trial protocols were

available.

Data collection

We developed a data extraction form using the Template for Intervention Description and

Replication (TIDieR checklist). The authors independently extracted data from the included

trials and assessed quality of reporting for each of the checklist components.

Results

We included 63 publications (45 results publications, 18 protocols). 89% of trials reported

using enabling or support activities. How these activities were provided (75.6%, n=34), and

how much was provided (71.1%, n=32), were the most frequently reported components. Less

than 20% (n=8) of the included trials reported that competency checking took place prior to

Page 2 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 3 -

implementation and data collection. 58% (n=26) of trials reported collecting measures of

fidelity. 40% (n=18) of the trials reported data from these measures.

Conclusions

Although enabling and support activities are reported in trials, important gaps exist when

assessed using the TIDieR checklist. Better reporting of the supports provided in

effectiveness trials will allow for informed decisions to be made on the financial and resource

implications of wide scale implementation of effective interventions.

Article summary

Strengths and limitations

• Enabling or support activities used to facilitate implementation of interventions in

effectiveness trials should be described in sufficient detail to allow interpretation of

results and future replication

• In this study we used a published checklist, TIDieR (the Template for Intervention

Description and Replication), to comprehensively assess the quality of reporting of

enabling or support activities in included trials

• We did not rate the quality of reporting, as other studies have, but assessed whether or

not authors reported details for each of the items of the TIDieR checklist. The figures

provided here may therefore present a more positive picture than would be the case if

we assessed the extent to which interventions could be replicated from the details

provided

• Our search strategy was designed to identify a sample of effectiveness trials

conducted in health services settings and a large number of eligible published trials

may not have been identified potentially limiting the generalisability of the results

Page 3 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 4 -

Introduction

A seminal publication in 2009 by Chalmers and Glasziou identified unusable research reports

as a primary contributor to avoidable waste in research production [1]. Despite

comprehensive guidelines to assist with the reporting of clinical trials, for example the

Consolidated Standards of Reporting Trials (CONSORT) 2010 statement [2] and the

Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013

statement [3], gaps in the completeness of reports remain. Trials of complex non-

pharmacological interventions have proved a particular challenge. These interventions have a

number of interacting components and have been found to be adequately described in only

39% of cases [4].

The development and evaluation of complex interventions occurs in distinct phases starting

with initial theory identification and modelling, progressing to pilot and exploratory studies

to test the feasibility of the developed intervention (and the feasibility of conducting a large

scale evaluation), before a final definitive evaluation in the form of a sufficiently powered

randomised controlled trial (RCT) [5]. Such trials have traditionally been designed to be more

explanatory in nature i.e. demonstrating the efficacy of developed interventions under

optimum conditions [6]. However, numerous challenges exist in attempting to translate

research evidence from explanatory trials into real life clinical practice. Most notably lack of

external validity, or generalisability, which has been a long standing criticism of the RCT

methodology [7]. To address this issue, there has been a dramatic increase in the number of

effectiveness trials being carried out which explore the degree of beneficial effect of

interventions when delivered under real world conditions [8]. The findings of these trials, as

one would expect, are of particular interest to clinicians and policy makers but are not

without their methodological limitations [9 10]. As a result of the growing interest in more

pragmatic trial designs, we have also seen increased use of cluster randomisation in health

Page 4 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 5 -

services research (i.e. where groups of patients rather than individual patients are

randomised). Cluster randomised designs are particularly appropriate when there is a risk of

contamination across trial groups when trial patients are managed within the same setting

[11].

Trials that aim to demonstrate the effectiveness of an intervention under real world

conditions, will often evaluate the intervention when it is delivered by routine staff as part of

routine care. Successful implementation therefore requires a change in the practices of these

staff [5]. An assessment as to whether or not an intervention has been “successfully”

implemented can be made by measuring implementation fidelity (the degree to which an

intervention is implemented as intended) [12]. To date the practice of reporting fidelity in

published studies has been poor [13-15]. However, more recently the importance of

monitoring and reporting fidelity has been highlighted through the development of influential

frameworks for addressing fidelity [12 16] and reporting guidelines. The ‘CONSORT’

extension for the reporting of RCTs of non-pharmacological treatments [17] and the

Template for Intervention Description and Replication (TIDieR) checklist [18], prompt

authors to report on any planned strategies used to maintain or improve fidelity and to report

actual implementation of the intervention being tested. The TIDieR checklist also prompts

authors to describe “any enabling or support activities” used to assist with the implementation

of the intervention (item 4).

Previous studies exploring the completeness of intervention descriptions in randomised

controlled trials have not differentiated between different trial designs and have not explicitly

assessed the completeness of reporting of enabling or support activities used to assist routine

staff in implementing the interventions being tested [4 19-21]. This is of increasing

importance due to the number of effectiveness trials being conducted and the increasing

emphasis being placed on implementation theory and research to inform the long term

Page 5 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 6 -

implementation of effective interventions. In this study we reviewed a sample of cluster

randomised controlled trials in which routine staff implemented the interventions being tested

as part of routine care to explore the quality of reporting of (i) enabling or support activities

provided to the staff during the trial, (ii) strategies used to monitor fidelity throughout the

trial and (iii) the actual implementation of the intervention.

Methods

Search strategy and selection of reports of trials

To identify a sample of trial reports for the review we searched MEDLINE for publications

with combinations of the search terms ‘randomised’, ‘cluster’, ‘trial’ ‘study’, ‘intervention’

and ‘implement*’ in their titles or abstracts. To manage the scope of the review we included

reports of cluster randomised controlled trials published between 2008 and 2014. This date

range was chosen as the Medical Research Council (MRC) guidance on developing and

evaluating complex interventions was published in 2008 [5], a seminal publication which has

influenced the quality of reporting in trials of complex interventions. We included trials in

which healthcare professionals (HCPs) implemented the intervention being tested as part of

their routine practice. We excluded trials (i) conducted in non-health services settings (e.g.

schools, universities), (ii) where the intervention being tested explicitly aimed to change the

behaviours of the HCPs (e.g. behaviour change interventions (BCIs) such as screening tools,

decision aids, audit and feedback) and (iii) if the trials were ongoing or for which only trial

protocols were available.

NM conducted the first round of screening to remove irrelevant trials based on the exclusion

criteria. The authors then independently screened the titles and abstracts of the remaining

trials for inclusion in pairs. Discrepancies were resolved by discussion or by a third party. In

instances where it was not possible to exclude trials based on title and abstract, full text

Page 6 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 7 -

versions were obtained and were independently assessed for inclusion by the authors.

Relevant publications related to the included trials (i.e. trial protocols, process evaluations)

were identified where possible by using the trial registration number or by manually

searching for publications where the primary investigator named on the trial registration was

an author.

Data extraction and analysis

A data extraction spreadsheet based on the TIDieR checklist [18] was developed for the purposes

of the review (See Appendix I – Data extraction form and completed example). Data was

extracted on: the rationale for the support or enabling activities, materials and procedures used,

where the activities were delivered, by who, when and how often and if the training was

accredited or competency checked. Any data on tailoring or modifications to the activities was

extracted along with planned strategies to monitor implementation fidelity, and actual

implementation of the intervention during the trial. In pairs the review authors independently

extracted the required information from a third of the trials (NM was part of two pairs). The

extracted data was cross-checked in the pairs and findings were discussed until consensus for

each section was reached and the authors agreed that all potentially relevant information had been

extracted.

Previous studies have rated intervention descriptions as adequate/complete or

inadequate/incomplete [4 20]. Due to the subjective nature of these types of ratings (i.e. there is

not yet consensus as to what information must be reported in what level of detail to constitute an

adequate description of an intervention) it was decided by the study team to assess each section of

the TIDieR checklist as ‘reported’ or ‘not reported’, and where information was reported to

extract examples of more detailed reporting and more typical reporting. All authors independently

assessed the included trials and crosschecking of the assessments was carried out. Microsoft

Excel was used to analyse the data descriptively.

Page 7 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 8 -

Results

We identified 630 citations from our initial search. This became 563 when duplicate citations

were removed. Trials that were conducted in a non-health service setting were excluded in

the first round of screening (n=240) and the eligibility criteria applied to the remaining 323

citations. A total of 63 citations were included for data extraction (45 trial results publications

and 18 related protocols). The reasons for exclusion are shown in Table 1. Process evaluation

publications were identified for 3 of the included trials [22-24]. Due to the small number of

process evaluations identified, and the nature of their content, we extracted data from trial

protocols and results publications only. All publications included in data extraction are

provided in Appendix II – Publications included in data extraction.

Table 1: Reasons for exclusion of trials

Reasons n

Testing behaviour change intervention targeting

professional practices of HCPs

155

No access 19

Non-HCP/mixed HCP and non-HCP 19

Non-health services setting 7

Not relevant 16

Ongoing/no published results 35

Intervention not delivered by routine staff (e.g.

delivered by research team)

19

Unclear who delivered the intervention 8

Total 278

Description of included trials

The trials included in the review span 22 countries. The country with the most included trials

was the Netherlands (n=7), followed by Australia (n=5), Spain (n=5) and the UK (n=5). The

trials results publications came from 33 different journals including the BMJ (n=5), PLoS

One (n=4) and the Lancet (n=3). The topics most frequently addressed were type 2 diabetes

Page 8 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 9 -

(n=5), asthma (n=4), cardiovascular disease, hypertension and stroke (n=4) and maternal and

child health (n=4). Just over half of the trials were conducted in primary care settings (n=23).

The interventions were primarily lifestyle/self-management interventions (n=22). The

healthcare professions implementing the interventions in the trials were nurses (n=12),

general practitioners (n=10), pharmacists (n=4), intensive care unit staff (n=2), community

teams (n=2), dieticians (n=1), hospital doctors (n=1) and allied health professions (n=1).

Mixed professions or multidisciplinary teams were responsible for implementing the

intervention in 12 of the trials.

Quality of reporting of enabling or support activities

40 of the 45 trials (88.8%) made reference to some form of enabling or support activities

conducted during the trial. Of the five trials that did not describe any enabling or support

activities, three of these were evaluating infection control interventions in hospital settings.

The number of trials providing information on each of the items of the TIDieR checklist, as

applied to enabling or support activities, is shown in Table 2. How activities were provided,

and how much, were the most frequently reported items (75.5% and 71.1% of trials

respectively). Reporting of the rationale for the chosen activities, any tailoring or

modifications of the activities and measures of planned and actual activities carried out

during the trial were the least frequently reported. Examples of reporting, and variation across

trials for the individual components are provided in Table 3.

Table 2: Number of trials reporting components of enabling or support activities

Components of enabling or support

activities

Reported

% n

Why 6.7 3

Who provided 51.1 23

What materials 55.6 25

Where 20 9

When 46.7 21

Page 9 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 10 -

How 75.6 34

How much 71.1 32

Tailoring 2.2 1

Modifications 2.2 1

Measure of fidelity for activities 2.2 1

Actual activities delivered 2.2 1

Competency checking 17.8 8

Page 10 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 11 -

Table 3: Examples of reporting, and variation across trials for the individual components

TIDieR

Checklist

Items

More detailed reports More typical reports

Why The development and evaluation of the training intervention have

taken place prior to this trial, and details of the theoretical

background and content have been published elsewhere. The planned

approach to training combines evidence-based approaches to

changing professional behaviour with approaches to ‘normalise’

those behaviours in current practice. [25 26]

The small-group format during the training sessions (a trainer, an

adolescent actor as a simulated patient and 4 participating

physicians) allowed direct observation, feedback and suggestions by

co-participants to monitor and improve adherence to the brief

intervention. [27]

The intervention components were selected

based on a review of the literature of

evidence-based strategies for integrating

smoking cessation into primary care settings.

[28]

Who provided The education is delivered by a Diabetes Specialist Nurse, Dietitian

and Doctor all of whom have undergone training in delivery of the

DAFNE curriculum. [29 30]

The training sessions were designed by 2 of us (D.M.H. and A.M.,

both family physicians trained in adolescent health; A.M. is also a

trainer in motivational interviewing). [27]

Before the start of the study, registered

dietitians had the opportunity to gain

experience with the intervention protocol

under supervision by the project team. [31

32]

Intervention site nursing and social work SBI

providers learned to deliver MI during a 1-

day workshop delivered by the study trainer

(CD). [33]

Page 11 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on September 17, 2020 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-008251 on 9 October 2015. Downloaded from



- 12 -

What materials A desktop resource was provided with a flow chart illustrating the

elements of counselling and general communication strategies. To

prompt counselling, a summary sheet of smoking-related

characteristics (e.g., cigarettes smoked per day, degree of nicotine

dependence, and stage of change to stop smoking) was generated by

practice assistants based on each patient’s questionnaire and given to

the practitioner. The summary sheet fitted the structure of the desktop

resource to guide the proper selection of counselling elements

adequate for each patient. Additionally, the summary sheet, which

was intended to be stored in the patient’s records, served as a

reminder for the follow-up intervention which was focused on

changes compared to the initial intervention. In addition to

interactive learning methods, video demonstrations with subtitles

identifying the counselling elements and techniques were used for

the practitioner training. [34]

Several aids have been developed to facilitate the use of the SCPD:

1. The SCPD screening tool. The questions for the screening a

caregiver's sense of competence and depressive symptoms are printed

on a handy plasticized pocket card.

2. The SCPD manual. The manual consists of the items to be

discussed during the training sessions (i.e., background information

and methods), and some supporting literature has been added.

3. The starter package and action list. Several forms have been

developed for the requested data. The starter package contains the

forms that professionals need to gather these data. The action list

contains 60 possible intervening and supportive actions that

The instructors were trained and provided

with all the material needed for the theoretical

and practical parts of the thematic meetings.

Also, they were paid for the time needed for

the training, for the actual meetings and for

making the telephone calls. [37 38]

Materials and resources were made available

as hard copy, online and on CDs, to provide

flexible learning opportunities. [39 40]

A training manual was written by the research

team and provided to all nurses. [41]

Page 12 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 13 -

professionals might undertake as a result of the screening. They are

divided into nine categories: intake, diagnostics, psychoeducation,

psychosocial care, medical care, how to hand over care, legal care,

case management, and crisis management. [35 36]

Details of the individual sessions that comprise the DAFNE course

are available on the DAFNE Collaborative website and all resource

literature is available to DAFNE graduates via the DAFNE Online

website. [29 30]

Where No examples were identified of more detailed reports on

location/setting for enabling or support activities. This item of the

TIDieR checklist prompts authors to describe the type(s) of

location(s) where the intervention occurred, including any necessary

infrastructure or relevant features

Training consisted of a distance learning

package (6 hours; reading, audit of patients,

four interactive group teleconference

sessions) and two one-hour interactive

practice visits delivered by an academic GP.

[42 43]

First, the team visited NH units before

crossing over to the intervention condition,

explained the program and delivered three

tailored packages to psychologists,

physicians, and nursing staff including unit

managers and activity therapists. [44]

When This training was conducted as pre-trial workshops (HS), during

which pharmacists were instructed about the key pamphlet messages

to reinforce and were advised about the necessity of delivering these

messages strictly in accordance with the pamphlet content.

Pharmacists were encouraged to request clarification and feedback in

All nurses had previously received training in

provision of HBV vaccinations. [41]

Before and during the intervention period the

Page 13 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 14 -

regards to delivery of the messages at the time of these workshops

and throughout the trial if/as required. [45]

The on-site workshop was followed over the next 6 months by four

trainer-led 30-minute telephone coaching sessions during which MI

skills were practised and subsequently boosted with written feedback

via e-mail. Following the 6-month training period, intervention site

SBI providers delivered bedside SBI to injured patients. [33]

GP and the pharmacist are offered support by

training, education and a web based

community of practice with a helpdesk on

pharmacotherapy. [46 47]

How Intervention site nursing and social work SBI providers learned to

deliver MI during a 1-day workshop delivered by the study trainer

(CD). The workshop training targeted a 20–30-minute MI that could

be delivered at bedside to injured in-patients by the full spectrum of

trauma center providers (e.g. social work and nursing providers). The

workshop emphasized MI skill development and the importance of

spending adequate time conducting interventions by the bedside with

injured in-patients. The on-site workshop was followed over the next

6 months by four trainer-led 30-minute telephone coaching sessions

during which MI skills were practised and subsequently boosted with

written feedback via e-mail. [33]

The Healthy Relationships Training programme. Training consisted

of a distance learning package (6 hours; reading, audit of patients,

four interactive group teleconference sessions) and two one-hour

interactive practice visits delivered by an academic GP (including

one visit with a simulated patient who role-played different readiness

to change scenarios). [42 43]

Several meetings about the aspects and basic

principles of the intervention protocol took

place (i.e. the screening procedure, self-

management principles, client centeredness,

motivational interviewing, interdisciplinary

collaboration, assessment tools, parts of the

toolbox and referrals). [31 32]

The professionals were provided with a two-

hour specific training on how to apply and

use the brief assessment form regarding

asthma-like symptoms and ETS exposure. [48

49]

All nurses in all groups participated in a 4-h

training session on how to establish a

treatment contract with hypertensive patients

or patients with diabetes. [50]

Page 14 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 15 -

Practices assigned to the MC intervention group received a multi-

component intervention program to support delivery of the 5As

strategies comprising four elements: (1) outreach facilitation visits;

(2) training; (3) real time provider prompts and patient tools; and (4)

audit and performance feedback. [28]

How much An initial 15 hours of telephone coaching training over two days, and

further support and training was provided to individual nurses after

assessment of their progress. [51 52]

The process of becoming a DAFNE centre involves 43 h of training

for a DAFNE doctor and 105 h of training for each of the (at least 2)

DAFNE educators. [29 30]

The training to teach professionals to use the Systematic Care

Program for Dementia consists of three sessions of 2 hours each. [35

36]

Physicians received 24-hour training on the

study protocol, counseling, and prescription

of physical activity. [53]

All therapists in both groups received 1½

days of training and continuing expert

consultation throughout the study. [54]

All nurses were supported with meetings

every third month during the intervention. [37

38]

Tailoring No examples were identified of more detailed reports on tailoring of

enabling or support activities. This item of the TIDieR checklist

prompts authors to describe if the intervention was planned to be

personalised, titrated or adapted, then describe what, why, when,

and how

First, the team visited NH units before

crossing over to the intervention condition,

explained the program and delivered three

tailored packages to psychologists,

physicians, and nursing staff including unit

managers and activity therapists. [44]

Modifications No examples were identified of more detailed reports on

modifications to the enabling or support activities during the course

of the trial. This item of the TIDieR checklist prompts authors to

Ensuring that training was acceptable at step

3 required considerable compromise in

restricting the length and content of training

Page 15 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 16 -

describe if the intervention was modified during the course of the

study, describe the changes (what, why, when, and how)

to match the time that practices were willing

to devote. [25 26]

Measure of

planned

support

activities

No examples were identified of more detailed reports on planned

measures of enabling or support activities provided. This item of the

TIDieR checklist prompts authors to describe if adherence or fidelity

was assessed, and if so to describe how and by whom, and if any

strategies were used to maintain or improve fidelity

Evaluations of teaching sessions: facilitators

evaluate the extent of the planned content of

each teaching session by completing a

standardized evaluation form. This ensures

the consistency of the program delivery at

each site. [55 56]

Actual support

activities

No examples were identified of more detailed reports on actual

enabling or support activities delivered. This item of the TIDieR

checklist prompts authors to describe the extent to which the

intervention (i.e. the enabling or support activities in this case) was

delivered as planned

All implementation strategies were provided

to almost all new staff members except for

new caregivers, who mostly did not follow a

new depression course but were supervised

by the unit managers. [44]

Competency

checking

The readiness of GPs to manage intimate partner abuse is assessed

before and after the training using PREMIS, a validated

questionnaire assessing knowledge, attitude and behaviours of

doctors with regard to intimate partner violence. [42 43]

Evaluation of the translation of knowledge and skills into practice

was built into the course, in the form of a MAWE reflective diary

(used to facilitate learning, but also as research data). / Before

recruiting trial participants, nurses were required to give the

researcher a verbal or written example of practicing each component

of the intervention. They then made a self-assessment of confidence

and competence to practice MAWE before recruiting into the trial.

This took a period of time ranging from one to four months per

nurse. [57]

The audit was based on an inquiry of the

trained nurses proving their knowledge from

training module, nonparticipating observation

of administration of SCION program, and

additional comparison of nursing records with

study documentation for 10% of all included

patients. [58]

Participating pharmacists underwent training

and subsequent assessment of knowledge and

skills. Pharmacists were required to

demonstrate a defined standard of

competence. [39 40]

Page 16 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 17 -

Quality of reporting of strategies used to monitor fidelity throughout the trial

and actual implementation of the intervention.

Just over half of the included trials (n=26) reported using a measure of fidelity to

assess the extent to which healthcare professionals delivered the intervention as

intended within the trial. The most frequently reported strategies included notes audits

(n=11), rating of random samples of audio recorded sessions (n=5), self-reported

measures of implementation (from healthcare professionals and study participants)

(n=3) and observation of practice (n=2). Data from these measures was reported in

40% (n=18) of the trials.

Discussion

Statement of principal findings

Almost all of the trials (89%) included in this review reported that enabling or support

activities were used to facilitate implementation of the intervention being tested.

However, large discrepancies were identified in the quality of reporting of the

components of these activities. Authors most frequently reported how enabling or

support activities were provided, and how much was provided, but details related to

who provided, where, when and the underpinning rationale for the content and

delivery of the activities were often omitted. Less than 20% of the included trials

reported that competency checking took place prior to implementation and data

collection. Although, 58% of the included trials reported using specific fidelity

measures to assess the extent to which the healthcare professionals had implemented

the intervention during the trial, data from these measures was only reported in 40%

of the trials.

Page 17 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 18 -

Strengths and weaknesses of the study

A strength of this study is the use of the published TIDieR checklist (the Template for

Intervention Description and Replication) to assess the quality of reporting of the

included trials [18]. TIDieR has been developed by Hoffmann and colleagues as an

extension to Item 5 of the CONSORT Statement and aims to guide authors in

describing interventions in a level of detail to allow for future replication. Although

the primary purpose of the TIDieR checklist is to assist in the describing of

interventions being tested, item 4 (what procedures) outlines that some interventions,

particularly complex ones, might require additional activities to enable or support the

intervention to occur, and these should also be described. We would argue, and have

demonstrated here, that these activities are interventions in themselves and therefore

can be reported as per the individual items of the TIDieR checklist.

A limitation of this review may be the included trials and the way in which they were

identified. Although a systematic process was used to identify effectiveness cluster

randomised controlled trials, a large number of published trials will not have been

identified by our search (for example due to the time period or search terms used).

The results of our review therefore may not be generalisable. However, it is also

unknown whether a larger sample would have influenced the overall results of the

study.

When attempting to identify secondary publications for the included trials it became

evident that authors do not always include the trial registration numbers. Despite best

efforts to ensure that all relevant publications were identified this is a limitation of the

study. It is also probable that details of the support activities omitted from the

protocol and trial results publications may be reported elsewhere. However, there is

an argument to be made that although secondary publications, such as process

Page 18 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 19 -

evaluations, provide a valuable insight into the trial, the primary publications should

provide some details on any efforts made to maximise implementation during the trial

and also whether or not the intervention was actually implemented as intended to

allow readers to interpret the outcome data presented.

As previously mentioned, similar trials [4 19-21] have not only looked at whether or

not details have been reported but have also made a judgement as to whether the

details provided were sufficient to merit replication. In this review we opted to assess

whether or not authors provided details on the items of the TIDieR checklist and

provide supporting examples. This may also be a limitation of the review as the

figures provided here without doubt present a more positive picture than would be the

case if we assessed the extent to which interventions could be replicated from the

details provided.

Meaning of the study

Despite improvements in the quality of reporting of clinical research in recent years,

complex trial designs, and interventions, have resulted in trial reports leaving readers

with many questions unanswered. In this review we have focused in particular on the

reporting of enabling and support activities and implementation fidelity within trials,

as this is important information for both interpreting the research implications but also

for clinicians wanting guidance on how to implement interventions in their practice.

We have shown that although these activities are reported in trials, there exist

important gaps when assessed using the TIDieR checklist. Implementation science is

a growing field of research, with much interest in how we can best translate

interventions shown to be effective in research into day-to-day clinical practice. In

order to progress this field of implementation research it is important that authors

Page 19 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 20 -

describe the rationale for their implementation activities, and the components of these

activities to contribute to the growing evidence-base.

Unanswered questions and future research

Effectiveness cluster randomised trials of complex non-pharmacological interventions

pose numerous challenges in their design, implementation and reporting. However,

there is a growing appreciation for the nuanced and iterative processes that occur

when introducing a new intervention into routine clinical practice and the role of

contextual and personal factors [59]. There is a need for more transparent reporting

about what is happening in trials to make informed decisions about the feasibility and

effectiveness of interventions, along with their likely financial and resource

implications. There is also a need to establish consensus about the level of detail that

should be reported for such large scale trials to ensure that what is reported is

meaningful and useful for patients, the public, clinicians and researchers.

Competing interests

All authors have completed the Unified Competing Interest form at

www.icmje.org/coi_disclosure.pdf (available on request from the corresponding

author) and declare that LC is supported by an NIHR Career Development Fellowship

which also funds NM; the authors have no relationships with other organisations that

might have an interest in the submitted work in the previous 3 years; their spouses,

partners, or children have no financial relationships that may be relevant to the

submitted work; and they have no non-financial interests that may be relevant to the

submitted work.

Page 20 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 21 -

Copyright

The Corresponding Author has the right to grant on behalf of all authors and does

grant on behalf of all authors, a worldwide licence to the Publishers and its licensees

in perpetuity, in all forms, formats and media to i) publish, reproduce, distribute,

display and store the Contribution, ii) translate the Contribution into other languages,

create adaptations, reprints, include within collections and create summaries, extracts

and/or, abstracts of the Contribution and convert or allow conversion into any format

including without limitation audio, iii) create any other derivative work(s) based in

whole or part on the on the Contribution, iv) to exploit all subsidiary rights that

currently exist or as may exist in the future in the Contribution, v) the inclusion of

electronic links from the Contribution to third party material where-ever it may be

located; and, vi) licence any third party to do any or all of the above.

Contributors

NM and LC were responsible for study conception and for conducting the original

search. All authors contributed to i) screening studies for inclusion, ii) developing,

piloting and refining the data extraction form and iii) completing data extraction. NM

was responsible for compiling the first draft of the manuscript. All authors were

involved in preparing the final manuscript for submission. NM and LC are the

guarantors. All authors had full access to all of the data in the study and take

responsibility for the integrity of the data and the accuracy of the data analysis.

Transparency declaration

NM affirms that the manuscript is an honest, accurate, and transparent account of the

study being reported; that no important aspects of the study have been omitted; and

that any discrepancies from the study as planned have been explained.

Page 21 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 22 -

Ethical approval

Not required.

Funding

LC is supported by an NIHR Career Development Fellowship that also funds NM.

This article presents independent research funded by the National Institute for Health

Research (NIHR). The views expressed are those of the authors and not necessarily

those of the NHS, the NIHR or the Department of Health.

Role of study sponsors

Not applicable.

Data sharing

Further details of included trials and data extraction is available on request from

[email protected].

References

1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of

research evidence. Lancet 2009;374:86-9.

2. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines

for reporting parallel group randomised trials. BMC Medicine 2010;8:18.

3. Chan A-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining

standard protocol items for clinical trials. Ann Intern Med. 2013;158:200-7.

4. Hoffmann TC, Erueti C, Glasziou PP. Poor description of non-pharmacological

interventions: analysis of consecutive sample of randomised trials. BMJ

2013;347.

5. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing

and evaluating complex interventions: the new Medical Research Council

guidance. BMJ 2008;337:a1655-a.

6. Flay BR. Efficacy and effectiveness trials (and other phases of research) in the

development of health promotion programs. Prev Med. 1986;15:451-74.

Page 22 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 23 -

7. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of

health promotion research to practice? Rethinking the efficacy-to-

effectiveness transition. Am J Public Health 2003;93:1261-7.

8. Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS. A simple and valid

tool distinguished efficacy from effectiveness studies. J Clin Epidemiol

2006;59:1040-8.

9. Kent DM, Kitsios G. Against pragmatism: on efficacy, effectiveness and the real

world. Trials 2009;10:48.

10. Möller HJ. Effectiveness studies: advantages and disadvantages. Dialogues Clin

Neurosci 2011;13:199.

11. Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster

randomized trials in primary care: a practical approach. Fam Pract

2000;17:192-6.

12. Bellg AJ, Borrelli B, Resnick B, et al. Enhancing treatment fidelity in health

behavior change studies: Best practices and recommendations from the NIH

behavior change consortium. Health Psychol. 2004;23:443-51.

13. Borrelli B, Sepinwall D, Ernst D, et al. A new tool to assess treatment fidelity and

evaluation of treatment fidelity across 10 years of health behavior research. J

Consult Clin Psychol 2005;73:852-60.

14. Moncher FJ, Prinz RJ. Treatment fidelity in outcome studies. Clin Psychol Rev

1991;11:247-66.

15. Dane AV, Schneider BH. Program integrity in primary and early secondary

prevention: Are implementation effects out of control. Clin Psychol Rev

1998;18:23-45.

16. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual

framework for implementation fidelity. Implement Sci 2007;2.

17. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, Grp C. Extending the

CONSORT statement to randomized trials of nonpharmacologic treatment:

Explanation and elaboration. Ann Intern Med 2008;148:295-309.

18. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions:

template for intervention description and replication (TIDieR) checklist and

guide. BMJ 2014;348.

19. Schroter S, Glasziou P, Heneghan C. Quality of descriptions of treatments: a

review of published randomised controlled trials. BMJ Open 2012;2.

20. Douet L, Milne R, Anstee S, Habens F, Young A, Wright D. The completeness of

intervention descriptions in published National Institute of Health Research

HTA-funded trials: a cross-sectional study. BMJ Open 2014;4:e003713.

21. Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions

of treatment in trials and reviews? BMJ 2008;336:1472.

22. Kennedy A, Rogers A, Chew-Graham C, et al. Implementation of a self-

management support approach (WISE) across a health system: a process

evaluation explaining what did and did not work for organisations, clinicians

and patients. Implement Sci 2014;9:129.

Page 23 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 24 -

23. Metzelthin SF, Daniels R, van Rossum E, et al. A nurse-led interdisciplinary

primary care approach to prevent disability among community-dwelling frail

older people: a large-scale process evaluation. Int J Nurs Stud 2013;50:1184-

96.

24. Hoddinott P, Britten J, Pill R. Why do interventions work in some places and not

others: a breastfeeding support group trial. Soc Sci Med 2010;70:769-78.

25. Bower P, Kennedy A, Reeves D, et al. A cluster randomised controlled trial of the

clinical and cost-effectiveness of a 'whole systems' model of self-management

support for the management of long- term conditions in primary care: trial

protocol. Implement Sci 2012;7:7.

26. Kennedy A, Bower P, Reeves D, et al. Implementation of self management

support for long term conditions in routine primary care settings: cluster

randomised controlled trial. BMJ 2013;346:f2882.

27. Haller DM, Meynard A, Lefebvre D, Ukoumunne OC, Narring F, Broers B.

Effectiveness of training family physicians to deliver a brief intervention to

address excessive substance use among young patients: a cluster randomized

controlled trial. CMAJ 2014;186:E263-72.

28. Papadakis S, McDonald PW, Pipe AL, Letherdale ST, Reid RD, Brown KS.

Effectiveness of telephone-based follow-up support delivered in combination

with a multi-component smoking cessation intervention in family practice: a

cluster-randomized trial. Prev Med 2013;56:390-7.

29. Dinneen SF, MC OH, Byrne M, et al. The Irish DAFNE study protocol: a cluster

randomised trial of group versus individual follow-up after structured

education for type 1 diabetes. Trials 2009;10:88.

30. Dinneen SF, O'Hara MC, Byrne M, et al. Group follow-up compared to individual

clinic visits after structured education for type 1 diabetes: a cluster randomised

controlled trial. Diabetes Res Clin Pract 2013;100:29-38.

31. Adachi M, Yamaoka K, Watanabe M, et al. Effects of lifestyle education program

for type 2 diabetes patients in clinics: study design of a cluster randomized

trial. BMC Public Health 2010;10:742.

32. Adachi M, Yamaoka K, Watanabe M, et al. Effects of lifestyle education program

for type 2 diabetes patients in clinics: a cluster randomized controlled trial.

BMC Public Health 2013;13:467.

33. Zatzick D, Donovan DM, Jurkovich G, et al. Disseminating alcohol screening and

brief intervention at trauma centers: a policy-relevant cluster randomized

effectiveness trial. Addiction 2014;109:754-65.

34. Meyer C, Ulbricht S, Gross B, et al. Adoption, reach and effectiveness of

computer-based, practitioner delivered and combined smoking interventions in

general medical practices: a three-arm cluster randomized trial. Drug Alcohol

Depend 2012;121:124-32.

35. Spijker A, Verhey F, Graff M, et al. Systematic care for caregivers of people with

dementia in the ambulatory mental health service: designing a multicentre,

cluster, randomized, controlled trial. BMC Geriatr 2009;9:21.

Page 24 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 25 -

36. Spijker A, Wollersheim H, Teerenstra S, et al. Systematic care for caregivers of

patients with dementia: a multicenter, cluster-randomized, controlled trial. Am

J Geriatr Psychiatry 2011;19:521-31.

37. Luoto R, Kinnunen TI, Aittasalo M, et al. Primary prevention of gestational

diabetes mellitus and large-for-gestational-age newborns by lifestyle

counseling: a cluster-randomized controlled trial. PLoS Med

2011;8:e1001036.

38. Luoto RM, Kinnunen TI, Aittasalo M, et al. Prevention of gestational diabetes:

design of a cluster-randomized controlled trial and one-year follow-up. BMC

Pregnancy Childbirth 2010;10:39.

39. Lau R, Stewart K, McNamara KP, et al. Evaluation of a community pharmacy-

based intervention for improving patient adherence to antihypertensives: a

randomised controlled trial. BMC Health Serv Res 2010;10:34.

40. Stewart K, George J, Mc Namara KP, et al. A multifaceted pharmacist

intervention to improve antihypertensive adherence: a cluster-randomized,

controlled trial (HAPPy trial). J Clin Pharm Ther 2014;39:527-34.

41. Weaver T, Metrebian N, Hellier J, et al. Use of contingency management

incentives to improve completion of hepatitis B vaccination in people

undergoing treatment for heroin dependence: a cluster randomised trial.

Lancet 2014;384:153-63.

42. Hegarty K, O'Doherty L, Taft A, et al. Screening and counselling in the primary

care setting for women who have experienced intimate partner violence

(WEAVE): a cluster randomised controlled trial. Lancet 2013;382:249-58.

43. Hegarty KL, Gunn JM, O'Doherty LJ, et al. Women's evaluation of abuse and

violence care in general practice: a cluster randomised controlled trial

(weave). BMC Public Health 2010;10:2.

44. Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S, Vernooij-Dassen MJ,

Koopmans RT. A structural multidisciplinary approach to depression

management in nursing-home residents: a multicentre, stepped-wedge cluster-

randomised trial. Lancet 2013;381:2255-64.

45. Slater H, Briggs AM, Watkins K, Chua J, Smith AJ. Translating evidence for low

back pain management into a consumer-focussed resource for use in

community pharmacies: a cluster-randomised controlled trial. PLoS One

2013;8:e71918.

46. Leendertse AJ, de Koning FH, Goudswaard AN, et al. Preventing hospital

admissions by reviewing medication (PHARM) in primary care: design of the

cluster randomised, controlled, multi-centre PHARM-study. BMC Health Serv

Res 2011;11:4.

47. Leendertse AJ, de Koning GH, Goudswaard AN, et al. Preventing hospital

admissions by reviewing medication (PHARM) in primary care: an open

controlled study in an elderly population. J Clin Pharm Ther 2013;38:379-87.

48. Hafkamp-de Groen E, Mohangoo AD, de Jongste JC, et al. Early detection and

counselling intervention of asthma symptoms in preschool children: study

design of a cluster randomised controlled trial. BMC Public Health

2010;10:555.

Page 25 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 26 -

49. Hafkamp-de Groen E, van der Valk RJ, Mohangoo AD, et al. Evaluation of

systematic assessment of asthma-like symptoms and tobacco smoke exposure

in early childhood by well-child professionals: A randomised trial. PLoS One

2014;9:e90982.

50. Labhardt ND, Balo JR, Ndam M, Manga E, Stoll B. Improved retention rates with

low-cost interventions in hypertension and diabetes management in a rural

African environment of nurse-led care: a cluster-randomised trial. Trop Med

Int Health 2011;16:1276-84.

51. Blackberry ID, Furler JS, Best JD, et al. Effectiveness of general practice based,

practice nurse led telephone coaching on glycaemic control of type 2 diabetes:

the Patient Engagement and Coaching for Health (PEACH) pragmatic cluster


52. Young D, Furler J, Vale M, et al. Patient Engagement and Coaching for Health:

The PEACH study--a cluster randomised controlled trial using the telephone

to coach people with type 2 diabetes to engage with their GPs to improve

diabetes care: a study protocol. BMC Fam Pract 2007;8:20.

53. Grandes G, Sanchez A, Sanchez-Pinilla RO, et al. Effectiveness of physical

activity advice and prescription by physicians in routine primary care: a

cluster randomized trial. Arch Intern Med 2009;169:694-701.

54. Law MC, Darrah J, Pollock N, et al. Focus on function: a cluster, randomized

controlled trial comparing child- versus context-focused intervention for

young children with cerebral palsy. Dev Med Child Neurol 2011;53:621-9.

55. Cabezas C, Advani M, Puente D, Rodriguez-Blanco T, Martin C, Group IS.

Effectiveness of a stepped primary care smoking cessation intervention:

cluster randomized clinical trial (ISTAPS study). Addiction 2011;106:1696-

706.

56. Cabezas C, Martin C, Granollers S, et al. Effectiveness of a stepped primary care

smoking cessation intervention (ISTAPS study): design of a cluster

randomised trial. BMC Public Health 2009;9:48.

57. Hopkinson JB, Fenlon DR, Okamoto I, et al. The deliverability, acceptability, and

perceived effect of the Macmillan approach to weight loss and eating

difficulties: a phase II, cluster-randomized, exploratory trial of a psychosocial

intervention for weight- and eating-related distress in people with advanced

cancer. J Pain Symptom Manage 2010;40:684-95.

58. Jahn P, Renz P, Stukenkemper J, et al. Reduction of chemotherapy-induced

anorexia, nausea, and emesis through a structured nursing intervention: a

cluster-randomized multicenter trial. Support Care Cancer 2009;17:1543-52.

59. Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is

not enough: evidence from an in-depth multiple case study on the untold role

and impact of context in randomised controlled trials of seven complex

interventions. Trials 2012;13:95-111.

Page 26 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 27 -

Appendices

I. Data extraction form and completed example

Data extraction form

First author of trial results

publication

Blackberry

Publication Title Effectiveness of general practice based, practice

nurse led telephone coaching on glycaemic control of

type 2 diabetes: the Patient Engagement and

Coaching for Health (PEACH) pragmatic cluster

randomised controlled trial.

Trial Acronym PEACH

Trial Registration ISRCTN50662837

Journal Citation Blackberry ID, Furler JS, Best JD, Chondros P, Vale

M, Walker C, et al. Effectiveness of general practice

based, practice nurse led telephone coaching on

glycaemic control of type 2 diabetes: the Patient

Engagement and Coaching for Health (PEACH)

pragmatic cluster randomised controlled trial. BMJ.

2013;347:f5272.

Journal BMJ

Year 2013

Study Design Cluster randomised controlled trial

Publication type Trial results

Other relevant trial

publications

PROTOCOL: Young D, Furler J, Vale M, Walker C,

Segal L, Dunning P, et al. Patient Engagement and

Coaching for Health: The PEACH study--a cluster

randomised controlled trial using the telephone to

coach people with type 2 diabetes to engage with

their GPs to improve diabetes care: a study protocol.

BMC Fam Pract. 2007;8:20.

Topic Type 2 Diabetes

Page 27 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 28 -

Aim To evaluate the effectiveness of goal focused

telephone coaching by practice nurses in improving

glycaemic control in patients with type 2 diabetes in

Australia.

Name/type of the

intervention

Patient Engagement and Coaching for Health

intervention (PEACH). (self-management)

HCPs implementing the

intervention

Nurses

Setting General practices

Enabling or support activities

Made reference to enabling

or support activities

Yes

Rationale for the activities Not described

Who provided The research team provides on-site assistance to the

PN in their first two baseline interviews and

questionnaire administration to ensure quality of data

collection.

What materials Practices were reimbursed for their nurses’ time

spent on the PEACH study, including for patient

recruitment, data collection, and delivery of the

intervention.

Where On-site assistance

When Unclear

How Each practice nurse from practices assigned to the

intervention group completed a two day training

programme in telephone coaching. / The research

team provided support to practice nurses during the

intervention period, including one visit to the

practice, monthly telephone calls, and a group

meeting, where all intervention practice nurses could

share their experience and learning from conducting

telephone coaching.

How much An initial 15 hours of telephone coaching training

Page 28 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 29 -

over two days, and further support and training was

provided to individual nurses after assessment of

their progress.

Tailoring None described

Modifications None described

Planned measures of fidelity

to the support activities (e.g. monitoring attendance at training

sessions)

Not described

Actual implementation of

support activities (e.g. had to increase number of sessions to

accommodate staff turnover)

Not described

Competency checking in

advance of implementation

/accreditation

None described

Fidelity of implementation of intervention being tested

Measure of fidelity used The "Head Coach" (MV) monitors the first telephone

coaching and random subsequent telephone

coaching.

Actual implementation of the

intervention

The intensity and fidelity of the intervention was

compromised so that the intended dose of the

intervention was not delivered to a high proportion of

patients and in many cases the key element of a focus

on medication intensification through negotiation

with the treating general practitioners was not

achieved. Intervention fidelity also seems to have

been compromised. Content analysis of calls showed

many consultations did not explicitly address

medication intensification, an important part of the

coaching programme.

Page 29 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 30 -

II. Publications included in data extraction

First author/year of

trial results publication

Trial registration

number Trial publications included in data extraction

1 Adachi 2013 UMIN000004049

• Adachi M, Yamaoka K, Watanabe M, et al. Effects of lifestyle education

program for type 2 diabetes patients in clinics: study design of a cluster

randomized trial. BMC Public Health 2010;10:742.

• Adachi M, Yamaoka K, Watanabe M, et al. Effects of lifestyle education

program for type 2 diabetes patients in clinics: a cluster randomized

controlled trial. BMC Public Health 2013;13:467.

2 Armour 2013 None identified

• Armour CL, Reddel HK, LeMay KS, et al. Feasibility and effectiveness of

an evidence-based asthma service in Australian community pharmacies: a

pragmatic cluster randomized trial. J Asthma 2013;50:302-9.

3 Blackberry 2013 ISRCTN50662837

• Young D, Furler J, Vale M, et al. Patient Engagement and Coaching for

Health: The PEACH study--a cluster randomised controlled trial using the

telephone to coach people with type 2 diabetes to engage with their GPs to

improve diabetes care: a study protocol. BMC Fam Pract 2007;8:20.

• Blackberry ID, Furler JS, Best JD, et al. Effectiveness of general practice

based, practice nurse led telephone coaching on glycaemic control of type

2 diabetes: the Patient Engagement and Coaching for Health (PEACH)

pragmatic cluster randomised controlled trial. BMJ 2013;347:f5272.

4 Cabezas 2011 NCT00125905

• Cabezas C, Martin C, Granollers S, et al. Effectiveness of a stepped

primary care smoking cessation intervention (ISTAPS study): design of a

cluster randomised trial. BMC Public Health 2009;9:48.

• Cabezas C, Advani M, Puente D, Rodriguez-Blanco T, Martin C, Group

IS. Effectiveness of a stepped primary care smoking cessation intervention:

cluster randomized clinical trial (ISTAPS study). Addiction

2011;106:1696-706.

Page 30 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 31 -

5 Chao 2014 None identified

• Chao YH, Usher K, Buettner PG, Holmes C. Cluster randomised

controlled trial: educational self-care intervention with older Taiwanese

patients with type 2 diabetes mellitus--impact on blood glucose levels and

diabetic complications. Collegian 2014;21:43-51.

6 Deales 2014 ACTRN12611000813987

• Deales A, Fratini M, Romano S, et al. Care manager to control

cardiovascular risk factors in primary care: the Raffaello cluster

randomized trial. Nutr Metab Cardiovasc Dis 2014;24:563-71.

7 Derde 2014 NCT00976638

• Derde LP, Cooper BS, Goossens H, et al. Interventions to reduce

colonisation and transmission of antimicrobial-resistant bacteria in

intensive care units: an interrupted time series study and cluster

randomised trial.[Erratum appears in Lancet Infect Dis. 2014

Jan;14(1):11]. Lancet Infect Dis 2014;14:31-9.

8 Dinneen 2013 ISRCTN79759174

• Dinneen SF, MC OH, Byrne M, et al. The Irish DAFNE study protocol: a

cluster randomised trial of group versus individual follow-up after

structured education for type 1 diabetes. Trials 2009;10:88.

• Dinneen SF, O'Hara MC, Byrne M, et al. Group follow-up compared to

individual clinic visits after structured education for type 1 diabetes: a

cluster randomised controlled trial. Diabetes Res Clin Pract 2013;100:29-

38.

9 Garcia-Cardenas 2013 NCT01085474

• Garcia-Cardenas V, Sabater-Hernandez D, Kenny P, Martinez-Martinez F,

Faus MJ, Benrimoj SI. Effect of a pharmacist intervention on asthma

control. A cluster randomised trial. Respir Med 2013;107:1346-55.

10 Grandes 2009 NCT00131079

• Grandes G, Sanchez A, Sanchez-Pinilla RO, et al. Effectiveness of

physical activity advice and prescription by physicians in routine primary

care: a cluster randomized trial. Arch Intern Med 2009;169:694-701.

11 Hafkamp-de Groen 2014 ISRCTN15790308

• Hafkamp-de Groen E, Mohangoo AD, de Jongste JC, et al. Early detection

and counselling intervention of asthma symptoms in preschool children:

study design of a cluster randomised controlled trial. BMC Public Health

2010;10:555.

Page 31 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 32 -

• Hafkamp-de Groen E, van der Valk RJ, Mohangoo AD, et al. Evaluation of

systematic assessment of asthma-like symptoms and tobacco smoke

exposure in early childhood by well-child professionals: A randomised

trial. PLoS One 2014;9:e90982.

12 Haller 2014 ACTRN12608000432314

• Haller DM, Meynard A, Lefebvre D, Ukoumunne OC, Narring F, Broers

B. Effectiveness of training family physicians to deliver a brief

intervention to address excessive substance use among young patients: a

cluster randomized controlled trial. CMAJ 2014;186:E263-72.

13 Hegarty 2010 ACTRN12608000032358

• Hegarty KL, Gunn JM, O'Doherty LJ, et al. Women's evaluation of abuse

and violence care in general practice: a cluster randomised controlled trial

(weave). BMC Public Health 2010;10:2.

• Hegarty K, O'Doherty L, Taft A, et al. Screening and counselling in the

primary care setting for women who have experienced intimate partner

violence (WEAVE): a cluster randomised controlled trial. Lancet

2013;382:249-58.

14 Hoddinott 2009 ISRCTN44857041

• Hoddinott P, Britten J, Prescott GJ, Tappin D, Ludbrook A, Godden DJ.

Effectiveness of policy to provide breastfeeding groups (BIG) for pregnant

and breastfeeding mothers in primary care: cluster randomised controlled

trial. BMJ 2009;338:a3026.

15 Hopkinson 2010 None identified

• Hopkinson JB, Fenlon DR, Okamoto I, et al. The deliverability,

acceptability, and perceived effect of the Macmillan approach to weight

loss and eating difficulties: a phase II, cluster-randomized, exploratory trial

of a psychosocial intervention for weight- and eating-related distress in

people with advanced cancer. J Pain Symptom Manage 2010;40:684-95.

16 Jahn 2009 None identified

• Jahn P, Renz P, Stukenkemper J, et al. Reduction of chemotherapy-induced

anorexia, nausea, and emesis through a structured nursing intervention: a

cluster-randomized multicenter trial. Support Care Cancer 2009;17:1543-

52.

Page 32 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 33 -

17 Kennedy 2013 ISRCTN90940049

• Bower P, Kennedy A, Reeves D, et al. A cluster randomised controlled

trial of the clinical and cost-effectiveness of a 'whole systems' model of

self-management support for the management of long- term conditions in

primary care: trial protocol. Implement Sci 2012;7:7.

• Kennedy A, Bower P, Reeves D, et al. Implementation of self management

support for long term conditions in routine primary care settings: cluster


18 Labhardt 2011 NCT00744458

• Labhardt ND, Balo JR, Ndam M, Manga E, Stoll B. Improved retention

rates with low-cost interventions in hypertension and diabetes management

in a rural African environment of nurse-led care: a cluster-randomised trial.

Trop Med Int Health 2011;16:1276-84.

19 Law 2011 None identified

• Law MC, Darrah J, Pollock N, et al. Focus on function: a cluster,

randomized controlled trial comparing child- versus context-focused

intervention for young children with cerebral palsy. Dev Med Child Neurol

2011;53:621-9.

20 Leendertse 2013 NTR 2647

• Leendertse AJ, de Koning FH, Goudswaard AN, et al. Preventing hospital

admissions by reviewing medication (PHARM) in primary care: design of

the cluster randomised, controlled, multi-centre PHARM-study. BMC

Health Serv Res 2011;11:4.

• Leendertse AJ, de Koning GH, Goudswaard AN, et al. Preventing hospital

admissions by reviewing medication (PHARM) in primary care: an open

controlled study in an elderly population. J Clin Pharm Ther 2013;38:379-

87.

21 Leonhardt 2008 None identified

• Leonhardt C, Keller S, Chenot JF, et al. TTM-based motivational

counselling does not increase physical activity of low back pain patients in

a primary care setting--A cluster-randomized controlled trial. Patient Educ

Couns 2008;70:50-60.

Page 33 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 34 -

22 Leontjevas 2013 NTR1477

• Gerritsen DL, Smalbrugge M, Teerenstra S, et al. Act In case of

Depression: the evaluation of a care program to improve the detection and

treatment of depression in nursing homes. Study Protocol. BMC Psychiatry

2011;11:91.

• Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S, Vernooij-

Dassen MJ, Koopmans RT. A structural multidisciplinary approach to

depression management in nursing-home residents: a multicentre, stepped-

wedge cluster-randomised trial. Lancet 2013;381:2255-64.

23 Luoto 2011 ISRCTN33885819

• Luoto RM, Kinnunen TI, Aittasalo M, et al. Prevention of gestational

diabetes: design of a cluster-randomized controlled trial and one-year

follow-up. BMC Pregnancy Childbirth 2010;10:39.

• Luoto R, Kinnunen TI, Aittasalo M, et al. Primary prevention of

gestational diabetes mellitus and large-for-gestational-age newborns by

lifestyle counseling: a cluster-randomized controlled trial. PLoS Med

2011;8:e1001036.

24 Magoma 2013 ACTRN12609000268246

• Magoma M, Requejo J, Campbell O, Cousens S, Merialdi M, Filippi V.

The effectiveness of birth plans in increasing use of skilled care at delivery

and postnatal care in rural Tanzania: a cluster randomised trial. Trop Med

Int Health 2013;18:435-43.

25 Metzelthin 2013 ISRCTN31954692

• Metzelthin SF, van Rossum E, de Witte LP, Hendriks MR, Kempen GI.

The reduction of disability in community-dwelling frail older people:

design of a two-arm cluster randomized controlled trial. BMC Public

Health 2010;10:511.

• Metzelthin SF, van Rossum E, de Witte LP, et al. Effectiveness of

interdisciplinary primary care approach to reduce disability in community

dwelling frail older people: cluster randomised controlled trial. BMJ

2013;347:f5264.

Page 34 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 35 -

26 Meyer 2012 NCT00679861

• Meyer C, Ulbricht S, Gross B, et al. Adoption, reach and effectiveness of

computer-based, practitioner delivered and combined smoking

interventions in general medical practices: a three-arm cluster randomized

trial. Drug Alcohol Depend 2012;121:124-32.

27 Morrell 2009 ISRCTN92195776

• Morrell CJ, Slade P, Warner R, et al. Clinical effectiveness of health visitor

training in psychologically informed approaches for depression in postnatal

women: pragmatic cluster randomised trial in primary care. BMJ

2009;338:a3045.

• Morrell CJ, Warner R, Slade P, et al. Psychological interventions for

postnatal depression: cluster randomised trial and economic evaluation.

The PoNDER trial. Health Technol Assess 2009;13:iii-iv, xi-xiii, 1-153.

28 Nokela 2010 None identified

• Nokela M, Arnlind MH, Ehrs PO, Krakau I, Forslund L, Jonsson EW. The

influence of structured information and monitoring on the outcome of

asthma treatment in primary care: a cluster randomized study. Respiration

2010;79:388-94.

29 Papadakis 2013 NCT00799279

• Papadakis S, McDonald PW, Pipe AL, Letherdale ST, Reid RD, Brown

KS. Effectiveness of telephone-based follow-up support delivered in

combination with a multi-component smoking cessation intervention in

family practice: a cluster-randomized trial. Prev Med 2013;56:390-7.

30 Pladevall 2010 ISRCTN35208258

• Pladevall M, Brotons C, Gabriel R, et al. Multicenter cluster-randomized

trial of a multifactorial intervention to improve antihypertensive

medication adherence and blood pressure control among patients at high

cardiovascular risk (the COM99 study). Circulation 2010;122:1183-91.

31 Pope 2008 None identified

• Pope DS, Deluca AN, Kali P, et al. A cluster-randomized trial of provider-

initiated (opt-out) HIV counseling and testing of tuberculosis patients in

South Africa. J Acquir Immune Defic Syndr 2008;48:190-5.

32 Quinn 2011 None identified

• Quinn CC, Gruber-Baldini AL, Shardell M, et al. Mobile diabetes

intervention study: testing a personalized treatment/behavioral

communication intervention for blood glucose control. Contemp Clin

Page 35 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 36 -

Trials 2009;30:334-46.

• Quinn CC, Shardell MD, Terrin ML, Barr EA, Ballew SH, Gruber-Baldini

AL. Cluster-randomized trial of a mobile phone personalized behavioral

intervention for blood glucose control. Diabetes Care 2011;34:1934-42.

33 Rat 2014 NCT01610531

• Rat C, Quereux G, Riviere C, et al. Targeted melanoma prevention

intervention: a cluster randomized controlled trial. Ann Fam Med

2014;12:21-8.

34 Roisin 2014 NCT00846105

• Roisin S, Laurent C, Denis O, et al. Impact of rapid molecular screening at

hospital admission on nosocomial transmission of methicillin-resistant

staphylococcus aureus: cluster randomised trial. PLoS One 2014;9:e96310.

35 Septimus 2014 None identified

• Septimus EJ, Hayden MK, Kleinman K, et al. Does chlorhexidine bathing

in adult intensive care units reduce blood culture contamination? A

pragmatic cluster-randomized trial. Infect Control Hosp Epidemiol 2014;35

Suppl 3:S17-22.

36 Slater 2013 ACTRN12611000053921

• Slater H, Briggs AM, Watkins K, Chua J, Smith AJ. Translating evidence

for low back pain management into a consumer-focussed resource for use

in community pharmacies: a cluster-randomised controlled trial. PLoS One

2013;8:e71918.

37 Sonnichsen 2010 ISRCTN27414162

• Sonnichsen AC, Rinnerberger A, Url MG, et al. Effectiveness of the

Austrian disease-management-programme for type 2 diabetes: study

protocol of a cluster-randomized controlled trial. Trials 2008;9:38.

• Sonnichsen AC, Winkler H, Flamm M, et al. The effectiveness of the

Austrian disease management programme for type 2 diabetes: a cluster-

randomised controlled trial. BMC Fam Pract 2010;11:86.

38 Spijker 2011 NCT00147693

• Spijker A, Verhey F, Graff M, et al. Systematic care for caregivers of

people with dementia in the ambulatory mental health service: designing a

multicentre, cluster, randomized, controlled trial. BMC Geriatr 2009;9:21.

• Spijker A, Wollersheim H, Teerenstra S, et al. Systematic care for

caregivers of patients with dementia: a multicenter, cluster-randomized,

Page 36 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 37 -

controlled trial. Am J Geriatr Psychiatry 2011;19:521-31.

39 Stewart 2014 ACTRN12609000705280

• Lau R, Stewart K, McNamara KP, et al. Evaluation of a community

pharmacy-based intervention for improving patient adherence to

antihypertensives: a randomised controlled trial. BMC Health Serv Res

2010;10:34.

• Stewart K, George J, Mc Namara KP, et al. A multifaceted pharmacist

intervention to improve antihypertensive adherence: a cluster-randomized,

controlled trial (HAPPy trial). J Clin Pharm Ther 2014;39:527-34.

40 van de Ven 2013 NTR2314

• van de Ven G, Draskovic I, Adang EM, et al. Improving person-centred

care in nursing homes through dementia-care mapping: design of a cluster-

randomised controlled trial. BMC geriatr 2012;12:1.

• van de Ven G, Draskovic I, Adang EM, et al. Effects of dementia-care

mapping on residents and staff of care homes: a pragmatic cluster-

randomised controlled trial. PLoS One 2013;8:e67325.

41 van den Bemt 2009 NCT00542061

• van den Bemt L, Schermer TR, Smeele IJ, et al. An expert-supported

monitoring system for patients with chronic obstructive pulmonary disease

in general practice: results of a cluster randomised controlled trial. Med J

Aust 2009;191:249-54.

42 Vicens 2014 ISRCTN13024375

• Vicens C, Socias I, Mateu C, et al. Comparative efficacy of two primary

care interventions to assist withdrawal from long term benzodiazepine use:

a protocol for a clustered, randomized clinical trial. BMC Fam Pract

2011;12:23.

• Vicens C, Bejarano F, Sempere E, et al. Comparative efficacy of two

interventions to discontinue long-term benzodiazepine use: cluster

randomised controlled trial in primary care. Br J Psychiatry 2014;204:471-

9.

43 Weaver 2014 ISRCTN72794493

• Weaver T, Metrebian N, Hellier J, et al. Use of contingency management

incentives to improve completion of hepatitis B vaccination in people

undergoing treatment for heroin dependence: a cluster randomised trial.

Page 37 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




- 38 -

Lancet 2014;384:153-63.

44 Wolfe 2010 ISRCTN10730637

• Wolfe CD, Redfern J, Rudd AG, Grieve AP, Heuschmann PU, McKevitt

C. Cluster randomized controlled trial of a patient and general practitioner

intervention to improve the management of multiple risk factors after

stroke: stop stroke. Stroke 2010;41:2470-6.

45 Zatzick 2014 NCT00607620

• Zatzick D, Donovan DM, Jurkovich G, et al. Disseminating alcohol

screening and brief intervention at trauma centers: a policy-relevant cluster

randomized effectiveness trial. Addiction 2014;109:754-65.

Page 38 of 38


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960




Activities to support the implementation of complex interventions as part of routine care: a review of the quality

of reporting in cluster randomised controlled trials

Journal: BMJ Open

Manuscript ID bmjopen-2015-008251.R1

Article Type: Research

Date Submitted by the Author: 21-Aug-2015

Complete List of Authors: McMahon, Naoimh; University of Central Lancashire, Clinical Practice Research Unit Holland, Emma-Joy; University of Central Lancashire, Clinical Practice

Research Unit Miller, Colette; University of Central Lancashire, Clinical Practice Research Unit Patel, Kulsum; University of Central Lancashire, Clinical Practice Research Unit Connell, Louise; University of Central Lancashire, Clinical Practice Research Unit

<b>Primary Subject Heading</b>:

Health services research

Secondary Subject Heading: Health services research

Keywords: n/a, n/a, n/a


BMJ Open on S

eptember 17, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2015-008251 on 9 O

ctober 2015. Dow

nloaded from



- 1 -

Activities to support the implementation of complex interventions as part

of routine care: a review of the quality of reporting in cluster randomised

controlled trials

Naoimh E McMahon, research physiotherapist, Emma-Joy Holland, senior research assistant,

Colette Miller, senior research assistant, Kulsum Patel, senior research assistant, Louise A

Connell, NIHR career development fellow

Clinical Practice Research Unit, College of Health and Wellbeing, University of Central

Lancashire, Preston, PR1 2HE, UK

Corresponding author:

Naoimh McMahon,

Clinical Practice Research Unit,

College of Health and Wellbeing,

University of Central Lancashire

[email protected]

01772 893654

Page 1 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 2 -

Abstract

Objective

To review a sample of cluster randomised controlled trials and explore the quality of

reporting of (i) enabling or support activities provided to the staff during the trial, (ii)

strategies used to monitor fidelity throughout the trial and (iii) the extent to which the

intervention being tested was delivered as planned.

Design

A descriptive review.

Data sources and study selection

We searched MEDLINE for trial reports published between 2008 and 2014 with

combinations of the search terms ‘randomised’, ‘cluster’, ‘trial’, ‘study’, ‘intervention’ and

‘implement*’. We included trials in which healthcare professionals (HCPs) implemented the

intervention being tested as part of routine practice. We excluded trials (i) conducted in non-

health services settings, (ii) where the intervention explicitly aimed to change the behaviours

of the HCPs and (iii) where the trials were ongoing or for which only trial protocols were

available.

Data collection

We developed a data extraction form using the Template for Intervention Description and

Replication (TIDieR checklist). Review authors independently extracted data from the

included trials and assessed quality of reporting for individual items.

Results

We included 70 publications (45 results publications, 25 related publications). 89% of trials

reported using enabling or support activities. How these activities were provided (75.6%,

n=34), and how much was provided (73.3%, n=33), were the most frequently reported items.

Less than 20% (n=8) of the included trials reported that competency checking occurred prior

Page 2 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 3 -

to implementation and data collection. 64% (n=29) of trials reported collecting measures of

implementation. 44% (n=20) of trials reported data from these measures.

Conclusions

Although enabling and support activities are reported in trials, important gaps exist when

assessed using an established checklist. Better reporting of the supports provided in

effectiveness trials will allow for informed decisions to be made about financial and resource

implications for wide scale implementation of effective interventions.

Strengths and limitations

• Enabling or support activities used to facilitate implementation of interventions in

effectiveness trials should be described in sufficient detail to allow interpretation of

results and future replication

• In this study we used a published checklist, TIDieR (the Template for Intervention

Description and Replication), to comprehensively assess the quality of reporting of

enabling or support activities in cluster randomised trials

• We did not rate the quality of reporting, as other studies have, but assessed whether or

not authors reported details for items of the TIDieR checklist. The figures provided

here may therefore present a more positive picture than would be the case if we

assessed the extent to which interventions could be replicated from the details

provided

• Our search strategy was designed to identify a sample of effectiveness trials

conducted in health services settings and a large number of eligible published trials

may not have been identified potentially limiting the generalisability of the results

Page 3 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 4 -

Introduction

A seminal publication in 2009 by Chalmers and Glasziou identified unusable research reports

as a primary contributor to avoidable waste in research production [1]. Despite

comprehensive guidelines to assist with the reporting of clinical trials, for example the

Consolidated Standards of Reporting Trials (CONSORT) 2010 statement [2] and the

Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013

statement [3], gaps in the completeness of reports remain. Trials of complex non-

pharmacological interventions have proved a particular challenge. Complex interventions are

those with numerous interacting components and have been found to be adequately described

in only 39% of trials [4] .

Randomised controlled trials (RCTs) have traditionally been designed to demonstrate the

efficacy of complex interventions under optimum conditions [5]. However, numerous

challenges exist in attempting to translate research evidence from efficacy trials into real life

clinical practice. Most notably the lack of external validity, or generalisability, which has

been a long standing criticism of the RCT methodology [6]. To address this issue, there has

been a dramatic increase in the number of effectiveness trials being conducted which explore

the degree of beneficial effect of interventions when delivered under real world conditions

[7]. RCTs are not considered to be either efficacy or effectiveness trials but can have

characteristics of both and thus are said to sit on a continuum [8].

As a result of the growing interest in more pragmatic trial designs, we have also seen an

increase in the use of cluster randomisation in health services research (i.e. where groups of

patients, rather than individual patients, are randomised). Cluster randomised designs are

particularly appropriate when there is a risk of contamination across trial groups when trial

patients are managed within the same setting [9]. Pragmatic cluster randomised trials will

Page 4 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 5 -

usually evaluate interventions when they are delivered by routinely employed healthcare

professionals. Successful implementation therefore requires a change in the practices of these

professionals [10]. The ‘CONSORT’ extension for the reporting of RCTs of non-

pharmacological treatments [11] and the Template for Intervention Description and

Replication (TIDieR) checklist [12], prompt authors to report on any activities that are used

to enable or support professionals to implement the interventions being tested and to report

actual implementation of the intervention. These enabling or support activities can be

considered to be interventions in themselves and therefore merit equal description in trial

reports.

Previous studies exploring the completeness of intervention descriptions in clinical trials have

not differentiated between different trial designs and have not explicitly assessed the

completeness of reporting of enabling or support activities used to assist professionals in

implementing the interventions being tested [4, 13-15]. In this study we reviewed a sample of

cluster randomised controlled trials, in which healthcare professionals employed to routinely

deliver care were responsible for implementing the interventions being tested. We explored

the quality of reporting of (i) enabling or support activities provided during the trial, (ii)

strategies used to monitor fidelity throughout the trial and (iii) the extent to which the

intervention was delivered as planned.

Methods

Search strategy and selection of reports of trials

To identify a sample of trial reports for the review we searched MEDLINE for publications

with combinations of the search terms ‘randomised’, ‘cluster’, ‘trial’ ‘study’, ‘intervention’

and ‘implement*’ in their titles or abstracts (Supplementary file 1). To manage the scope of

the review we included reports of cluster randomised controlled trials published between

Page 5 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 6 -

2008 and 2014. This date range was chosen as the Medical Research Council (MRC)

guidance on developing and evaluating complex interventions was published in 2008 [10], a

seminal publication which has influenced the quality of reporting in trials of complex

interventions. We included trials, accessible online and published in English, in which

healthcare professionals (HCPs) implemented the intervention being tested as part of their

routine practice. We excluded trials (i) conducted in non-health services settings (e.g.

schools, universities), (ii) where the intervention being tested explicitly aimed to change the

behaviours of the HCPs (e.g. behaviour change interventions (BCIs) such as screening tools,

decision aids, audit and feedback) and (iii) if the trials were ongoing or for which only trial

protocols were available. In instances where the intervention being tested serviced a dual

purpose of impacting on patient outcomes and also changing the behaviours of healthcare

professionals to implement best practice we made decisions on a case by case bases. Trials

were included in which healthcare professionals were responsible for implementing defined

actions or components of the interventions being tested. Where it was not possible to identify

defined actions or components of the interventions to be implemented these trials were

excluded.

NM conducted the first round of screening to remove irrelevant trials based on the exclusion

criteria. In pairs, all authors independently screened the titles and abstracts of the remaining

trials for inclusion. Discrepancies were resolved by discussion or by a third party. In

instances where it was not possible to exclude trials based on title and abstract, full text

versions were obtained and were assessed for inclusion by the authors in the same way.

Relevant publications related to the included trials (i.e. trial protocols, process evaluations)

were identified where possible by using the trial registration number or by manually

searching for publications where the primary investigator named on the trial registration was

an author.

Page 6 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 7 -

Data extraction and analysis

A data extraction form, based on the TIDieR checklist [12], was developed for the purposes of

the review. All of the TIDieR items were applied to enabling and support activities. Data on

fidelity was extracted at two levels (i) fidelity of implementation of the enabling and support

activities and (ii) fidelity of implementation of the intervention being tested. Throughout data

extraction we made minor iterative changes to the way in which the items of the TIDieR checklist

were organised to facilitate ease of use. For example, we reordered some of the items to fit with

how they most often appeared in publications, and we separated “when” and “how much”. Some

additional information which is highlighted in the TIDieR checklist publication was coded as a

separate item as it was reported in a number of trials (i.e. competency checking is discussed under

“who provided” in the original TIDieR publication and we extracted this as a stand-alone item).

The final data extraction form is provided in Supplementary file 2. Data were extracted on: the

rationale for the support or enabling activities, materials and procedures used, where the activities

were delivered, by who, when and how often and if the training was accredited or competency

checked. Data on tailoring or modifications to the activities were extracted along with planned

strategies to monitor implementation fidelity, and actual implementation of the intervention

during the trial. Three pairs of authors independently extracted data from 15 trials each where one

author (NM) was a member of two pairs. Extracted data were cross-checked in the pairs and

findings were discussed until consensus for each section was reached and the authors agreed that

all potentially relevant information had been extracted.

Previous studies have rated intervention descriptions as adequate/complete or

inadequate/incomplete [4, 14]. Due to the subjective nature of these types of ratings (i.e. there is

not yet consensus as to what information must be reported in what level of detail to constitute an

adequate description of an intervention) it was decided by the study team to assess each section of

the TIDieR checklist as ‘reported’ or ‘not reported’, and where information was reported to

extract examples of more detailed reporting and more typical reporting. Items were assessed as

Page 7 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 8 -

“not reported” if there was no mention of that item at all in the trial publications. Where authors

made any reference to an item it was assessed as “reported”. A limitation of this approach to

assessment is that it may overstate the quality of reporting in included studies. All authors

independently reviewed the extracted data for each trial and made an assessment for each item

extracted. Examples of variation in reporting were also highlighted by each of the review authors

at this stage. Crosschecking of the assessments was carried out by the lead author. Microsoft

Excel was used to analyse the data descriptively.

Results

We identified 630 citations from our initial search. This became 563 when duplicate citations

were removed. Trials that were conducted in a non-health service setting were excluded in

the first round of screening (n=240) and the eligibility criteria applied to the remaining 323

citations. A total of 70 citations were included for data extraction (45 trial results publications

and 25 related publications). Where available we used trial registration numbers to identify

publications that may have been relevant for data extraction. We identified trial registration

numbers for 35 of the 45 included trials. Known trial registration numbers were used to

search PubMed identifying protocols and trial results publications for 26 of the included

trials. The remaining publications could not be retrieved using the trial registration number

alone. Identification of related publications through trial registration number search was not

always successful but searching using a combination of author name, trial name and

keywords identified 7 additional process evaluation publications. For the 35 trials with

registration numbers, 16 registrations had no records of publications, 6 had a results

publication but no protocol, 12 had both the protocol and results publications and one was not

in English. Related publications for the 9 trials where registration numbers were not

identified were retrieved through additional manual searching. In all but one of the related

publications there was an obvious link to the original trial i.e. the trial was named or cited in

Page 8 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 9 -

the publication. One potentially related publication was not included as it was not possible to

link the content to the original trial [16]. A flow diagram of studies through the review is

provided in Figure 1. Citations excluded as “not relevant” were secondary trial publications

(e.g. cost-effectiveness publications), trials where the healthcare professionals employed to

deliver routine care did not take an active role in implementing an intervention (e.g. system

wide quality improvement) and trials where healthcare professionals were recipients of an

education or training intervention (e.g. train-the-trainer initiative). All publications included

in data extraction are provided in Supplementary file 3.

<Insert Figure 1. PRISMA flow diagram about here>

Characteristics of included trials

The trials included in the review span 22 countries. The country with the most included trials

was the Netherlands (n=7), followed by Australia (n=5), Spain (n=5) and the UK (n=5). The

trial results publications came from 33 different journals including the BMJ (n=5), PLoS One

(n=4) and the Lancet (n=3). The conditions most frequently studied were type 2 diabetes

(n=5), asthma (n=4), cardiovascular disease, hypertension and stroke (n=4) and maternal and

child health (n=4). Over half of the trials were conducted in primary care settings (n=23) and

primarily evaluated lifestyle/self-management interventions (n=22). Nurses were responsible

for implementing the intervention in 12 trials, general practitioners in 10 of the trials and

pharmacists in four. Further details on the characteristics of the included studies are provided

in Supplementary file 4).

Quality of reporting of enabling or support activities

40 of the 45 trials (88.8%) made reference to some form of enabling or support activities

conducted during the trial. Of the five trials that did not describe any enabling or support

activities, three of these were evaluating infection control interventions in hospital settings.

The number of trials providing information on each of the items of the TIDieR checklist,

Page 9 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 10 -

adapted for enabling or support activities, is shown in Table 1. How activities were provided,

and how much, were the most frequently reported items (75.6% and 73.3% of trials

respectively). Reporting of the rationale for the chosen activities, any tailoring or

modifications of the activities and measures of planned and actual activities carried out

during the trial (e.g. attendance at training sessions) were the least frequently reported.

Table 1: Number of trials reporting components of enabling or support activities

Components of enabling or support activities Reported

% n

Why 11.1 5

Who provided 53.3 24

What materials 57.8 26

Where 22.2 10

When 51.1 23

How 75.6 34

How much 73.3 33

Tailoring 4.4 2

Modifications 4.4 2

Measure of fidelity 2.2 1

Actual activities delivered 6.7 3

Competency checking 17.8 8

Quality of reporting of strategies used to monitor fidelity throughout the trial and the

extent to which the intervention was delivered as planned

Over half of the included trials (n=29) reported monitoring the extent to which healthcare

professionals delivered the intervention as planned within the trial. The ways in which

fidelity was most frequently monitored included notes audits (n=12), rating of random

samples of audio recorded sessions (n=5), self-reported measures of implementation (from

healthcare professionals and study participants) (n=4) and observation of practice (n=2). Of

the trials that reported collecting measures of fidelity, data from these measures was reported

Page 10 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 11 -

in in 44.4% (n=20) of the trials. Examples of reporting, and variation across trials for the

individual items are provided in Supplementary file 5.

Discussion

Statement of principal findings

Almost all of the trials (89%) included in this review reported that enabling or support

activities were used to facilitate implementation of the intervention being tested. However,

large discrepancies were identified in the quality of reporting of the components of these

activities. Authors most frequently reported how enabling or support activities were provided,

and how much was provided, but details related to who provided, where, when and the

underpinning rationale for the content and delivery of the activities were often omitted. Less

than 20% of the included trials reported that competency checking took place prior to

implementation and data collection. Although, 29 trials reported using specific fidelity

measures to assess the extent to which the healthcare professionals had implemented the

intervention as planned during the trial, data from these measures was not reported in nine

instances.

Strengths and weaknesses of the study

A strength of this study is the use of the published TIDieR checklist (the Template for

Intervention Description and Replication) to assess the quality of reporting of the included

trials [12]. TIDieR has been developed by Hoffmann and colleagues as an extension to Item 5

of the CONSORT Statement and aims to guide authors in describing interventions in a level

of detail to allow for future replication. Although the primary purpose of the TIDieR

checklist is to assist in the describing of interventions being tested, a number of items in

TIDieR discuss how some interventions, particularly complex ones, might require additional

activities to enable or support the intervention to occur, and these should also be described.

We would argue, and have demonstrated here, that these activities are interventions in

Page 11 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 12 -

themselves and therefore can be reported with the same level of detail and attention afforded

to the intervention under investigation.

A limitation of this review may be the included trials and the way in which they were

identified. Although a systematic process was used to identify effectiveness cluster

randomised controlled trials, a large number of published trials will not have been identified

by our search (for example due to the time period or search terms used). The results of our

review therefore may not be generalisable. However, it is also unknown whether a larger

sample would have influenced the overall results of the study. When attempting to identify

secondary publications for the included trials it became evident that authors do not always

include the trial registration numbers in the trial results publications. We carried out

additional searching using publication titles, trial names and author names but were unable to

locate trial registration numbers for ten of the included trials. It is not possible to conclude

whether these trials were not registered or whether the search efforts were unable to locate

registration details. We endeavoured to retrieve all relevant publications, including process

evaluations and reports on training procedures. However, it is possible that details of support

activities may be reported in publications not identified during our searches. It was interesting

to note than in a number of instances the process evaluations did not provide additional

information on actual implementation during the trial or associated measures as the focus of

these publications was on providing in depth qualitative accounts on stakeholders experiences

of the implementation process. Medical Research Council guidance on process evaluations

was published earlier this year which may results in more consistency as to the type of

information reported in these evaluations [17].

As previously mentioned, similar trials [4, 13, 14] have not only looked at whether or not

details have been reported but have also made a judgement as to whether the details provided

were sufficient to merit replication. In this review we opted to assess whether or not authors

Page 12 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 13 -

provided details on the items of the TIDieR checklist and provide supporting examples. This

may also be a limitation of the review as the figures provided here without doubt present a

more positive picture than would be the case if we were to make subjective judgements on

the extent to which interventions could be replicated from the details provided.

Meaning of the study

Despite improvements in the quality of reporting of clinical research in recent years, complex

trial designs, and interventions, have resulted in trial reports leaving readers with many

questions unanswered. In this review we have focused in particular on the reporting of

enabling and support activities and implementation fidelity within trials, as this is important

information for both interpreting the research findings and also for clinicians wanting

guidance on how to implement interventions in their practice. We have shown that although

these activities are reported in trials, there exist important gaps when assessed using items of

the TIDieR checklist. Implementation science is a growing field of research, with much

interest in how we can best translate interventions that have been shown to be effective in

research into day-to-day clinical practice. In order to progress this field of research it is

important that authors describe the rationale for their implementation activities, and the

components of these activities in order to contribute to the growing evidence-base.

Publication restrictions pose a particular challenge in achieving this goal and therefore novel

methods of efficiently communicating these details in useable formats are urgently needed.

Unanswered questions and future research

Effectiveness cluster randomised trials of complex non-pharmacological interventions pose

numerous challenges in their design, implementation and reporting. However, there is a

growing appreciation for the nuanced and iterative processes that occur when introducing a

new intervention into routine clinical practice and the role of contextual and personal factors

Page 13 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 14 -

[18]. There is a need for more transparent reporting about what is happening in trials to make

informed decisions about the feasibility and effectiveness of interventions, along with their

likely financial and resource implications. There is also a need to establish consensus about

the level of detail that should be reported for such large scale trials to ensure that what is

reported is meaningful and useful for patients, the public, clinicians and researchers.

Copyright

The Corresponding Author has the right to grant on behalf of all authors and does grant on

behalf of all authors, a worldwide licence to the Publishers and its licensees in perpetuity, in

all forms, formats and media to i) publish, reproduce, distribute, display and store the

Contribution, ii) translate the Contribution into other languages, create adaptations, reprints,

include within collections and create summaries, extracts and/or, abstracts of the Contribution

and convert or allow conversion into any format including without limitation audio, iii) create

any other derivative work(s) based in whole or part on the on the Contribution, iv) to exploit

all subsidiary rights that currently exist or as may exist in the future in the Contribution, v)

the inclusion of electronic links from the Contribution to third party material where-ever it

may be located; and, vi) licence any third party to do any or all of the above.

Transparency declaration

NM affirms that the manuscript is an honest, accurate, and transparent account of the study

being reported; that no important aspects of the study have been omitted; and that any

discrepancies from the study as planned have been explained.

Ethical approval

Not required.

Page 14 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 15 -

Contributors

NM and LC were responsible for study conception and for conducting the original search. All

authors contributed to i) screening studies for inclusion, ii) developing, piloting and refining

the data extraction form and iii) completing data extraction. NM was responsible for

compiling the first draft of the manuscript. All authors were involved in preparing the final

manuscript for submission. NM and LC are the guarantors. All authors had full access to all

of the data in the study and take responsibility for the integrity of the data and the accuracy of

the data analysis.

Competing interests

All authors have completed the Unified Competing Interest form at

www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and

declare that LC is supported by an NIHR Career Development Fellowship which also funds

NM; the authors have no relationships with other organisations that might have an interest in

the submitted work in the previous 3 years; their spouses, partners, or children have no

financial relationships that may be relevant to the submitted work; and they have no non-

financial interests that may be relevant to the submitted work.

Funding

LC is supported by an NIHR Career Development Fellowship that also funds NM. This

article presents independent research funded by the National Institute for Health Research

(NIHR). The views expressed are those of the authors and not necessarily those of the NHS,

the NIHR or the Department of Health.

Page 15 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 16 -

Data sharing

Further details of included trials and data extraction is available on request from

[email protected].

References

1. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research

evidence. Lancet 2009;374:86-9.

2. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for

reporting parallel group randomised trials. BMC Medicine 2010;8:18.

3. Chan A-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 statement: defining standard

protocol items for clinical trials. Ann Intern Med. 2013;158:200-7.

4. Hoffmann TC, Erueti C, Glasziou PP. Poor description of non-pharmacological

interventions: analysis of consecutive sample of randomised trials. BMJ 2013;347.

5. Flay BR. Efficacy and effectiveness trials (and other phases of research) in the

development of health promotion programs. Prev Med. 1986;15:451-74.

6. Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of

health promotion research to practice? Rethinking the efficacy-to-effectiveness

transition. Am J Public Health 2003;93:1261-7.

7. Gartlehner G, Hansen RA, Nissman D, Lohr KN, Carey TS. A simple and valid tool

distinguished efficacy from effectiveness studies. J Clin Epidemiol 2006;59:1040-8.

8. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic–explanatory continuum

indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol

2009;62:464-75

9. Campbell MK, Mollison J, Steen N, Grimshaw JM, Eccles M. Analysis of cluster

randomized trials in primary care: a practical approach. Fam Pract 2000;17:192-6.

10. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and

evaluating complex interventions: the new Medical Research Council guidance. BMJ

2008;337:a1655.

11. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, Grp C. Extending the

CONSORT statement to randomized trials of nonpharmacologic treatment:

Explanation and elaboration. Ann Intern Med 2008;148:295-309.

12. Hoffmann TC, Glasziou PP, Boutron I, et al. Better reporting of interventions:

template for intervention description and replication (TIDieR) checklist and guide.

BMJ 2014;348.

13. Schroter S, Glasziou P, Heneghan C. Quality of descriptions of treatments: a review

of published randomised controlled trials. BMJ Open 2012;2.

14. Douet L, Milne R, Anstee S, Habens F, Young A, Wright D. The completeness of

intervention descriptions in published National Institute of Health Research HTA-

funded trials: a cross-sectional study. BMJ Open 2014;4:e003713.

Page 16 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 17 -

15. Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of

treatment in trials and reviews? BMJ 2008;336:1472.

16. Papadakis S, Gharib M, Hambleton J, Reid RD, Assi R, Pipe AL. Delivering

evidence-based smoking cessation treatment in primary care practice: experience of

Ontario family health teams. Can Fam Physician 2014;60:e362-71.

17. Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions:

Medical Research Council guidance. BMJ 2015;350:h1258.

18. Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not

enough: evidence from an in-depth multiple case study on the untold role and impact

of context in randomised controlled trials of seven complex interventions. Trials

2012;13:95-111.

Page 17 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



Figure 1. PRISMA flow diagram 158x214mm (300 x 300 DPI)

Page 18 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 1 -

Supplementary file 1: Search strategy (22.09.2014):

1. (randomised or randomized).m_titl. (109896)

2. implement*.ti,ab. (261884)

3. (cluster or multi-site or multi-centre).m_titl. (22294)

4. "intervent*".m_titl. (87510)

5. 2 or 4 (341756)

6. ((trial or study) not protocol).m_titl. (1070741)

7. 1 and 3 and 6 (2093)

8. 5 and 7 (761)

9. limit 9 to yr="2008 - 2014" (630)

Page 19 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 1 -

Supplementary file 2: Data extraction form and completed example

Data extraction form

First author of trial results publication

Blackberry

Publication Title Effectiveness of general practice based, practice nurse led telephone coaching on glycaemic control of type 2 diabetes: the Patient Engagement and Coaching for Health (PEACH) pragmatic cluster randomised controlled trial.

Trial Acronym PEACH

Trial Registration ISRCTN50662837

Journal Citation Blackberry ID, Furler JS, Best JD, Chondros P, Vale M, Walker C, et al. Effectiveness of general practice based, practice nurse led telephone coaching on glycaemic control of type 2 diabetes: the Patient Engagement and Coaching for Health (PEACH) pragmatic cluster randomised controlled trial. BMJ. 2013;347:f5272.

Journal BMJ

Year 2013

Study Design Cluster randomised controlled trial

Publication type Trial results

Other relevant trial publications

PROTOCOL: Young D, Furler J, Vale M, Walker C, Segal L, Dunning P, et al. Patient Engagement and Coaching for Health: The PEACH study--a cluster randomised controlled trial using the telephone to coach people with type 2 diabetes to engage with their GPs to improve diabetes care: a study protocol. BMC Fam Pract. 2007;8:20.

Topic Type 2 Diabetes

Page 20 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 2 -

Aim To evaluate the effectiveness of goal focused telephone coaching by practice nurses in improving glycaemic control in patients with type 2 diabetes in Australia.

Name/type of the intervention

Patient Engagement and Coaching for Health intervention (PEACH). (self-management)

HCPs implementing the intervention

Nurses

Setting General practices

Enabling or support activities

Made reference to enabling or support activities

Yes

Rationale for the activities Not described

Who provided The research team provides on-site assistance to the practice nurse in their first two baseline interviews and questionnaire administration to ensure quality of data collection.

What materials Practices were reimbursed for their nurses’ time spent on the PEACH study, including for patient recruitment, data collection, and delivery of the intervention.

Where On-site assistance

When Unclear

How Each practice nurse from practices assigned to the intervention group completed a two day training programme in telephone coaching. / The research team provided support to practice nurses during the intervention period, including one visit to the practice, monthly telephone calls, and a group meeting, where all intervention practice nurses could share their experience and learning from conducting telephone coaching.

Page 21 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 3 -

How much An initial 15 hours of telephone coaching training over two days, and further support and training was provided to individual nurses after assessment of their progress.

Tailoring None described

Modifications None described

Measures of fidelity for the enabling or support activities (e.g. monitoring attendance at training sessions)

Not described

Actual implementation of support activities (e.g. had to increase number of sessions to accommodate staff turnover)

Not described

Competency checking in advance of implementation /accreditation

None described


Strategy used to monitor fidelity

The "Head Coach" (MV) monitors the first telephone coaching and random subsequent telephone coaching.

Extent to which the intervention was delivered as planned

The intensity and fidelity of the intervention was compromised so that the intended dose of the intervention was not delivered to a high proportion of patients and in many cases the key element of a focus on medication intensification through negotiation with the treating general practitioners was not achieved. Intervention fidelity also seems to have been compromised. Content analysis of calls showed many consultations did not explicitly address medication intensification, an important part of the coaching programme.

Page 22 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 1 -

Supplementary file 3: Publications included in data extraction

ID First author/year of trial results publication Trial registration number No. Trial publications included in data extraction

1 Adachi 2013 UMIN000004049 2

PROTOCOL: • Adachi M, Yamaoka K, Watanabe M, et al. Effects of lifestyle

education program for type 2 diabetes patients in clinics: study design of a cluster randomized trial. BMC Public Health 2010;10:742.

RESULTS: • Adachi M, Yamaoka K, Watanabe M, et al. Effects of lifestyle

education program for type 2 diabetes patients in clinics: a cluster randomized controlled trial. BMC Public Health 2013;13:467.

2 Armour 2013 None identified 1

RESULTS: • Armour CL, Reddel HK, LeMay KS, et al. Feasibility and

effectiveness of an evidence-based asthma service in Australian community pharmacies: a pragmatic cluster randomized trial. J Asthma 2013;50:302-9.

3 Blackberry 2013 ISRCTN50662837 2

PROTOCOL: • Young D, Furler J, Vale M, et al. Patient Engagement and

Coaching for Health: The PEACH study--a cluster randomised controlled trial using the telephone to coach people with type 2 diabetes to engage with their GPs to improve diabetes care: a study protocol. BMC Fam Pract 2007;8:20.

RESULTS: • Blackberry ID, Furler JS, Best JD, et al. Effectiveness of general

practice based, practice nurse led telephone coaching on glycaemic control of type 2 diabetes: the Patient Engagement and Coaching for Health (PEACH) pragmatic cluster randomised controlled trial. BMJ 2013;347:f5272.

Page 23 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 2 -

4 Cabezas 2011 NCT00125905 2

PROTOCOL: • Cabezas C, Martin C, Granollers S, et al. Effectiveness of a

stepped primary care smoking cessation intervention (ISTAPS study): design of a cluster randomised trial. BMC Public Health 2009;9:48.

RESULTS: • Cabezas C, Advani M, Puente D, Rodriguez-Blanco T, Martin C,

Group IS. Effectiveness of a stepped primary care smoking cessation intervention: cluster randomized clinical trial (ISTAPS study). Addiction 2011;106:1696-706.

5 Chao 2014 None identified 1

RESULTS: • Chao YH, Usher K, Buettner PG, Holmes C. Cluster randomised

controlled trial: educational self-care intervention with older Taiwanese patients with type 2 diabetes mellitus--impact on blood glucose levels and diabetic complications. Collegian 2014;21:43-51.

6 Deales 2014 ACTRN12611000813987 1

RESULTS: • Deales A, Fratini M, Romano S, et al. Care manager to control

cardiovascular risk factors in primary care: the Raffaello cluster randomized trial. Nutr Metab Cardiovasc Dis 2014;24:563-71.

7 Derde 2014 NCT00976638 1

RESULTS: • Derde LP, Cooper BS, Goossens H, et al. Interventions to reduce

colonisation and transmission of antimicrobial-resistant bacteria in intensive care units: an interrupted time series study and cluster randomised trial.[Erratum appears in Lancet Infect Dis. 2014 Jan;14(1):11]. Lancet Infect Dis 2014;14:31-9.

Page 24 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 3 -

8 Dinneen 2013 ISRCTN79759174 2

PROTOCOL: • Dinneen SF, MC OH, Byrne M, et al. The Irish DAFNE study

protocol: a cluster randomised trial of group versus individual follow-up after structured education for type 1 diabetes. Trials 2009;10:88.

RESULTS: • Dinneen SF, O'Hara MC, Byrne M, et al. Group follow-up

compared to individual clinic visits after structured education for type 1 diabetes: a cluster randomised controlled trial. Diabetes Res Clin Pract 2013;100:29-38.

9 Garcia-Cardenas 2013 NCT01085474 1

RESULTS: • Garcia-Cardenas V, Sabater-Hernandez D, Kenny P, Martinez-

Martinez F, Faus MJ, Benrimoj SI. Effect of a pharmacist intervention on asthma control. A cluster randomised trial. Respir Med 2013;107:1346-55.

10 Grandes 2009 NCT00131079 1

RESULTS: • Grandes G, Sanchez A, Sanchez-Pinilla RO, et al. Effectiveness

of physical activity advice and prescription by physicians in routine primary care: a cluster randomized trial. Arch Intern Med 2009;169:694-701.

11 Hafkamp-de Groen 2014 ISRCTN15790308 2

PROTOCOL: • Hafkamp-de Groen E, Mohangoo AD, de Jongste JC, et al. Early

detection and counselling intervention of asthma symptoms in preschool children: study design of a cluster randomised controlled trial. BMC Public Health 2010;10:555.

RESULTS: • Hafkamp-de Groen E, van der Valk RJ, Mohangoo AD, et al.

Evaluation of systematic assessment of asthma-like symptoms and tobacco smoke exposure in early childhood by well-child professionals: A randomised trial. PLoS One 2014;9:e90982.

Page 25 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 4 -

12 Haller 2014 ACTRN12608000432314 1

RESULTS: • Haller DM, Meynard A, Lefebvre D, Ukoumunne OC, Narring

F, Broers B. Effectiveness of training family physicians to deliver a brief intervention to address excessive substance use among young patients: a cluster randomized controlled trial. CMAJ 2014;186:E263-72.

13 Hegarty 2010 ACTRN12608000032358 2

PROTOCOL: • Hegarty KL, Gunn JM, O'Doherty LJ, et al. Women's evaluation

of abuse and violence care in general practice: a cluster randomised controlled trial (weave). BMC Public Health 2010;10:2.

RESTULTS: • Hegarty K, O'Doherty L, Taft A, et al. Screening and counselling

in the primary care setting for women who have experienced intimate partner violence (WEAVE): a cluster randomised controlled trial. Lancet 2013;382:249-58.

14 Hoddinott 2009 ISRCTN44857041 2

RESULTS: • Hoddinott P, Britten J, Prescott GJ, Tappin D, Ludbrook A,

Godden DJ. Effectiveness of policy to provide breastfeeding groups (BIG) for pregnant and breastfeeding mothers in primary care: cluster randomised controlled trial. BMJ 2009;338:a3026.

PROCESS EVALUATION: • Hoddinott P, Britten J, Pill R. Why do interventions work in

some places and not others: a breastfeeding support group trial. Soc Sci Med 2010;70:769-78

15 Hopkinson 2010 None identified 1

RESULTS: • Hopkinson JB, Fenlon DR, Okamoto I, et al. The deliverability,

acceptability, and perceived effect of the Macmillan approach to weight loss and eating difficulties: a phase II, cluster-randomized, exploratory trial of a psychosocial intervention for weight- and eating-related distress in people with advanced cancer. J Pain Symptom Manage 2010;40:684-95.

Page 26 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 5 -

16 Jahn 2009 None identified 1

RESULTS: • Jahn P, Renz P, Stukenkemper J, et al. Reduction of

chemotherapy-induced anorexia, nausea, and emesis through a structured nursing intervention: a cluster-randomized multicenter trial. Support Care Cancer 2009;17:1543-52.

17 Kennedy 2013 ISRCTN90940049 3

PROTOCOL: • Bower P, Kennedy A, Reeves D, et al. A cluster randomised

controlled trial of the clinical and cost-effectiveness of a 'whole systems' model of self-management support for the management of long- term conditions in primary care: trial protocol. Implement Sci 2012;7:7.

RESULTS: • Kennedy A, Bower P, Reeves D, et al. Implementation of self

management support for long term conditions in routine primary care settings: cluster randomised controlled trial. BMJ 2013;346:f2882.

PROCESS EVALUATION: • Kennedy A, Rogers A, Chew-Graham C, et al. Implementation

of a self-management support approach (WISE) across a health system: a process evaluation explaining what did and did not work for organisations, clinicians and patients. Implement Sci 2014;9:129.

18 Labhardt 2011 NCT00744458 1

RESULTS: • Labhardt ND, Balo JR, Ndam M, Manga E, Stoll B. Improved

retention rates with low-cost interventions in hypertension and diabetes management in a rural African environment of nurse-led care: a cluster-randomised trial. Trop Med Int Health 2011;16:1276-84.

Page 27 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 6 -

19 Law 2011 None identified 1

RESULTS: • Law MC, Darrah J, Pollock N, et al. Focus on function: a cluster,

randomized controlled trial comparing child- versus context-focused intervention for young children with cerebral palsy. Dev Med Child Neurol 2011;53:621-9.

20 Leendertse 2013 NTR 2647 2

PROTOCOL: • Leendertse AJ, de Koning FH, Goudswaard AN, et al.

Preventing hospital admissions by reviewing medication (PHARM) in primary care: design of the cluster randomised, controlled, multi-centre PHARM-study. BMC Health Serv Res 2011;11:4.

RESULTS: • Leendertse AJ, de Koning GH, Goudswaard AN, et al.

Preventing hospital admissions by reviewing medication (PHARM) in primary care: an open controlled study in an elderly population. J Clin Pharm Ther 2013;38:379-87.

21 Leonhardt 2008 None identified 1

RESULTS: • Leonhardt C, Keller S, Chenot JF, et al. TTM-based motivational

counselling does not increase physical activity of low back pain patients in a primary care setting--A cluster-randomized controlled trial. Patient Educ Couns 2008;70:50-60.

22 Leontjevas 2013 NTR1477 2

PROTOCOL: • Gerritsen DL, Smalbrugge M, Teerenstra S, et al. Act In case of

Depression: the evaluation of a care program to improve the detection and treatment of depression in nursing homes. Study Protocol. BMC Psychiatry 2011;11:91.

RESULTS: • Leontjevas R, Gerritsen DL, Smalbrugge M, Teerenstra S,

Vernooij-Dassen MJ, Koopmans RT. A structural multidisciplinary approach to depression management in nursing-home residents: a multicentre, stepped-wedge cluster-randomised trial. Lancet 2013;381:2255-64.

Page 28 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 7 -

23 Luoto 2011 ISRCTN33885819 2

PROTOCOL: • Luoto RM, Kinnunen TI, Aittasalo M, et al. Prevention of

gestational diabetes: design of a cluster-randomized controlled trial and one-year follow-up. BMC Pregnancy Childbirth 2010;10:39.

RESULTS: • Luoto R, Kinnunen TI, Aittasalo M, et al. Primary prevention of

gestational diabetes mellitus and large-for-gestational-age newborns by lifestyle counseling: a cluster-randomized controlled trial. PLoS Med 2011;8:e1001036.

24 Magoma 2013 ACTRN12609000268246 1

RESULTS: • Magoma M, Requejo J, Campbell O, Cousens S, Merialdi M,

Filippi V. The effectiveness of birth plans in increasing use of skilled care at delivery and postnatal care in rural Tanzania: a cluster randomised trial. Trop Med Int Health 2013;18:435-43.

25 Metzelthin 2013 ISRCTN31954692 3

PROTOCOL: • Metzelthin SF, van Rossum E, de Witte LP, Hendriks MR,

Kempen GI. The reduction of disability in community-dwelling frail older people: design of a two-arm cluster randomized controlled trial. BMC Public Health 2010;10:511.

RESULTS: • Metzelthin SF, van Rossum E, de Witte LP, et al. Effectiveness

of interdisciplinary primary care approach to reduce disability in community dwelling frail older people: cluster randomised controlled trial. BMJ 2013;347:f5264.

PROCESS EVALUATION: • Metzelthin SF, Daniels R, van Rossum E, et al. A nurse-led

interdisciplinary primary care approach to prevent disability among community-dwelling frail older people: a large-scale process evaluation. Int J Nurs Stud 2013;50:1184-96.

Page 29 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 8 -

26 Meyer 2012 NCT00679861 1

RESULTS: • Meyer C, Ulbricht S, Gross B, et al. Adoption, reach and

effectiveness of computer-based, practitioner delivered and combined smoking interventions in general medical practices: a three-arm cluster randomized trial. Drug Alcohol Depend 2012;121:124-32.

27 Morrell 2009 ISRCTN92195776 3

RESULTS: • Morrell CJ, Slade P, Warner R, et al. Clinical effectiveness of

health visitor training in psychologically informed approaches for depression in postnatal women: pragmatic cluster randomised trial in primary care. BMJ 2009;338:a3045.

HEALTH TECHNOLOGY ASSESSMENT REPORT: • Morrell CJ, Warner R, Slade P, et al. Psychological interventions

for postnatal depression: cluster randomised trial and economic evaluation. The PoNDER trial. Health Technol Assess 2009;13:iii-iv, xi-xiii, 1-153.

TRAINING OF HEALTH VISITORS: • Morrell CJ, Ricketts T, Tudor K, Williams C, Curran J, Barkham

M. Training health visitors in cognitive behavioural and person-centred approaches for depression in postnatal women as part of a cluster randomised trial and economic evaluation in primary care: the PoNDER trial. Prim Health Care Res Dev 2011;12:11-20.

28 Nokela 2010 None identified 1

RESULTS: • Nokela M, Arnlind MH, Ehrs PO, Krakau I, Forslund L, Jonsson

EW. The influence of structured information and monitoring on the outcome of asthma treatment in primary care: a cluster randomized study. Respiration 2010;79:388-94.

Page 30 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 9 -

29 Papadakis 2013 NCT00799279 1

RESULTS: • Papadakis S, McDonald PW, Pipe AL, Letherdale ST, Reid RD,

Brown KS. Effectiveness of telephone-based follow-up support delivered in combination with a multi-component smoking cessation intervention in family practice: a cluster-randomized trial. Prev Med 2013;56:390-7.

30 Pladevall 2010 ISRCTN35208258 1

RESULTS: • Pladevall M, Brotons C, Gabriel R, et al. Multicenter cluster-

randomized trial of a multifactorial intervention to improve antihypertensive medication adherence and blood pressure control among patients at high cardiovascular risk (the COM99 study). Circulation 2010;122:1183-91.

31 Pope 2008 None identified 2

RESULTS: • Pope DS, Deluca AN, Kali P, et al. A cluster-randomized trial of

provider-initiated (opt-out) HIV counseling and testing of tuberculosis patients in South Africa. J Acquir Immune Defic Syndr 2008;48:190-5.

NURSES EXPERIENCES OF IMPLEMENTATION: • Pope DS, Atkins S, DeLuca AN, et al. South African TB nurses'

experiences of provider-initiated HIV counseling and testing in the Eastern Cape Province: a qualitative study. AIDS Care 2010;22:238-45.

32 Quinn 2011 None identified 2

PROTOCOL: • Quinn CC, Gruber-Baldini AL, Shardell M, et al. Mobile

diabetes intervention study: testing a personalized treatment/behavioral communication intervention for blood glucose control. Contemp Clin Trials 2009;30:334-46.

RESULTS: • Quinn CC, Shardell MD, Terrin ML, Barr EA, Ballew SH,

Gruber-Baldini AL. Cluster-randomized trial of a mobile phone personalized behavioral intervention for blood glucose control. Diabetes Care 2011;34:1934-42.

Page 31 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 10 -

33 Rat 2014 NCT01610531 1

RESULTS: • Rat C, Quereux G, Riviere C, et al. Targeted melanoma

prevention intervention: a cluster randomized controlled trial. Ann Fam Med 2014;12:21-8.

34 Roisin 2014 NCT00846105 1

RESULTS: • Roisin S, Laurent C, Denis O, et al. Impact of rapid molecular

screening at hospital admission on nosocomial transmission of methicillin-resistant staphylococcus aureus: cluster randomised trial. PLoS One 2014;9:e96310.

35 Septimus 2014 None identified 1

RESULTS: • Septimus EJ, Hayden MK, Kleinman K, et al. Does

chlorhexidine bathing in adult intensive care units reduce blood culture contamination? A pragmatic cluster-randomized trial. Infect Control Hosp Epidemiol 2014;35 Suppl 3:S17-22.

36 Slater 2013 ACTRN12611000053921 1

RESULTS: • Slater H, Briggs AM, Watkins K, Chua J, Smith AJ. Translating

evidence for low back pain management into a consumer-focussed resource for use in community pharmacies: a cluster-randomised controlled trial. PLoS One 2013;8:e71918.

37 Sonnichsen 2010 ISRCTN27414162 2

PROTOCOL: • Sonnichsen AC, Rinnerberger A, Url MG, et al. Effectiveness of

the Austrian disease-management-programme for type 2 diabetes: study protocol of a cluster-randomized controlled trial. Trials 2008;9:38.

RESULTS: • Sonnichsen AC, Winkler H, Flamm M, et al. The effectiveness

of the Austrian disease management programme for type 2 diabetes: a cluster-randomised controlled trial. BMC Fam Pract 2010;11:86.

Page 32 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 11 -

38 Spijker 2011 NCT00147693 3

PROTOCOL: • Spijker A, Verhey F, Graff M, et al. Systematic care for

caregivers of people with dementia in the ambulatory mental health service: designing a multicentre, cluster, randomized, controlled trial. BMC Geriatr 2009;9:21.

RESULTS: • Spijker A, Wollersheim H, Teerenstra S, et al. Systematic care

for caregivers of patients with dementia: a multicenter, cluster-randomized, controlled trial. Am J Geriatr Psychiatry 2011;19:521-31.

ASSESSMENT OF IMPLEMENTATION: • Spijker A, Teerenstra S, Wollersheim H, Adang E, Verhey F,

Vernooij-Dassen M. Influence of adherence to a systematic care program for caregivers of dementia patients. Am J Geriatr Psychiatry 2013;21:26-36.

39 Stewart 2014 ACTRN12609000705280 2

PROTOCOL: • Lau R, Stewart K, McNamara KP, et al. Evaluation of a

community pharmacy-based intervention for improving patient adherence to antihypertensives: a randomised controlled trial. BMC Health Serv Res 2010;10:34.

RESULTS: • Stewart K, George J, Mc Namara KP, et al. A multifaceted

pharmacist intervention to improve antihypertensive adherence: a cluster-randomized, controlled trial (HAPPy trial). J Clin Pharm Ther 2014;39:527-34.

Page 33 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 12 -

40 van de Ven 2013 NTR2314 2

PROTOCOL: • van de Ven G, Draskovic I, Adang EM, et al. Improving person-

centred care in nursing homes through dementia-care mapping: design of a cluster-randomised controlled trial. BMC geriatr 2012;12:1.

RESULTS: • van de Ven G, Draskovic I, Adang EM, et al. Effects of

dementia-care mapping on residents and staff of care homes: a pragmatic cluster-randomised controlled trial. PLoS One 2013;8:e67325.

41 van den Bemt 2009 NCT00542061 1

RESULTS: • van den Bemt L, Schermer TR, Smeele IJ, et al. An expert-

supported monitoring system for patients with chronic obstructive pulmonary disease in general practice: results of a cluster randomised controlled trial. Med J Aust 2009;191:249-54.

42 Vicens 2014 ISRCTN13024375 2

PROTOCOL: • Vicens C, Socias I, Mateu C, et al. Comparative efficacy of two

primary care interventions to assist withdrawal from long term benzodiazepine use: a protocol for a clustered, randomized clinical trial. BMC Fam Pract 2011;12:23.

RESULTS: • Vicens C, Bejarano F, Sempere E, et al. Comparative efficacy of

two interventions to discontinue long-term benzodiazepine use: cluster randomised controlled trial in primary care. Br J Psychiatry 2014;204:471-9.

43 Weaver 2014 ISRCTN72794493 1

RESULTS: • Weaver T, Metrebian N, Hellier J, et al. Use of contingency

management incentives to improve completion of hepatitis B vaccination in people undergoing treatment for heroin dependence: a cluster randomised trial. Lancet 2014;384:153-63.

Page 34 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 13 -

44 Wolfe 2010 ISRCTN10730637 1

RESULTS: • Wolfe CD, Redfern J, Rudd AG, Grieve AP, Heuschmann PU,

McKevitt C. Cluster randomized controlled trial of a patient and general practitioner intervention to improve the management of multiple risk factors after stroke: stop stroke. Stroke 2010;41:2470-6.

45 Zatzick 2014 NCT00607620 2

RESULTS: • Zatzick D, Donovan DM, Jurkovich G, et al. Disseminating

alcohol screening and brief intervention at trauma centers: a policy-relevant cluster randomized effectiveness trial. Addiction 2014;109:754-65.

EVALUATION OF TRAINING: • Darnell D, Dunn C, Atkins D, Ingraham L, Zatzick D. A

Randomized Evaluation of Motivational Interviewing Training for Mandated Implementation of Alcohol Screening and Brief Intervention in Trauma Centers. J Subst Abuse Treat 2015 [Epub ahead of print].

Page 35 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 1 -

Supplementary file 4: Characteristics of included trials

Categories n

Condition

Type 2 diabetes 5 Asthma 4

Cardiovascular disease, hypertension, stroke 4 Maternal/child health (Breastfeeding, gestational

diabetes mellitus (GDM), postnatal depression) 4

Cancer 3 Infection control 3

Smoking cessation 3 Dementia 2

Multiple long term conditions 2 Low back pain 2

Other (intimate partner violence, frailty) 2 Substance use 2

Pharmacotherapy 2 Alcohol consumption 1

Chronic obstructive pulmonary disease 1 Depression 1

HIV 1 Cerebral palsy 1

Physical activity 1 Type 1 diabetes 1

Categories n

Healthcare professionals

implementing the intervention

Nurses 12 Mixed 12

GPs 10 Pharmacists 4

ICU staff 2 Community teams (mental health, palliative care) 2

Dieticians 1 Hospital doctors 1

Allied health professionals 1 Categories n

Setting

Primary care 23 Specialist centres/services 5

Community pharmacies 4 Community teams 2

Hospitals 2 Intensive care units 2

Outpatient clinics 2 Mixed 2

Other (trauma unit, dispensary, German centre) 2 Nursing homes 1

Page 36 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 2 -

Categories n

Type of intervention

Lifestyle/Education/Self-management (e.g. counselling, motivational interviewing, smoking

cessation) 22 Multi-/interdisciplinary case management 6

Screening and brief intervention 4 Infection control strategies 3

Other 2 Pharmacotherapy (e.g. discontinuation of

benzodiazepine) 2 Treatment contracts/incentives (e.g. vaccinations) 2

Breastfeeding policy 1 COPD monitoring system with expert

recommendations 1 Psychosocial intervention in cancer care 1

Neurological rehabilitation 1 Categories n

Country

The Netherlands 7 Australia 5

Spain 5 UK 5

Germany 3 USA 3

Canada 2 Austria 1

Belgium 1 Cameroon 1

Europe 1 Finland 1 France 1 Ireland 1

Italy 1 Japan 1

Scotland 1 South Africa 1

Sweden 1 Switzerland 1

Taiwan 1 Tanzania 1

Page 37 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 1 -

Supplementary file 5: Examples of reporting, and variation across trials for the individual components The purpose of this table is to provide examples of the variation in reporting across trials. Quotes provided may relate to a number of items but

have been provided with the item to which they are most closely related. Trial ID numbers (Supplementary file 3) are also provided.

Item More detailed reports More typical reports

Why The development and evaluation of the training intervention have taken place prior to this trial, and details of the theoretical background and content have been published elsewhere. The planned approach to training combines evidence-based approaches to changing professional behaviour with approaches to ‘normalise’ those behaviours in current practice. #17

The intervention components were selected based on a review of the literature of evidence-based strategies for integrating smoking cessation into primary care settings. #29

Who provided The education is delivered by a Diabetes Specialist Nurse, Dietitian and Doctor all of whom have undergone training in delivery of the DAFNE curriculum. #8

The training sessions were designed by 2 of us (D.M.H. and A.M., both family physicians trained in adolescent health; A.M. is also a trainer in motivational interviewing). #12

Before the start of the study, registered dietitians had the opportunity to gain experience with the intervention protocol under supervision by the project team. #1

What materials Several aids have been developed to facilitate the use of the SCPD:

1. The SCPD screening tool. The questions for the screening a caregiver's sense of competence and depressive symptoms are printed on a handy plasticized pocket card.

2. The SCPD manual. The manual consists of the items to be discussed during the training sessions (i.e., background

Materials and resources were made available as hard copy, online and on CDs, to provide flexible learning opportunities. #39

A training manual was written by the research team and provided to all nurses. #41

Page 38 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 2 -

information and methods), and some supporting literature has been added.

3. The starter package and action list. Several forms have been developed for the requested data. The starter package contains the forms that professionals need to gather these data. The action list contains 60 possible intervening and supportive actions that professionals might undertake as a result of the screening. They are divided into nine categories: intake, diagnostics, psychoeducation, psychosocial care, medical care, how to hand over care, legal care, case management, and crisis management. #38

Where No examples were identified of more detailed reports on location/setting for enabling or support activities. This item of the TIDieR checklist prompts authors to describe the type(s) of location(s) where the intervention occurred, including any necessary infrastructure or relevant features

Training consisted of a distance learning package (6 hours; reading, audit of patients, four interactive group teleconference sessions) and two one-hour interactive practice visits delivered by an academic GP. #13

First, the team visited NH units before crossing over to the intervention condition, explained the program and delivered three tailored packages to psychologists, physicians, and nursing staff including unit managers and activity therapists. #22

When This training was conducted as pre-trial workshops (HS), during which pharmacists were instructed about the key pamphlet messages to reinforce and were advised about the necessity of delivering these messages strictly in accordance with the pamphlet content. Pharmacists were encouraged to request clarification and feedback in regards to delivery of the messages at the time of these workshops and throughout the trial if/as required. #36

All nurses had previously received training in provision of HBV vaccinations. #43

Before and during the intervention period the GP and the pharmacist are offered support by training, education and a web based community of practice with a helpdesk on pharmacotherapy. #20

Page 39 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 3 -

The on-site workshop was followed over the next 6 months by four trainer-led 30-minute telephone coaching sessions during which MI skills were practised and subsequently boosted with written feedback via e-mail. Following the 6-month training period, intervention site SBI providers delivered bedside SBI to injured patients. #45

How Intervention site nursing and social work SBI providers learned to deliver MI during a 1-day workshop delivered by the study trainer (CD). The workshop training targeted a 20–30-minute MI that could be delivered at bedside to injured in-patients by the full spectrum of trauma center providers (e.g. social work and nursing providers). The workshop emphasized MI skill development and the importance of spending adequate time conducting interventions by the bedside with injured in-patients. The on-site workshop was followed over the next 6 months by four trainer-led 30-minute telephone coaching sessions during which MI skills were practised and subsequently boosted with written feedback via e-mail. #45

The Healthy Relationships Training programme. Training consisted of a distance learning package (6 hours; reading, audit of patients, four interactive group teleconference sessions) and two one-hour interactive practice visits delivered by an academic GP (including one visit with a simulated patient who role-played different readiness to change scenarios). #13

Several meetings about the aspects and basic principles of the intervention protocol took place (i.e. the screening procedure, self-management principles, client centeredness, motivational interviewing, interdisciplinary collaboration, assessment tools, parts of the toolbox and referrals). #1

The professionals were provided with a two-hour specific training on how to apply and use the brief assessment form regarding asthma-like symptoms and ETS exposure. #11

All nurses in all groups participated in a 4-h training session on how to establish a treatment contract with hypertensive patients or patients with diabetes. #18

Page 40 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 4 -

How much An initial 15 hours of telephone coaching training over two days, and further support and training was provided to individual nurses after assessment of their progress. #3

The process of becoming a DAFNE centre involves 43 h of training for a DAFNE doctor and 105 h of training for each of the (at least 2). #8

All nurses were supported with meetings every third month during the intervention. #23

Tailoring No examples were identified of more detailed reports on tailoring of enabling or support activities. This item of the TIDieR checklist prompts authors to describe if the intervention was planned to be personalised, titrated or adapted, then describe what, why, when, and how

Consistent with the person-centred approach to education, the training course paid attention to developing trust, participation, acceptance, excitement, and curiosity, partly by having check in, group and process time at the beginning and end of each day, which encouraged personal sharing and processing; and partly by being flexible about the order of the content of the course. #27

Modifications No examples were identified of more detailed reports on modifications to the enabling or support activities during the course of the trial. This item of the TIDieR checklist prompts authors to describe if the intervention was modified during the course of the study, describe the changes (what, why, when, and how)

Ensuring that training was acceptable at step 3 required considerable compromise in restricting the length and content of training to match the time that practices were willing to devote. #17

Measure of planned support activities

No examples were identified of more detailed reports on planned measures of enabling or support activities provided. This item of the TIDieR checklist prompts authors to describe if adherence or fidelity was assessed, and if so to describe how and by whom, and if any strategies were used to maintain or improve fidelity

Evaluations of teaching sessions: facilitators evaluate the extent of the planned content of each teaching session by completing a standardized evaluation form. This ensures the consistency of the program delivery at each site. #4

Actual support activities

No examples were identified of more detailed reports on actual enabling or support activities delivered. This item of

All implementation strategies were provided to almost all new staff members except for new caregivers, who

Page 41 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 5 -

the TIDieR checklist prompts authors to describe the extent to which the intervention (i.e. the enabling or support activities in this case) was delivered as planned

mostly did not follow a new depression course but were supervised by the unit managers. #22

Competency checking

The readiness of GPs to manage intimate partner abuse is assessed before and after the training using PREMIS, a validated questionnaire assessing knowledge, attitude and behaviours of doctors with regard to intimate partner violence. #13

Evaluation of the translation of knowledge and skills into practice was built into the course, in the form of a MAWE reflective diary (used to facilitate learning, but also as research data). / Before recruiting trial participants, nurses were required to give the researcher a verbal or written example of practicing each component of the intervention. They then made a self-assessment of confidence and competence to practice MAWE before recruiting into the trial. This took a period of time ranging from one to four months per nurse. #15

Participating pharmacists underwent training and subsequent assessment of knowledge and skills. Pharmacists were required to demonstrate a defined standard of competence. #39


Item More detailed reports More typical reports

Strategy used to monitor fidelity

Two research nurses in each ICU were centrally trained to do hand hygiene observations. 15 observation sessions, of 15–30 min, were done every month throughout the study at the ICUs. / We checked 10% of all electronic case report forms against data collected at each ICU, and checked all procedures for consistency once during phase 1 and once during phase. #7

Intervention group interviews were recorded randomly in order to ensure the quality of the sessions and the compliance with techniques learned during the training. #4

Page 42 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 6 -

Extent to which the intervention was delivered as planned

Sixty-nine percent of the dyads were exposed to the intervention. They received an average of 3.09 (SD: 2.97) counselling sessions, including the assessment procedure. Fifty-six percent of these dyads received an average of 19.22 (SD: 16.65) supportive actions after the screening. / The professionals did not systematically screen the caregivers. Thirty-one percent of the dyads in the intervention group had not received a systematic assessment. However, adjusting for the extent to which professionals carried out the intervention as planned did not yield different effects on institutionalization. #38

However, the correct use of the checklist was promoted and monitored during training. According to these reports, most of the patients were screened for substance use and exposed to at least 2 of the 5 elements of the brief intervention. #12

Page 43 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 1 -

PRISMA Checklist

Section/topic # Checklist item Reported on page #

TITLE

Title 1 Identify the report as a systematic review, meta-analysis, or both.

- Identified as a review in the title and descriptive review in the abstract 1

ABSTRACT

Structured summary 2

Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.

- We did not have a systematic review registration number due to the nature of the review

2

INTRODUCTION

Rationale 3 Describe the rationale for the review in the context of what is already known. 4-6

Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS). 4-6

METHODS

Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if

available, provide registration information including registration number. No

Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. 7

Page 44 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 2 -

Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched. 7

Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated. S1

Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis). 7

Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate)

and any processes for obtaining and confirming data from investigators. 8

Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made. 8

Risk of bias in individual studies 12

Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

Not included

Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). N/A

Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency (e.g., I2) for each meta-analysis. 9

Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication

bias, selective reporting within studies). Not included

Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating which were pre-specified. N/A

RESULTS

Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram. 9

Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations. S4

Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item

12). Not included

Page 45 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from



- 3 -

Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data

for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot. N/A

Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. N/A

Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). N/A

Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). N/A

DISCUSSION

Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider

their relevance to key groups (e.g., healthcare providers, users, and policy makers). 12-15

Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias). 14-15

Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. 15-17

FUNDING

Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review. 19

Page 46 of 46


BMJ Open

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

on Septem


http://bmjopen.bm

j.com/

BM



ownloaded from


BMJ OpenPR1 2HE, UK Corresponding author: ... Naoimh McMahon, Clinical Practice Research Unit,...

Documents

Transcript of BMJ OpenPR1 2HE, UK Corresponding author: ... Naoimh McMahon, Clinical Practice Research Unit,...