Journal of Clinical Epidemiology 67 (2014) 296e304

Paper-based and web-based intervention modeling experimentsidentified the same predictors of general practitioners’

antibiotic-prescribing behavior

Shaun Treweeka,*, Debbie Bonettib, Graeme MacLennana, Karen Barnettc, Martin P. Ecclesd,Claire Jonese, Nigel B. Pittsf, Ian W. Rickettsg, Frank Sullivanh, Mark Weali, Jill J. Francisj

aHealth Services Research Unit, University of Aberdeen, Health Sciences Building, Foresterhill, Aberdeen AB25 2ZD, UKbDental Health Services Research Unit, University of Dundee, Kirsty Semple Way, Dundee DD2 4BF, UK

cCentre for Population Health Sciences, University of Edinburgh, Medical Quad, Teviot Place, Edinburgh EH8 9AG, UKdInstitute of Health & Society, Newcastle University, Baddiley Clark Building, Richardson Road, Newcastle Upon Tyne NE2 4AX, UK

eHealth Informatics Centre, University of Dundee, Kirsty Semple Way, Dundee DD2 4BF, UKfDental Institute, Kings College London, Strand, London WC2R 2LS, UK

gSchool of Computing, University of Dundee, Queen Mother Building, Dundee DD1 4HN, UKhQuality, Safety & Informatics Research Group, University of Dundee, Kirsty Semple Way, Dundee DD2 4BF, UKiSchool of Electronics and Computer Science, University of Southampton, Highfield, Southampton SO17 1BJ, UK

jSchool of Health Sciences, City University London, Northampton Square, London EC1V 0HB, UK

Accepted 24 September 2013; Published online 31 December 2013

Abstract

Objectives: To evaluate the robustness of the intervention modeling experiment (IME) methodology as a way of developing and testingbehavioral change interventions before a full-scale trial by replicating an earlier paper-based IME.

Study Design and Setting: Web-based questionnaire and clinical scenario study. General practitioners across Scotland were invited tocomplete the questionnaire and scenarios, which were then used to identify predictors of antibiotic-prescribing behavior. These predictorswere compared with the predictors identified in an earlier paper-based IME and used to develop a new intervention.

Results: Two hundred seventy general practitioners completed the questionnaires and scenarios. The constructs that predicted simulatedbehavior and intention were attitude, perceived behavioral control, risk perception/anticipated consequences, and self-efficacy, which matchthe targets identified in the earlier paper-based IME. The choice of persuasive communication as an intervention in the earlier IME was alsoconfirmed. Additionally, a new intervention, an action plan, was developed.

Conclusion: A web-based IME replicated the findings of an earlier paper-based IME, which provides confidence in the IME method-ology. The interventions will now be evaluated in the next stage of the IME, a web-based randomized controlled trial. � 2014 Elsevier Inc.All rights reserved.

Keywords: Intervention modeling experiments; Behavior change; Randomized controlled trials; Intervention development; Prescribing; Primary care

1. Introduction

Improving health care is not only about developing newtreatments and therapies but also requires that existingknowledge of effective interventions be put into clinical

Conflict of interest: The authors declare no conflicts of interest.

Funding: WIME was funded by the Chief Scientist Office, grant num-

ber CZH/4/610. The Health Services Research Unit, University of Aber-

deen, is core funded by the Chief Scientist Office of the Scottish

Government Health Directorates.

* Corresponding author. Tel.: þ44-777-901-6955; fax: þ44 1224

438165.

E-mail address: streweek@mac.com (S. Treweek).

0895-4356/$ - see front matter � 2014 Elsevier Inc. All rights reserved.

http://dx.doi.org/10.1016/j.jclinepi.2013.09.015

practice. This can be challenging. Without active imple-mentation, there is a danger that potentially useful researchevidence will languish in obscurity (the ‘‘bench to book-shelf’’ phenomenon) or will diffuse only very slowly intopractice [1]. Although some interventions have been shownto be effective in changing the behavior of health profes-sionals [1e4], the literature provides little information toguide the choice, or to optimize the components, of theseinterventions for use in different contexts [5,6]. Interven-tions can be effective (eg, reminder systems, audits), butthe evidence is conflicting and the reason for this is largelyunknown [2]. However, many interventions are developedwithout an explicit theoretical rationale for why and how

297S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

What is new?

Key findings� A web-based intervention modeling experiment

(IME) replicated the findings of an earlier paper-based IME on general practitioners’ antibiotic-prescribing behavior.

� The constructs that predicted both simulatedbehavior and intention were attitude, perceivedbehavioral control, risk perception/anticipated con-sequences, and self-efficacy, which matched thoseidentified in the earlier paper-based IME.

What this adds to what was known?� The IME methodology has been used for paper-

based experiments, but there had been no replica-tion studies to test the methodology itself. Thisstudy replicated an earlier paper-based IME andwas expected to identify the same predictors ofbehavior, which it did.

What is the implication and what should changenow?� The IME methodology is a robust choice for

exploratory work developing and evaluating com-plex behavior change interventions before evalu-ating them in a full-scale trial.

the intervention might be expected to have an effect, whichmay help to explain why the effectiveness of behaviorchange interventions can appear somewhat hit and miss.To address this, the UK Medical Research Council frame-work for developing and evaluating complex interventionshas argued for more and better theoretical and exploratorywork before a full-scale trial as a means of improving inter-vention development [7].

One way of carrying out this exploratory work is touse an intervention modeling experiment (IME) [8]. In anIME, key elements of the intervention are delivered (usinga randomized design) in a manner that approximates thereal world but where the measured outcome is generallyan interim outcome, a proxy for the clinical behavior ofinterest. To date, IMEs have been conducted using paper-based materials [8e10], but this may limit their efficiency,acceptability, and ecological validity. Web-based IMEs(WIMEs) have the potential to provide much richer simula-tions of clinical encounters (eg, through presentation ofvideo clips of patientephysician consultations) and alloweasy measurement of key process variables such as timetaken to make a decision.

To evaluate the robustness of the IME methodology, weconducted a web-based IME study [11] that replicated an

earlier paper-based IME, which evaluated theory-basedinterventions to reduce antibiotic prescribing for upperrespiratory tract infections (URTIs) in primary care[9,10]. We will refer to the earlier study [9] as ‘‘thepaper-based IME’’ throughout this article; we will callthe web-based study ‘‘WIME.’’ This article describes theprocess that we used to identify predictors of prescribingbehavior in the WIME, a comparison of these with thepredictors identified in the paper-based IME [9], andhow we used predictors from WIME to develop a newintervention.

2. Specifying the target behavior and selectinga theoretical framework

The IME methodology has been described elsewhere[9e11]. Briefly, there are three stages. The first stage usu-ally involves qualitative work to provide information on therange of perceptions and beliefs among future participants[eg, general practitioners (GPs)] about the behavior ofinterest (eg, managing patients with URTI without usingantibiotics). These beliefs are used in the second stage todevelop theory-based questionnaire items relevant to thebehavior, together with clinical scenarios that can be usedto simulate situations in which the target behavior may beperformed. The responses of individuals to the question-naire and scenarios are used to identify predictors of thebehavior of interest, and an intervention that targets theseis developed, based on the identified theories and theirevidence base. The final stage of the IME is to evaluatethe new intervention in a randomized trial, again usinga questionnaire and clinical scenarios. This article describesstage 2 of an IME, identifying predictors of GPs’ antibiotic-prescribing behavior and developing an intervention. Stage1 was done in the earlier work [9], and stage 3 will be thefocus of a future publication.

As we were seeking to replicate, as far as possible, thepaper-based IME, we were interested in the same targetbehavior as that used by Hrisos et al. [9]: ‘‘managingpatients presenting with uncomplicated URTI without pre-scribing an antibiotic.’’ The authors identified three the-ories that included factors predictive of GPs’ prescribingbehavior for URTI: theory of planned behavior (TPB)[12], social cognitive theory (SCT) [13,14], and operantlearning theory [15]. The TPB [12] proposes that peopleare more likely to perform a behavior (eg, eat a healthy dietor follow a guideline recommendation) if they feel moti-vated (intend) to do so, if they believe that performing thatbehavior will result in a valued consequence (have a posi-tive attitude), if they believe that other people think thatthey should do the behavior (high subjective norm), andif they believe they can overcome any significant barriersthat may prevent them from performing the behavior (highperceived control). SCT [13,14] proposes that people aremore likely to perform or change their behavior if they

298 S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

are confident that they can (high self-efficacy) and if theiroutcome expectancies (beliefs about the consequences ofperforming a behavior) support them in doing so (eg, if Istop smoking, I will feel better). Operant learning theory[15] proposes that behaviors with positive consequencesfor the individual are likely to be repeated, and the moreoften a behavior is performed in the same context, the morelikely it is to be performed again in the same context, thatis, it becomes habitual. The principle that positive conse-quences promote repetition of behavior is well establishedand has been widely and successfully used to understandbehavior and behavior change [15].

Together, these theories explain behavior in terms of fac-tors that influence motivation (through ‘‘preintentional pro-cesses’’) and action (through ‘‘postintentional processes’’)[16]. They also explain behavior in terms of factors thatare amenable to change (eg, attitude, self-efficacy); theyinclude nonvolitional components (eg, external constraints)that acknowledge that individuals do not always have com-plete control over their actions. A summary of the theoriesand constructs is given in Table 1, together with the WIMEquestionnaire items linked to them. The full questionnaireis available in the Appendix at www.jclinepi.com, togetherwith a list of questionnaire items used to operationalizeeach construct.

The paper-based IME included two interventions: apersuasive communication and a graded task [9]. Thepersuasive communication addressed beliefs about the con-sequences (eg, including ‘‘attitude’’ from the TPB and‘‘outcome expectancies’’ from SCT) of managing patientswith uncomplicated URTI without prescribing antibiotics.It was effective in reducing the number of antibiotic pre-scriptions in the paper-based IME’s prescribing scenarios.The paper-based IME’s second intervention, a graded task,did not influence prescribing decisions or the proposedmediating construct, self-efficacy [10]. A part of the presentwork was therefore to develop a new intervention using thepredictors identified in the WIME to replace the gradedtask.

Before doing the work described in this article, we hadtwo predictions for how the WIME results would comparewith those of the paper-based IME:

1. The WIME would identify the same predictors ofGPs’ behavior regarding prescribing of antibioticsfor URTIs as the paper-based IME.

2. The WIME would again suggest a persuasivecommunication as an intervention to change GPs’prescribing behavior.

If these predictions were supported, then we would havegreater confidence that the IME methodology is a robustchoice for exploratory work developing and evaluatingcomplex behavior change interventions before a full-scaletrial.

3. Methods

The development of theory-based interventions tochange behavior involves at least four steps [9,17,18]:

1. Specifying the target behavior.2. Selecting a theoretical framework to guide an empir-

ical investigation.3. Conducting a predictive study to identify the key

theoretical constructs (eg, attitude, self-efficacy) thatpredict the target behavior.

4. Using a ‘‘mapping’’ process to select interventioncomponents that are proposed to change the predict-ing constructs.

As we were replicating earlier work, steps 1 and 2 hadalready been done, and we focused on steps 3 and 4, whichin the context of our WIME study meant

3. Conducting a predictive study with GPs to identifyconstructs that predict the targeted behavior and arepotentially modifiable.

4. Mapping the selected constructs onto behaviorchange techniques that are known to (or likely to)change the predicting constructs and are feasible tooperationalize in a primary care context.

The earlier interviewwork [9] provided information on therange of perceptions and beliefs among GPs about managingpatients with URTI without using antibiotics. These beliefswere used to develop questionnaire items that were relevantto the behavior and operationalized the constructs of our cho-sen theories (eg, TPB: the questionnaire had an item askingGPswhether they thought patients expected them to prescribean antibiotic, which is linked to the theory’s ‘‘subjectivenorm’’ construct). Constructs such as attitude and perceivedbehavioral control were measured ‘‘indirectly’’ by askingGPs about their specific beliefs (eg, prescribing antibioticsmay reassure the patient or may increase the likelihood ofantibiotic resistance in the community) and ‘‘directly’’ byasking GPs to report their beliefs at a more general level(eg, Do the benefits of managing patients with URTIs withoutprescribing antibiotics outweigh the harms?).

These constructs were used to predict two outcomes:

1. Behavioral intentiondstrength of motivation orintention to perform the target behavior.

2. Behavioral simulationdclinical decisions in thecontext of simulated clinical situations that were pre-sented as a set of eight clinical scenarios.

In addition to the scenarios, the questionnaire asked 20questions about prescribing behavior and finished with 4general questions about the GP’s background. We tookthese materials and put them into a web-based delivery sys-tem. We did not modify the content of the materials,although we did make minor changes to formatting to better

Table 1. Summary of the internal consistency of constructs (a) and correlations with behavioral simulation and intention

Theoretical constructNumberof items a

Correlation withintention

Correlation withbehavioral simulation

Agrees withpaper IME?a Questionnaire items

Theory of planned behavior (TPB)Attitude (direct measure) 3 0.63 0.48*** 0.25*** Y: bothb 12Attitude (indirect measure; TPB measured this

using the same questionnaire items as foroutcome expectancies in SCT)

8 0.52 0.31*** 0.12 Y: intention 3 and 13

Intention 3 0.89 d 0.34*** Y 8, 16, 17, and 18Perceived behavioral control (direct measure) 3 0.65 0.25*** 0.05 Y: intention 11Perceived behavioral control (indirect measure) 6 0.86 0.40*** 0.31*** Y: both 9 and 10Subjective norm 5 0.63 0.09 �0.06 Y: both 2 and 14

Social cognitive theory (SCT)Risk perception (OLT measured this using the

same items as for anticipated consequences)3 0.64 0.39*** 0.24*** Y: both 4 and 5

Outcome expectancy: procedure (SCTmeasured this using behavioral beliefsitems from attitude indirect in TPB)

2 �0.05c �0.15* �0.05 d 3 and 13

Outcome expectancy: patient (SCT measuredthis using behavioral beliefs items fromattitude indirect in TPB)

5 0.54 0.18** 0.19** d 3 and 13

Outcome expectancy: clinician (SCT measuredthis using behavioral beliefs items fromattitude indirect in TPB)

2 0.33c 0.29*** 0.08 d 3 and 13

Self-efficacy 6 0.86 0.37*** 0.22** Y: both 6 and 15Operant learning theory (OLT)Anticipated consequences (TPB measured this

using the same questionnaire items as for riskperception)

3 0.64 0.39*** 0.24*** Y: both 4 and 5

Evidence of habite 3 0.832 0.78*** 0.37*** Y: both 7

Abbreviations: IME, intervention modeling experiment; Y, yes; SCT, social cognitive theory.This table corresponds to Table 3 in the original paper-based IME [9].*P � 0.05; **P � 0.01; ***P � 0.001.a We did not expect the delivery method (paper or web) to affect the correlation between constructs and behavioral simulation and intention.

‘‘Yes: both’’ means that the statistical significance of the correlation measured in the web-based IME agreed with that seen in the paper-based IMEfor both behavioral simulation and intention. ‘‘Y: intention’’ means that the statistical significance of the correlation measured in the web andpaper-based IMEs agreed for only intention.

b Predictors marked in bold were selected as targets for a new intervention.c These are correlation coefficients because of there being fewer than three items in the scale.d Outcome expectancy was split into two parts in the paper-based IME but had poor internal consistency. We attempted to address this in the

WIME study but stay as close to the original study as possible by simply adding one more dimension to it (ie, splitting it three ways rather than two)in the hope that this would overcome the internal reliability and consistency issues of this measure. However, this alternative approach did notgenerate factors with desirable psychometric properties. Thus, similar to the paper-based IME, outcome expectancy was not flagged as a potentiallysuitable candidate construct to develop an intervention. A direct comparison between paper and web-based IMEs for these items is thereforeinappropriate.

e Although habitual behavior was highly correlated with behavioral simulation and intention, it was not selected as intervention targets. Habit isnot a causal determinant but an attribute of behavior and is modified indirectly by targeting other causal aspects of behavior.

299S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

fit electronic rather than paper delivery. A PDF file contain-ing a full set of screenshots of the web-delivered question-naire is available in the Appendix at www.jclinepi.com.

3.1. Sample size

The sample size strategy was driven by the subsequentintervention study for which we needed to recruit250 GPs. For the intervention development and predictivestudy, we assumed that we would need an adequate samplesize for a multiple linear regression with at least 14 predic-tor variables. We allowed for 14 predictor variables to facil-itate incorporating additional variables, slightly more thanthe 11 included predictor variables in the study by Hrisoset al. [9] because of the potential for empirical evidence

of multiple latent constructs within our proposed theoreticalconstructs. The rule of thumb from the study by Green [19]recommends that the minimum sample size required is50 þ 8� the number of predictor variables, which was162 (50 þ 112) in this case. As such, the recruitment targetof the subsequent intervention study meant the sample sizewas more than adequate for the predictive modeling aspectbeing reported here.

3.2. Recruitment

GPs from 12 Scottish health boards were identified bythe Scottish Primary Care Research Network (SPCRN;www.sspc.ac.uk/spcrn/) using a combination of publiclyavailable information provided by Information Services

300 S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

Division Scotland (http://www.isdscotland.org/isd/3793.html) and restricted information held on the NHS.net data-base, the latter to provide e-mail addresses. It was not clearwhether GPs would be more likely to respond to a postal oran e-mail invitation, so we embedded a methodologicalstudy of how best to contact GPs by randomly allocatingGPs to either a postal or an e-mail invitation. The resultsof that study are published elsewhere, [20] but in summary,e-mails were as effective as postal invitations and werequicker and cheaper to send.

The study statistician (G.M.) generated a list of randomnumbers, and participant identifications were broken downinto mailing blocks, which SPCRN used to randomly allo-cate GPs (on a 1:1 basis without stratification) to receiveeither an e-mail or a postal invitation. Blocks of invitationswere sent out until the number of GPs recruited met or ex-ceeded the required sample size of 250 GPs. All researchstaff, except SPCRN staff, were blinded to GP recruitmentallocation until the study database was locked.

The invitations contained a one-page letter and a two-page information sheet or a link to the information sheetin e-mail invitations. Together with general information,the letter contained a link to the WIME questionnaire.We sent two reminders to nonresponders at 2 and 4 weeks,using the same contact method as used for the initial invi-tation. Finally, GPs were offered a £20 gift voucher forcompleting the questionnaire.

3.3. Statistical analysis for the predictor study

Categorical data are described using numbers and per-centages, continuous data using mean (standard deviation)and/or median (interquartile range), as appropriate. Internalconsistency was assessed using Cronbach a. Correlation be-tween constructs and the two outcomes (behavioral simula-tion and intention) was assessed using Pearson correlationcoefficient. Analysis was carried out using Stata 12 (Stata-Corp LP, StataCorp. 2009, Stata Statistical Software: release11, College Station, TX, USA) and PASW Statistics 18(SPSS, Hong Kong).

We used least angle regression to reduce the set of con-structs to a subset that explained the maximum variation inbehavioral simulation scores. We then took the construct thathad the strongest relationship with behavioral simulationand broke it down to the item level to explore which itemor items had the greatest influence on behavioral simulation.These would then be targeted by the new intervention.

4. Intervention development: using the theoreticaldomain framework to identify behavior changestrategies

We used the methods proposed by Michie et al. [21] tomap constructs onto behavior change techniques. First, weclassified the constructs identified as predictors of behavior

into ‘‘theoretical construct domains’’ [22,23]. Theoreticaldomains (eg, beliefs about consequences, knowledge, socialinfluence) are clusters of similar constructs, and constructswithin a domain are likely to be modifiable using the samebehavior change techniques. Behavior change techniquesproposed to affect each domain have been identified froman expert consensus process and presented as a tool forintervention developers [21]. We used the tool to selectbehavior change techniques that could change the con-structs identified as predictors of our target behavior ofnot prescribing an antibiotic. These behavior techniquesthen became the basis of intervention components. Thiswas expected to lead to one or more potential interventionsfor evaluation.

We then operationalized the selected behavior changetechniques in a form that could be delivered using web-based materials. The clusters of techniques were thusplanned to be delivered together as a complex (ie, multifac-eted) intervention.

5. Results

Between January 27, 2011 and May 15, 2011, 293 GPslogged onto the WIME system, of which 270 completed theWIME materials for the predictor study. Further details onrecruitment are published elsewhere [20].

The constructs that predicted both simulated behaviorand intention were attitude, perceived behavioral control,risk perception/anticipated consequences, and self-efficacy(Table 1). These targets match those identified in the earlierpaper-based IME for simulated behavior and intention. Todetermine the appropriate behavior change strategy, theseconstructs were mapped onto the theoretical domain frame-work (TDF). The results are presented in Table 2, alongwith their associated behavior change strategy (selected us-ing a published mapping tool [21]), chosen by expertconsensus (D.B. and J.J.F.) as likely to be feasible to oper-ationalize within the constraints of an IME.

The results suggested that the theoretical domains totarget in the new intervention were beliefs about conse-quences, beliefs about capabilities, and behavioral regula-tion. A behavior change technique known to influence thelast 2 of these 3 domains is action planning. An action planis an explicit statement of where, when, and how a behaviorwill be performed. Action plans are proposed to work bysetting up environmental cues to remind an individual toperform the behavior [24]. Furthermore, repeated perfor-mance of a behavior in response to the cue increases thelikelihood that a behavior may become a ‘‘good’’ habit.

In addition to action planning, we included the followingbehavior change techniques (as defined in a recent taxon-omy [25,26]):

1. Goal setting (behavior; targeting behavioral regu-lation)

2. Prompts/cues (targeting behavioral regulation)

Table 2. Behavior change techniques by theoretical domains identified as intervention targets

Theoretical domain targeted Behavior change technique and definition How it was delivered in the WIME

Behavioral regulation Action planningPrompt detailed planning of performance of thebehavior (included at least one of context,frequency, duration, and intensity).

GPs were asked to make an action plan followinga template, which included context andfrequency.

Goal setting (behavior)Set or agree on a goal defined in terms of thebehavior to be achieved.

The action plan template asked GPs to set theirbehavioral goal to be ‘‘not prescribing anantibiotic.’’

Prompts/cuesIntroduce or define environmental or socialstimulus with the purpose of prompting orcueing the behavior.

The examples of action plans we provided weredesigned to make salient the most likelysituations for cueing ‘‘not prescribing anantibiotic.’’

Self-monitoring of behaviorEstablish a method for the person to monitor andrecord their behavior(s) as part of a behaviorchange strategy.

The action plan template included a sectionrequesting that GPs determine how they wouldmonitor the success of their plan.

Beliefs about capabilities (includesperceived behavioral control construct)

Demonstration of the behaviorProvide an observable sample of theperformance of the behavior, directly in personor indirectly, for example, via film, pictures,for the person to aspire to or imitate.

The action plan examples described situationsthat previous research had identified as mostlikely to result in inappropriate prescribingbehavior and included alternative behaviors toprescribe an antibiotic that they could doinstead.

Behavioral practice/rehearsalPrompt practice or rehearsal of the performanceof the behavior one or more times in a contextor at a time when the performance may not benecessary, to increase habit and skill.

Asking GPs to actually write out their action planenabled them to rehearse what they would doas an alternative to prescribe in the situationsthey would usually prescribe.

Beliefs about consequences (includesattitude construct)

Social rewardArrange verbal or nonverbal reward if and only ifthere has been effort and/or progress inperforming the behavior.

On downloading the action plan, the GPsreceived the following message with the actionplan template: ‘‘Great! By writing out your ownaction plan and following it through, you arelikely to follow more evidence-basedprescribing practice in the future.’’

Abbreviations: WIME, Web-based intervention modeling experiment; GP, general practitioner.

301S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

3. Feedback on behavior (targeting beliefs about capa-bilities and behavioral regulation)

4. Self-monitoring of behavior (targeting behavioralregulation)

The results suggested that this intervention should alsoinclude

5. Behavioral practice/rehearsal (targeting perceivedbehavioral control/beliefs about capabilities)

6. Modeling (targeting perceived behavioral control/be-liefs about capabilities)

7. Social reward (targeting attitude/beliefs aboutconsequences)

These behavior change techniques were operationalizedas follows: GPs were asked to devise an action plan for thetwo scenarios from the baseline questionnaire that gave thegreatest variation in their scores:

1. A patient with an URTI specifically asks for an antibi-otic or clearly expects to be prescribed an antibiotic.

2. A patient with an URTI (or the patient’s parent(s) ifthe patient is a child) is very distressed about thesymptoms.

Some examples of action plans were given. These exam-ples were not just a template for an action plan but alsodescribed some behaviors that could be alternatives to anti-biotic prescribing in these situations (see the action plan inthe Appendix at www.jclinepi.com). The presentation ofthe intervention was designed as a manageable one screenper page of A4 length.

6. Discussion

We aimed to test the IME methodology and develop anew intervention to influence GPs’ antibiotic-prescribingbehavior for URTIs [11]. Our key research interest with re-gard to IME methodology is whether the delivery mecha-nism of the IME (paper or web) affects predictors of GPbehavior. This is important information because for theIME methodology to be useful, it needs to be a robustand reliable method to support trialists with their interven-tion modeling work. This is indeed what we found: our re-sults involving 270 GPs from around Scotland showed thatthe WIME identified 8 of 10 predictors of prescribingbehavior identified in the paper-based IME. For the

302 S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

remaining two constructs, attitude (indirect) and perceivedbehavioral control (direct), disagreement was not completebecause they did agree for prediction of intention. Why thepaper-based IME and WIME should differ for the perceivedbehavioral control (direct) construct is unclear, although itis not too difficult to imagine that this reflects some real dif-ference in GPs from different parts of the United Kingdomtaking part in studies 5 years apart. On the other hand, it ispossible that scores for perceived behavioral control wouldbe related to generalized self-efficacy and that GPs withhigher self-efficacy may have been more inclined to takepart in a WIME than a paper-based IME. However, if thiswas the case, it would in a sense strengthen the main find-ings as two samples with different self-efficacy nonethelessshow very similar patterns of prediction overall.

One element of the original paper-based IME could notbe compared in the WIME study, which was outcome ex-pectancies. In the paper-based IME, outcome expectancieswere addressed using factor analysis to explore potentialempirical factor loadings within the items. This resultedin two factors being derived, neither of which was consid-ered a suitable candidate construct to develop an interven-tion and also had problems with internal consistency. In theWIME study, when we were faced with the same problem,we considered three options: (1) replicating the paper-basedIME factor analysis approach, (2) using the factors identi-fied in the paper-based IME report as constructs, or (3)grouping the items together using a theoretical rationaleto create theory-based subconstructs of outcome expec-tancy. We attempted to stay as close to the original paper-based study as possible, simply adding one more dimensionto it (ie, splitting it three ways rather than two) in the hopethat this would overcome the internal reliability and consis-tency issues of this measure. This alternative approach didnot generate factors with desirable psychometric properties.Thus, similar to the paper-based IME, outcome expectancywas not flagged as a potentially suitable candidate constructto develop an intervention.

The WIME study also replicated the paper-based IME’sfinding that a persuasive communication would be a sensi-ble choice of intervention. By identifying beliefs about con-sequences (in the form of attitude and risk perception) as adomain that was key to prediction of intention, persuasivecommunication was once again identified as a potentiallyeffective technique for change. The design of the new inter-vention, the action plan, involving postintentional pro-cesses comes straight from the predictor data (seeTables 1 and 2) and a rigorously developed tool for map-ping predictors of behavior to behavioral change techniquesknown to affect those predictors [21]. This is importantbecause it provides a rationale with which to explain‘‘why and how’’ we expect the intervention to have aneffect and will help us to interpret the results of the nextstage of the WIME project, a trial of the persuasivecommunication and action plan against a ‘‘no intervention’’comparator.

The study had two limitations. The first is inherent in theIME methodology: we used vignettes to provide clinicalscenarios, and how well these approximate real clinical sit-uations is open to debate. Vignettes are widely used in med-ical research [27e30], and although strong evidence of theexternal validity of vignettes is rather sparse, studies thathave looked at this have been favorable toward the use ofvignettes. Peabody et al. [31], for example, compared vi-gnettes with standardized patients and found vignettes tobe a valid tool for measuring the quality of clinical practice.WIMEs offer the possibility of much richer vignettes usingvideo and audio, together with greater opportunities forinteraction, which might provide better simulations of realclinical situations. We did not explore this in the presentwork (the second limitation of the study) because we werereplicating a previous paper-based IME and did not want tointroduce differences beyond web-based delivery. We do,however, think richer web-based vignettes are worth evalu-ating in future studies.

How might WIMEs support future studies? A Cochranesystematic review of interventions to improve antibiotic-prescribing practices in primary and community careidentified 39 studies of interventions including printededucational materials for physicians, audit and feedback,educational meetings, educational outreach visits, financialand health-care system changes, physician reminders,patient-based interventions, and multi-faceted interventions[32]. No mention was made of an explicit rationale beingbehind any of these interventions, although it is possiblethat some did have one. It is more likely that all or mostof the interventions seemed like sensible ideas with someempirical support from earlier work and so they were putinto a trial. The review authors concluded ‘‘No single inter-vention can be recommended for all behaviours in anysetting. Multi-faceted interventions . may be successfullyapplied to communities after addressing local barriers tochange. Future research should focus on which elementsof these interventions are the most effective.’’ The IMEmethodology is well suited to identify local barriers tochanging prescribing behavior and, moreover, provides aclear rationale for intervention components. We believeWIMEs, alone or in combination with qualitative methodsof identifying barriers and facilitators such as focus groupsand stakeholder interviews, have a valuable role to play inthe more informed development of complex behaviorchange interventions.

7. Conclusion

We have replicated, in a web-based system, an IMEdelivered initially on paper, and we found high levels ofagreement between the two predictive studies in terms ofthe predictors of behavior they identified. This gives usconfidence in the IME methodology. A WIME systemopens up more possibilities for choice of behavior change

303S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

techniques (eg, we were able to use social reward inresponse to an action, which is not possible with paper-based systems) and how intervention components are puttogether. We propose that the WIME system will reducethe overall cost of trials by requiring fewer iterations offull-scale trial/analysis/revision before an optimized inter-vention is produced, thereby making better use of limitedresearch budgets for trials. Finally, a web-based systemmeans individuals from further afield, and/or more of them,can be involved because the cost of doing so is minimal, ex-panding the range of opinions available for interventiondesign and development and enhancing both the generaliz-ability of IMEs and the implementation potential of qualityimprovement initiatives.

Acknowledgments

The authors thank the GPs who took part in this study.The authors also thank Marie Pitkethly and Gail Morrisonfor their help and support in recruiting GPs to the study.The work described here was funded by the Chief ScientistOffice, grant number CZH/4/610.

Ethical approval: WIME was approved by the TaysideCommittee on Medical Research Ethics A, Research EthicsCommittee reference 10/S1401/54 and received NHSResearch & Development approval from the 12 NationalHealth Service (NHS) Health Boards involved.

The trial of which this study is part is registered:ClinicalTrials.gov number NCT01206738.

Authors’ contributions: All authors contributed to thedesign of the study. K.B. and G.M. did the analysis of predic-tor data. K.B., G.M., D.B., J.J.F., and S.T. discussed the re-sults and D.B. and J.J.F. designed the new intervention. Allauthors contributed to the discussion of intervention develop-ment. S.T. was the chief investigator of the study and wrotethe first draft of the article. All authors contributed to the finalversion. All authors have approved the final manuscript.

Appendix

Supplementary data

Supplementary data associated with this article can befound, in the online version, at http://dx.doi.org/10.1016/j.jclinepi.2013.09.015.

References

[1] Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge

translation of research findings. Implement Sci 2012;7:1.

[2] Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L,

et al. Changing provider behavior: an overview of systematic reviews

of interventions. Med Care 2001;39(Suppl 2):II2e45.[3] Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J,

French SD, et al. Audit and feedback: effects on professional practice

and healthcare outcomes. Cochrane Database Syst Rev 2012;(6):

CD000259. http://dx.doi.org/10.1002/14651858.CD000259.pub3.

[4] O’Brien MA, Rogers S, Jamtvedt G, Oxman AD, Odgaard-Jensen J,

Kristoffersen DT, et al. Educational outreach visits: effects on pro-

fessional practice and health care outcomes. Cochrane Database

Syst Rev 2007;(4):CD000409. http://dx.doi.org/10.1002/14651858.

CD000409.pub2.

[5] Walker AE, Grimshaw J, Johnston M, Pitts N, Steen N, Eccles M.

PRIMEdPRocess modelling in ImpleMEntation research: selecting

a theoretical basis for interventions to change clinical practice.

BMC Health Serv Res 2003;3:22.

[6] Foy R, Eccles M, Jamtvedt G, Young J, Grimshaw J, Baker R. What

do we know about how to do audit and feedback? BMC Health Serv

Res 2005;5:50.

[7] Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M.

Developing and evaluating complex interventions: the new Medical

Research Council guidance. BMJ 2008;337:a1655.

[8] Bonetti D, Eccles M, Johnston M, Steen N, Grimshaw J, Baker R,

et al. Guiding the design and selection of interventions to influence

the implementation of evidence-based practice: an experimental

simulation of a complex intervention trial. Soc Sci Med 2005;60:

2135e47.[9] Hrisos S, Eccles M, Johnston M, Francis J, Kaner EF, Steen N, et al.

Developing the content of two behavioural interventions: using

theory-based interventions to promote GP management of upper res-

piratory tract infection without prescribing antibiotics #1. BMC

Health Serv Res 2008;8:11.

[10] Hrisos S, Eccles M, Johnston M, Francis J, Kaner EF, Steen N, et al.

An intervention modelling experiment to change GPs’ intentions to

implement evidence-based practice: using theory-based interventions

to promote GP management of upper respiratory tract infection

without prescribing antibiotics #2. BMC Health Serv Res 2008;8:10.

[11] Treweek S, Ricketts IW, Francis J, Eccles M, Bonetti D, Pitts NB,

et al. Developing and evaluating interventions to reduce inappropriate

prescribing by general practitioners of antibiotics for upper respira-

tory tract infections: a randomised controlled trial to compare

paper-based and web-based modelling experiments. Implement Sci

2011;6:16.

[12] Ajzen I. The theory of planned behaviour. Organ Behav Hum Decis

Process 1991;50:179e211.[13] Bandura A. Self-efficacy: the exercise of control. New York, NY:

Freeman; 1997.

[14] Bandura A. Health promotion from the perspective of social

cognitive theory. In: Norman P, Abraham C, Conner M, editors. Un-

derstanding and changing health behaviour: from health beliefs to

self-regulation. Amsterdam: Harwood Academic Publishers; 2000:

299e339.

[15] Skinner BF. Science and human behavior. New York, NY: Macmil-

lan; 1953.

[16] Wiedemann AU, Sch€uz B, Sniehotta F, Scholz U, Schwarzer R. Dis-

entangling the relation between intentions, planning, and behavior: a

moderated mediation analysis. Psychol Health 2009;24:67e79.

[17] Foy R, Francis JJ, Johnston M, Eccles M, Lecouturier J, Bamford C,

et al. The development of a theory-based intervention to promote

appropriate disclosure of a diagnosis of dementia. BMC Health Serv

Res 2007;7:207.

[18] French SD, Green SE, O’Connor D, McKenzie JE, Francis JJ,

Michie S, et al. Developing theory-informed interventions to imple-

ment evidence into practice: a systematic approach using the Theo-

retical Domains Framework. Implement Sci 2012;7:38.

[19] Green S. ‘‘How many subjects does it take to do a regression anal-

ysis?’’. Multivariate Behav Res 1991;26:499e510.[20] Treweek S, Barnett K, MacLennan G, Bonetti D, Eccles M,

Francis JJ, et al. E-mail invitations to general practitioners were as

effective as postal invitations and were more efficient. J Clin Epide-

miol 2012;657:793e7.[21] Michie S, Johnston M, Francis JJ, Hardeman W, Eccles MP. From

theory to intervention: mapping theoretically derived behavioural de-

terminants to behaviour change techniques. Special Issue on

304 S. Treweek et al. / Journal of Clinical Epidemiology 67 (2014) 296e304

‘‘Conceptualizing Theory-Based Health Behavior Change Research’’

of Applied Psychology: An International Review 2008; 57: 660e680.

[22] Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A,

‘Psychological Theory’ Group. Making psychological theory useful

for implementing evidence based practice: a consensus approach.

Qual Saf Health Care 2005;14:26e33.

[23] Francis JJ, O’Connor D, Curran J. Theories of behaviour change syn-

thesised into a set of theoretical groupings: introducing a thematic se-

ries on the theoretical domains framework. Implement Sci 2012;7:35.

[24] Sniehotta FF. Towards a theory of intentional behaviour change: plans,

planning, and self-regulation. Br J Health Psychol 2009;14:261e73.

[25] Michie S, Abraham C, Eccles MP, Francis JJ, Hardeman W,

Johnston M. Strengthening evaluation and implementation by speci-

fying components of behaviour change interventions: a study proto-

col. Implement Sci 2011;6:10.

[26] Michie S, Richardson M, Johnston M, Abraham C, Francis J,

Hardeman W, et al. The Behavior Change Technique Taxonomy

(v1) of 93 hierarchically-clustered techniques: building an interna-

tional consensus for the reporting of behavior change interventions.

Ann Behav Med 2013;46:81e95.

[27] Agoritsas T, Deom M, Perneger TV. Study design attributes influ-

enced patients’ willingness to participate in clinical research: a ran-

domized vignette-based study. J Clin Epidemiol 2011;64:107e15.

[28] Veloski J, Tai S, Evans AS, Nash DB. Clinical vignette-based sur-

veys: a tool for assessing physician practice variation. Am J Med

Qual 2005;20:151e7.

[29] Cleopas A, Villaveces A, Charvet A, Bovier PA, Kolly V,

Perneger TV. Patient assessments of a hypothetical medical error: ef-

fects of health outcome, disclosure, and staff responsiveness. Qual

Saf Health Care 2006;15:136e41.

[30] Ryan M, Farrar S. Using conjoint analysis to elicit preferences for

health care. BMJ 2000;320:1530e3.[31] Peabody JW, Luck J, Glassman P, Jain S, Hansen J, Spell M, et al.

Measuring the quality of physician practice by using clinical vi-

gnettes: a prospective validation study. Ann Intern Med 2004;141:

771e80.[32] Arnold SR, Straus SE. Interventions to improve antibiotic pres-

cribing practices in ambulatory care. Cochrane Database Syst Rev

2005;(4):CD003539. http://dx.doi.org/10.1002/14651858.CD003539.

pub2.