Provocations Roundtable, EHPS, Bordeaux, 2013

Most Findings in Health Psychology

Are Not Believable

James C. CoyneUniversity of Groningen, University Medical Center Groningen (UMCG), The Netherlands@CoyneoftheRealm

Psychology and psychiatry articles 5 times more likely to report positive findings than articles in physics and other hard sciences.

Can we trust what we read in journals?

Ioannidis, J. P. (2005). Why most published research findings are false. PLoS medicine, 2(8), e124.

Shun-Shin, M. J., & Francis, D. P. (2013). Why Even More Clinical Research Studies May Be False: Effect of Asymmetrical Handling of Clinically Unexpected Values. PLOS ONE, 8(6), e65323.

Copyright ©2005 American Heart Association

Modified from Loscalzo, J. Circulation 2005;112:3026-3029

Incentives and Pressures For Incentives and Pressures For Confirmatory BiasConfirmatory Bias

Ioannidis, J. (2012). Scientific inbreeding and same-team replication: Type D personality as an example. Journal of Psychosomatic Research.

Obedient replication: Investigators feel that the prevailing views are so dominant that finding consistent results is a sign of being a good scientist and there is no room for dissenting results and objections.

Ioannidis, J. (2012). Scientific inbreeding and same-team replication: Type D personality as an example. Journal of Psychosomatic Research.

Obliged replication: Proponents of dominant view are so strong in controlling the publication venues that they can largely select and mold the results, wording, and interpretation of studies eventually published.

Telling It Like It Ain’t: How to Telling It Like It Ain’t: How to Succeed in Health PsychologySucceed in Health Psychology

Have Lots of Endpoints and Ignore Negative Results in Main Analyses of Primary Endpoints.

Favor Secondary Analyses, Subgroup Analyses, and Endpoints Developed Post Hoc over Negative Findings for Primary Analyses.

Ignore Methodological Shortcomings that Would Make Trial or Meta Analyses Invalid.

Telling It Like It Ain’t: How to Telling It Like It Ain’t: How to Succeed in Health PsychologySucceed in Health Psychology

• Present Negative Findings as if Positive in Subsequent Publications, Exaggerate Positive Findings.

Assess Multiple Endpoints, Treat any Significant Finding as a Replication of Past Findings.

Create a False Consensus and Seeming Unanimity in the Literature by Cherrypicking Findings that can be Construed as Positive, Ignoring the Rest.

Do we modify Sackett’s definition of evidence-based medicine as "the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients”

To

Evidence-based health psychology is ‘the careful selection, analysis, and interpretation of results to put theories and interventions in the best possible light, upholding the credibility of the field.’

Change what is viewed as “for the good of the field”

Need to stop viewing producing evidence upholding dominant theories and efficacy of interventions as overarching goal of publication.

Need to recognize null findings, nonreplication, and criticism of dominant views as accomplishments in need of protecting and publishing.