The Slight, the Fair and the Very Strong: the Meaning of Evidence in

Public Health from Bradford Hill to the

Campbell Commission

R.E.G. UpshurDepartment of Family and Community Medicine, Public Health Sciences and

Joint Centre for Bioethics, University of Toronto

Outline

Historical prelude: Fisher vs Hill Definitions of Evidence Characteristics of Evidence Models of Evidence Precaution and the Moral Epistemology of

Public Health Conclusions

Lord Kelvin

I often say that when you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be.

Nietzsche

Against that positivism which stops before the phenomena, saying “there are only facts” I should say: no, it is precisely facts that do not exist, only interpretations…

Gottlob Frege

It cannot be required that everything should be proved, since that is impossible; but one can see to it that all propositions that are being used without being proved, are expressly stated as such, so that it is clearly known on what the whole structure rests.

Ronald Aylmer Fisher

Fisher: Cigarettes, Cancer and Statistics, 1958

Before one interferes with the peace of mind and habits of others, it seems to me that the scientific evidence-the exact weight of the evidence free from emotion-should be rather carefully examined.

Requirements are: replication, randomization and control

Austin Bradford Hill

Hill: The Case for Action

…in “real life” we shall have to consider what flows from that decision. On scientific grounds we should do no such thing. The evidence is there to be judged on its merits and the judgment should be utterly independent of what hangs upon it or who hangs because of it. But in another and more practical sense we may surely ask what is involved in our decision.

Definitions of Evidence

The Oxford English Dictionary gives several senses of the term evidence:

The quality or condition of being evident Manifestation That which makes evident, mark, trace Ground for belief; that which tends to prove

or disprove any conclusion

Medical Definitions

Evidence is a property of data that makes us alter our beliefs about how the world around us is working. Another way to say this is that evidence is the basis upon which we derive inferences. (Goodman and Royall 1988)

In its simplest form, the available scientific evidence consists of the published report of a single piece of original research. (Miettinen 1998)

Legal definitions

Any species of proof or probative matter legally presented at a trial of an issue, by the act of the parties and through the medium of witnesses, records, documents, exhibits, concrete objects for the purpose of inducing beliefs in the minds of the court or the jury as to their contention

All means by which any alleged matter of fact, the truth of which is submitted to investigation,is established or disproved.

Evidence Based Public Health

Evidence based public health can be defined as a public health endeavour in which there is an informed, explicit, and judicious use of evidence that has been derived from any of a variety of science and social science research and evaluation methods.

Canadian Health Services Research Foundation

Evidence is information that comes closest to the facts of a matter. The form it takes depends on context. The findings of high-quality, methodologically appropriate research are the most accurate evidence. Because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research. The evidence base for a decision is the multiple forms of evidence combined to balance rigour with expedience—while privileging the former over the latter.

Characteristics of EvidenceUpshur JECP 2000

Provisional, Defeasible, Emergent Incomplete Constrained: Ethical, Economic, Computational Collective Asymmetric Historical Market Influenced

Provisional, Defeasible, Emergent

For example Peptic Ulcer Disease 3 distinct pathogentic and therapeutic models

in last three decades 1970’s: stress, V&P, White Diet 1980’s: Hypersecretion, H2 Blockers 1990’s: Infection and Eradication

Warren and Marshall

Ethical Constraints

Ethical: Some things we will not know because it is unethical to perform certain studies (Is this an absolute or relative restraint?

Fisher: It is not the fault of Hill or Doll that they cannot produce evidence in which a thousand children of teen age have been laid under a ban that they shall never smoke and a thousand more chosen at random from the same age group have been under a compulsion to smoke at least thirty cigarettes a day. If that type of experiment could be done, there would be no difficulty.

Economic Constraints

Economic: Had we but money enough and time, this lack of evidence would be no crime

Evidence of certain kinds is expensive (see cohort studies)

Computational Constraints Computational: There are limits in terms of what can be known

given the resources required to perform the calculations. C. David Naylor: Another difficulty arises from the Malthusian

growth of uncertainty when multiple technologies are combined into clinical strategies. Take two technologies and they can be used in two different sequences; take five and the number of possible sequences is one hundred and twenty. Furthermore the elements in a clinical strategy are usually tested in separate studies leaving few data on the chains of conditional probabilities that link sequences of tests, treatments and outcomes.

Computational Constraints

Hierarchical, serial clinical trials would permit identification of the optimum combination of these agent classes for Alzheimer disease through 127 trials, enrolling 63,500 patients, requiring 286 years; for ischemic stroke through 31 trials, enrolling 186,000 patients, requiring 155 years.

Saver JL. Kalafut M. Combination therapies and the theoretical limits of evidence-based medicine. Neuroepidemiology. 20(2):57-64, 2001 May.

Collective

Evidence is collective Volume of information to great for any one

person to master Evidence, then exists as a collective property

Asymmetric

There are differences in evidence access both within and between disciplines. This relates to both funding and access to intellectual resources. Some issues are not sufficiently prevalent to warrant full scale attention (rare diseases); some lack appeal from the point of view of funders (vitamins, public health interventions eg restaurant inspection).

Historical

Many health care professions do not have an established history of commitment to research. Indeed, it is only in the last 30 years that the RCT and in the past 10 years that the systematic review assumed prominence in medical conceptions of evidence. Other disciplines have neither the intellectual tradition, nor commitment to the production of researchers.

Markets: Alolaprilodine

Market forces create redundancies However, evidence claims are frequently

made (e.g. receptor blockade) Evidential indiscernables

Time Constraints

There is a necessary trade off between the need to search for information and make decisions

The conflicting hierarchy challenge

First used by Periodic Task Force for Preventive Care

Now a proliferation of such hierarchies None are based on research, all are based

on expert judgement or consensus

Conflicting HierarchiesUpshur CMAJ 2003

Highest Level of Evidence for a treatment or Intervention

Grade (A) of Recommendation

The Canadian Task Force on Preventive Health Care 1

1: Evidence from at least 1 properly randomized controlled trial (RCT).

A=Good evidence to support the recommendation that the condition be specifically considered in a PHE.

Osteoporosis guidelines2 1+: systematic review or meta-analysis of randomized controlled trials1: one randomized trial with adequate power

A= need supportive level 1 or 1+ evidence plus consensus

Centre for Evidence Based Medicine Oxford3 1a: Systematic review with homogeneity of randomized controlled trials1b: Individual RCT with narrow confidence interval1c: all or none

A= consistent level 1 studies

Scottish Intercollegiate Guidelines Network grading system 4

1++ High quality meta-analyses, systematic reviews of randomised controlled trials (RCTs), or RCTs with a very low risk of bias 1+ Well conducted meta-analyses, systematic reviews of RCTs, or RCTs with a low risk of bias 1- Meta-analyses, systematic reviews or RCTs, or RCTs with a high risk of bias

A= At least one meta-analysis, systematic review, or RCT rated as 1++ and directly applicable to the target population; or a systematic review of RCTs or a body of evidence consisting principally of studies rated as 1+ directly applicable to the target population and demonstrating overall consistency

Evidence Based Public Health Descriptive: to identify the qualities and distributions of variables;

Taxonomic: to compare and classify variables into related groups or categories;

Analytic: to examine associations between variables—these may be hypothesised causal or therapeutic relations;

Interpretive: to identify and explain meanings, usually from a particular perspective;

Explanatory: to make observations intelligible and understandable; and

Evaluative: to determine quality and worth—often assessing the relevance, effectiveness, and consequences of activities.

GeneralParticular

Meaning

Measurement

Q1Concrete/Historical

Q2Historical/Social

Q4Personal/

Mathematical

Q3ImpersonalMathematical

CHSRF

CHSRF

Krever Commission

In The Commission of Inquiry on the Blood System in Canada, Mr. Justice Krever said:

Where there is reasonable evidence of an impending threat to public health, it is inappropriate to require proof of causation beyond a reasonable doubt before taking steps to avert the threat.

Campbell Commission

The Commission therefore recommends: That the precautionary principle, which states that action to reduce risk need not await scientific certainty, be expressly adopted as a

guiding principle throughout Ontario’s health, public health and worker safety systems by way of policy statement, by explicit reference in all relevant operational standards and directions, and by way of inclusion, through preamble, statement of principle, or otherwise, in the Occupational Health and Safety Act, the Health Protection and Promotion Act, and all relevant health statutes and regulations.

That in any future infectious disease crisis, the precautionary principle guide the development, implementation and monitoring of procedures, guidelines, processes and systems for the early detection and treatment of possible cases.

Where does this leave us?

Is precaution appropriate to all public health interventions or only communicable diseases?

Would thresholds vary between communicable diseases, chronic diseases, health promotion, environmental health?

Which formulation of the precautionary principle?