Research Designs

Murray W. Enns Professor of Psychiatry

Broad Strokes

The design is the structure of any investigation. It gives direction and systematizes the  research. Each design has advantages and disadvantages.

There are two main approaches to a research problem:

Quantitative ResearchQualitative Research

Qualitative Research

Qualitative research is a method of inquiry employed in many different academic disciplines particularly in the social sciences

The form of data, by definition is not “quantitative measurement” and traditional statistics do not apply

Strong theoretical foundation – e.g. Grounded Theory

Grounded theory

A systematic methodology social sciences involving the construction of theory through the analysis of data.

Begins with a question, or a collection of qualitative data. The data is reviewed to define repeated ideas, concepts or elements and tagged with codes, which have been extracted from the data. As more data is collected, codes are grouped into categories.

These categories may become the basis for new theory.

Descriptive research

Aim: Observe and describeE.gs. Case studyCase seriesSurveys

The “lowly” case reportCase reports have a time-honored and rich tradition in

medical publication.

Case reports can be relevant, timely, and important in advancing medical knowledge especially of rare phenomena.

The observation of a single patient can add to our understanding of etiology, pathogenesis, natural history, and treatment,

Excellent training opportunity for junior investigators

Appropriate use of case series data

Proof (or Disproof) of concept for a new hypothesis

Reporting of sentinel eventsToxicities of therapiesRecognition of epidemicsInitial identification of previously unrecognized

syndromesStudying outcomes of rare diseases or new

treatments

Examples of case reports

Four weeks after having performed the first human heart transplantation on December 3, 1967, Christian Barnaard published a case report in the South African Medical Journal

In 1981, a series of case report publications noted unusual infections and neoplasms occurring in young men leading to the discovery of a new syndrome, AIDS

Reviewing other research

Aim: Understand/explain what is knownLiterature reviewSystematic reviewMeta-analysis

Systematic review

A systematic review can/should be a piece of true research done to establish whether a hypothesis is correct or not.

It should begin with a structured protocol describing the aims and objectives, a clear search strategy, and an identified primary outcome

Inclusion and exclusion of studies needs to be based on pre-determined criteria

Accurate records - reflected in a flow diagram

Meta-analysisMeta analysis is a formal statistical method for

aggregating the results of multiple studies

Major advantage is the increased “N” which may increase power OR give better estimates of the sizes of effects

Challenges:

Cherry-picking versus systematic inclusion

Garbage in, garbage out

Publication biases – the file drawer effect

Funnel PlotsUnbiased publications vs. Biased publications

PILOT study before full-scale

Aim: to find out if the design works and is feasible

Typical objectives of a pilot study: sample size calculation; a practice run of the protocol; testing data collection forms; testing randomization procedures; determining recruitment and consent rates; examining the acceptability of the intervention; selection of the most appropriate primary outcome measure.

When is a pilot study not a pilot study?Recent review: Shanyinde et al, 2011:

Random sample of 50 “pilot” studies taken from MEDLINE

56% of papers- methodological issues discussed in substantial depth,

18% of papers- discussed future trials and only

12% of authors were actually conducting a future trial

Conclusion: Sometimes a “pilot study” is just a confession of inconclusive results or lack of adequate sample size.

Correlational studies

Aim: PredictCase-control studyObservational studyLongitudinal studyCross sectional study

Secondary Data AnalysisThis refers to the analysis of pre-existing data sets that

were collected for another “primary” purpose

Many large-scale MH epidemiological studies have had their data “liberated” for secondary analyses

E.g. NCS-R, NEMESIS, NESARC, CCHS

Huge benefits/efficiency with the “right questions”

Major risk – inflation of risk of false positives due to multiple analyses being conducted

E.g. the “post-hoc a-priori” hypothesis

Administrative Data AnalysisRecords collected for other administrative purposes

(e.g. hospital records, physician billing records, DPIN) can be subject to analysis particularly if data from different sources is LINKED.

*E.g. Manitoba Centre for Health Policy*1.Health - Administrative, Clinical, Survey data

2. Education

3. Social - Healthy Child Manitoba, Community Services,

Social Survey data

4. Justice

Experimental designs

Aim: determine causesTrue experimental design

e.g.Pre-test post-testUsing a control groupRandomization including RCTsBetween subjects or within-subjects

SO, how do you choose?What information do you want? (aims of the study)

The nature of the phenomenon –

Is it feasible to collect the data?

Is data already available elsewhere?

How reliable does the information need to be?

Is it ethical to conduct the study?

The cost of the design

Is there little or much current scientific theory and literature on the topic?