Rigour and Robustnessin Research

Dr Jennifer LokePHD; MBA; MSC; PGDIP; BN; ADVDIP,

SFHEA; LPE; RN

© J. Loke 2014

Aims

• To gain insight to the meaning of rigour and its importance for robustness in all research.

• To gain understanding on how rigour can be maintained to achieve robustness in the different stages of the research process of the different research approaches (paradigms).

• To appreciate the strategies to achieve rigour in the different stages of research.

© J. Loke 2014

Learning Outcomes

At the end of the session the student will be able to: o understand the meaning and the importance of rigour in

research;o understand how rigour is judged in the different research

paradigms;o describe ways to achieving rigour in research studies;o identify some of the common pitfalls in the process of

achieving rigour in research;o highlight research knowledge and skills as appropriate to

address these pitfalls.

© J. Loke 2014

© J. Loke 2014

By John Stevens PUBLISHED:MailOnline 14:03, 20 June 2012 | UPDATED: 12:27, 21 June 2012

One McBurger with lies, please: Fast-food firm's own video reveals why items on the menu never look like what you are served

© J. Loke 2014

Still?!!

© J. Loke 2014

The quality of being extremely thorough and careful

Rigour Robustness

© J. Loke 2014

Replicability not

reproducibility

Trouble at the lab. The Economist. October 2013

© J. Loke 2014

Types of Research

QUANTITATIVE QUALITATIVEMixed-Methods

© J. Loke 2014

Research paradigms

Quantitative designs Qualitative approachesMixed methodology

approaches

LOKE, J. Jan2012

Located in a continuum

The goal of Quantitative research

© J. Loke 2014

Generally, to quantify the problem and understand how prevalent it is by looking

for projectable results to a larger population

Rigour in Quantitative ResearchReliabilityValidity

© J. Loke 2014

The goal of Qualitative research

© J. Loke 2014

Generally, to define the problem or develop an approach to the problem.

Aim to contribute to an ongoing dialogue, in the form of debate, discussion or

argument about a phenomenon

Rigour in Qualitative Research

CredibilityTrustworthiness

© J. Loke 2014

Dependability

Rigour in Qualitative Research – drawing parallels with quantitative research

ReliabilityValidity

CredibilityTrustworthiness

© J. Loke 2014

Validity

Dependability

Reliability

Strategies to achieve rigour and robustness

Research approach/paradigm• Methodological approach• Sampling techniques• Data collection • Data analysis

Strategies to achieve rigour and robustness

Research approach/paradigm• Professional/Personal bias– Implicitly expressed (quantitative research) – Explicitly expressed (qualitative research)

Rigour through proper selection of research paradigm/methodological approaches

• Personal/Professional Bias – prevent highlighting some issues while obscuring others?

• Address ethical aspects of research – which sometimes can be compromised due to political reasons

Reflexivity – qualitative research

© J. Loke 2014

Strategies to achieve rigour and robustness

Methodological approach

Quantitative vs Qualitative

Research methodologies

Randomised control Trial

(RCT)

Quasi RCT (cluster/group

randomisation

Survey-based approach

Discourse analytic work:

Corpus analytic work

Grounded theory

Discourse analytic work

Life history

Semiotic analytic work

Ethnography

Ethno-methodology

Phenomenology

Quantitative

in nature

Qualitative

in nature

LOKE, J. Jan2012

Selecting an appropriate research design

• Quantitative• Qualitative

– Triangulation of theoretical framework

• Mixed-Methods - approach

Credibility

Theoretical framework 1

Theoretical framework 2 Data Sources

© J. Loke 2014

Strategies to achieve rigour and robustness

Sampling Techniques

Participants selection

Document selection

Event selection

Representativeness

Strategies to achieve rigour and robustness

Data collection

Participants selection

Document selection

Event selection

what, where, when and how

Data collection in research

© J. Loke 2014

Common techniques for producing data

Indirect Observations

Direct Observations Elicitation Techniques

Mixed Methods

Continuous or Spot

monitoring

Scientific tools, Unstructured, Semi-

Structured, Structured, Mixed

Case studies, Participants

Observations

What Technique?

Strategies to achieve rigour

© J. Loke 2014

• Knowledge and experience of data collector

• Appropriateness of the collection methods for the research objectives and settings

• Relationship between data collector and data (participants/ texts/historical documents)

• Addressing ethical issues

Consider the different quantitative and qualitative techniques in Data Collection

• Data Collection

Reliability

Data Source 1

Data Source 3

Data Source 2

Credibility

Observer 2

Observer 3

Observer 1

Trustworthiness

Observer

Participant/member checking

Data source

© J. Loke 2014

Where/When/How

Strategies to achieve rigour and robustness

Data Analysis

Depends on the research approach

Key qualitative and quantitative distinctions

DATA

Analysis Quantitative Qualitative

Quantitative Statistical and mathematical analysis of the numerical

data as presented

Assigning meaning of words based on scoring system

E.g. Content analysis, word counts, free lists, pile sorts

Qualitative Search for and presentation of meaning in results of quantitative processing

Interpretative test studiesE.g. Hermeneutics, Grounded Theory, Phenomenology

© J. Loke 2014

Consider the different quantitative and qualitative techniques in Data Analysis

• Appropriate types of statistical tests for different types of quantitative data

• Appropriate level of statistical tests based on the size of data

• Appropriate analytic framework for the different theoretical framework

• Triangulation of analytic framework

Reliability

Data Source 1

Data Source 3

Data Source 2

© J. Loke 2014

Rigour for Robustness

Analyses involve attending to inconsistencies and diversity

Present the richness of detail in data, researcher’s assumptions must be clarified upfront and context of the study must be

explained

Validation through reference to coherence and fruitfulness of findings, participants’ orientation and

to new research problems

Seek out negative/deviant cases

© J. Loke 2014

Rigour through reporting on findings and making conclusions

• Personal/Professional Bias – prevent highlighting some issues while obscuring others?

• Address ethical aspects of research – which sometimes can be compromised due to political reasons

Reflexivity – qualitative research

© J. Loke 2014

Ways to address rigour

• Consider the quantitative and qualitative techniques in:

a. Data sources/collectionb. Data analysis

© J. Loke 2014

Selecting an appropriate research design

*Research Question

Further readings• http://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-

alarming-degree-it-not-trouble• http://cogprints.org/7691/7/ICMLws09.pdf• http://www.sonoma.edu/users/k/koshar/n300/rigor_table.htm• Tobin, G. A., Begley, C. M. 2004. Methodological rigour within a qualitative framework.

Journal of Advanced Nursing. 48(4), 388-96.

© J. Loke 2014