Evidence-Based HR Management & Systematic Reviews

Evidence-Based Management

What is it?

Why do we need it?

How does it look like in practice?

PhD Consortium of the 7th International

Conference of the Dutch HRM network

What is it?

Where does it come from?

Why do we need it?

What is stopping us?

How does is look like in practice?

Postgraduate Course

Evidence-Based Management

1. Evidence based management:

What is it?

Postgraduate Course

Four propositionsPostgraduate Course

Research produced by management scholars could be useful to

organizations

Drawing on available evidence (including research produced by

academics) is likely to improve decisions

Managers and organizations do not appear to be strongly aware

of nor use research findings

We need to increase awareness of and access to research

findings

What is EBMgt?Postgraduate Course

Evidence-based management is about making

decisions through the conscientious, explicit,

and judicious use of four sources of information:

practitioner expertise and judgment, evidence

from the local context, a critical evaluation of

the best available research evidence, and the

perspectives of those people who might be

affected by the decision.

(Briner, Denyer, Rousseau, 2009)


The problem is that management is not as

evidence-based as it should be nor as it could be


It is not a completely new idea – managers and

organizations use evidence all the time

EBMgt is different because it’s about:

Increasing the types of evidence we use

Using it more thoughtfully and carefully

(conscientious, judicious, explicit)

Its only purpose is to help us make better

decisions through more and more systematic

use of evidence

Who says it’s a problem?Postgraduate Course

People in many other fields (medicine, social

work, criminology, politicians) say it’s a

problem in their fields

Not everyone in management but some of

people think it’s a problem

Some academics and researchers

Some managers and organizations

Some professional associations

Some commentators and journalists

Origins of EBMgtPostgraduate Course

Management not the only field where there are these concerns. What field is this?

“a research-user gap” “practitioners do not read academic journals” “the findings of research into what is an effective intervention are

not being translated into actual practice” “academics not practitioners are driving the research agenda” “the relevance, quality and applicability of research is

questionable” “practice is being driven more by fads and fashions than

research” “many practices are doing more harm than good” “the collective wisdom from research is being lost”

Evidence-Based PracticePostgraduate Course

1991Medicine

1998Education

1998Probation service

1999Housing policy

1999Social care

2000Nursing

2000Criminal justice

????Management?

Academic interest in EBMgtPostgraduate Course

Similar ideas around for a long time

2003 – Systematic reviews of evidence

2006 – Rousseau EBMgt Presidential Address and

Pfeffer & Sutton book

2007-2009 – Rousseau EBMgt Collaborative

2008-2011 – Several conferences

2011 – Center for Evidence-Based Management

(CEBMa) in Amsterdam

2011 – EBMgt Handbook

But many researchers not interested at all in EBMgt

Manager & practitioner interest in EBMgt

Some HRM and professional bodies express

interest (SHRM, SIOP, VOV Learning Network)

Some Universities running courses for

practitioners (e.g., Amsterdam, Ghent)

Some publications for practitioners

But managers are: Used to working in a different way

Under pressure to adopt fads and fashions

Have high expectations of evidence

Managers used to working in a different way:

Need to act quickly: Speed more

important than accuracy

Organizational politics

Formal authority and hierarchies

Over-emphasize experience

Rewarded for getting things done not

doing what works

Sometimes we are evidence-based

Try to gather data and information

Invest time and effort in trying to

understand and apply it

Question our and others’ assumptions and

logic

Are sceptical about what appear to be fads

Resist the temptation to act quickly

Sometimes we are not so evidence-based

Act on gut feeling (though intuition can be

important for some decisions)

Copy other people who appear successful

(benchmarking)

Think there is one ideal way (best practice)

Let the ‘solution’ frame and define our ‘problem’

and create need (kitchen gadgets)

Want to fit in and be as cool as everyone else

(fashion)


Where does it come from?

Postgraduate Course

McMaster University Medical School, Canada

Medicine: Founding fathers

David Sackett Gordon Guyatt

Postgraduate Course

Problem I: too much information

More than 1 million articles in 40.000 medical journals per

year (= 1995; now probably more than 2 million). For a

specialist to keep up this means reading 25 articles every

day (for a GP more than 100!)

Most of the new insights and treatment

methods don’t reach the target group

Postgraduate Course

Problem II: persistent convictions

if you’re hyperventilating

breathe into a bag

Postgraduate Course

Problem III: jumping to conclusions

people who have an irregular heartbeat are much more likely to die

of coronary disease

give them a drug that reduces the number of

irregular beats

Postgraduate Course

Examples of mechanistic reasoning gone wrong:

Oestrogen replacement therapy to reduce cardiac events and stroke

in post-menopausal women.

Treatment of measles with antibiotics.

Rest for recovery.

Placing babies on their fronts to prevent Sudden Infant Death

Syndrome (SIDS).

Debriefing after psychological trauma

Postgraduate Course

Problem IV

David Sackett

Half of what you learn in medical school will be

shown to be either dead wrong or out-of-date

within 5 years of your graduation; the trouble is that

nobody can tell you which half.

The most important thing to learn is how to learn

on your own.

(Remember that your teachers are as full of bullshit

as your parents)

Postgraduate Course

David Sackett

“Good doctors use both individual clinical expertise and

the best available external evidence, and neither alone is

enough. Without clinical expertise, practice risks

becoming tyrannized by evidence, for even excellent

external evidence may be inapplicable to or inappropriate

for an individual patient. Without current best evidence,

practice risks becoming rapidly out of date, to the

detriment of patients.”

Evidence based decisionPostgraduate Course

Management: Founding MotherPostgraduate Course

Jeffrey Pfeffer Robert Sutton

Management: Founding FathersPostgraduate Course

2. Evidence-based management:

Why do we need it?

Postgraduate Course

EBMgt: some basic assumptionsPostgraduate Course

Research produced by management scholars could be useful to

organizations

Drawing on available evidence (including research produced by

academics) is likely to improve decisions

Organizations do not appear to be strongly aware of nor use

research findings

EBMgt is a potentially useful way of thinking about how we can

incorporate research evidence into decision-making

Reason 1:

Errors and Biases of Human Judgment

Postgraduate Course

Seeing order in randomness

Mental corner cutting

Misinterpretation of incomplete data

Halo effect

False consensus effect

Reinterpreting evidence

Group think

Self serving bias

Sunk cost fallacy

Cognitive dissonance reduction

Postgraduate Course


Confirmation bias

Authority bias

In-group bias

Recall bias

Anchoring bias

Inaccurate covariation detection

Distortions due to plausibility

Seeing order in randomness Mental corner cutting


Halo effect



Group think

Self serving bias

Sunk cost fallacy

Cognitive dissonance reduction

Postgraduate Course


Confirmation bias

Authority bias

In-group bias

Recall bias

Anchoring bias



We are predisposed to see order, pattern and causal

relations in the world.

Patternicity: The tendency to find meaningful patterns in

both meaningful and meaningless noise.

Postgraduate Course


We are pattern seeking primates: association learning

Postgraduate Course


Postgraduate Course

Points of impact of V-1 bombs in London

A Type I error or a false positive, is

believing a pattern is real when it is not

(finding a non existent pattern)

A Type II error or a false negative, is

not believing a pattern is real when it is

(not recognizing a real pattern)

Postgraduate Course


Dr. Michael Shermer

(Director of the Skeptics Society)

A Type I error or a false positive: believe that the

rustle in the grass is a dangerous predator when it is

just the wind (low cost)

Postgraduate Course


A Type II error or a false negative: believe that the

rustle in the grass is just the wind when it is a

dangerous predator (high cost)

Postgraduate Course


Pattern detection problem

Assessing the difference between a Type I and

Type II error is highly problematic (especially in

split second ‘life and death’ situations), so the

default position is to assume

all patterns are real.

Postgraduate Course


Postgraduate Course


Jennifer Whitson, University of Texas Austin, corporate environments

Postgraduate Course


Erroneous beliefs plaque both experienced

professionals and less informed laypeople alike.

stress peptic ulcer

Peptic ulcer – an infectious disease!

This year's Nobel Prize in Physiology or Medicine goes to Barry Marshall and Robin

Warren, who with tenacity and a prepared mind challenged prevailing dogmas. By

using technologies generally available (fibre endoscopy, silver staining of

histological sections and culture techniques for microaerophilic bacteria), they

made an irrefutable case that the bacterium Helicobacter pylori is causing disease.

By culturing the bacteria they made them amenable to scientific study.

In 1982, when this bacterium was discovered by Marshall and Warren, stress and

lifestyle were considered the major causes of peptic ulcer disease. It is now

firmly established that Helicobacter pylori

causes more then 90% of duodenal ulcers.

The link between Helicobacter pylori

infection and peptic ulcer disease has been

established through studies of human

volunteers, antibiotic treatment studies and

epidemiological studies.

Oct 2005

Postgraduate Course


Doctors, teachers, lawyers and managers hold many

erroneous beliefs, not because they are ignorant or

stupid, but because they seem to be the most sensible

conclusion consistent with the available evidence.

They hold such beliefs because they seem to be the

irresistible products of their own professional experience.

They are the products, not of irrationality, but of flawed

rationality

Managers seem to be extremely good at generating

ideas, theories, and explanations that have the ring of

plausibility. They may be relatively deficient, however,

in evaluating and testing those ideas once they are

formed.

This requires that we think critically about experience,

question our assumptions, and challenge what we

think we know

(Show me the evidence!)

Postgraduate Course


3. Evidence-based management:

What is stopping us?

Postgraduate Course

Postgraduate Course

Quick fixes (1)

What is the quick fix? A ‘solution’ which Focuses on style and presentation not content

Is not evaluated

Is always slower than we hoped

Usually doesn’t work

Is followed by another quick fix

Everybody forgets and becomes subject to

organizational amnesia

Postgraduate Course

Quick fixes (2)

So why do we do quick fixes?

Can be career-enhancing for managers

(e.g., issue selling, kick-ass CEOs)

Speed is often valued over accuracy

Heavily sold and marketed

Are we all looking for quick and easy

solutions?

So who needs or wants academic research?

• 57

• 58

• 59

• 60

• 61

• 62

• 63

• 64

Postgraduate Course

Management Fads (1)

The nearly-forgotten fads Scientific Management/Taylorism

Business Process Reengineering

Management by results

Excellence

Total Quality Management

Learning Organizations

Knowledge Management

Postgraduate Course

Management Fads (2)

The fads that haven’t been forgotten (yet) Talent management

Management development

Executive coaching

Emotional intelligence

Employee engagement

Myers Briggs Type Indicator

Belbin Team Roles

General concern about the destructive impact of fads

from both practitioners and researchers

• 67

• 68

• 69

• 70

• 71

• 72

• 73

• 74

• 75

• 76

FADS* SEEM TO BE ATTRACTIVE, COMPELLING AND IRRESISTIBLE

Promise to deliver a lot and fast Appear simple New and shiny Will make everything alright and help contain

anxieties around intractable problems Help user feel effective and cutting edge Bits of some fads may work in some contexts

So who needs or wants academic research?

*Evidence-based management not a fad!

• 78

• 79

• 80

dDwtNzI3

http://2.bp.blogspot.com/_V4Xflj0uLaE/TNhWcCcwpGI/AAAAAAAAAu8/dj6W2_DNT18/s1600/1980s-fashion.jpg

• 81


Mental corner cutting


Halo effect



Group think

Postgraduate Course


Confirmation bias

Authority bias In-group bias

Recall bias

Anchoring bias



Postgraduate Course

Under pressure to adopt fads

“And there we see the power of any big

managerial idea (or fad). It may be smart, like

quality, or stupid, like conglomeration. Either

way, if everybody's doing it, the pressure to do it

too is immense. If it turns out to be smart, great.

If it turns out to be stupid, well, you were in good

company and most likely ended up no worse off

than your competitors. Your company's board

consists mostly of CEOs who were probably

doing it at their companies. How mad can they

get?

Postgraduate Course

The true value of conventional management

wisdom is not that it's wise or dumb, but that it's

conventional. It makes one of the hardest jobs in

the world, managing an organization, a little

easier. By following it, managers everywhere see

a way to drag their sorry behinds through

another quarter without getting fired. And isn't

that, really, what it's all about?”

(Colvin, 2004, Fortune)

Under pressure to adopt fads


How does it look like in practice?

Postgraduate Course

Postgraduate Course

Four sources

Postgraduate Course

JAMA, 1992

Postgraduate Course

Push vs Pull

Push: teaching (management) principles

based upon a convergent body of

research and telling students what to do.

Pull: teaching (managers) how to find,

appraise and apply the outcome of

research (evidence) by themselves

Postgraduate Course

The 5 steps of ‘pull’ EBP

1. Formulate an answerable question

2. Search for the best available evidence

3. Critically appraise the evidence

4. Integrate the evidence with your managerial

expertise and organisational concerns and apply

5. Monitor the outcome

Answerable question

Postgraduate Course

I am a consultant, my client a large health-care

organization. The board of directors has plans for a

merger with a smaller healthcare organization. However,

it’s been said that the organizational culture differs widely

between the two organizations. The board want’s to

know if this can impede a successful outcome.

Postgraduate Course

P = Population or problem

I = Intervention or successfactor

C = Comparison

O = Outcome

C = Context

Answerable question: PICO(C)

Answerable question: PICOC

Postgraduate Course

P: What kind of Population are we talking about? Middle managers,

back-office employees, medical staff, clerical staff?

O: What kind of Outcome are we aiming for? Employee productivity,

return on investment, profit margin, competitive position, innovation

power, market share, customer satisfaction?

P/C: And how is the assumed cultural difference assessed? Is it the

personal view of some managers or is it measured by a validated

instrument?

Postgraduate Course








Where do we search?Postgraduate Course

•Postgraduate School

Where do we search?

Postgraduate Course








Critical appraisal

How to read a research article?

Postgraduate Course

Critical appraisalPostgraduate Course

1. Study designs

2. Levels of evidence

3. Bias / confounding

4. Effect sizes

5. External validity

Postgraduate Course

Which study for which question?

Research designs

The “best” evidence depends on the question type !

Postgraduate Course

Levels of evidence

Postgraduate Course





4. Integrate the evidence with your

managerial expertise and organisational

concerns and apply


Postgraduate Course

1. Is your organization / division / population so different from those in the study that its results cannot apply?

2. How relevant is the study to what you are seeking to understand or decide?

3. What are your organization’s potential benefits and harms from the intervention?

4. Is the intervention feasible in your setting?

Organization concerns

Always ask yourself to what extent the evidence is applicable in your situation:

Postgraduate Course








Postgraduate Course

Monitor the outcome

posttest?

pretest?

control group?

Postgraduate Course

Do a trial!

(*). Why should academics be interested in EBMgt?

Postgraduate Course

(5). Evidence based management:

What can you do as a PhD student?

Postgraduate Course

Postgraduate Course

Barriers

Postgraduate Course

Best available evidence

=

systematic reviews

Barriers

Postgraduate Course

Better than a single study: a replication study

Better than a replication study:a systematic review / meta analysis

If there were 100 RCT’s, 99 of which gave a ‘negative’ result (where, say, the new intervention appeared to be harmful), while one had a ‘positive’ result (were the intervention appeared helpful), it would obviously be a mistake to consider only the single positive RCT.

Postgraduate Course

• 115

TYPES OF LITERATURE REVIEW

Explicit systematic: Explicit use of rigorous method - can vary as least as much as the range of methods in primary research

Implicit systematic: rigorous method but not statedFalse systematic: described as systematic but with little evidence of

explicit rigorous methodArgument/thematic: a review that aims to explore and usually

support a particular argument or theme with no pretension to use an explicit rigorous method (though thematic reviews can be systematic)

Expert or ad hoc review: informed by the skill and experience of the reviewer but no clear method so open to hidden bias.

Rapid evidence assessment: a rapid review that may or may not be rigorous and systematic. If it is systematic then in order to be rapid it is likely to be limited in some explicit aspect of scope.(Gough 2007)

• 116

LITERATURE REVIEWS IN MANAGEMENTHow many here have had training in reviewing literature?Are we really “standing on the shoulders of giants”?Do you recognize these sort of unqualified statements?• “Previous studies have shown that…”• “It has been demonstrated that…”

But how many studies? Demonstrated how? Did other studies find something else?

Very few systematic reviews in management

WHAT QUALITIES SHOULD LITERATURE REVIEWS HAVE?

• 118

WHAT QUALITIES SHOULD LITERATURE REVIEWS HAVE?Comprehensive?Reader-friendly?Informative?Balanced?Insightful?Critical?Rigorous?Accessible?User led?Up-to-date?

Focused?Exploratory?Inclusive? (of different types of evidence)Transparent? Accurately referenced?Objective?Replicable?Interesting?Standardized?

• 119

WHAT IS A SYSTEMATIC REVIEW?It’s research on existing research

With a clear, explicit and replicable methodology• Clear review question• Search strategy• Quality criteria

Allows us to draw reliable conclusions about what we know and do not know about a given question or problem

Systematic review

The intention behind a systematic review is to identify as fully

as possible all the scientific studies of relevance to a particular

subject and to assess the validity and authority of the evidence

of each study separately. As the name indicates, a systematic

review takes a systematic approach to identifying studies and

has the methodological quality critically appraised by multiple

researchers independently of each other, as a consequence of

which the review is transparent and reproducible and can be

monitored. The use of statistical analysis techniques in a

systematic review to pool the results of the individual studies

numerically in order to achieve a more accurate estimate of

the effect is termed a “meta-analysis”.

Postgraduate Course

• 122

THE QUESTIONS SRs ANSWER

For any given specific problem:What do we know?

What do we not know?

What are we not sure about?

How do we know we know or don’t know or are not sure that…?

What is the basis for our claims? (e.g., How much evidence? What quality?)

Stages of a SR

Existing research

studies

SYSTEMATIC REVIEW

1. problem formulation;

2. locating studies;

3. study selection and evaluation;

4. analysis and synthesis;

5. reporting of the results

What do we

know?

What we do not know?

Informs practice

Informs future

research questions

Practice-

relevant

question

Stages of a systematic reviewPostgraduate Course

1. Formulate a focussed review

question


3. Select relevant studies


5. Synthesise the findings

6. Report what is known

• 125

WHAT SORT OF QUESTIONS CAN BE ADDRESSED IN A SR?

Each would require much more specificity

Does team-building work?

Can you improve emotional intelligence?

Do increases in EI lead to performance improvements?

Does management development improve the performance of managers?

Does employee engagement predict organizational performance?

Is 360 degree feedback effective?

Can potentially great leaders be identified?

Is coaching effective?

Does team-building work?

How would you make this question more specific

Postgraduate Course

P = Population

I = Intervention or factor

C = Comparison

O = Outcome

C = Context

PICOC & CIMO

C = Context

I = Intervention or

factor

M = Mechanism

O = Outcome

• 128

WHAT SORT OF QUESTIONS CAN BE ADDRESSED IN A SR?Does team-building work?• What is meant by ‘team’? And what is not included as a ‘team’?• What kind of teams?• In which particular contexts or settings?• What is ‘team building’? And what is not ‘team building’?• What does ‘work’ mean?• ‘Work’ compared to any other team intervention? No intervention?• What outcomes are relevant?• What are the mechanisms, processes and theory which might

account for possible effects of team building on outcomes?• What time periods are relevant for observing any possible effects?• What about possible negative effects or harm?• What types of data from what sorts of designs would in principle

provide good quality, medium quality and poor quality evidence?

Postgraduate School

Which study for which question?

Research questionQualitative

studiesSurveys

Observational studies

Controlled studies

Effectiveness: does it work?, does A work better than B? + ++Process: how does it work, why does it work? ++ +Context: in what circumstances does it work, for whom? ++ + +Safety: will it do more good than harm? + + ++Acceptability: will the target group accept the intervention / new method of working? ++ + +Cost effectiveness: does it reduce costs? is A cheaper than B? ++Appropriateness: is this the right intervention / method for this target group? ++ ++

Satisfaction: is the target group satisfied with the new method of working? ++ ++ +


1. Formulate a focussed review question






• 131

MAKING DECISIONS ABOUT SEARCH STRATEGYWhat sources of evidence?

What sources will you include or exclude and why?

How iterative can you be?• Test the question doing some simple searches• Does the question work?• Does the search strategy work?

Postgraduate School

Two types of search strategies

Search strategy

Building blocks methodSnowball method

Postgraduate School

Snowball method

Starting from one book or article, you search for other literature on the same topic.

Snowballing to older publications by finding out which publications were used by the author (see bibliography of book or article).

Snowballing to more recent publications by finding out how often that book or article has been cited by other authors (see Web of Knowledge).

Postgraduate School

Snowball method

ISI Web of Knowledge

Postgraduate School

Synonyms or

related terms

• ….

• ….

• ….

• ….

Synonyms or

related terms

• ….

• ….

• ….

• ….

Synonyms or

related terms

• ….

• ….

• ….

• ….

Building blocks method

Synonyms or

related terms

• ….

• ….

• ….

• ….

Keyword 1 Keyword 2 Keyword 3 Keyword 4

AND AND AND

OR OR OR

Postgraduate School

Postgraduate School

Building blocks method




3. Select relevant studies4. Critically appraise the evidence



•Postgraduate School

Standard appraisal questions

1. Did the study adress a clearly focused issue?

2. Is the sample size justified?

3. Is the design appropriate to the stated aims?

4. Are te measurements likely to be valid and reliable?

5. Are the statistical methods described?

6. Did untoward events occur during the study?

7. Were the basic data adequately described?

8. Do the numbers add up?

9. Was the statistical significance assessed?

10. What do the findings mean?

11. Are important effects overlooked?

12. What implications does the study have for your practice?

Once a body of evidence has been collated….

How relevant is this to what we are seeking to understand or decide?

How representative is this of the population that concerns us?

How reliable, how well-founded theoretically, empirically is it?

‘These are tough but necessary tests for evidence based policy and practice’ Solesbury 2004

What does synthesis mean in MOS?

Analysis,• “…is the job of systematically breaking down something into its

constituent parts and describing how they relate to each other – it is not random dissection but a methodological examination”

• Is the aim is to extract key data, ideas, theories, concepts [arguments] and methodological assumptions from the literature?

Synthesis,• “…is the act of making connections between the parts identified in

analysis. It is about recasting the information into a new or different arrangement. That arrangement should show connections and patterns that have not been produced previously”

(Hart, 1998: p.110)

• 146

Summarizing evidence

What does the evidence say?

Consistency of evidence?

Quality of evidence

Quantity of evidence (avoiding double counting)

The example of management and leadership development

Does leadership development work?

What do we know?

What is the ‘best’ research evidence available?

How can this evidence be ‘put together’?

What are the strengths and weaknesses of different approaches to synthesis?

What methods of synthesis are available? (1/2)

Aggregated synthesis

Analytic induction

Bayesian meta analysis

Case Survey

Comparative case study

Constant targeted comparison

Content analysis

Critical interpretive synthesis

Cross design synthesis

Framework analysis

Grounded theory

Hermeneutical analysis

Logical analysis

Meta analysisMeta ethnographyMeta narrative mappingMeta needs assessmentMeta synthesisMetaphorical analysisMixed method synthesisNarrative synthesisQuasi statisticsRealist synthesisReciprocal analysisTaxonomic analysisThematic synthesisTheory driven synthesis

What methods of synthesis are available? (2/2)

Synthesis by aggregation• extract and combine data from separate studies to increase the

effective sample size.

Synthesis by integration• collect and compare evidence from primary studies employing

two or more data collection methods.

Synthesis by interpretation• translate key interpretations / meanings from one study to

another.

Synthesis by explanation • identify causal mechanisms and how they operate.

• (Rousseau, Manning, Denyer, 2008)

Van Buren & Erskine, 2002 (building on Kirkpatrick)

Organizations reported collecting data on:

78% reaction (how participants have reacted to the programme)

32% learning (what participants have learnt from the programme)

9% behaviour (whether what was learnt is being applied on the job)

7% results (whether that application is achieving results)

Postgraduate School

What do you do with this?

Does management and leadership development work

Overall, the results suggest a medium to large effect size for learning and behaviour (largely based on self report)

Absence of evidence of impact of business impact

Do the results of synthesis create clarity - or confusion, conflict and controversy?

“A wide variety of program outcomes are reported in the literature – some that are

effective, but others that are failing. In some respects the lessons for practice can be found in the wide variance reported in these studies. The range of effect sizes clearly shows that it

is possible to have very large positive outcomes, or no outcomes at all” (p. 240/241)

(Collins and Holton, 1996: 240/241)

Do the results of synthesis create clarity - or confusion, conflict and controversy?

“Organizations should feel comfortable that their managerial leadership development programmes will produce substantial results, especially if they offer the right development programs for the right people at the

right time. For example, it is important to know whether a six-week training session is enough or the

right approach to develop new competencies that change managerial behaviours, or it is individual feedback from a supervisor on a weekly basis

regarding job performance that is most effective?”

(Collins and Holton, 1996: 240/241)

systematic reviewPostgraduate Course

Example

• 157

Flexible working conditions and their effects on employee health and wellbeing (Joyce et al, 2010)

Background: Flexible working conditions are increasingly popular in developed countries but the effects on employee health and wellbeing are largely unknown.

Objectives: To evaluate the effects (benefits and harms) of flexible working interventions on the physical, mental and general health and wellbeing of employees and their families.

Search strategy: Our searches (July 2009) covered 12 databases including the Cochrane Public Health Group Specialized Register, CENTRAL; MEDLINE; EMBASE; CINAHL; PsycINFO; Social Science Citation Index; ASSIA; IBSS; Sociological Abstracts; and ABI/Inform. We also searched relevant websites, hand searched key journals, searched bibliographies and contacted study authors and key experts.

• 158


Selection criteria: Randomized controlled trials (RCT), interrupted time series and controlled before and after studies (CBA), which examined the effects of flexible working interventions on employee health and wellbeing. We excluded studies assessing outcomes for less than six months and extracted outcomes relating to physical, mental and general health/ill health measured using a validated instrument. We also extracted secondary outcomes (including sickness absence, health service usage, behavioral changes, accidents, work-life balance, quality of life, health and wellbeing of children, family members and co-workers) if reported alongside at least one primary outcome.

Data collection and analysis: Two experienced review authors conducted data extraction and quality appraisal. We undertook a narrative synthesis as there was substantial heterogeneity between studies.

• 159


Main results: Ten studies fulfilled the inclusion criteria. Six CBA studies reported on interventions relating to temporal flexibility: self-scheduling of shift work (n = 4), flexitime (n = 1) and overtime (n = 1). The remaining four CBA studies evaluated a form of contractual flexibility: partial/gradual retirement (n = 2), involuntary part-time work (n = 1) and fixed-term contract (n = 1). The studies retrieved had a number of methodological limitations including short follow-up periods, risk of selection bias and reliance on largely self-reported outcome data. Four CBA studies on self-scheduling of shifts and one CBA study on gradual/partial retirement reported statistically significant improvements in either primary outcomes (including systolic blood pressure and heart rate; tiredness; mental health, sleep duration, sleep quality and alertness; self-rated health status) or secondary health outcomes (co-workers social support and sense of community) and no ill health effects were reported. Flexitime was shown not to have significant effects on self-reported physiological and psychological health outcomes. Similarly, when comparing individuals working overtime with those who did not the odds of ill health effects were not significantly higher in the intervention group at follow up. The effects of contractual flexibility on self-reported health (with the exception of gradual/partial retirement, which when controlled by employees improved health outcomes) were either equivocal or negative. No studies differentiated results by socio-economic status, although one study did compare findings by gender but found no differential effect on self-reported health outcomes.

• 160


Authors’ conclusions: The findings of this review tentatively suggest that flexible working interventions that increase worker control and choice (such as self scheduling or gradual/partial retirement) are likely to have a positive effect on health outcomes. In contrast, interventions that were motivated or dictated by organizational interests, such as fixed-term contract and involuntary part-time employment, found equivocal or negative health effects. Given the partial and methodologically limited evidence base these findings should be interpreted with caution. Moreover, well-designed intervention studies are needed to delineate the impact of flexible working conditions on health, wellbeing and health inequalities.

Evidence-Based HR Management & Systematic Reviews

Education

Transcript of Evidence-Based HR Management & Systematic Reviews