Evidence-Based Management

Evidence-Based Practice Insights from Key Domains

University of Edinburgh, May 2nd 2014

Eric Barends Rob Briner

A brief history

Current developments

Future developments

Challenges

Implications for practice & research

2

Evidence-Based Management

3

Evidence based management:

A brief history

4

Manager > ‘Manage’

introduced mid 16th century: from Italian maneggiare, “to handle”, based on Latin manus, “hand”.

Management?

(we’re all managers)

History of management/business schools

Late 1800s - started as trade schools

1950s - Ford and Carnegie reports criticised business

schools for not being sufficiently academic or scientific

1980s onwards – criticised for producing research

irrelevant to practice and students without relevant skills

or knowledge

5

On the one hand they fear . . . the scorn of other, more

traditional academic subjects. On the other hand, they

often stand accused of being less than relevant to

business. (Grey, 2001)

2000s onwards – criticized for producing ideas and

students that lead to financial crisis and poor ethical

behaviour (e.g., Enron, only one US president has MBA,

MBA students cheat more than other graduate students)

6

History of management/business schools

Management schools are big business

Nat. Center for Education Statistics (USA, 2011-’12)

20% of all Batchelor’s Degrees

25% of all Master’s Degrees

 Higher Education Statistics Agency (UK, 2012-’13)

27% of all Higher Degrees awarded

18% of all Higher Education qualifications obtained

 UK Council for International Student Affairs (2011-’12)

36% of all business school students are international

30% of all international students studying business7

EBP in Management

Many similar ideas in past (e.g., Mode 1-Mode 2,

collaborative research, rigour and relevance)

Rousseau (2005) Presidential address

Peffer and Sutton (2006) Hard Facts, Dangerous Half-

Truths, and Total Nonsense: Profiting from Evidence-

based Management

8

EBP in Management

EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon

Wiley Evidence-Based Management Insights (2007-’08)

(attempt to create Cochrane database for management)

Briner and Denyer

Presentation to AoM Board of Governors (2008) (attempt to

persuade to develop systematic reviews in management)

Center for Evidence-Based Management – more later

9

Evidence based management:

What is it?

10

11

Evidence-based practice

Central Premise:

Decisions should be based on a

combination of critical thinking

and the ‘best available evidence‘.

12

Evidence?

findings from scientific research,

organizational facts & figures,

benchmarking, best practices,

professional experience

13

All managers base their

decisions on ‘evidence’

14

But…many managers pay

little or no attention to the

quality of the evidence they

base their decisions on

15

Trust me, 20 years of management experience

16

SO ...

17

Teach managers how to

critically evaluate the validity,

and generalizability of the

evidence and help them find

‘the best available’ evidence

18

Evidence-based practice is about making decisions throughthe conscientious, explicit and judicious use of

the best available evidence from multiple sources by

1. Asking: translating a practical issue or problem into an answerable question

2. Acquiring: systematically searching for and retrieving the evidence

3. Appraising: critically judging the trustworthiness and relevance of the evidence

4. Aggregating: weighing and pulling together the evidence

5. Applying: incorporating the evidence in the decision-making process

6. Assessing: evaluating the outcome of the decision taken

to increase the likelihood of a favorable outcome.

Definition

19

Evidence based decision

Professional experience and

judgment

Organizational data, facts and figures

Stakeholders’ values and concerns

Scientific research findings

AskAcquire

AppraiseAggregate

ApplyAssess

20

Evidence-based practice:

Focus on the decision making process

Think in terms of probability

21

Differences?

Research: low internal validity

Controlled vs uncontrolled studies

Differences?

Research: low internal validity

Lack of evidence summaries

Focus on multiple sources

Organizational (BIG) data

Stakeholders’ concerns

How evidence-based are we (managers)?

“I’ve never thought I need more

evidence before making a

decision;

I know what needs to be done,

we get on with it and we get

results.”

25

How evidence-based are we?

959 (US) + 626 (Dutch) HR professionals

35 statements, based on an extensive body of

evidence

true / false / uncertain

HR Professionals' beliefs about effective human resource practices: correspondence between research and practice, (Rynes et al, 2002, Sanders et al 2008)

26

1. Incompetent people benefit more from feedback than

highly competent people.

2. Task conflict improves work group performance while

relational conflict harms it.

3. Encouraging employees to participate in decision

making is more effective for improving organizational

performance than setting performance goals.

True (likely) or false (not likely)?

27

Outcome: not better than random chance28

Evidence based management:

Current developments

29

Postgraduate Course

31

Postgraduate Course

Postgraduate Course

Current developments

Education

Access to research databases

REAs

Building a community

Evidence based management:

Future developments

34

Postgraduate Course

CEBMa Database of Evidence Summaries

Online learning modules

Accreditational bodies

Future developments: practice

Postgraduate Course

Some move towards systematic review methods in management

Limited interest in teaching EBP in management schools

Management schools still a bit trapped: Cash cows for universities; focus on ‘top quality’ academic research; few incentives to be relevant

Future developments: academia

36

Postgraduate Course

Though issues around accountability, ethics, corporate social responsibility could in part be resolved by research and teaching EBP

Need for a professional doctorate?

Future developments: academia

Evidence based management:

Challenges

38

In general

incentives to do non-EBP and punishment (or no incentives) for doing EBP

39

Challenges

(same for practitioners and academics)

Incentives for managers

Not rewarded for doing what ‘works’ – few evaluations

Speed and action valued more highly than accuracy and analysis

Managing and understanding power and politics to get things done more valued than understanding and using evidence to make decisions

It may be too late to change existing senior managers…

40

Barriers from academic / buss school context• In publishing and research

• In teaching content and teaching style

Barriers from organizational contexts• Managers love fads and quick fixes

• Power and politics

41

Other challenges

In general: academics don’t like EBMgt

Ambivalence about the value and applicability of management research

Few incentives to get involved

Primary research (collecting new data) valued more highly than secondary research (reviewing existing data)

EBMgt not academics‘responsibility – this is about practice not research

Some concern that systematic reviews will expose the limited nature of management research

Some academics are like ‘gurus’ and feel that EBMgt might show their claims to be untrue

Why don’t academics like EBMgt?

Espoused and more implicit goals of management school educatorsESPOUSED GOALS To develop critical thinking To help students understand the

full body of knowledge To discuss gaps and limitations

of our knowledge To challenge thinking and

assumptions To educate To teach students how to think

for themselves To maintain quality standards

IMPLICIT GOALS To help students feel successful To select those bits of that are

interesting or digestible or ‘cutting edge’ fads

To reassure students that what we’re teaching them is solid

To make sure students are satisfied

To entertain (edutainment) Teach students what they need

to pass assessments To give ever-higher grades

44

ESPOUSED GOALS To advance scientific

understanding Using the best research

techniques Publishing all results and

replications – unbiased Focus on what’s important Being honest about existing

evidence To disseminate all our

evidence and make publically available

Collaboration & cooperation

IMPLICIT GOALS To advance career Use whatever techniques will

get you published Publishing (mostly) only

positive results, no replications Identifying ‘new’ or trendy

topics – creating empires Exaggerating how much we

know Locking up our evidence

behind publishers’ pay walls Competition for resources,

slots in journals, between universities 45

Espoused and more implicit goals of management school researchers

Implications for research

Less focus on collecting new primary data

More focus on systematic reviews and understanding what we know and don’t know

Focusing research efforts on what is academically and/or practically important without compromising ‘academic freedom’

Broader-based research training – management researchers highly and narrowly specialized

46

WHAT’S STOPING US?In general: managers don’t like EBMgt

Undermines formal authority

They feel it constrains freedom to make managerial decisions

Speed valued and rewarded more than accuracy

Feel they cannot use their own experience and judgment (not true)

Managers not necessarily rewarded for doing what works (organizations rarely evaluate)

THEY LOVE FADS & QUICK FIXES

Why don’t managers like EBMgt?

Huge (peer) pressure to adopt fads & quick fixes

Fads & quick fixes

Postgraduate Course

Postgraduate Course

51

Postgraduate Course

How are fads a problem?

“The main problem…is their lack of any solid

intellectual foundation. Implicit in each fad is a

cause effect statement that is rarely made

explicit and never properly supported.”

52(Donaldson & Hilmer, 1998)

53

54

55George BuckleyJames McNerney

QUICK FIXES

Because quick fixes / fads

Can be career-enhancing for managers

Speed is often valued over accuracy

Do we crave quick and easy solutions?

So who needs or wants academic research?

So why do managers love quick fixes?

Implications for management practice

Focus on accountability

Focus on next generation

Management = profession

EBP > Professional standard >Accreditation

57

Think about this

How can a profession / domain be evidence-based

if it is not managed in an evidence-based way?

58

“No job is more vital to our society than that of a manager. It is the manager who determines whether our social institutions serve us well or whether they squander our talents and resources.” Henry Mintzberg