Management field experiments Nick Bloom (Stanford and NBER)
www.stanford.edu/~nbloom AOM, August 3 rd 2012
Slide 2
Management field experiments Review of current experimental
literature Our projects in India and China Thoughts on running
field experiments
Slide 3
The reason is running management experiments is expensive Surge
in management experiments, but mainly in: (A) micro-firms (1 or 2
person) in developing countries (B) individual larger firms
Slide 4
Developing countries, micro-firm experiments Karlan &
Valdivia in Peru; Bruhn, Karlan & Schoar in Mexico; Karlan
& Udry in Ghana; McKenzie & Woodruff in Sri Lanka etc.
Provide limited (50 hours) of basic trainings to small firms:
accounting, marketing, pricing, strategy etc. Training is provided
at random and data collected before & after So far finding
not-much impact. I see two potential explanations -management does
not matter in tiny firms, or -intervention is very poor
quality
Slide 5
Single firms in developed countries, (1/2) Growing literature
(surveyed in Bloom & Van Reenen, 2010, Handbook of Labour
Economics) Classic examples include: Lazaer (2000, AER) on
incentive pay at Safelite Glass, Shearer (2004, REStud) on tree
planters and Hamilton et al (2003, JPE) on group incentives in
factories
Slide 6
Single firms in developed countries (2/2) Recently Bandiera,
Barankay and Rasul have an impressive set of papers. Run
experiments on incentives for workers and managers, team selection,
and task division on a fruit farm Introduce changes way through
season (using last season as the control), finding for example
Worker incentive pay increases their performance, especially
absolute (rather than relative) incentives Manager incentive pay
improves team selection (less favoritism) and the effort they put
into monitoring workers
Slide 7
Management field experiments Review of current experimental
literature Our projects in India and China India China Thoughts on
running field experiments
Slide 8
Does management matter? Evidence from India Nick Bloom
(Stanford) Benn Eifert (Berkeley) Aprajit Mahajan (Stanford) David
McKenzie (World Bank) John Roberts (Stanford GSB)
http://www.stanford.edu/~nbloom/DMM.pdf
Slide 9
Management score Source:
www.worldmanagementsurvey.comwww.worldmanagementsurvey.com
2.62.833.23.4 US Japan Germany Sweden Canada Australia UK Italy
France New Zealand Mexico Poland Republic of Ireland Portugal Chile
Argentina Greece Brazil China India One motivation for looking at
management is that country management scores are correlated with
GDP
Slide 10
Source:
www.worldmanagementsurvey.comwww.worldmanagementsurvey.com
Management score US (N=695 firms) India (N=620 firms) Density And
firm management spreads look like TFP spreads
Slide 11
But does management cause any of these TFP differences between
firms and countries? Massive literature of case-studies and surveys
but no consensus Syverson (2011, JEL) no potential driving factor
of productivity has seen a higher ratio of speculation to empirical
study.
Slide 12
We ran an experiment on large firms to investigate the impact
of modern management practices on TFP Experiment on 20 plants in
large multi-plant firms (average 300 employees and $7m sales) near
Mumbai making cotton fabric Randomized treatment plants got 5
months of management consulting intervention, controls got 1 month
Consulting was on 38 specific practices tied to factory operations,
quality and inventory control Collect weekly performance data from
2008 to August 2010, and long-run size and management data from
2008 to 2011
Slide 13
Exhibit 1: Plants are large compounds, often containing several
buildings.
Slide 14
Fabric weaving Exhibit 2: Plants operate continuously making
cotton fabric from yarn
Slide 15
Exhibit 3: Many parts of these Indian plants were dirty and
unsafe Garbage outside the plantGarbage inside a plant Chemicals
without any coveringFlammable garbage in a plant
Slide 16
Exhibit 4: The plant floors were often disorganized and aisles
blocked Instrument not removed after use, blocking hallway. Tools
left on the floor after use Dirty and poorly maintained machines
Old warp beam, chairs and a desk obstructing the plant floor
Slide 17
Yarn piled up so high and deep that access to back sacks is
almost impossible Exhibit 5: The inventory rooms had months of
excess yarn, often without any formal storage system or protection
from damp or crushing Different types and colors of yarn lying
mixed Yarn without labeling, order or damp protection A crushed
yarn cone, which is unusable as it leads to irregular yarn
tension
Slide 18
Intervention aimed to improve 38 core textile management
practices in 6 areas for example Targeted practices in 6 areas:
operations, quality, inventory, HR and sales & orders
Slide 19
Months after the diagnostic phase.2.3.4.5.6
-10-8-6-4-2024681012 Adoption of the 38 management practices rose
Treatment plants Control plants Share of 38 practices adopted
Non-experimental plants in treatment firms Months after the start
of the diagnostic phase
Slide 20
In terms of performance looked at four outcomes we have weekly
data for Quality Inventory Output Productivity (defined as: Log(VA)
0.42*log(K) 0.58*log(L)) Use weekly data from March 2008 until
August 2010 (after which some firms started upgrading to Jacquard
looms)
Slide 21
Poor quality meant 19% of labor went on repairs Workers spread
cloth over lighted plates to spot defectsLarge room full of repair
workers (the day shift) Defects lead to about 5% of cloth being
scrappedDefects are repaired by hand or cut out from cloth
Slide 22
22 Previously mending was recorded only to cross- check against
customers claims for rebates
Slide 23
Now mending is recorded daily in a standard format, so it can
analyzed by loom, shift, design & weaver
Slide 24
The quality data is now collated and analyzed as part of the
new daily production meetings Plant managers meet with heads of
departments for quality, inventory, weaving, maintenance, warping
etc.
Slide 25
Quality improved significantly in treatment plants Control
plants Treatment plants Weeks after the start of the experiment
Quality defects index (higher score=lower quality) Note: solid
lines are point estimates, dashed lines are 95% confidence
intervals
Slide 26
Stock is organized, labeled, and entered into the computer with
details of the type, age and location. Organizing and racking
inventory enables firms to substantially reduce capital stock
Slide 27
Many treated firms have also introduced basic initiatives to
organize the plant floor Marking out the area around the model
machine Snag tagging to identify the abnormalities
Slide 28
28 Spare parts were also organized, reducing downtime (parts
can be found quickly) Nuts & bolts Tools Spare parts
Slide 29
Production data is now collected in a standardized format, for
discussion in the daily meetings Before (not standardized, on loose
pieces of paper) After (standardized, so easy to enter daily into a
computer)
Slide 30
TFP rose in treatment plants vs controls Control plants
Treatment plants Weeks after the start of the experiment Total
factor productivity Note: solid lines are point estimates, dashed
lines are 95% confidence intervals
Slide 31
Why do badly managed firms exist? Competition heavily
restricted by trade restrictions, the difficulty of new firms
entering (finance is hard to raise), and the difficulty of good
current firms expanding (limited by family size) Information is
limited: firms either not aware of modern practices or simply do
not believe they matter (not worth the it)
Slide 32
Management field experiments Review of current experimental
literature Our projects in India and China India China Thoughts on
running field experiments
Slide 33
Does Working from Home Work? Evidence from a Chinese Experiment
Nick Bloom (Stanford) James Liang (Ctrip & Stanford) John
Roberts (Stanford) Jenny Ying (Stanford)
http://www.stanford.edu/~nbloom/WFH.pdf
Slide 34
34 CTrip, a large NASDAQ listed Chinese multinational, wondered
about introducing working from home CTrip, Chinas largest
travel-agent (13,000 employees, and $5bn value on NASDAQ). James
Liang is the co-founder, first CEO and Chairman
Slide 35
The randomization into working from home was done publicly and
also shown on the firm intranet Open lottery over even/odd
treatment Working at Home Working at home Working at Home
Slide 36
Experiment yielded four learnings for the firm: (1)
Working-from-home works (on average) Normalized calls per week
Before the experimentDuring the experiment Control Treatment
Slide 37
Experiment yielded four learnings for the firm: (2) Better
& worse workers both improve when WFH Normalized calls per
week: difference between home and work Before experimentDuring
experiment
Slide 38
Experiment yielded four learnings for the firm: (3) Selection:
Worker choice increases WFH impact Normalized calls per week:
difference between home and work During the experiment After the
experiment (roll-out) Before the experiment
Slide 39
Experiment yielded four learnings for the firm: 4) Employees
value WFH as attrition down 50% Note: average daily commute is 1.41
hours and cost $0.96
Slide 40
Experiment so successful that CTrip is rapidly rolling out WFH
across the firm Profit increase per employee WFH about $2,000 per
year: Rent: $1,200 per year Retention: $400 per year Labor costs:
$300 per year
Slide 41
Management field experiments Review of current experimental
literature Our projects in India and China India China Thoughts on
running field experiments
Slide 42
Thoughts on experiments 1.Expensive and hard, but worthwhile
for the right question 2.Risky for junior faculty as can take many
years 3.Think about both measure and intervention both can be tough
(in India measuring the control firms was tough) 4.Works best as a
team effort design, funding and execution all best as joint
production 5.Running pilots and spending time on the ground
invaluable for effective operation, analysis and presentation
Slide 43
Management field experiments Nick Bloom (Stanford and NBER)
www.stanford.edu/~nbloom AOM, August 2012