Public policy innovation at the interface between central ... · between central steering and local...
Transcript of Public policy innovation at the interface between central ... · between central steering and local...
PhD Training School, Paris, 1st October 2013
Public policy innovation at the interface
between central steering and local
autonomy
Jostein Askim, Department of Political Science, University of Oslo
Outline of the argument
• Policy innovation under multi-level governance– at the
interface between central steering and local autonomy
• Problem: trade-off between fostering innovative
practices at the local level and producing generalizable
knowledge
• Two ways of thinking about knowledge production and
policy innovation – relative strengths and weaknesses
• Mixed model – empirical manifestations – lessons
Policy innovation and MLG
• Policy innovation = generating new policies (interventions),
testing and verifying their results, and diffusing effective ones
• Policy innovation is always challenging but some challenges
are especially present in MLG settings
• MLG: central and local governments share responsibility for the
development, implementation and improvement of policies
• Central governments have primary responsibility for policies,
but have
– ideational limitations due to lack of proximity to practice and
– restricted authority and
– legitimacy to install change locally
Policy innovation and MLG (cont.)
• Local governments can generate new interventions but have
– restricted overview and
– their informal horizontal learning is error prone (de- and re-coding
experiences)
• “The laboratory of local governments” works in the sense that
new solutions are developed and implemented, and cross-polity
learning is active
• … but lessons have poor internal and external validity, and
lesson-drawing is therefore risky
Challenge for constructive research
• Most definitions of innovation equate innovation and change -
play down the newness and effectiveness of what is diffused
• Challenge: How to encourage innovative practices at local level
AND extract generalizable, sticky knowledge from innovative
practices
• Aid to thought: Two models of knowledge production
contrasting design templates for policy innovation programs
mixed model with empirical manifestations we can learn from
• Synthesis of literatures: innovation, evaluation, local
government, sociological neo-institutionalism
www.akf.dk
Experimental model Embedded model
Knowledge
objective
Better knowledge of the effects of
interventions
Better knowledge of the development of
interventions and their applications
Process of
discovery
Randomized, controlled experiment, ex
post to the development of intervention
Heuristic. Experience based problem
solving, learning and discovery
Relation to
generation of
interventions
Intervention is designed ex ante to the
experiment. The source of the
innovation is not important.
Intervention is designed in the
experiment. The source of the innovation
is the participants’ praxis and experience.
Criteria for valid
knowledge
Solid knowledge of effects and
causality
True and inspiring depiction of process,
outcome and context
Functional
requirements
Control the experiment.
Participants selected in a randomized
way; distinction between effect and
control group.
Measurement validity, internal validity
(causality versus correlation), external
validity and replicability
Nurture the innovative and
entrepreneurial spirits of participants
Participants self-select/volunteer based
on interest in and ability to conduct the
experiment.
Models of experience-based knowledge production
www.akf.dk
Experimental model Embedded model
Knowledge
objective
Better knowledge of the effects of
interventions
Better knowledge of the development of
interventions and their applications
Process of
discovery
Randomized, controlled experiment, ex
post to the development of intervention
Heuristic. Experience based problem
solving, learning and discovery
Relation to
generation of
interventions
Intervention is designed ex ante to the
experiment. The source of the
innovation is not important.
Intervention is designed in the
experiment. The source of the innovation
is the participants’ praxis and experience.
Criteria for valid
knowledge
Solid knowledge of effects and
causality
True and inspiring depiction of process,
outcome and context
Functional
requirements
Control the experiment.
Participants selected in a randomized
way; distinction between effect and
control group.
Measurement validity, internal validity
(causality versus correlation), external
validity and replicability
Nurture the innovative and
entrepreneurial spirits of participants
Participants self-select/volunteer based
on interest in and ability to conduct the
experiment.
Models of experience-based knowledge production
www.akf.dk
Experimental model Embedded model
Knowledge
objective
Better knowledge of the effects of
interventions
Better knowledge of the development of
interventions and their applications
Process of
discovery
Randomized, controlled experiment, ex
post to the development of intervention
Heuristic. Experience based problem
solving, learning and discovery
Relation to
generation of
interventions
Intervention is designed ex ante to the
experiment. The source of the
innovation is not important.
Intervention is designed in the
experiment. The source of the innovation
is the participants’ praxis and experience.
Criteria for valid
knowledge
Solid knowledge of effects and
causality
True and inspiring depiction of process,
outcome and context
Functional
requirements
Control the experiment.
Participants selected in a randomized
way; distinction between effect and
control group.
Measurement validity, internal validity
(causality versus correlation), external
validity and replicability
Nurture the innovative and
entrepreneurial spirits of participants
Participants self-select/volunteer based
on interest in and ability to conduct the
experiment.
Models of experience-based knowledge production
www.akf.dk
Experimental model Embedded model
Knowledge
objective
Better knowledge of the effects of
interventions
Better knowledge of the development of
interventions and their applications
Process of
discovery
Randomized, controlled experiment, ex
post to the development of intervention
Heuristic. Experience based problem
solving, learning and discovery
Relation to
generation of
interventions
Intervention is designed ex ante to the
experiment. The source of the
innovation is not important.
Intervention is designed in the
experiment. The source of the innovation
is the participants’ praxis and experience.
Criteria for valid
knowledge
Solid knowledge of effects and
causality
True and inspiring depiction of process,
outcome and context
Functional
requirements
Control the experiment.
Participants selected in a randomized
way; distinction between effect and
control group.
Measurement validity, internal validity
(causality versus correlation), external
validity and replicability
Nurture the innovative and
entrepreneurial spirits of participants
Participants self-select/volunteer based
on interest in and ability to conduct the
experiment.
Models of experience-based knowledge production
www.akf.dk
Experimental model Embedded model
Knowledge
objective
Better knowledge of the effects of
interventions
Better knowledge of the development of
interventions and their applications
Process of
discovery
Randomized, controlled experiment, ex
post to the development of intervention
Heuristic. Experience based problem
solving, learning and discovery
Relation to
generation of
interventions
Intervention is designed ex ante to the
experiment. The source of the
innovation is not important.
Intervention is designed in the
experiment. The source of the innovation
is the participants’ praxis and experience.
Criteria for valid
knowledge
Solid knowledge of effects and
causality
True and inspiring depiction of process,
outcome and context
Functional
requirements
Control the experiment.
Participants selected in a randomized
way; distinction between effect and
control group.
Measurement validity, internal validity
(causality versus correlation), external
validity and replicability
Nurture the innovative and
entrepreneurial spirits of participants
Participants self-select/volunteer based
on interest in and ability to conduct the
experiment.
Models of experience-based knowledge production
Trade-offs
Experimental model Embedded model
Main
strength
Improve field
Production of knowledge with
high internal and external validity
Improve entity
Production of local knowledge
Legitimacy emanating from praxis
Main
weakness
Distance to practice Low internal and external validity
Risk of learning the wrong lesson
Key success
factor Control the experiment Nurture the spirits of inventors and
entrepreneurs
Design templates for MLG policy innovation
Experimental model Embedded
If necessary:
Definition of
exemptions
(problem)
Coordinating agent selects regulations
from which exemptions can be granted.
Coordinating agent agrees exemptions
with line ministries prior to local level’s
involvement
Local level selects regulations from which
exemptions are necessary to develop policy
solutions.
Exemptions agreed case-by-case in dialogue
between applicants and line ministries,
brokered by coordinating agent.
Definition of
intervention
(solution)
Interventions (policy solutions) are
defined ex ante by coordinating agent
and applied at the local level
Interventions are created as part of local
experiments. Some but not all necessitate
exemptions from regulations
Selection of
participants
Selection based on criteria that facilitate
generalization of results (median or
critical case, etc.). Limited number of
participants is necessary to facilitate tight
monitoring of experiments.
Self-selection facilitates inclusion of
enthusiastic and able participants. Such field
leaders enable ex post diffusion of
innovations. Large number of participants
unproblematic.
Verification of
results
Evaluation as a scientific study of the
causal effects of interventions.
Experiments – de-contextualization to
distinguish “signal from noise”.
Evaluation as context-sensitive
documentation and description of the
experimental process and outcome.
Design templates for MLG policy innovation
Experimental model Embedded model
If necessary:
Definition of
exemptions
(problem)
Coordinating agent selects regulations
from which exemptions can be granted.
Coordinating agent agrees exemptions
with line ministries prior to local level’s
involvement
Local level selects regulations from which
exemptions are necessary to develop policy
solutions.
Exemptions agreed case-by-case in dialogue
between applicants and line ministries,
brokered by coordinating agent.
Definition of
intervention
(solution)
Interventions (policy solutions) are
defined ex ante by coordinating agent
and applied at the local level
Interventions are created as part of local
experiments. Some but not all necessitate
exemptions from regulations
Selection of
participants
Selection based on criteria that facilitate
generalization of results (median or
critical case, etc.). Limited number of
participants is necessary to facilitate tight
monitoring of experiments.
Self-selection facilitates inclusion of
enthusiastic and able participants. Such field
leaders enable ex post diffusion of
innovations. Large number of participants
unproblematic.
Verification of
results
Evaluation as a scientific study of the
causal effects of interventions.
Experiments – de-contextualization to
distinguish “signal from noise”.
Evaluation as context-sensitive
documentation and description of the
experimental process and outcome.
Respective foci and strengths
Discovery Testing Diffusion
Embedded model
Experimental model
Self-fuelled
High-risk
Low-risk
Needs fuel
Experimental model Embedded model
Definition of
exemptions
(problem)
Coordinating agent selects regulations
from which exemptions can be granted.
Coordinating agent agrees exemptions
with line ministries prior to local level’s
involvement
Local level selects regulations from which
exemptions are necessary to develop policy
solutions.
Exemptions agreed case-by-case in dialogue
between applicants and line ministries,
brokered by coordinating agent.
Definition of
intervention
(solution)
Interventions (policy solutions) are
defined ex ante by coordinating agent
and applied at the local level
Interventions are created as part of local
experiments. Some but not all necessitate
exemptions from regulations
Selection of
participants
Selection based on criteria that facilitate
generalization of results (median or
critical case, etc.). Limited number of
participants is necessary to facilitate tight
monitoring of experiments.
Self-selection facilitates inclusion of
enthusiastic and able participants. Such field
leaders enable ex post diffusion of
innovations. Large number of participants
unproblematic.
Verification of
results
Evaluation as a scientific study of the
causal effects of interventions.
Experiments – de-contextualization to
distinguish “signal from noise”.
Evaluation as context-sensitive
documentation and description of the
experimental process and outcome.
Mixed
model
Mixed model
• Square the circle?
• Combine strengths of embedded and experimental models of
knowledge production?
Empirical manifestations of mixed-model
knowledge production
• Benchmarking programs
• Piloting schemes
• «Right to challenge» schemes (DK)
• Free commune experiments (FCEs) – most
• More on FCEs
– What is it?
– Does it work?
– Lessons for policy innovation in MLG settings
FCE = a recipe for policy innovation at the
central steering/local autonomy interface
Program theory:
• A limited number of LGs are granted temporal exemptions from
selected national regulations, in order to try out ”illegal” policy
interventions. Successful innovations diffused.
• LGs invited to apply to be included in the program CG
selects participants based on criteria.
• Then LGs apply for waivers to conduct local experiments. CG
assesses these based on criteria. Approvals made valid for all
program participants
• Five cases: Sweden 84-92, Denmark I 85-93, Norway 86-92,
Finland 88-96, Denmark II 2012-
FCE experiences and lessons
• Borrowed legitimacy from both science and praxis
• Varieties in pre-definition of scope for local experiments,
selection and evaluation practices, predominantly embedded
approaches
• Some diffusion vertically though national rulemaking and local
implementation, both on the back of- and absent systematic
evaluation
• A lot of horizontal, voluntary diffusion, LGs readily learn from
context-sensitive (success) stories
• Highly effective as generators of new policy interventions, less
effective as test-beds for their effects
• Potential for generalization – limited by political context
Lessons: Getting the mix right (1/2)
Discovery and verification
• How to design policy innovation programs that can both
encourage innovation and extract generalizable results and
thereby encourage less risky diffusion of sticky innovations?
• Sequential combination strategies
• Two-step or interlinked programs
• Cf. Finnish FCE experience
• Simultaneous combination strategies
• Layered programs
• Cf. Swedish FCE experience
• Avoid selection bias in favor of field-leadning municipalites –
representativeness vs. legitimacy of results
• More pre-defining and down-scoping of problems and solutions
for local experiments – cf. signal/noise and ”evaluability”
• Evaluation practice that combines effects analysis and thick
description – legitimcacy
Lessons: Getting the mix right (2/2)
Safer diffusion of sticky innovations
Summary
• Policy innovation under multi-level governance involves
trade-off between fostering innovative practices at the
local level and producing generalizable knowledge
• Two ways of thinking about knowledge production and
policy innovation – relative strengths and weaknesses
• Mixed model – empirical manifestations – lessons
• Necessary to have critical perspective on both purely
embedded and purely experimental approaches to
policy innovation
FCEs and templates for policy innovation (1/2)
Experimental Embedded Definition of
exemptions
(problem)
Coordinating agent selects
regulations from which exemptions
can be granted.
Coordinating agent agrees
exemptions with line ministries prior
to local level’s involvement
Local level selects regulations from
which exemptions are necessary to
develop policy solutions.
Exemptions agreed case-by-case in
dialogue between applicants and line
ministries, brokered by coordinating
agent.
Definition of
intervention
(solution)
Interventions (policy solutions) are
defined ex ante by coordinating
agent and applied at the local level
Interventions are created as part
of local experiments. Some but
not all necessitate exemptions from
regulations
FCEs and templates for policy innovation (2/2)
Experimental Embedded Selection of
participants
Selection based on criteria that
facilitate generalization of results
(median or critical case, etc.).
Limited number of participants is
necessary to facilitate tight
monitoring of experiments.
Self-selection facilitates inclusion of
enthusiastic and able participants. Such
field leaders enable ex post diffusion of
innovations. Large number of
participants unproblematic.
Verification of
results
Evaluation as a scientific study of
the causal effects of interventions.
Experimental to distinguish “signal
from noise”.
Evaluation as context-sensitive
documentation and description of the
experimental process and outcome.
?