From Flexible to Formal

Campaign Infrastructure:

The Implication for Evaluators

Evaluation 2015: Exemplary Evaluations in a Multicultural World

November 13, 2015

Presenters:

Deepti Sood, TCC Group @DSood26

Kate Locke, TCC Group @TCCKate

Session Goals

1) Provide background on considerations for

evaluating campaigns

2) Introduce 5 types of evaluator roles

3) Share examples of real campaigns and the

pitfalls and successes of the evaluator role

Our History

• 35 years engaging in evaluation, capacity building, and strategic planning

• Evaluation practice has a range of advocacy clients

• Our roles as campaign evaluators range in depth, involvement, and focus

Campaign Considerations 101

• Who is funding the work?

• Who are you accountable to?

• Who is the audience for the evaluation work?

• How many campaign members are there?

Campaign Considerations 101

• How engaged are campaign members?

• What is the purpose of the evaluation?

• What is the scope of the campaign? What are the targets? What is the overall goal?

5 Evaluation Models in Campaigns

1) Retrospective

2) Formative

3) Developmental

4) Technical Assistance

5) Program Officer

Model 1: Retrospective

• Role

– Establish what happened and how the campaign contributed

• Benefits

– Know endgame

– Not urgent

– Engage in interpretation with funder

• Challenges

– Benefit to the grantees?

– Quality of data --- mostly grant reports?

– Access/reconstructing from the outside

– Perspective

• Considerations

– Intention of funder

– Applicable to other campaigns?

Model 1: Retrospective

Model 2: Formative

• Role – Provide timely evaluation

feedback to inform campaign strategy

• Benefits – Greater opportunity for

impact – Greater opportunity for

grantees to buy-into the organization

– Data sources can be stronger

• Challenges – Timeliness of data

collection and analysis – Conclusions may be

inaccurate – Creating a flexible scope of

work – Digesting the implications

• Considerations – How willing are the

funders/actors willing to make course corrections?

– Will it be 100% formative?

Model 2: Formative

Model 3: Developmental

• Role – Serve as an embedded

evaluator and thought partner

• Benefits – Being a thought partner – Build relationships and

trust – Being responsive to

emerging needs

• Challenges – Role clarity – Structuring time & level of

engagement – Internal knowledge

management – Balancing data collection

burden with the campaign – Sensitivity to power dynamics

• Considerations – Are you 100%

developmental? – Campaign reliance/lack of

capacity building

Model 3: Developmental

Model 4: Technical Assistance

• Role – Build the campaign’s

capacity to track and analyze data

• Benefits – Helping build campaign

capacity and sustainability – Clear benefit for campaign

actors

• Challenges – A lot of upfront time/work – Different starting places – Campaign member

accountability – Buy-in

• Considerations – Can you give

recommendations? – Who gets to participate? – How much does TA benefit

organizations vs the campaign?

Model 4: Technical Assistance

Model 5: Program Officer

• Role

– Establish monitoring frameworks and grantee reporting tools

• Benefits

– Ability to scale tracking systems

– Ensuring data quality

– Ensuring comparability

– Control and relevance of the metrics

• Challenges

– Go between

– Limited buy-in

– Campaign accountability to providing data

• Considerations

– What happens if have recommendations?

– Acknowledging campaign capacity for data collection

Model 5: Program Officer

Discussion Retrospective:

Establish what happened and how the campaign

contributed

Formative: Provide timely evaluation

feedback to inform campaign strategy

Developmental: Serve as an embedded

evaluator and thought partner

Technical Assistance: Build the campaign’s capacity to track and analyze data

Program Officer: Establish monitoring frameworks and grantee reporting tools

CONTACT US

Deepti Sood dsood@tccgrp.com @DSood26

Kate Locke klocke@tccgrp.com

@TCCKate

@TCCGRP #EVAL15