The Most Significant Change Technique (MSC) Dr Jessica Dart Clear Horizon.
-
Upload
harmony-openshaw -
Category
Documents
-
view
214 -
download
0
Transcript of The Most Significant Change Technique (MSC) Dr Jessica Dart Clear Horizon.
The Most Significant The Most Significant Change Technique (MSC) Change Technique (MSC)
Dr Jessica DartDr Jessica DartClear HorizonClear Horizon
MSCMSC
– Form of qualitative, participatory M&E– Based on ‘stories’ of significant change– Developed by Davies 1996 - Bangladesh – Now used in numerous development
programs and in the public sector
QualitativeQualitative vs quantitative monitoringvs quantitative monitoring
Quantitative
• Focus on measurement• Closed questions • About ‘proving’• Easy to aggregate• Deductive• Static• Goal displacement can be
a problem
Qualitative
• Focus on questioning• Open questions • About learning• Hard to aggregate• Inductive• Dynamic• Goal displacement is not
an issue
• Goal displacement • Creaming• Not about learning• Don’t tell you what you don’t know you
need to know
Limitations of indicator Limitations of indicator based monitoringbased monitoring
Qualitative monitoringQualitative monitoring
• Can be used in conjunction with conventional output monitoring
• Is usually more aimed at learning than accountability
Why stories?Why stories?• People tell stories naturally - indigenous
• Stories can deal with complexity and context
• People remember stories
• Stories can carry hard messages /undiscussables
• But stories not known for accuracy/truth
Use of stories in MSCUse of stories in MSC
• Collection of stories + systematic, collective interpretation = storytelling can be effectively harnessed for participatory evaluation
• Because interpretations tell another story & process has beneficial outcomes for evaluation utilisation
Overview of MSCOverview of MSC
1. Determine sorts of change to monitor
2. Collect stories
3. Review & filter stories regularly
4. Collate ‘selected’ stories for funders review
5. Monitor the process and verify the stories
Overview of MSCOverview of MSC
1. Determine sorts of change to monitor
2. Collect stories
3. Review & filter stories regularly
4. Collate ‘selected’ stories for funders review
5. Monitor the process and verify the stories
ExampleExample
• Target 10 Dairy Extension Project
• Four regions in Victoria, 50 staff
• 1999-2000 trail of the approach
• Still continues today
Step 1- Selection of ‘domains’ of Step 1- Selection of ‘domains’ of changechange
• 3 broad ‘domains’ of changes to be monitored at the project level:
• Changes in on-farm practice • Changes in farmer-decision making skills• Changes in profitability • Any other type of change
• * Not precisely defined
Step 2 - Collect storiesStep 2 - Collect stories• “During the last month, in your opinion, what do you think was the most significant change that took place as a result of the project?”
•The respondent (farmer, extension worker or industry rep) answers in 2 parts:
• 1) descriptive 2) explanatory
Step 3 –Step 3 –Review & filtering processReview & filtering process
• The stories were reviewed by:– The regional committees (every 2-3 months)– Statewide Executive (every 2-3 months)
• The stories are reviewed using a facilitated process at the state and funder levels
Funder meeting
State meetings
Region 1 Region 2 Region 3 Region 4
Story tellers
feedbackflow ofstories
Step 4 - Collate & review Step 4 - Collate & review selected storiesselected stories
1. In total 134 stories were collected - 80% from extension staff
2. A booklet containing 24 selected stories accompanied by the selection criteria & comments
3. Purchasers provide feedback to project after reviewing booklet
Step 5 – Step 5 – Monitor process & verify storiesMonitor process & verify stories
• A database was developed to keep track of all stories
• Secondary analysis at end of reporting period
• In this case selected stories were not verified
• Storytellers were asked to check final stories in report
Impact of MSCImpact of MSC
• Staff gained more fully shared vision
• Process boosted their morale
• Process saw farmers, staff, collaborators sitting together and interpreting qualitative data & casting evaluative judgements
• Project committees became better at conceptualising impact
Use of storiesUse of stories
In addition to reporting, stories were used:– To improve planning– To help explain a point to a farmer– To recruit new participants – To help explain a point to another member
of staff– For PR
MSCMSC vs quantitative monitoringvs quantitative monitoring
MSC• Focus on measurement• Closed questions • Project out• About ‘proving’• Deductive• Static• Inclusive• Central tendencies
Qualitative• Focus on questioning• Open questions • Context in• About learning• Inductive• Dynamic• Selective• Outer edges of experience
Goals
Goal-based evaluation
change
change
change
change
change
change
From the viewpoint
of the Participants
Extent to which they were achieved
From the view point of the program staff + consultation
Program out Context in
From the viewpoint of the Participants
Program
Purpose of MSC in M&E Purpose of MSC in M&E Primary purpose to facilitate improvement by:
focusing direction of work towards explicitly valued directions
eg. what do we really want to achieve and how will we produce more of it?
Contributes to summative evaluation: Information about unexpected outcomes Performance information concerning very best
success stories Can inform criteria used to judge projects
MSCMSC• Creates space for stakeholders to reflect, to make sense of
complex changes
• Provides dialogue to help make sense of each other’s values
• Facilitates dynamic dialogue ie. “what do we really want to achieve and how will we produce more of it?”
• Excellent for participatory programs with diverse, complex outcomes, & multiple stakeholders