Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open...
-
Upload
alexa-daly -
Category
Documents
-
view
215 -
download
0
Transcript of Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open...
![Page 1: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/1.jpg)
Creating Robust Monitoring Systems
Dr Susan DenmanHead of Research, Monitoring & Evaluation
Open Days Workshops “Managing Regional Development”
28-29 September 2004Brussels
![Page 2: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/2.jpg)
Creating Robust Monitoring Systems
How can we ensure that we get high quality data which are useful for programme management and evaluation purposes?
www.wefo.wales.gov.uk
![Page 3: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/3.jpg)
Structure of Presentation
• Welsh context
• Definitions
• Issues at design and planning stages
• Targeting projects for quality
• Issues at data collection and entry stages
• Reporting
www.wefo.wales.gov.uk
![Page 4: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/4.jpg)
www.wefo.wales.gov.uk
![Page 5: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/5.jpg)
The Welsh Context (1)
• Three main programmes:Objective 1
Objective 2 Objective 3
• Four Community Initiatives: INTERREG IIIA, URBAN II, LEADER+ and
EQUAL
www.wefo.wales.gov.uk
![Page 6: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/6.jpg)
The Welsh Context (2)
• Total European Funding of €2,180 million
• Total value of €4,675 million (including private sector match funding)
Largest programme is Objective 1
Main aims:
• to create 43,000 net additional jobs
• to contribute to raising the Gross Domestic Product (GDP)
• reduce economic inactivity
www.wefo.wales.gov.uk
![Page 7: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/7.jpg)
ED&T Group of Welsh Assembly Government
WEFO
ERDFINTERREG
Project Management Policy and Strategy
ESFEAGGF
FIFGLEADER+
EQUAL
LCDURBAN
Finance and Corporate Services
Paying Authority
Finance Payments Article 4 Article 10 IT
RM&EPolicy
BranchCommunic
ationsAssemblyBusiness
![Page 8: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/8.jpg)
RM&E Research, Monitoring & Evaluation Branch
Monitoring and Reporting
4 staff: 1 researcher,3 administrators
Responsibilities:PMC Papers
Annual Monitoring Business PlansAnnual Implementation Reports
Local Partnership Reports
Evaluation
1 researcher,2 research and
evaluation specialists
Responsibilities:Data Quality
Programme Level EvaluationM&E advice and guidance
Head of Branch
Corporate
1 administrator, 1 researcher
Responsibilities:Training
DisseminationSynthesis Papers
Ex-Post EvaluationClosure Reports
![Page 9: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/9.jpg)
Definitions
Monitoring is the regular and systematic examination of the resources, activities and results to inform management decisions
Audit is the verification of the legality and regularity of the implementation of resources.
Evaluation is a broad concept concerning the judgement made on the value of the initiative, which can encompass the needs which have to be met and the effects produced by it. Evaluation concerns not only outcomes but the processes by which they have been produced.
www.wefo.wales.gov.uk
![Page 10: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/10.jpg)
Design and Planning
• Consider monitoring and evaluation at outset• start with ‘what’ not ‘how’• involve users from start• choose indicators with care• ensure hierarchy of indicators is coherent• build capacity
www.wefo.wales.gov.uk
![Page 11: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/11.jpg)
Indicators
Helps if they are:• clearly linked with goal, objectives or targets• cover monitoring and evaluation needs • measurable on a regular basis using reliable
methods• provide clear and simple information• measurable against baselines• congruent with other monitoring indicator
systems
www.wefo.wales.gov.uk
![Page 12: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/12.jpg)
Data Provided by Projects
Projects should:• understand purpose of centrally collected
data• collect other useful data• have access to training, support and
guidance• be clear on definitions• be checked for quality of their plans
www.wefo.wales.gov.uk
![Page 13: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/13.jpg)
Project Sponsor
Gateway
Project development
Submission of application
Project appraisal
projectapproval
Delivery and review
Payments Audit
Closure
WEFO guidance
Advice on prioritisation from Thematic Advisory
Groups
WEFO advise
WEFOappraise
Advice from thematic groups, cross-cutting theme advisors and local partnerships
WEFO monitor
Advice from local partnerships and project officers
Assistance from local partnerships and project officers
Support and aftercare from local
partnerships and project officers
The Project Lifecycle and Monitoringwww.wefo.wales.gov.uk
![Page 14: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/14.jpg)
Data Collection and Storage
• timing and methods• dealing with inaccuracies, gaps, double
counting and under-reporting • designing a good IT system • testing and evaluating the whole monitoring
system• management checks of data entry• liaison with global grants/grant schemes
www.wefo.wales.gov.uk
![Page 15: Creating Robust Monitoring Systems Dr Susan Denman Head of Research, Monitoring & Evaluation Open Days Workshops Managing Regional Development 28-29 September.](https://reader035.fdocuments.net/reader035/viewer/2022070305/5515290255034673228b542a/html5/thumbnails/15.jpg)
Use of data
• involve users in report design• co-ordinate release of data• ensure users understand data limitations • ensure that definitions are known and
understood by all• analyse the data collected• help with data interpretation
www.wefo.wales.gov.uk