Curtis Cartwright Programme Evaluation Jan2010
description
Transcript of Curtis Cartwright Programme Evaluation Jan2010
![Page 1: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/1.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 1
Evaluation
Dr Geoff CurtisDr Rob Hawtin
29 January 2010
Institutional Innovation Programme
![Page 2: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/2.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 2
Overview
• Institutional Innovation Programme (IIP)
• Evaluation questions
• Headlines from evaluation of Round 1 projects
• Initial findings
• Lessons identified
• Evaluation of Round 2 projects
• Approach
• Your contribution
• Self-evaluation guidance
![Page 3: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/3.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 3
Institutional Innovation Programme
• Issues with JISC approach to ICT in education and research
• Often applies to specific problems of department / institution
– Not the sector
• Solution often technology- rather than business-led
• Not necessarily sustainable
![Page 4: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/4.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 4
IIP concept
• Institutional approach
• Build on advances from JISC innovation activities
– Proven practices, technologies, standards and services
– Often substantial internal business change for institution
– Supported by significant senior management involvement
• Fund range of substantial ICT deployments/business change activities
• Use as exemplars to demonstrate innovation and good practice
• Extend benefits to sector
– Benefits realisation activities
• Problem areas identified
– professional bodies
– other initiatives
![Page 5: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/5.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 5
Top-level evaluation questions
• 1: Is IIP concept useful as general approach for institution-wide
problems?
• 2: Is IIP delivering desired outcomes at institutional level for key
areas of institutional ICT concern?• Improved community engagement• Improved and sustained JISC leadership to the sector• Improved capacity, knowledge and skills and diversity of background;• Sustainable technological solutions• Development of good practice approaches• Evidence base for good practice and dissemination• Overall sustainability of outcomes
• 3: Is IIP being managed well?
![Page 6: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/6.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 6
Evaluation
• Assessment of evaluation questions
• Lessons identified?
• Improvements?
• What else?
• Evaluating programme not the projects
![Page 7: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/7.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 7
Initial findings –IIP concept
• Assessment
• Useful general approach for creating sustainable change
• More general applicability
• Lesson identified
• Need to select/manage projects to meet IIP characteristics
• Areas for improvement
• Benefits realisation activities
• More stringent selection of areas of institutional concern and projects
• Training on concept and implementation
• Review approach to planning/funding of benefits realisation activities
![Page 8: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/8.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 8
Initial findings – IIP delivery
• Assessment
• Too early for definitive assessment
– SSBR had only just started
– Likely to mature post project completion
– Aspirational nature of outcomes
• Continue to focus on desired outcomes
![Page 9: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/9.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 9
Initial findings – IIP management
• The programme is generally well managed
• Lack of programme management resource and overall scale of Round 2
projects might be a risk to programme
• Focus role of SSBR
![Page 10: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/10.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 10
Initial findings – summary
Question After Round 1 Potential
1 IIP concept
2 IIP delivery
3 IIP management
Overall
![Page 11: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/11.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 11
Evaluation of IIP – phase 2
• Approach
• Phase 1 was by interviews against detailed questions
– Many more Round 2 projects
• Evaluation based on
– Questionnaires– Detailed evaluation questions as before– Quantitative and qualitative– Stakeholders– Project teams
– Follow-up interviews, where necessary
– Other available information– Programme meetings– Final and completion reports
– Self-evaluation
![Page 12: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/12.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 12
Evaluation of Round 2 projects
• Your contribution
• Please find time to respond to questionnaires
– End February
• Please do your final and completion reports
– Good things
– Lessons identified
![Page 13: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/13.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 13
Self-evaluation
• JISC evaluation handbook
• Based on your evaluation plans
• Tips on self-evaluation
• Outcomes
![Page 14: Curtis Cartwright Programme Evaluation Jan2010](https://reader035.fdocuments.net/reader035/viewer/2022081413/5492104db47959e9728b4597/html5/thumbnails/14.jpg)
www.curtiscartwright.co.uk CC382D006 – 1.0 - 14
Contact us
• [email protected] 01483 685023
• [email protected] 01483 685024