Post on 27-Dec-2015
Using Outcome Mapping as a Using Outcome Mapping as a Research Methodology for Information Research Methodology for Information Communication Technology ProjectsCommunication Technology Projects
Chris Morris African Advanced Institute for Information & Communications Technology (AAIICT)
Comparative study of “first mile” and Comparative study of “first mile” and “first inch” technology in different low “first inch” technology in different low
density contexts.density contexts.
Project funded by the IDRC
MozambiqueMozambique
AngolaAngola
South AfricaSouth Africa
ZimbabweZimbabwe
FMFI Projects & PartnersFMFI Projects & Partners
Angonet (Huambo)
Wireless Mesh (Peebles Valley)
CSIR Telehealth (Eastern Cape)
UWC MuTi (Eastern Cape)
CSIR HLT (Openphone)
Translate.org (HTML Pub)
Power Line Comms (Rooiwal)
SchoolNet Mozambique (Inhambane)
MICTI (Maputo)
UCM (Pemba)
ISPU (Quelimane)Wireless
(Harare)
ObjectivesObjectives
The objectives of this project are: To develop innovative ICT technologies and to implement “first
mile” solutions To develop business models for first mile solutions To research behaviour in the use of ICTs – how the use of ICTs
has changed community life Scalability and replicability of technologies – the viability of
rolling out the solution• Influence on policy and regulation – demonstration of project
benefits to the regulator To publish a reference book for “first mile” and “first inch”
implementation in rural Africa.
Research Question? Research Question?
• What is the sector you want to contribute to through your project? How does it link to the overall vision of the First Mile/First Inch Network?
• What is it that you want to make known through your project? (What is the research question?) Why?
• Who are you going to influence through your project? • What kinds of influence are you intending to have on your boundary
partners’ behaviours? • What are the main methods by which you are going to answer your
research question? Why did you select these methods? (relates to issue of validity)
• What are the main strategies you are going to use to influence your boundary partners? Why did you select these strategies? (relates to issue of relevance)
• How would you propose to monitor and report on the project process as it unfolds?
Choice?Choice?
• research methodology that was simple to apply• robust in being able to provide an implementation framework but
flexible enough to adapt• we needed to be participatory with our project partners• to understand the development process and learn more about
human behaviour and how that influences technology choice• we are a learning organisation, we wanted to look at the change
taking place, not simply make a claim to it. • a tool that helped us document the journey & understand the
processes - what worked and what did not work and why• one that would develop a space for social learning and creative
dialogue
Choice contd. ?Choice contd. ?
• we needed to be able to reflect and review our plans and make changes when necessary.
• at the end of the project we needed to know where we have been and what were the lessons learned.
• we needed to know how to improve next time. • to be accountable to our stakeholders and donors.• we needed to undertake comparative research across projects,
across countries.• we needed a dissemination strategy - identify Primary Intended
Users and Primary Intended Uses of research results.
Why Outcome Mapping?Why Outcome Mapping?
Outcome Mapping methodology for planning, monitoring, and evaluation in order to document, learn, and report results
Development is accomplished by, and for, people
Focuses on outcomes as behavioural change
Not impact but contributions to outcomes
Boundary partners control change
Outcomes MappingOutcomes Mapping
OM provides a method for development programmes to plan for and assess the capacities that they are helping to build in the people, groups, and organizations that will ultimately be responsible for improving the well-being of their communities.
Outcomes are defined as changes in the behaviour, relationships, activities, or actions of the people, groups, and organizations with whom a program works directly
Boundary partners are those individuals, groups, and organizations with whom the program interacts directly and with whom the program anticipates opportunities for influence
Outcome Mapping…Cont.Outcome Mapping…Cont.
Outcome Mapping asks the following questions :
Why? (What is the vision to which the program wants to contribute?);
Who? (Who are the program's boundary partners?);
What? (What are the changes that are being sought?); and
How? (How will the program contribute to the change process?).
Why?Why?
• VISION This inspired ambitious action and commitment, - but was not an
yardstick by which the project would be measured
Who? - Boundary Partner MapWho? - Boundary Partner Map
Tsilitwa Health Project
Doctor(Hospital)
Clinic sister
DistrictManager
ProvincialDoH
NationalDoH
MEC
Community(Patients)
USAICASA
CSIRResearchers
WorldBank IDRC
Outcome ChallengesOutcome ChallengesBoundary partner – Clinic sister
The project intends to see the clinic sister empowered by the use of the technology and sharing her knowledge with other clinic sisters in the area and seeking advice from specialist. The clinic sister will build a trust relationship with the doctors and be recognised by other health officials for inputs into decision-making. The sister is able to clearly articulate a vision of telehealth activities and goals that is relevant to her community needs. She acts as a champion for the roll-out of telehealth in the province and motivates others.
Boundary partner – hospital doctor
The project intends to see the doctors supporting and participating in the use of the technology. The doctors will join the online community of health specialists and share case studies and lessons learnt. The doctor will build a trust relationship with the clinic sister and help build her skills and knowledge.The doctor will access health information on the internet and continue with his professional training online. The doctor will use the internet to keep in professional and social contact with colleagues and friends around the world thereby reducing some of the feelings of isolation experienced from being in a deep rural location.
Boundary partner – East London hospital specialist
The project intends to see the dermatologist based at the East London hospital responding to the clinic sister’s e-mails and providing advice. on a timely basis. One would also like to see a proactive stance from the doctor in terms of devolving first line diagnostic decisions to the clinic level, based on the learning experience of clear-cut cases. The dermatologist will build up case studies and share and promote the technology at conferences and medical events.
Boundary partner – Provincial Department of Health
The project intends to see Dr Rajeev become an enthusiastic supporter for the telehealth technology. He will influence the Department to adopt the Tsilitwa telehealth project and seek to replicate it throughout the Eastern Cape Province..The rural telehealth solution will be included in the Department’s strategy and budget. The Department will develop training modules for the clinic sisters and provide ongoing support. Dr Rajeev will become a champion for rural connectivity and inspire other provinces and the National Department of Health to replicate the technology.
What? - Progress markersWhat? - Progress markers
Outcomes have been assigned to each boundary partner & progress markers are used to monitor behavioural changes of boundary partners in achieving the desired outcome. The Progress Markers for the Outcome Challenges are categorised according to the current status where:
• Expect to see = Behaviours the project expects to see
• Like to see = Behaviours the project would like to see
• Love to see = The behaviours associated with achieving the best outcome
Progress Marker: Clinic SisterProgress Marker: Clinic Sister
Clinic Sister
Expect to See Like to See Love to See
Clinic sister not afraid to use the system and
feels like it is a benefit Sister becoming proficient in use of the technology and suggests improvements
Sisters becomes a champion for rural telehealth, she develops self-confidence - talking at conferences, , motivating others, advocacy and training
Sister feels confident to communicates with the Nessie Knight hospital doctor using the technology
Some communication with other provincial doctors and specialists (Umtata, East London)
Sister builds up experience and a body of case studies, reading and ongoing skills development on the internet (Consulting medical documents for improved health care) sharing new knowledge with other health workers
The sister’s experience grows and she performs simple diagnoses of patients with some support by the doctor
The clinic sister feels confident and empowered showing off and training other healthworkers.
Sister gives inputs into provincial telehealth
discussions and influences telehealth plans
The community are not afraid to be diagnosed through the use of the technology
Sister accesses the internet for information and uses it for ongoing skills development
Sister articulates a vision of telehealth to the community
How? - Strategy MapHow? - Strategy Map
Strategy Causal I-1 Persuasive I-2 Supportive I-3
Strategies and activities aimed at a specific individual or group
Supply ICT equipment
Deliver training to the
sister
Make sure the system
is reliable
Provide technical
support
E-mail from doctors
requesting cases
Regular contact with
sister from CSIR
Brief the DoH
(Hospital super,
matron, DM, Dr
Rajeev, MEC with
regard to the system
to ensure ongoing
support for the sister
E-1 E-2 E-3
Strategies and activities aimed at individual’s or group’s environment
Include Telehealth in
sisters KRAs
Telehealth ICT policy
Frequent visitors to see
telehealth UNDP country visits
Promote the project through the media
Collaboration with other organisations
OM 3 STAGESOM 3 STAGES
• Intentional Design• Why? Who? What? How?
• Outcome & Performance Monitoring• Outcome journals• Strategy journals• Performance journal
• Evaluation Planning
Organisational PracticesOrganisational Practices
• Prospecting for new ideas• Opportunities and resources• Seeking feedback from key informants• Obtaining support from your next highest power• Assessing and redesigning products, services, systems and procedures• Checking on those already served to add value• Sharing your best wisdom in the world• Experimenting to remain innovative• Engaging in organisational reflection
Evaluation Evaluation
• Assess contribution to development outcomes• Understand linkages between behavioural changes and
strategies• User uptake, benefits of ICTs• Influence on regulatory policy
• Documentation of lessons learned• Understand intended uses and intended users• By using OM as a framework it allows comparison
across cases, across countries
Dissemination StrategyDissemination Strategy PI Users PI Uses PI Users PI Uses
Influence on regulatory policy : Department of Communication• Create license exemption for use of wifi in research
• Create license exmption for use of wifi for non-commercial use e.g. health & education
• Dissemination strategy – inner circle, LINK, DoC, NGOs, University
Influence on health policy : Department of Health Eastern Cape• Support for the use of wifi for telehealth in rural clinics
• Demonstrate value of eHealth training module
• Focus group key DoH decision makers,Provincial Head, Supply Chain Manager
ConclusionConclusion• OM as a research tool enabled us to answer the research questions – what is it that
you want to make known that was not known before? The research produced primary data in discovering something new. OM provided the route map and data gathering tools to produce evidence based results. The evaluation of these research results enabled us to answer the research question user uptake, influence on policy
• OM provided a methodology for the comparative analysis between projects, between countries
• OM provided the framework for planning, documenting the journey, and capturing the lessons learned during the life of the project. It helped the project team document “where it had been”, understand the development process, social learning & creative dialogue
• OM enabled the project team to review progress and adjust strategies when necessary recognising development does not have fixed goals but is a moving target.
• OM provides a framework for understanding who are the boundary partners with whom the project seeks to influence and can influence, what are the uses and who are the intended users of the research results.
Thank YouThank You
Chris Morris ICTs for Development
Tel: +27 (12) 841-2509
Cell: 082 892 9349
E-mail: cmorris@csir.co.za
URL: http://www.csir.co.za
URL: http://www.fmfi.org.za