INNOVATIVE USE OF TECHNOLOGY FOR DATA COLLECTION IN YOUTH PROGRAMS

September 12, 2012

Small group discussion questions

Scenario 1: You are implementing a youth project in a post-conflict highly volatile area. You need to regularly collect data on your trainees’ performance. The travel is restricted and your beneficiaries are very mobile. How can you use technology to help you?

Scenario 2: Your project is building a system of basic education combined with vocational training for out of school youth in a developing country. You are working closely with the ministry of education and need to have your solution sustainable. How can you use technology to help you?

Scenario 3: You are implementing a youth project in a poorly developed area, and as part of your programming you need to run a quarterly community  needs assessment. Your resources are extremely limited, but you need to survey 400 randomly selected people every quarter. How can you use technology to help you?

MOBILE TECHNOLOGY IN RESEARCH AND EVALUATION

Scott Frick, EDCSeptember 12, 2012

Challenge

You run a program that provides demand-driven livelihood training to youth ages 15-35. This livelihood program is in a fragile state, and reaches approximately 12,000 youth countrywide through a network of affiliated implementing partner organizations. You have field offices in major towns but the team has few ways of safely traveling to sites outside of these urban areas.

You must report on several indicators: Number of people participating in a given livelihood programNumber of people completing a given livelihood programParticipant placement levels in jobs, internships, self employment and further education after participating in a given livelihood programParticipant satisfaction after participating in a given livelihood programParticipant completion percentages.

Would traditional M&E procedures be adequate to gather data on these indicators? How might you use technology to increase accuracy and efficiency?

Somali Youth Livelihood Program (SYLP)

Solutions

Shaqodoon

What: Provide youth with access to training, internships, work, and self employment.

Why: Productively engage youth & add to the development of the region.

Who: Targets: 8700 Youth Trained, 80% Placed

ShaqodoonHow:

Worked with employers to identify demand.

Assisted education and training providers to develop programs more focused on meeting demand.

Mobile Technology

SMS-based Info Match

Educational Audio on Demand

EDC Server At Local Telecom

Mobile phone use among youth is booming: Mobile phone ownership grew 1600% between 2002 – 2007.

Financial Literacy Entrepreneurship

Interactive Audio Instruction

Mobile Infrastructure Used for M&E in 3 Main Ways:

1. An InfoMatch Survey (SMS)2. A placement satisfaction survey

(traditional call)3. An interactive quiz (touch tone

response)

InfoMatch Survey (SMS)

SYLP conducted a one-day cell phone based survey to determine the effectiveness and ease of use of InfoMatch system. (n=200).  

Youth were randomly selected from the SYLP monitoring and evaluation database to participate in the survey. 

SYLP sent six questions in Somali language via SMS.  A total of US$100 in credit vouchers was split

between the first nine respondents as an incentive to complete the survey. 

Eighty percent of the youth responded within one hour of the survey being sent.  The total response rate was 90 percent.

Placement Satisfaction Survey

Using the existing database of learners and cell phones, SYLP conducted phone interviews of learners who had completed the program in Burao.

Interactive Quiz (Touch Tone)

Findings: Preliminary Mobile Feedback Data

Interactive Audio Instruction (IAI)

IAI Financial Literacy Series (40 episodes): 757 calls received from 18 devices across Somaliland, Puntaland, South Galkayo for 1400 listeners

50% of groups placed 40+ calls during program period

DEVELOPING ELECTRONIC ASSESSMENT

Yvette TanSeptember 12, 2012

Challenge

Conduct student reading assessment to measure program efficacy

Limited hardware and software options

Needed quick turnaround of results in order to make informed changes/enhancements to program interventions

Pen & paper

Data entry, cleaning and analysis took a long time and involved hiring a number of temporary staff

Took 8 – 10 months to turn around results

Can we do better?

Electronic Solution

Developed an electronic version of the assessment using Excel that runs on macros.

Instructions were standardized by adding an audio file to make sure that every student gets the same instructions.

Electronic Solution

Has a built in timer to automatically calculate correct letters and words read. This is important because most data input mistakes are made through human errors on this calculation.

Electronic Solution

Has a built-in database and a corresponding results page for immediate viewing of results.

Advantages

Less costly – did not have to hire encoders

Can run on any netbooks and laptops

Easy to use – administrators were trained in a day on how to use the software

Results were turned around in less than a month – very useful in informing program design/re-design

Additional information

Visit http://eegra.edc.org/

MOBILE TECHNOLOGY IN RESEARCH AND EVALUATION

Elena VinogradovaSeptember 12, 2012

Challenge

Quarterly community needs assessment in a remote village in Kenya Low M&E budget Team: one manager, 8data collectors

Pen & paper solution

Pen & paper survey administration : Data collection: 16 person days Data processing: 10 person days Data set preparation for analysis: 2 days

Total: 28 person days, at least three week lag between data capture and data analysis. In reality – a few months lag

Can we do better?

Mobile Solution

A solution that we tested:

by

Advantages

Inexpensive (¢11 per survey) Compatible with Windows and Android Runs on mobile devices + computers Full encryption of data Hosted server Easy to use

So how does it work?

Step One: Program survey in

workstation application SurveyToGo

Studio

Step Two: Download

SurveyToGo Mobile App and your survey to

the mobile device or computer

Step Three: Go get your

data!

Step Four: Your data is

ready for analysis!

Back to Kenya: administration using SurveyToGo with Lenovo tabletsTASK Pen and paper STG with tablets

Survey programming 0 1 day

Data collection 16 days 6 days

Data processing 10 days 0

Prep for analysis 2 days 0

TOTAL 28 person days 7 person days

Total cost savings: up to 75% Increase in data quality: priceless

As the screen print above shows, the software collects information on survey administration that allows us to easily weed out fabricated data

Testimonial from the field: Kenya“You can’t compare using paper administration

and using the tablets. The tablets have saved us time with surveys being conducted faster as interviewers do not have to think about the instructions of skipping questions. This has also reduced the errors that we initially had with the paper administrations.

For me the best part about the tablets is not having to go through each questionnaire to check for errors at the end of each day of data collection, and not having to spend at least a month on entering data, verification and scanning of the hard copies. ”

M&E Manager

Integrating technology in the M&E structure

In all one-on-one data collection paper is replaced with tablets

All data entry occurs via tablets

Results are immediately available for management decisions.

EDC Project in Pakistan: the first project with the fully integrated technology in the M&E system:

OR using technology for the single administration

Survey administration in Rwanda in February of 2012:

Short timeline Target: 900 surveys Challenge: illiterate respondents Solution: 26 trained interviewers collect

data using tablets

Testimonial from the field: Rwanda“The time and money this application saved us is

huge. We were able to significantly cut our daily paper usage by a large percentage, (imagine the amount of paper  that would have been used for 900 surveys !!!). It was great to conduct surveys without the delays of printing and shipping paper forms. We did not need to worry about paper forms being lost or damaged.

With tablets we did not need to re-enter data into a computer because its digital to start with. So there were no delays in that regard.”

M&E Manger of Akaze Kenozi in Rwanda

Functionality

Question branching, skipping and looping Photo, Video & Voice Recording that can

be attached to individual surveys Data export to SPSS, MS Excel, MS

Access, MS Word, XML Offline and online data capture Fast and easy data processing Sophisticated user support Multi language support Data integrity assessment

Conclusions

Use of mobile technology with SurveyToGo application allows us Reduce cost associated with data capture

and processing Increase control over the process of data

capture and processing Eliminate time lag between data capture and

data analysis Improve data quality by reducing human

interaction with data Improve overall quality of evaluations by

investing in the development and analysis rather than data collection and processing