Literacy Landscape Assessment Guidebook

NOVEMBER 2019

This paper was made possible by the support of the American people through the United States Agency for International Development (USAID). The paper was prepared for

USAID’s Building Evidence and Supporting Innovation to Improve Primary Grade Reading Assistance for the Office of Education (E3/ED), University Research Co., LLC, Contract

No. AID-OAA-M-14-00001, MOBIS#: GS-10F-0182T.

On the cover: Photo caption: Left – Students in Cambodia; Right – Students in the Philippines

Photo: Left Credit – Global Partnership for Education;

Right Credit: The Enabling Writers Project /University of San Jose-Recoletos

Literacy Landscape Assessment Guidebook

November 2019

Prepared by: Hank Healey, Emily Morris, and Emily Kochetkova

With support of: Jennifer Bowser Gerst, Reading within Reach

RIGHTS AND PERMISSIONS

Except as otherwise noted, Literacy Landscape Assessment Guidebook is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0), https://creativecommons.org/licenses/by-nc-nd/4.0/.

Under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, you are free to copy and redistribute the material in any medium or format under the following conditions:

Attribution: You must give appropriate credit, provide a link to the licensed material and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. A suggested citation appears at the bottom of this page, and please also note: This document was developed by Reading within Reach, through the support of the U.S. Agency for International Development (USAID) and licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Non-Commercial: You may not use the material for commercial purposes.

NoDerivatives: If you remix, transform, or build upon the material, you may not distribute the modified material.

Third-Party Content: USAID does not necessarily own each component of the content contained within the work. USAID therefore does not warrant that the use of any third party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to reuse a component of the work, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures or images.

Disclaimer: The views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development.

Suggested citation: Healey, Hank, Emily Morris, and Emily Kochetkova. Literacy Landscape Assessment Toolkit. Washington, D.C.: USAID (2019). Prepared by University Research

Co., LLC (URC) under the Reading within Reach (REACH) initiative for USAID’s Building Evidence and Supporting Innovation to Improve Primary Grade Assistance for the Office of

Education (E3/ED).

Acknowledgments

This resource was developed by Reading within Reach (REACH), which is supported by the U.S. Agency for International Development (USAID). REACH works to

facilitate the development, dissemination and application of evidence-based resources, research and practices to increase the impact, scale and sustainability of

early grade reading (EGR) and literacy programs. As its Secretariat, REACH also provides support to develop and sustain the Global Reading Network (GRN), an

interactive hub and resource repository to connect and mobilize stakeholders interested in early grade reading.

The authors wish to thank each of the individuals who helped conceptualize, advise and review the production of this toolkit. Thank you to Jennifer Gerst of the

Reading within Reach (REACH) initiative at University Research Co., LLC (URC) for her tireless leadership. We are grateful to Melissa Chiappetta, Rebecca Rhodes,

Eirini Gouleta, Elena Walls, and Nate Haight of USAID for their constructive guidance and leadership.

Thank you to URC staff and consultants, Stephanie Kenner, Alison Pflepsen, Merle Schwartz, Amy Pallangyo, and Andy Smart for their conceptual and visual

guidance. The authors also thank the GRN, Basic Education Coalition (BEC), Education in Conflict and Crises Network (ECCN) and Rapid Education and Risk

Analysis (RERA) team members for their formative ideas, careful review, and real examples. These include: Cambridge Education (Sabine Kube-Barth), Catholic

Relief Services (Aminata Jalloh), Chemonics (Jordene Hale and Jennifer Swift-Morgan), Creative Associates (Susan Hirsch-Ayari), EDC (Rachel Cristina, Simon

James, and Daniel Lavan), FHI 360 (Julia Frazier, Nathalie Louge, Emily Miksic, and Guitelle Nicoleau), International Rescue Committee (Shahida Maheen), MSI

(Gaelle Simon), Room to Read (Christabel Pinto), RERA Toolkit team (Jim Rogan, Gwen Hean, and Ashley Henderson), RTI International (Luis Crouch), School to

School International (Mark Lynd), SIL-LEAD (Paul Frank), Another Option (Dee Bennet and Rebecca Martinez), and Ash Hartwell.

List of Acronyms CBO Community-Based Organization DDL Development Data Library DEC Development Experience Clearinghouse DFID Department for International Development E3/ED Bureau for Economic Growth, Education, and Environment, Office of Education EO Education Officer EGR Early Grade Reading EGRA Early Grade Reading Assessment EMIS Education Management Information Systems ESA Education Sector Analyses FGD Focus Group Discussion GER Gross Enrollment Ratio HR Human Resources IRB Institutional Review Board KII Key Informant Interview KM Knowledge Management LLA Literacy Landscape Assessment LO Learning Objective M&E Monitoring and Evaluation MERL Monitoring, Evaluation, Research, and Learning MOF Ministry of Finance NER Net Enrollment Ratio NGO Non-Governmental Organization PISA Programme for International Student Assessment PTA Parent Teacher Association REACH Reading within Reach SACMEQ The Southern and Eastern Africa Consortium for Monitoring Educational Quality SMC School Management Committee SRGBV School-related Gender-based Violence TE Teacher Education (pre-service teacher education) TLM Teaching and Learning Materials TPD Teacher Professional Development (in-service teacher training) TTC Teacher Training Colleges TWG Technical Working Group USAID United States Agency for International Development

List of Figures, Boxes, and Tables

Figures Figure 1: USAID Reading MATTERS Framework: Key Elements of a Reading Program 17 Figure 2: Summary of the Four Phases of the LLA 20 Figure 3: Literacy Domains 21 Figure 4: Interconnected Data Source Contexts 22 Figure 5: Core Principles of the LLA 26 Figure 6: Equity Analyses 27 Figure 7: System Map Example 57 Figure 8: Distinctions Between Quantitative and Qualitative Approaches 66 Figure 9: Distinctions Between Survey, Interview, and FGD Methods (Tools) 67 Figure 10: Mechanisms Driving Learning Coherence 99 Figure 11: Date Use for Improved Learning 101 Figure 12: Lines of Accountability 102 Figure 13: Assessing a Program’s Viability in Another Context 117

Boxes Box 1: LLA Objectives 19 Box 2: LLA Design Document Template 37 Box 3: Data Triangulation 55 Box 4: Education System Mapping (Systemigram) Exercise: Sample Agenda 61 Box 5: Conducting a Language Mapping Exercise to Answer Critical Questions 64 Box 6: Qualitative Sampling (Interviews, FGDs, Observations 69 Box 7: Site Planning: Collaborating, Learning, and Adapting to the Context 69 Box 8: LLA Primary Data Collection Tools 1-11 (Example) 70 Box 9: Translation Tips 71 Box 10: The Role of the Facilitator 74 Box 11: Ethical Issues Checklist for Primary Data Collection Activities 75 Box 12: Nodes (Categories) of Enablers, Barriers, and Recommendations 82 Box 13: Stakeholder Meeting to Refine and Revise Recommendations: Sample Agenda 88 Box 14: LLA Final Report Outline (Example) 93

Tables Table 1: LLA Journey Map (Phases and Steps) 24

Table 2: LLA Objective, Example Entry 31

Table 3: LLA Parameters 31

Table 4: LLA Team Tasks and Deliverables 33

Table 5: LLA Team Roles and Qualifications 34

Table 6: TWG Members 39

Table 7: Key Documents and Resources 41

Table 8: Education Systems Landscape 42

Table 9: Background Education Statistics 44

Table 10: Inventory of Literacy Initiatives 46

Table 11: Stakeholder Analysis 48

Table 12: Guiding Questions Outline for Education Systems (Context 1) and

Learning Environment (Context 2) 57

Table 13: Guiding Questions Outline for Home and Community (Context 3) 59

Table 14: Primary Data Collection Planning Table for Contexts 2 and 3 (Context 3 Example) 63

Table 15: Pilot Planning Table 72

Table 16: Recommendations Prioritization Table (Example from Context 3) 90

Table 17: Context 1 Standards 103

Table 18: EGR and Literacy Initiatives Costs 115

Table 19: Analytical Framework Approaches for Qualitative Data

(FGDs and Key Informant Interviews) 120

Table 20: Supporting Resources Across Literacy Domains 122

9

Contents Acknowledgments 5

List of Acronyms 6

List of Figures, Boxes, and Tables 7

Figures 7

Boxes 7

Tables 7

Introduction 15

Background 15

The Strategic Objective of the LLA and the Reading MATTERS Framework 16

Figure 1: USAID Reading MATTERS Framework: Key Elements of a Reading Program 17

Research Questions 18

The LLA Process: Using the Toolkit 18

Box 1: LLA Objectives ........................................................................................................................................................................................... 19

LLA Mapped Elements: ........................................................................................................................................................................................ 20

Figure 2: Summary of the Four Phases of the LLA 20

Literacy Domains .............................................................................................................................................................................................. 21

Figure 3: Literacy Domains 21

Figure 4: Interconnected Data Source Contexts 22

Table 1: LLA Journey Map: Phases and Steps .................................................................................................................................................. 24

Principles of the LLA 26

10

Figure 5: Core Principles of the LLA 26

Figure 6: Equity Analyses 27

Key Terms 28

PHASE ONE: Planning and Procurement 30

Step 1: Define the objective(s) of the LLA 30

Table 3: LLA Parameters ...................................................................................................................................................................................... 31

Step 3: Develop a Scope of Work to procure an LLA 33

Table 4: LLA Team Tasks and Deliverables ........................................................................................................................................................... 33

Step 4: Select and train the LLA Team 34

Step 4a: Select the LLA Team ............................................................................................................................................................................... 34

Step 4b: Facilitate LLA training for LLA Team ...................................................................................................................................................... 35

PHASE TWO: Design and Background Data 36

Step 1: Develop the LLA Design 36

Box 2: LLA Design Document Template ............................................................................................................................................................... 37

Step 2: Form a Technical Working Group (TWG) 38

Table 6: TWG Members ...................................................................................................................................................................................... 39

Step 3: List key resources, record education landscape information and background data, and inventory literacy initiatives 39

Step 3a: Identify key documents and resources .................................................................................................................................................. 39

Table 7: Key Documents and Resources .............................................................................................................................................................. 41

Rapid Education and Risk Analysis (RERA) 41

Education Sector Analysis 41

Step 3b: Record the Education Systems Landscape ............................................................................................................................................ 42

11

Table 8: Education Systems Landscape................................................................................................................................................................ 42

Step 3c: Record the background education statistics .......................................................................................................................................... 43

Table 9: Background Education Statistics ............................................................................................................................................................ 44

Step 3d: Conduct an inventory of all the literacy programs (government, NGO, CBO, etc.) .............................................................................. 46

Table 10: Inventory of Literacy Initiatives ............................................................................................................................................................ 46

Example of entry into Template 10: Inventory of Literacy Initiatives in Workbook Phase Two. ......................................................................... 46

Step 4: Conduct an initial stakeholder analysis 47

Table 11: Stakeholder Analysis ............................................................................................................................................................................ 47

Step 5: Convene a stakeholder design workshop 49

PHASE THREE: Implementation of Data Collection 54

Box 3: Data Triangulation ..................................................................................................................................................................................... 55

Step 1: Answer the Guiding Questions for all Contexts (columns 1a-1d) with existing data 56

Table 12: Guiding Questions Outline for Education Systems (Contexts 1) and Learning Environment (Context 2) ........................................... 57

Step 2: Map out the education system through a stakeholder meeting (Context 1 only) 60

Box 4: Education System Mapping (Systemigram) Exercise: Sample Agenda ..................................................................................................... 61

Figure 7: System Map Example. This figure depicts what the results of a mapping exercise might look like. 62

Figure 7: System Map Example 62

Step 3: Plan the primary data collection 63

Table 14: Primary Data Collection Planning Table for Contexts 2 and 3 (Context 3 Example) ............................................................................ 63

Step 3a. Determine which Guiding Questions will be asked ............................................................................................................................... 63

Box 5: Conducting a Language Mapping Exercise to Answer Critical Questions ................................................................................................. 64

Step 3b. Determine the methodological approach(es), tools (methods), and sample sizes ............................................................................... 65

file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724008file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724008file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724008file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724012file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724012file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724012file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724014file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724014file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724014file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724018file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724018file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724018

12

Figure 8: Distinctions Between Quantitative and Qualitative Approaches 66

Figure 9: Distinctions Between Survey, Interview, and FGD Methods (Tools) 67

Box 7: Site Planning: Collaborating, Learning, and Adapting ............................................................................................................................... 69

Box 6: Qualitative Sampling (Interviews, FGDs, Observations) ........................................................................................................................... 69

Step 4: Create data collection tools (survey, interview, and focus group discussion) 70

Step 4a. Draft the data collection tools ............................................................................................................................................................... 70

Box 8: LLA Primary Data Collection Tools 1-11 (Example) ................................................................................................................................... 70

Step 4b. Translate and pilot the tools .................................................................................................................................................................. 71

Box 9: Translation Tips ......................................................................................................................................................................................... 71

Table 15: Pilot Planning Table .............................................................................................................................................................................. 72

Step 4c. Train the facilitators and standardize how the data will be recorded ................................................................................................... 73

Box 10: The Role of the Facilitator ....................................................................................................................................................................... 74

Box 11: Ethical Issues Checklist for Primary Data Collection Activities ............................................................................................................... 75

PHASE FOUR: Analyzing, Reporting, and Disseminating 79

Step 1: Analyze and synthesize findings 80

Analyze the primary and secondary data ............................................................................................................................................................ 81

Step 1a. Analyze the extent, enablers, and barriers (Guiding Question Tables columns 4a-6)........................................................................... 81

Box 12: Nodes (Categories) of Enablers, Barriers, and Recommendations ......................................................................................................... 82

Step 1b. Synthesize findings and enablers and barriers across literacy domains (bottom of Guiding Questions Tables) .................................. 84

Step 1c. Visualize Context 1 data in system maps to compare with Phase 2 - Step 5 “ideal maps” ................................................................... 85

Step 2: Formulate preliminary recommendations from Guiding Question Tables, column 7a) ......................................................................... 86

Step 3: Conduct a stakeholder meeting to revise and refine the preliminary recommendations (Guiding Question Tables column 7b) ........ 87

file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724022file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724022file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724022file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724023file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724023file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724023file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724028file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724028file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724028file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724031file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724031file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724031file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724032file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724032file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724032

13

Box 13: Stakeholder Meeting to Refine and Revise Recommendations: Sample Agenda .................................................................................. 88

Step 4: Conduct a TWG meeting to prioritize the recommendations ................................................................................................................. 89

Step 4a. Conduct a prioritization analysis ............................................................................................................................................................ 89

Table 16: Recommendations Prioritization Table (Example from Context 3) ..................................................................................................... 90

Step 4b. Determine a final list of recommendations ........................................................................................................................................... 92

Step 5: Write a final LLA report 92

Box 14: LLA Final Report Outline (Example) ........................................................................................................................................................ 93

Step 6: Disseminate findings 95

ANNEXES—LLA RESOURCES, TOOLS, AND EVIDENCE 96

Annex A: The Learning Coherent System 96

Mechanisms that Drive Coherence 97

Protocols .............................................................................................................................................................................................................. 98

Systems Maps ...................................................................................................................................................................................................... 99

Figure 10: Mechanisms Driving Learning Coherence 99

Figure 11: Data Use for Improved Learning 101

Figure 12: Lines of Accountability 102

Table 17: Context 1 Standards ........................................................................................................................................................................... 103

Annex B: Key Terms and Concepts 106

Annex C: Host Country Government Capacity and Commitment (Context 1) 114

Table 18: EGR and Literacy Initiatives Costs ...................................................................................................................................................... 115

Figure 13: Assessing a Program’s Viability in Another Context 117

Annex D: Guidance for Qualitative Data Collection (FGDs and Interviews) 119

file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724042file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724042file:///C:/Users/JGerst/Desktop/GRN%20Literacy%20Landscape%20Assessment%20Guidebook%2011_2019.docx%23_Toc25724042

14

Table 19: Analytical Framework Approaches for Qualitative Data (FGDs and Key Informant Interviews) ....................................................... 120

Annex E: Supporting Resources by Literacy domain 122

Table 20: Supporting Resources Across Literacy Domains ................................................................................................................................ 122

Annex F: Evidence 128

Annex G: Language Mapping Experience and Resources 129

Afghanistan .................................................................................................................................................................................................... 129

Ghana ............................................................................................................................................................................................................. 130

Mozambique .................................................................................................................................................................................................. 131

References 132

15

Introduction

Background

In the USAID Education Policy launched in November 2018, the United States Agency for International Development (USAID) set a priority of

ensuring that “children and youth gain literacy, numeracy, and social-emotional skills that are foundational to future learning and success”1. As

of 2018, USAID has supported early grade reading and literacy programs in 45 countries through a range of activities, such as teaching and

learning materials development, teacher education and professional development, government education management and information

systems strengthening, and community and family engagement in literacy learning2. Evaluation and research on progress made to date shows

that while reading improvements are observable, there are also notable areas where further progress could be made to ensure all children can

read and comprehend3. For instance, in many countries, evidence shows that children are learning to decode text but cannot actually

comprehend what they decode4.

The Literacy Landscape Assessment Toolkit (LLA)

The Literacy Landscape Assessment Toolkit (LLA) recognizes that country context and environment matter in the success of literacy

programs, and their ability to be scaled and sustained.

• The toolkit was developed as an exercise for capturing and analyzing the different landscapes and contextual factors that enable literacy

acquisition for all children, as well as factors that may be hindering desired progress.

• It can be adapted to the various countries or contexts where USAID works, including in conflict or crisis-affected areas, as well as to other

learning outcomes of interest, such as numeracy.

• The LLA outlines practical steps in gathering, analyzing, and using contextual data germane to literacy acquisition in the primary grades (or

equivalent) to better inform the design and adaptation of reading interventions.

1 USAID. USAID Education Policy. (Washington, D.C.: USAID, 2018), 4 2 USAID. USAID Education Policy. (Washington, D.C.: USAID, 2018), 4 3 Thomas Alvares de Azevedo et al. Synthesis of Findings and Lessons Learned from USAID-funded Evaluations (Washington, D.C.: Management Systems International, 2018). 4 Young-Suk Grace Kim et al. The Landscape Report on Early Grade Literacy (Washington, D.C.: USAID, 2016).

16

The Strategic Objective of the LLA and the Reading MATTERS Framework

The strategic objective of the LLA is to provide USAID missions and teams, host-country governments, implementing partners, and other key

stakeholders with critical information and data about the literacy and early grade reading landscape. This data and information are intended to

assist in designing, planning, implementing, or adapting effective literacy efforts and programming in USAID-recipient countries, and to ensure

that the literacy needs of all children can be addressed at scale. The LLA Toolkit is based on existing evidence (see Annex) and reflects USAID’s

Reading MATTERS Framework. The LLA incorporates data collected per other assessments, such as Education Sector Analyses (ESA), Rapid

Education and Risk Analysis (RERA), and Early Grade Reading Assessments (EGRA).

The USAID Reading MATTERS conceptual framework, as shown in Figure 1 below, includes the key elements of a reading program.5 This

assessment exercise can: identify literacy domains in the landscape that enable or inhibit literacy acquisition and progress, indicate the extent to

which elements of the system work together in a learning coherent manner, and gauge government capacity and commitment for necessary

reforms. This assessment process relies on key data collection stages: a) secondary data analysis of existing documents and evidence, and b)

primary data collection through interviews and focus group discussions (FGD), as well as surveys as needed. The final analysis of these data is

meant to inform decisions on where USAID programs might best leverage resources, such as Basic Education Funds, and build on strengths of

the host country education system to help governments and key stakeholders ensure that all children gain and continue to gain the literacy skills

that are foundational to future learning and success.

5 For more information on the Reading MATTERS Framework, see https://www.edu-links.org/resources/usaid-reading-matters-conceptual-framework

https://www.edu-links.org/resources/usaid-reading-matters-conceptual-framework

17

Figure 1: USAID Reading MATTERS Framework: Key Elements of a Reading Program

Figure 1: USAID Reading Matters Framework: Key Elements of

a Reading Program. This framework takes into consideration

systems theory6 and asserts that many programmatic domains

of reading programs must be functioning individually and in

alignment with one another for the goal of all children reading

to be realized. Research and adaptation are critical to ensuring

this system is relevant to the contexts. The outer circle

indicates that reading programs must build on host-country

government capacity and commitment in order to be scaled

and sustained.

6 Lant Pritchett. “Creating Education Systems Coherent for Learning Outcomes: Making the Transition from Schooling to Learning.” RISE Working Paper.

Oxford, United Kingdom: Research on Improving Systems of Education, December 2015.

18

Research Questions

The research questions guiding the LLA are generally as follows:

➢ Which aspects of the education system enable and ensure quality teaching and learning for all children in primary school? Which aspects could be strengthened to ensure learners are attaining minimum grade-level proficiency in reading at the end of primary school?

➢ How are early grade literacy skills being taught in the learning environment? What are the system enablers and barriers for learners attaining minimum grade level proficiency in reading at the end of primary school?

➢ How are early grade literacy skills being developed and furthered in the home and community environment?

The LLA Process: Using the Toolkit

The LLA Toolkit is comprised of two documents:

1) this LLA Guidebook, which describes the LLA process and outlines the phases and steps involved, and

2) a separate LLA Workbook (multiple segments) with templates in which to document findings per

detailed instructions. The two documents are to be used in tandem.

The LLA Toolkit contains

multiple tools and activities

that can be applied in part or as

a whole to suit the needs, time,

and budget of its users.

19

The LLA is designed for use in any country, region, or lower-level jurisdiction (i.e., province, district, or county) where early grade literacy data7

exists or will soon be collected, and can be adapted to several contexts, including in crisis or conflict areas.8

The LLA is designed for use towards two main objectives, as explained below:

Box 1: LLA Objectives

Objectives Assessment and Examples Timing

Inform the design of a new program

Revealing what already exists to enable the system to function well with regards to building proficiency in literacy, as well as identifying key elemental areas that can be strengthened.

Example: a USAID mission is designing a new program and is considering what components at the system, classroom, and community levels to focus on in the program.

Prior to procuring a new reading program.

Inform the adaptation or scaling of an existing program(s)

Providing contextual information and insight into why results and outcomes may have occurred or to document major changes (policy, political, etc.) in the literacy environment.

Example 1: A reading program is not producing the intended results, and the mission and/or implementer wants to adapt the program for greater impact, or a mission wants to assess the enablers and barriers in an environment before bringing a program to scale.

Example 2: A significant education policy or systemic change occurs, and a mission wants to understand the implications across the literacy landscape.

During implementation of existing programs and/or after a major shift in the country’s education landscape has occurred.

7 Literacy data can include any national or representative data collected per the designated regions covered by the toolkit. Data can be collected through

national assessment tools, early grade reading assessments (EGRA), or other literacy assessments conducted within the last 5 years. If there is no early grade reading data for the region or population of learners and teachers targeted by the LLA exercise, this data will need to be collected before or concurrent to the LLA exercise. Literacy data is critical as it provides an overarching picture of whether or not countries are able to meet their literacy objectives and to what extent system reform is needed.

8 Per the USAID Rapid Education and Risk Analysis (RERA) Toolkit (USAID, 2019), crisis-affected includes areas affected by natural disasters and health

epidemics and also can include climate vulnerabilities and crime and violence activity. Conflict-affected includes areas affected by active armed conflict or post conflict (see Annex B for more definitions). If a RERA has been conducted, the LLA should build on information already collected as will be indicated throughout the toolkit.

20

LLA Mapped Elements:

➢ Phases (4), each with steps and templates

➢ Programmatic Domains for Reading & Literacy (10)

➢ Contexts (3)

Phases: The LLA is divided into four phases summarized in Figure 2 and outlined below in Table 1, The LLA Journey Map with accompanying

steps.

Figure 2: Summary of the Four Phases of the LLA

Figure 2: Summary of the Four Phases of the LLA. The phases of the LLA inform the design of a new program or the adaptation or scaling

of an existing program or programs.

Phase 1

(4-8 weeks)Planning &

ProcurementIdentifies the parameters of the LLA and team.

Phase 2

(6-8 weeks)

Design & Background

Data

Includes the background info needed and stakeholder consultation to promote host-country ownership and

implementing partner involvement.

Phase 3

(10-20 weeks)Implementation of Data

CollectionGathering primary and secondary data.

Phase 4

(10-20 weeks)

Analyzing, Reporting, &

Disseminating

Ensures key objectives and questions proposed in design have been answered.

21

Literacy Domains

There are also seven programmatic literacy domains based on the Reading MATTERS Framework.

For the LLA, one additional political economy domain has been added: Host Country Government

Capacity and Commitment.

Figure 3: Literacy Domains

Programmatic Literacy Domains (Not Sequential)

Domain Detail

Standards Evidence-based standards, norms, and policies

Texts High-quality teaching and learning materials

Teachers Effective teachers and classroom instruction

Mentors Effective coaches and mentors

Administrators Quality administrative support and supervision

Regular Assessments Assessments inform instruction

Host Country Government Capacity and

Commitment Government financing for early grade literacy

Extra Practice Extra practice and support outside of school

Figure 3: Literacy Domains. The phases of the LLA are mapped to these literacy domains, which are not sequential.

22

Figure 4: Interconnected Data Source Contexts

Figure 4: Interconnected Data Source Contexts. The LLA builds on secondary data from existing reports, documents, and sources, and also

includes primary data collection.

CONTEXT ONE, Education Systems: The elements of the system in place to facilitate literacy learning.

CONTEXT TWO, Learning Environment (formal and non-formal): The teaching and learning infrastructure, practices, beliefs, and attitudes

around literacy learning.

CONTEXT THREE, Home and Community: The infrastructure, practices, beliefs, and attitudes around literacy learning outside of the learning

environment (how literacy acquisition is supported at home and in the community). Context three corresponds to the Extra Practice tile from the

Reading MATTERS Framework. As revealed in the research, the key literacy domains that influence reading and literacy acquisition in the Home

and Community Context are: 1) availability of text and literacy programs; 2) familial demographics and cultural contexts; and 3) caregiver

engagement in reading and literacy acquisition.

Context 1: Education Systems

Context 3: Home and

Community

Context 2: Learning

Environment

23

Each LLA Context is critical to supporting children’s literacy acquisition, but the stakeholders, factors considered, and documents analyzed will

differ across these Contexts. Completing the exercises for all three Contexts will provide the most comprehensive understanding of the

landscape. While the Education Systems Context should be completed in its entirety, the LLA is flexible in that, within the Learning Environment

and Home and Community Contexts, teams will decide which of the literacy domains to analyze for their particular context.

24

Table 1: LLA Journey Map: Phases and Steps NOTE FOR USER: Create a local team workspace for this toolkit. Each Phase and Step of the LLA below references Excel-based Workbook Resources in the form of fillable templates, sample agendas or forms. Once the LLA Team sets up a workspace, for ease of use the team may wish to ensure the templates referenced below are hyperlinked to the corresponding Workbooks and that at each icon in the LLA Guidebook labelled “Back to Journey Map” hyperlinks to Table 1 below.

Step Workbook Resource

LLA WORKBOOK

Phase 1

Planning and Procurement (4-8 weeks)

Step 1: Define the objective(s) of the LLA Template 2: LLA Objectives

Step 2: Set the parameters of the LLA Template 3: LLA Parameters

Step 3: Develop a Scope of Work to procure an LLA

Template 4: LLA Scope of Work

Step 4: Select and train the LLA Team Template 5: LLA Team Roles and Qualifications

LLA WORKBOOK

Phase 2

Design and Background Data (6-8 weeks)

Step 1: Develop the LLA design Template Box 2: Design Outline

Step 2: Form a Technical Working Group (TWG) Template 6: TWG Members

Step 3: List key resources, record education landscape information and background data, and inventory literacy initiatives

Template 7: Key Documents and Resources Template 8: Education Systems Landscape Template 9: Background Education Statistics Template 10: Inventory of Literacy Initiatives

Step 4: Conduct an initial stakeholder analysis Template 11: Stakeholder Analysis

Step 5: Convene a stakeholder design workshop Annex A

LLA WORKBOOK

Phase 3

Implementation of Data Collection (10-20 weeks)

Step 1: Answer the Guiding Questions for all Contexts (columns 1a-1d) with secondary data

Templates 12: Guiding Questions Outline for Education Systems (Context 1) and Learning Environment (Context 2) Template 13: Guiding Questions Outline for Home and Community (Context 3)

Step 2: Map out the education system through a stakeholder meeting (Context 1 only)

Template Box 4: Education System Mapping Sample Agenda

Step 3: Plan the primary data collection Template 14: Primary Data Collection Planning Table for Contexts 2 and 3 (Context 3 Example)

Step 4: Create data collection tools (survey, interview, and focus group discussion)

Template 15: Pilot Planning Table

Step 5: Collect and record the primary data in the Guiding Questions Templates

Templates 12A, 12B and 13, Data Collection Section

LLA WORKBOOK

Phase 4

Analyzing, Reporting, and Disseminating (10-20 weeks)

Step 1: Analyze and synthesize findings Templates 12A, 12B and 13, Analysis Section

Step 2: Formulate preliminary recommendations

Refer to Templates 12A, 12B and 13

Step 3: Conduct a stakeholder meeting to revise and refine the preliminary recommendations

Template Box 10: Stakeholder Meeting to Refine and Revise Recommendations: Sample Agenda AND Refer to Template 12 and 13

Step 4: Conduct a TWG meeting to prioritize the recommendations

Template 16: Final Recommend

Step 5: Write a final LLA report Template Box 13: Final Report Outline

Step 6: Disseminate findings

https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%201.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%201.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%201.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-1.xlsxhttps://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%202.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%202.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%202.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-2.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-annex.pdfhttps://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%203.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%203.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%203.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-3.xlsxhttps://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%204.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%204.xlsx?web=1https://chsglobal.sharepoint.com/sites/LLAGuidebookandWorkbooks/Shared%20Documents/LLA%20WORKBOOK%20Phase%204.xlsx?web=1https://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsxhttps://www.urc-chs.com/sites/default/files/urc-grn-lla-4.xlsx

25

The LLA steps are designed to be conducted both remotely (from anywhere) and in-country. The total time required will vary by context (i.e., size of population and crises or conflict environment) and defined parameters (i.e., number of literacy domains covered in Phase 3, whole country or regions, etc.). The timeframes for each phase (estimated above in Table 1, the LLA Journey Map) will vary depending on the parameters set in Phase 1. In countries where multiple LLAs must be prepared (i.e., in Tanzania for mainland and Zanzibar, which have separate Ministries of Education), Phase 1 may take longer than the estimate provided. Likewise, in situations affected by conflicts and/or crises the timeline and process will have to be adapted. The implementation of the LLA is facilitated by a team of experts and advised by a Technical Working Group (TWG). These experts and

members of the TWG are identified by the USAID Mission per criteria described in Phase 1.

26

Principles of the LLA

The principles undergirding the LLA reflect the principles articulated within USAID’s 2018 Education Policy as well as USAID’s commitment to

collaborating, learning, and adapting (CLA).

Figure 5: Core Principles of the LLA

Figure 5: Core Principles of the LLA. The LLA is based on three core principles.

The goal:

All children reading

•The LLA exists to uncover and illuminate factors that can hinder or support children’s literacy acquisition, and to provide information to guide governments, donors, and implementers in removing hindrances and strengthening supports so that all children, regardless of background and demographics, are reading and continue doing so over time.

Host country ownership

•It is critical for key government stakeholders to be integrally involved in each phase of the LLA so that recommendations are developed collaboratively.

•This means that all units and levels of government that are assessed by the LLA should be involved in work-planning, data collection, and dissemination of findings.

Focus on inclusion and

equity

•Access to resources and the learning needs of girls and boys--children with disabilities; indigenous children; children who speak minority languages; children in remote and rural areas; children living in crisis-affected or conflict-affected areas; or children living in vulnerable circumstances are not always adequately considered in policy and practice.

•The LLA explicitly considers these groups across spaces and reading domains.

27

Figure 6: Equity Analyses

Figure 6: Equity Analyses. There are a number of factors that must be considered when conducting equity analyses. If equity analyses have been conducted per the RERA, all tables and tools should be adapted to reflect the major equity issues revealed. 9

9 USAID’s Education Policy states that marginalized children and vulnerable groups include “girls, children affected by or emerging from armed conflict or humanitarian crises, children with disabilities, children in remote or rural areas (including those who lack access to safe water and sanitation), religious or ethnic minorities, indigenous individuals, orphans and children affected by HIV/AIDS, child laborers, married adolescents, and victims of trafficking” (USAID, 2018, p. 46).

Language issues

Disability

Gender

Minority and/or indigenous groups

Crisis/conflict contexts

ICT (Access)

Equity Analyses

28

Key Terms

Here we provide definitions of a few key terms; Annex B contains a full list of LLA terms and their definitions.

LITERACY: “Literacy is defined as the ability to read and write to gain and produce meaning in context. Early grade reading or literacy refers to instruction that takes place in primary school grades 1-3”10. LEARNING COHERENT SYSTEM: An education system that is learning coherent is one in which there is alignment between all aspects, or literacy domains. That is, the teaching and learning materials, the teacher education and teacher professional development courses and programs, the coaching practice, and student assessment are all aligned with and support the achievement of the learning objectives outlined in the curriculum. (More detail can be found in Annex A.) READING MATTERS STANDARDS: A government’s evidence-based policies and norms related to literacy, as reflected in their curricula, syllabi, and other guiding frameworks. LLA STANDARDS: The evidence-based ideal (based on learning coherence and all children reading) used in all literacy domains to measure the extent to which something is happening in the education system or learning environment. LEARNING ENVIRONMENT: The learning environment constitutes the formal and informal spaces where teaching and learning take place. In the LLA, the learning environment includes the teaching and learning infrastructure as well as educators’ practices, beliefs, and attitudes around literacy learning.

COACHING/MENTORING: “Teacher educators commonly define coaching as ‘onsite, job-embedded, sustained professional development for teachers’ (Bean, 2014)11. Through coaching, teachers receive continuous support to help them acquire and master new knowledge and skills to improve their instruction and, ultimately, student achievement”12.

10 Young-Suk Grace Kim et al. The Landscape Report on Early Grade Literacy (Washington, D.C.: USAID, 2016). 11 Rita M. Bean. The Power of Coaching: Improving Early Grade Reading Instruction in Developing Countries: Final Report. Report produced for USAID

(Silver Spring, MD: Aguirre Division of JBS International, Inc., 2014).

12 Alison Pflepsen, Coaching in Early Grade Reading Programs, resource developed by REACH and the GRN (Chevy Chase, MD: URC, 2019).

29

ASSESSMENT: “Assessment is an essential component of effective instruction as it provides information about students’ achievement levels and reasons for the performance level. There are multiple types and purposes of student assessments such as screening (to identify some students who might be at risk for literacy failure and require further diagnostic assessment), diagnostic (to identify specific areas of strengths and weaknesses), formative and progress monitoring (to reveal whether students are learning adequately), and summative (overall achievement level)”13.

13 Kim et al. Landscape Report, 2016.

30

PHASE ONE

Back to Journey Map

PHASE ONE: Planning and Procurement

Step 1: Define the objective(s) of the LLA

Purpose: To identity why the LLA is being conducted and to develop consensus as to how it will be used by the USAID Mission.

Who leads: USAID Mission, with input from selected host-country education representatives.

☐ Look at Box 1 in the introduction and decide which of the two objectives the LLA will be addressing; include country-specific information.

☐ Complete Template 2 in the Workbook Phase One. An example follows.

Overview

The Planning and Procurement process consists of four steps that lay the groundwork for the LLA. This section of the

Guidebook provides details for each step, the first three of which are to be carried out by USAID Mission staff in advance

of procuring a team to lead the LLA.

☐ Step 1: Define the objective(s) of the LLA

☐ Step 2: Set the parameters of the LLA

☐ Step 3: Develop a Scope of Work to procure an LLA

☐ Step 4: Select and train the LLA Team

31

PHASE ONE

Back to Journey Map

Table 2: LLA Objective, Example Entry

Objective Assessment and Examples Timing

Inform the design of a new program

e.g., to see how the Ministry’s current education system is operating in regards to literacy and to understand how the new policy changes of local language (L1) instruction are influencing teaching and learning. Also, to understand home and community reading and literacy practices for informing the new literacy program design.

After EGRA baseline is complete and before project design is complete

Example of entry for Template 2: LLA Objectives in Workbook Phase One.

Step 2: Set the parameters of the LLA Purpose: To set the parameters of LLA and what geographical areas, grade levels, and other considerations will be covered. Who leads: USAID Mission, with input from selected host-country education representatives.

☐ Complete the LLA parameters table in Template 3 of the Workbook Phase One. Top level questions are listed here below and in more detail in the Workbook.

Table 3: LLA Parameters

Questions

1. What contextual assessments have been conducted to date and how will this LLA exercise expand these analyses? (e.g. Education Sector Analyses, Rapid Education and Risk Analysis (RERA), and Early Grade Reading Assessments (EGRA)).

2. How often will the LLA be implemented?

3. Where will the LLA be implemented?

4. Which grades/classes within pre-primary/primary (or equivalent) will the LLA cover?

5. Which modalities will the LLA cover (formal or nonformal education)?

32 Back to

Journey Map

6. Which kinds of learning institutions will the LLA cover? Who operates these schools/centers?

7. What populations and key issues will be considered in this LLA process?

8. Is this country/region currently in a crises/conflict affected area?

9. Which Contexts will be completed?

33

PHASE ONE

Back to Journey Map

Step 3: Develop a Scope of Work to procure an LLA

Purpose: To procure a team that can undertake the technical and operational tasks required for conducting the LLA.

Who leads: USAID Mission.

When: After the objectives and parameters of the LLA have been determined.

☐ Modify the Scope of Work Template 4 found in the Workbook Phase One, adding details about the objectives, parameters, and the country or regional context. The overall length of an LLA, as mentioned, can vary and is likely to require at least 6 months and as many as 12 months.

☐ Modify the time frame according to the parameters set, keeping in mind things that will be outside of the LLA team’s control, such as contract approvals, availability of stakeholders for meetings and interviews, timing of data collection in relation to school calendar, etc. USAID Mission personnel will determine the approach for procurement.

Table 4: LLA Team Tasks and Deliverables

Phase Deliverables Time Frame

Phase 1: Form LLA Team ▪ LLA Team contracted 2-4 weeks

Phase 2: Design and Background Data

▪ Steps 1 and 2 completed: LLA Design and Implementation Plan; final report outline (both finalized with TWG)

▪ Steps 3 and 4 completed: landscape, background information, and key resources; stakeholder analysis

6-8 weeks

Phase 3: Implementation of Data Collection

▪ Steps 1 through 3 completed: Secondary data analysis and plans for primary data collection across Contexts

▪ Steps 4 and 5 completed: Primary data collected and entered into Guiding Questions tables across Contexts

10-20 weeks

Phase 4: Analyzing, Reporting, and Disseminating

▪ Steps 1 and 2 completed: Data analysis and preliminary recommendations

▪ Steps 3 and 4 completed: Refine and prioritize recommendations

▪ Steps 5 and 6 completed: Final report and dissemination

10-20 weeks

Summary of information going into Template 4: LLA Scope of Work in Workbook Phase One.

34

PHASE ONE

Back to Journey Map

Step 4: Select and train the LLA Team

Step 4a: Select the LLA Team

Purpose: To select a team with the requisite experience, skills, and expertise to carry out the work of the LLA. Who leads: USAID Mission.

☐ Review proposals or applications for executing the work per the defined the LLA Team roles in the Scope of Work (Table 5).

☐ Select the most suitable team. Contracting options could include a standalone contract or an addition (subcontract) to an existing implementation contract. USAID may also need to develop a Memorandum of Understanding (MOU) with the government that outlines the timeframe and local counterpart commitment to the LLA. Table 5: LLA Team Roles and Qualifications

Position Role Qualifications

USAID Mission Lead With the Ministry/Department of Education co-lead, oversee the LLA process and facilitate communication with government and other donor stakeholders.

Experience working in the education sector and managing assessments or evaluations.

Host-country Government Education Lead

With the USAID co-lead, oversee the LLA process and facilitate communication with government and other donor stakeholders.

High-level experience and knowledge of the education system.

LLA Team Lead Coordinate all steps and activities in the LLA process; manage timeline; provide quality control; communicate with USAID; oversee synthesis of reporting.

Demonstrated experience managing mixed methods research projects.

Country-Level Education Advisors

Work closely alongside international experts on the LLA team, providing contextual insight and technical knowledge related to various lines of inquiry, contributing to LLA outputs, and facilitating connections as needed.

A deep internal understanding of the country’s education context and the technical expertise needed to collaborate with international counterparts on the LLA team (i.e., literacy, equity, M&E, etc.).

Education Systems Expert Lead all activities under Context 1. Demonstrated experience leading system analysis efforts, with emphasis on financial and cost-analysis expertise.

35 Back to

Journey Map

PHASE ONE

Literacy/Teaching and Learning Expert

Lead all activities under Contexts 2 and 3. Demonstrated expertise in literacy and research and assessment in teaching and learning in developing country contexts.

Research and CLA Expert Oversee and manage the primary data collection and analysis activities.

Demonstrated expertise leading all aspects of quantitative and qualitative research (research design and methodologies, tools development, data collection, analysis, and writing) in developing country contexts.

Equity (Social and Gender Inclusion) Expert(s), ideally with Experience in Crises and Conflict Environments

Support all Contexts by contributing to secondary data analysis and primary data collection and analysis related to equity analyses. Depending on the size of the country and team, you may need multiple experts that specialize in disability, gender, ICT, crises and conflict, etc.

Demonstrated experience leading equity research and analysis related to education. If RERA has been conducted in this environment, ideally a person familiar with the RERA data.

Summary of LLA Team Roles and Qualifications to be entered with identified team members into Template 5 in Workbook Phase One.

Step 4b: Facilitate LLA training for LLA Team

Purpose: To provide training for the selected LLA Team, including the USAID Mission Lead and host-country Government Education Lead. Who leads: Individuals with LLA expertise.

☐ Convene LLA Team and USAID and host-country Government Leads for a 4-day training.

36

PHASE TWO

Back to Journey Map

PHASE TWO: Design and Background Data

Step 1: Develop the LLA Design

Purpose: To create a document that encompasses all of the different components of the LLA plan, including the background, objectives, data collection tools, and plans for the report. This document can be submitted to USAID Mission staff, host-country government, and other stakeholders for approval. Who leads: LLA team, who then shares with Technical Working Group (see Step 2 below) and USAID Mission for feedback.

Overview Phase 2 of the LLA involves four main steps that guide the design of the LLA. These steps are listed sequentially but can occur in the order that makes the most sense for the LLA team. Phase 2 takes approximately 6-8 weeks.

☐ Step 1: Develop the LLA Design

☐ Step 2: Form a Technical Working Group (TWG)

☐ Step 3: List key resources, record education landscape information and background data, and inventory literacy initiatives

☐ Step 4: Conduct an initial stakeholder analysis

37

PHASE TWO

Back to Journey Map

☐ Complete the Box 2: LLA Design Document, found in the Workbook. It integrates the templates and tools from the Workbook as indicated implementing in Box 2 below. This document will evolve throughout Phase One and Phase Two as these steps are completed.

Box 2: LLA Design Document Template

Table of Contents, List of Tables, List of Figures, Acronyms, etc.

LLA purpose and objectives • Write up the LLA objectives (Template 2) and the intended audience. If the LLA is part of a larger activity (i.e., part of an existing effort), describe how this fits into these larger efforts.

• State how this LLA aligns with USAID’s Reading MATTERS framework.

Background • Describe the context in which the LLA will take place, including the regions, grade levels, etc. per the parameters (Template 3, Workbook Phase One).

• Include critical information on the education systems landscape (Template 8) and education statistics (Template 9) for the above context. Integrate data from Education Sector Analyses, Rapid Education and Risk Analysis (RERA), Early Grade Reading Assessments (EGRA) and other contextual assessments conducted.

• Include any information on key literacy initiatives in the landscape that need to be considered (Template 11). • Describe the LLA team and their different roles (in Annex D the actual team’s staffing and LoE will be designated).

Methodology for data collection • Describe which literacy domains in the three Contexts will be covered by the LLA exercise (provide an outline of the key literacy domains as in Templates 13 and 14 in Phase Three).

• Describe how the secondary data collection will be conducted and name key documents and resources (Template 10).

• If applicable, describe how educational system mapping will be conducted (systemigram described in Step 2 in Phase 2).

• Outline stakeholder participations, including what steps they will be included in and who these stakeholders are (Template 12, Phase Three).

• Explain the proposed process for collecting primary data and which tools will be used (Template 15, Phase 3). Note that this plan will evolve and may be different in the final report.

• Summarize the steps to be taken to recruit and train data collection staff for primary data collection. • Identify any known limitations to the LLA design, including staffing limitations, logistical issues, political or social

constraints, etc.

Data analysis approach • Describe how the data will be analyzed (including disaggregation, triangulation, and coding and sorting).

38

PHASE TWO

Back to Journey Map

Work plan • Create a work plan (e.g., Gantt Chart) with details on when, where, and how deliverables and activities will be carried out. Ensure all human resources on the LLA team are reflected in the work plan.

Recommendations and final report • Include information on how the recommendations will be devised, including the stakeholder analysis and TWG prioritization.

• Describe the parameters of the final report and how it will be disseminated.

Annex A LLA Scope of Work (Template 4, Phase One)

Annex B LLA Team Roles and Qualifications (Template 5, Phase One)

Annex C Data Collection Tools (KII, Survey, Interviews, or FGDs) and IRB documents (as applicable)

Box 2: LLA document to be completed located in Workbook.

Step 2: Form a Technical Working Group (TWG)

Purpose: To identify a group of international and host-country experts (including government representatives) that ultimately advise the LLA exercise and: a) provide feedback on the LLA Design Document (Step 1 above); b) determine the list of final recommendations (Phase 3, Step 4); c) provide feedback on the Final Report Outline and Final Report (Phase 3, Step 5) ; and d) ensure on-going participation, collaboration, and ownership of the LLA process by the host-country. Who leads: LLA Team Lead with host-country government partners and in consultation with USAID Mission.

☐ Identify and coordinate with existing education sector TWGs, Local Education Groups, Steering Committees, etc. to either form a new TWG to provide oversight for the LLA, or to orient an existing TWG with the LLA.

TWG members should include education sector stakeholders from government, parastatal, non-government, and civil society institutions/organizations that support education and literacy efforts in the country/region. Example participants might include ministry officials, representatives from non-governmental organizations (NGO) and community-based organizations (CBOs), teacher unions, parent alliances (parent teacher associations, school management committees), teachers and head teachers, a book publisher representative, and other private sector and donor representatives as appropriate. If other assessments have been conducted, such as Education Sector Analyses, Rapid Education and Risk Analysis (RERA), and Early Grade Reading Assessments (EGRA), representatives from these assessment teams should be included in the

39

PHASE TWO

Back to Journey Map

TWG. While robust participation is ideal, the number of participants for this can vary greatly depending on stakeholder interest and time during the course of the LLA, and the culture of other TWGs in the country.

☐ List TWG members in Template 6 of the Workbook, following the example provided below.

Table 6: TWG Members

Name Affiliation Description of Role

e.g., Mr. Ezekiel Stanley MoE, Standards and Curriculum

Director of Department; oversees changes and revisions to standards and curriculum.

e.g., Ms. Mariam Ali MoE, Teacher Training Institute

Director of TTI; oversees hiring and professional development of TTI faculty; manages teams that revise changes to TTI curricula

Example of entries for Template 6: TWG Members in Workbook Phase Two.

Step 3: List key resources, record education landscape information and background data, and inventory

literacy initiatives

In this step, users collect data that help inform the Design Document and provide the overall context of the country or region where the LLA will be conducted. This data will also be used in the background section of the Final Report.

Step 3a: Identify key documents and resources

Purpose: Identify documents and data sources critical to answer the guiding questions in each Context (for Part Two). This table will serve as the basis for the Reference list in the final report and should be updated throughout the LLA process as the data analysis is conducted. This is included in the Planning and Design stage as it is important to include in the Design Document so the TWG and Government partners can add to this list. Who leads: LLA Team Lead and host-country education representatives.

40 Back to

Journey Map

When: After the parameters have been set and the team formed in Steps 2 and 3 and before any in-depth document analysis has commenced. Steps 5a-c can be completed simultaneously.

☐ Identify and complete the Key Documents and Resources table (example next page) in the Workbook Phase 2, Template 7.

☐ Keep updating the template throughout the process, as each can be consolidated with the others and serve as the references section in the final report.

41

PHASE TWO

Back to Journey Map

Table 7: Key Documents and Resources

a. Title of Resource b. Source/Links c. Year d. Author e. Information

Country Context

Rapid Education and Risk Analysis (RERA)

Education Sector Analysis

Education Systems, Context 1

e.g., Education Law Kenya Basic Education Act 2013 Parliament Passed into law in 2013; revised in 2017

e.g., National Education Plan

Learning Environment, Context 2

e.g., Academic research and studies Implementing large-scale instructional technology in Kenya: Changing instructional practice and developing accountability in a National Education System

2017 Piper, B., Oyanga, A. Mejia, J. and Pouezevara, S.

Impact of the National Tablets Program to increase the number of classroom observations done by coaches and to improve student learning outcomes.

e.g., EGRA reports, national exam results*

Kenya Tusome Midline EGRA Report

2017 MSI, Inc. Results of midline evaluation of USAID-and DFID- funded EGR program

Home and Community, Context 3

e.g., Academic research and studies

e.g., School Management Committee (SMC) and Parent-Teacher Association (PTA) policies, reports, and strategies

* As described in the introduction section, literacy assessment data are critical to this LLA, and if there are no data for the country or region where the LLA is being conducted, this

should be prioritized in the LLA Design.

Example of entries for Template 7: Key Documents and Resources in Workbook Phase Two.

http://kenyalaw.org/lex/rest/db/kenyalex/Kenya/Legislation/English/Acts%20and%20Regulations/B/Basic%20Education%20Act%20No.%2014%20of%202013/docs/BasicEducationActNo14of2013.pdfhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-and%22HYPERLINK%20%22http:/shared.rti.org/content/implementing-large-scale-instructional-technology-kenya-changing-instructional-practice-andhttp://shared.rti.org/content/tusome-external-evaluation-midline-reporthttp://shared.rti.org/content/tusome-external-evaluation-midline-report

42

PHASE TWO

Back to Journey Map

Step 3b: Record the Education Systems Landscape

Purpose: The Education System Landscape table documents the basic structure of the education system and how it functions. The table provides important contextual information about the system’s basic structure, the level of decentralization, and how it is financed. This contextual information will aid in understanding the system’s assumptions about literacy learning in the primary grades as well as how the system is organized. This information will help inform the background sections of the Design Document and Final Report. While Context 1 Guiding Questions provide a deep dive into aspects of the system, as they align to Reading MATTERS literacy domains, this general information is important for guiding the refinement of the different questions in the Guiding Questions tables and providing overarching data before data collection commences. Most of this information can likely be gathered based on country-level experts’ knowledge, the overarching Education Policy, and the education authorities’ websites. Integrate relevant ESA and RERA data if previously collected. Who leads: LLA Team Lead and Country-Level Education Advisors.

☐ Complete the full Education Systems Landscape Table in Workbook Phase Two (Template 8). Modify as needed. An example is provided below.

Table 8: Education Systems Landscape

Education System Questions Details Source (Link)

1. Basic structure of the system: Number of years and official entry age for each level.

e.g., 12 years of basic education, 2-6-4 (2 pre-primary, 6 primary, 4 lower secondary). Upper secondary and tertiary are beyond the basic education system. Entry age for pre-primary is 4; grade 1 is 6.

Education Policy

2. Early grade literacy is established during which grades per the national standards and curriculum (i.e., preschool to grade 2 or kindergarten-grade 1, etc.)? What literacy instructional approach is used? What languages are used for instruction, and in what grades?

•Early grade reading starts in pre-primary 1 and extends through Standard 4 (therefore six years). Pre-primary taught in Kiswahili with English as a subject (and Arabic as a language taught in religious schools). •Pre-literacy starts with syllables (agglutinated language) and phonics (name of letters, syllable sounds, and shape).

Education Policy. National curriculum and review by SIL report

43 Back to

Journey Map

PHASE TWO

3. Teacher education (pre-service). What division is responsible for pre-service teacher education?

•National teacher training institute provides pre-service for all government schools. Non-formal learning center provided by Department of Adult and Non-formal Education.

Education Policy and MoE website

4. Professional development (in-service). What division is responsible for administering in-service education?

•Teacher Training Centers in all of the decentralized district offices. Non-formal learning center in-service training largely provided by NGOs in collaborate with Department of Adult and Non-formal Education.

Education Policy and MoE website

5. Assessment. In what grades is literacy assessed annually? What literacy constructs are tested? What assessment tools are used?

School administered: Grade 2 Centrally administered: Grade 4 Regional assessment (cross-5 countries): Grade 5

Education Policy. National Exam Board’s website and strategy

6. Decentralization. Describe the degree to which the education system is decentralized. At what levels of the system are particular decisions made?

Central Ministry oversees the policy, pre-service teacher training, curriculum and materials developed, and examinations and assessments (as well as inspections and monitoring). The district education offices oversee in-service education training, mentoring and coaching.

Education Strategy and Education Policy

Example of entries for Template 8: Education Systems Landscape in Workbook Phase Two.

Step 3c: Record the background education statistics

Purpose: The Background Education Statistics table below consolidates the data on critical education and related indicators and contextualizes the education system. Again, this information will help inform the background sections of the Design Document and Final Report. Who leads: LLA Team Lead and Country-Level Education Advisors.

☐ Complete the full Background Education Statistics Table in the Workbook Phase Two (Template 9). Users should complete the table to the extent that data exists. If data has been collected per the RERA or an ESA, integrate the data into the table. Many countries will not have disaggregated data on language, disabilities, and gender, but gaps will provide a picture of the extent to which the data is gender and disability-responsive. An example follows.

44

PHASE TWO

Back to Journey Map

Accompanying tools/resources: Data can be accessed from national education reports and datasets as well as the Demographic Health Survey

and international agency datasets at http://databank.worldbank.org, http://data.uis.unesco.org/, https://knoema.com, and

http://www.who.int/gho/countries/en/.

Table 9: Background Education Statistics

Education Statistic (National) Data (Year) Source (Link) Education Statistic Data (Year) Source (Link)

Pre-primary education

Pre-primary gross enrollment (or intake) ratio (GER), by sex and disability

Total: 112.5% Male: 113.5% Female:111.5% (2018)

http://uis.unesco.org/country/GH

Pre-primary school net enrollment ratio (NER) by sex

Total: 72.2% Male: 73.8% Female: 70.6% (2018)

http://uis.unesco.org/country/GH

Pupil teacher ratio for pre-primary (policy and actual national average)

32.8 (2016 avg.) https://www.indexmundi.com/facts/ghana/pupil-teacher-ratio

Gender parity indices for pre-primary GER or NER

1 (2016) https://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.html

Percent of pre-primary students enrolled in government schools, by sex and disability

Total: 72.8%

(2015) https://ourworldindata.org/pre-primary-education

Actual (average) entry ages in pre-primary

3 (2016) https://ourworldindata.org/pre-primary-education

Primary education

Percent of primary school students who have attended pre-primary, by sex and disability

Percent of primary school students who have attended government-run pre-primary, by sex and disability

Primary school gross enrollment ratio (GER), by sex and disability

Primary school net enrollment ratio (NER), by sex and disability

Gender parity indices for primary GER or NER

Primary dropout rates, by sex and disability (if data)

Survival to last grade of primary (%), by sex and disability

Primary to secondary school transition rate, by sex and disability

http://databank.worldbank.org/http://data.uis.unesco.org/https://knoema.com/http://www.who.int/gho/countries/en/http://uis.unesco.org/country/GHhttp://uis.unesco.org/country/GHhttp://uis.unesco.org/country/GHhttps://www.indexmundi.com/facts/ghana/pupil-teacher-ratiohttps://www.indexmundi.com/facts/ghana/pupil-teacher-ratiohttps://www.indexmundi.com/facts/ghana/pupil-teacher-ratiohttps://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.htmlhttps://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.htmlhttps://tradingeconomics.com/ghana/school-life-expectancy-pre-primary-gender-parity-index-gpi-wb-data.htmlhttps://ourworldindata.org/pre-primary-educationhttps://ourworldindata.org/pre-primary-educationhttps://ourworldindata.org/pre-primary-educationhttps://ourworldindata.org/pre-primary-education

45 Back to

Journey Map

Repetition rates for primary school, by sex and disability

Reading data for primary (i.e., % of children reading at grade level per EGRA), by sex and disability

Enrollment trends by type of disability (and gender if data)

# of students (or %) with disabilities enrolled in primary

Pupil textbook ratio for primary (policy and actual national average)

Pupil teacher ratio for primary (policy and actual national average)

Actual (average) entry ages in primary

Available data on school-related gender-based violence (SRGBV)

Out-of-school children (and adults)

Percent of primary school age children not in school, by sex and disability

National literacy/literacy rate, by sex (and age group) and disability (useful for the Context 3 context)

Education Financing

Government expenditure on education as % of GDP

Government expenditure on education as % of total government expenditure

% of education budget spent on salaries, non-personnel recurrent expenditures (i.e., textbooks) and capital expenditures

Government expenditure per student for pre-primary

Government expenditure per student for primary

Example of entries for Template 9: Background Education Statistics Table in Workbook Phase Two.

46

PHASE TWO

Back to Journey Map

PHASE TWO

Step 3d: Conduct an inventory of all the literacy programs (government, NGO, CBO, etc.)

Purpose: To identify the key early grade reading and literacy initiatives (preschool to primary) being implemented in the country/region that

directly align with the literacy domains in the Reading MATTERS framework.

Who leads: LLA Team Lead and Country-Level Education Advisors.

☐ Using Template 10 in Workbook Phase Two, include key programs in the country or region that directly align with the programmatic literacy domains in the Reading MATTERS framework. Initiatives beyond these literacy domains, such as infrastructure development of schools, do not need to be captured in this inventory; nor do programs that are very small-scale and/or do not have demonstrated results.

Table 10: Inventory of Literacy Initiatives

Example of entry into Template 10: Inventory of Literacy Initiatives in Workbook Phase Two.

Agency/ Organization

Initiative Name (Years)

Initiative Type & MATTERS Literacy domain

Target Audience

Donors/ Partners

Literacy Focus Area(s)/ Activities

LLA Context(s)

Government

e.g., National Literacy Programme 2010-present

Curriculum & TLM (evidence-based standards, norms and policies)

All grade 1-3 children

USAID, UNICEF

Reaches all government schools nationally and guides grade-level literacy instruction. Emphasizes phonemic awareness. Delivers in 5 languages depending on region.

1, 2

USAID & Implementing Partners

Multilateral (e.g. UNICEF, etc.)

Non-state schools & education providers

47 Back to

Journey Map

Step 4: Conduct an initial stakeholder analysis

The host-country government is critical to the LLA. At least two host-country experts are part of the

LLA Team, and host-country government representatives are part of the TWG. In addition to the

host-country government representatives from central and decentralized offices, representatives

from non-governmental organizations (NGO), community-based organizations (CBOs), teacher

unions, parent alliances (parent teacher associations, school management committees), and

teachers and head teachers are key stakeholders. Where stakeholders have been identified in an

ESA or RERA, integrate these stakeholders into the analysis below.

Purpose: To identify key actors and stakeholders across the three Contexts, and to ensure that

these stakeholders are included in the stakeholder worksho