© SMARTEDGE Improving educational outcomes for children in low-income countries HOW THE PROGRESS AND PITFALLS OF MODERNIZING EDO © SMARTEDGE STATE’S EDUCATION PROVIDE LESSONS FOR THE FUTURE AUTH OR: Martin Elias De Simone 2025 © 2025 International Bank for Reconstruction and Development/The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Early-Grade Literacy Instruction in Edo State Contents 5 Acknowledgments 7 Acronyms 8 Executive Summary 12 I. Introduction Methodology 13 of the report Structure  14 15 II. The context in Edo State 18 III. Evidence-based interventions: the EdoBEST program The structured pedagogy program 18 EdoBEST@Home: c  omplementing learning in school with learning at home 41 in the last few The stronger focus on foundational learning  years of the program 44 48 IV. Is EdoBEST working? inputs and outputs EdoBEST is improving  48 to be improving learning EdoBEST appears  50 56 V. Political commitment, leadership, and financing political economy challenges Institutional and  58 3 Early-Grade Literacy Instruction in Edo State 63 VI. A focus on implementation capacity Data as a key element 63 of the program Scalability  66 A public-private partnership  for implementation 68 of communities The role  72 The delivery team 74 The cost of EdoBEST  and its sustainability 76 79 VII. Conclusion and key lessons learned 82 References 4 Improving educational outcomes for children in low-income countries Acknowledgments Acknowledgments T his report was written by Martin De Simone under the guidance of Scherezad Latif, Education Practice Manager for Western and Central Africa. The team is grateful for the critical contributions of the previous Practice Manager, Halil Dundar, whose leadership was instrumental in the project’s initial stages. The report reflects the efforts and contributions of numerous individuals and organizations. Thomas O’Brien and Juan Manuel Moreno Olmedilla provided invaluable expert feedback at multiple stages. Many discussions with Luis Benveniste and Jaime Saavedra also helped shaped multiple sections of the report. The team would like to extend our sincere gratitude to the government authorities of Edo State, Nigeria, as well as federal level authorities. Special thanks go to His Excellency Godwin Obaseki, Executive Governor of Edo State, for his steadfast commitment to educational reform. We sincerely appreciate the leadership and dedication of Dr. Joan Oviawe, Honorable Commissioner of Education, who envisioned the EdoBEST program, and Mrs. Eyitayo Salami, Chair of the State Universal Basic Education Board (SUBEB), who continued with its implementation over the last few years. The Secretary of SUBEB, Imuentinyan Akin-Omoyajowo, has been a key partner. The collaboration with the delivery team of the EdoBESST project has been critical to this report. We particularly recognize Mr. Dele Salami (Project Coordinator) for his exemplary leadership. We also extend our appreciation to all members of the EdoBESST team for their significant contributions, including Elliot Uhuonoma (Deputy Project Coordinator), Dayo Forsythe (M&E Officer), Christiana Oshomoh (Project Accountant), James Owolabi (Procurement Officer), Hauwa Etsenmuhie (Assistant Procurement Officer), Isoken Osemwimgie (Social Safeguard Officer), Queen Abusonmwan (Grievance Redress Officer), Esther Ogunode (Environmental Safeguard Officer), Friday Asogbaha (Finance Officer), Unyinme Ajah (DLI 3 Focal Person), Boniface Aliu (DLI 4 Focal Person), Sade Franklin - Osakwe (DLI 5 Focal Person), Peter Obaraye (DLI 6 Focal Person), Asemota Izoduwa (DLI 7& 8 Focal Person), Samuel Abada (Head, PFMU), Jude Adamu (Accountant, PFMU), Maureen Obarisiagbon (Internal Auditor), Ruth Aliu (Accelerator Program focal point), and Bashiru Imuekemhe (Assistant Project Accountant). The team is indebted to numerous experts who have provided technical expertise that shaped the recommendations and analysis in this report, many of whom contributed to the Mid-Term Review of the World Bank-supported project. The list of experts includes Michael Crawford, Steven Bakker, Fernando Cartwright, Gustavo Arcia, Angélica Rivera-Olvera, Maria Rebeca Barron, Agatha Gikunda, and Melissa Diane Kelly. Special thanks go to Wuraola Mosuro for her exceptional support throughout the implementation of EdoBESST and to Dilip Parajuli for his constant guidance. Tolulope Oluseun Idowu supported the 5 Improving educational outcomes for children in low-income countries Acknowledgments team constantly with great versatility. Federico Tiberti led the data analysis aspects of the report, and Federico Manolio contributed from the project’s monitoring and evaluation perspective. Eliot Jolomi Dikoru supported the qualitative aspects of the report, including conducting multiple focus groups and in-depth interviews. The report also benefits from other studies conducted together with colleagues from the American Institute of Research (AIR), including Anasthasie Liberiste-Osirus, Kelsey Woodrick, Jill Pritts, and Pooja Nakamura. The report would not have been possible without the support of the Accelerator Program funded by the Bill & Melinda Gates Foundation and the discussions over the years with Benjamin Piper, Nic Spaull, Clio Dintilhac, and Prasangi Ranaweera. The team also acknowledges the EdoBESST team from the World Bank, including Gloria Joseph-Raji (co-Task Team Leader of the project), Cindy Ikeaja, Ida Mboop, Gladys Omone Fayomi, Abdulgafar Olukayode Tijani, Ese Joan Adeluyi, and Sunday Osoba. The team also thanks Saamira Halabi, Alison Grimsland, and Tihtina Zenebe Gebre, who peer-reviewed the document and provided valuable feedback to help strength it. 6 Improving educational outcomes for children in low-income countries Acronyms Acronyms COVID-19 Coronavirus Disease 2019 LDOs Learning and development officers DIBELS Dynamic Indicators of Basic Early LGEA Local Government Education Literacy Skills Authority EdoBESST Edo Basic Education Sector and MICS Multiple Indicator Cluster Surveys Skills Transformation NEI+ Northern Education Initiative Plus EdoBEST Edo Basic Education Sector NPA National Personnel Audit Transformation PforR Program for Results ELAS Edo Learning Assessment System PIU Project Implementation Unit EMIS Education Management Information System PPP Public-Private Partnership FCDO Foreign, Commonwealth & SBMC School-Based Management Development Office Committees GBV Gender-Based Violence SDG4 Sustainable Development Goal 4 GEEAP Global Education Evidence SUBEB State Universal Basic Education Advisory Panel Board GEPD Global Education Policy UBEC Universal Basic Education Dashboard Commission GPE Global Partnership for Education UNICEF United Nations Children’s Fund GPF Global Proficiency Framework USAID United States Agency for International Development GRM Grievance Redress Mechanism 7 Early-Grade Literacy Instruction in Edo State Executive Summary © SMARTEDGE T he world is grappling with a learning crisis, which is particularly severe in developing countries. Understanding the experiences of successful education reforms is critical for designing and implementing more impactful programs. In this spirit, this report explores the lessons learned from the Edo Basic Education Sector Transformation (EdoBEST) program, an initiative implemented in Edo State, Nigeria, since 2018. Edo is a state in the South of the country with a population of approximately 5 million people. Before the reforms started, the state was dealing with multiple education problems including low teaching quality, and the absence of a coherent set of instructional materials. Widespread teacher absenteeism compounded the problem, associated with a lack of effective accountability and oversight frameworks. At the school level, inadequate infrastructure presented significant barriers, while the education system as a whole struggled with weak governance and accountability, alongside a limited focus on improving learning outcomes. The main component of EdoBEST is a technology-enabled structured pedagogy program, a coherent package of investments that includes lesson plans, learning materials, skills-based ongoing teacher coaching, continuous classroom observations, and teacher mentoring that are carefully coordinated to reinforce each other. A plethora of recent evidence has identified structured pedagogy as one of the most cost-effective ways to improve learning, especially in contexts of low teacher capacity. 8 Improving educational outcomes for children in low-income countries Executive Summary The objective of this report is twofold. First, to understand what has worked and what needs improvement in Edo, enabling the fine-tuning of interventions for a more impactful and accelerated effect on learning outcomes. Second, to distill lessons that can be useful in other settings where education reforms are being designed or implemented, particularly those focused on improving foundational learning outcomes. Some of the main conclusions of the report are highlighted below. Impact on learning ● Although EdoBEST is still an ongoing reform program, its six years of implementation provide a solid foundation from which to draw critical lessons. Learning outcomes in the state are still low, but findings suggest there are noteworthy, encouraging features upon which to continue to build. ● Learning outcomes are improving as a result of the program interventions. A large dataset shows that for primary school pupils, average test scores in quantitative reasoning have risen by 12.5 percent between 2021 and 2023, and in verbal reasoning by 13.3 percent. Furthermore, the effect of the program is increasing over time. This trajectory of improvement underscores the importance of sustained efforts and long-term commitment to education reforms. The political economy of reforms ● Central to EdoBEST’s success has been the political leadership and entrepreneurship that identified opportunities to implement these reforms and build consensus around them. Rather than merely increasing funding, the program focused on strategic investments, particularly in “start-up” costs, which remained within feasible budgetary limits. Financing ● Policy makers must be prepared to meet a significant cost to fund the program. These costs include up-front investment, plus a 24–40 percent increase at the outset in the per pupil cost of education provision in primary education and an 18–30 percent increase for secondary education. Despite the costs, these interventions are among the most cost-effective to improve learning. In Edo, the unit cost of the interventions is going down—from around US$65 per pupil at the outset to US$16 in 2023. Evidence-based interventions ● The structured pedagogy program, grounded in evidence-based practices, underscores the importance of continuous improvement and policy refinement. For instance, while the lesson plans used in Edo are based on the science of reading, there is room for further enhancement to make the program more effective and responsive to the needs of students. Edo also shows that highly scripted approaches for lesson plans can be effective in contexts of low teacher capacity. ● The experience shows that education reforms that focus on learning outcomes are complex, and it is not possible to tackle all challenges simultaneously. Educational authorities must therefore regularly monitor and adjust priorities as needed. In Edo, for instance, language of instruction was not among the main priorities initially, but it might need to be in the future to deliver even larger uplifts in educational performance. 9 Improving educational outcomes for children in low-income countries Executive Summary Teachers and educational leaders ● Teachers play a pivotal role in the success of major reform programs. Indeed, the attendance rate in primary schools has improved dramatically—from 63 percent in 2018/19 to 82 percent in 2023/24. Over 16,000 teachers—the vast majority of the workforce—have benefited from targeted training in EdoBEST. Despite the support provided through coaching and scripted lessons, teacher capacity remains a challenge. Nonetheless, there has been a noticeable improvement in teaching practices, highlighting the need for systemic reforms to ensure high- quality teacher recruitment and development from the outset. This dual focus on immediate support and long-term capacity building is essential for sustainable success. ● School managers, and especially headteachers, have been an essential component of the success of the program. School management shows strong capabilities in providing instructional leadership and managing core functions effectively. This strength positions school management as a valuable partner in tackling educational challenges and implementing further reforms. Educational technologies ● Education technology must be tailor-made to the environment in which it is deployed. With that, it has proven to be a powerful tool when integrated coherently with other interventions, such as teacher training and school modernization. It should not be viewed as a standalone solution, but rather as a complement to a broader, integrated approach. This integrated approach ensures that technology enhances rather than distracts from the core educational objectives. Technologies can also be a way to elevate the status of frontline providers and secure their support. ● Technology for learning outside the classroom, or “@Home technology”, shows promise, especially in contexts where education disruptions are frequent, such as due to insecurity or climate-induced shocks. However, the successful implementation of these technologies requires overcoming substantial challenges, as evidenced by the experience in Edo. Having a robust alternative delivery mechanism can be invaluable in maintaining the continuity of learning. Implementation capacity ● While careful design is crucial for an educational reform program like EdoBEST, the implementation process is equally or perhaps even more significant. Drawing from the principles of the science of implementation, it is evident that the success of such reforms hinges on the fidelity and quality of their execution. For EdoBEST, this involves not only the initial roll-out but also continuous monitoring, adaptation, and support to address emerging challenges and contextual nuances. Effective implementation requires building strong local capacity, fostering stakeholder collaboration, and ensuring that the program’s components are delivered as intended. Additionally, it necessitates a robust feedback mechanism to inform iterative improvements that enhance the program’s impact over time. 10 Improving educational outcomes for children in low-income countries Executive Summary ● In Edo, the program was implemented through a public-private partnership (PPP), where a private provider managed key aspects of public schools while maintaining critical resources under public control. This model offers significant advantages for scaling up interventions. However, its success depends on careful implementation to ensure strong accountability mechanisms for the service provider, prevent the exclusion of vulnerable students from the program’s benefits, and uphold government ownership of essential inputs and outputs, such as lesson plans and collected data. PPPs should always be implemented based on a careful analysis of costs, benefits, and institutional capacity. A focus on sustainability must be embedded from the beginning of the reforms. Public-private partnerships can be effective in contexts of low institutional capacity and where rapid results are needed, but these should be coupled with measures to ensure the effective transfer of knowledge and capacity to the state bureaucracy. This approach ensures that the benefits of the reform are not only immediate but enduring. Edo’s experience shows that it is possible to build capacity in a relatively short period, even starting from a very low baseline. ● Data is fundamental to the success of education reforms. For data to be truly helpful, it needs to be actively used. In more advanced stages, making data public can enhance accountability and encourage contributions from think tanks and civil society, fostering original research and new intervention avenues. However, data privacy and security must be diligently respected to protect the integrity of the information and the individuals to whom it may relate. The educational challenges faced by so many low-income countries in Africa are daunting. Yet the need to do better—much better—for children across the continent is more important than ever. EdoBEST is still an ongoing, work-in-progress reform program, but its six years of implementation are showing promise and shining a light on ingredients that can help drive meaningful and lasting improvements in educational outcomes. 11 Early-Grade Literacy Instruction in Edo State I. Introduction © SMARTEDGE W hich factors contribute to the success of educational reforms in a developing context? How can evidence-based strategies be adjusted to specific contexts? In what ways can technology enhance the learning experience for students? And what role does the political economy play in the implementation and sustainability of educational initiatives? These questions are at the heart of understanding the complexities and challenges of improving educational outcomes worldwide. The urgency of the situation is underscored in the current context, where the world grapples with a severe learning crisis. In low- and middle-income countries, a staggering 70 percent of 10-year-olds are unable to comprehend a simple age-appropriate text, placing them in learning poverty (World Bank et al., 2022). What is more alarming is that despite significant strides in education access globally, learning levels among enrolled students are declining (Le Nestour et al., 2022). Sub-Saharan Africa bears the brunt of this crisis, with an estimated 89 percent of children in learning poverty. While Nigeria lacks an official estimate, available surveys indicate very low learning outcomes and the highest number of out-of-school children worldwide. In this context, the need for successful education reforms that can deliver tangible results in a relatively short period is more pressing than ever. 12 Improving educational outcomes for children in low-income countries I. Introduction In Edo State, Nigeria, an ambitious program known as EdoBEST (Edo Basic Education Sector Transformation) provides valuable insights to answer these questions. Launched with the vision of overhauling the education sector, EdoBEST represents a comprehensive approach to educational reform, aiming to improve the quality of teaching and learning across the state. By leveraging evidence-based methodologies, integrating technology into classrooms, and putting a strong focus on monitoring, EdoBEST strives to create a scalable and sustainable model for educational reforms. This report delves into the lessons learned from the EdoBEST program, exploring how it has navigated the challenges of reforming education in a complex setting. The objective is twofold. First, to understand what has worked and what needs to be improved in Edo, to fine-tune interventions for a more impactful and faster effect on learning outcomes. Second, to distill lessons that can be useful in other settings where education reforms are at design or implementation stages and with a focus on improving learning outcomes—particularly foundational learning outcomes. Methodology The methodology underpinning this report is multifaceted and rigorous, designed to provide a comprehensive analysis of the EdoBEST program in the context of educational reform in Edo State, Nigeria. It is informed by the World Development Report 2018 (World Bank, 2018), whose recommendations are structured around three policy actions to address the learning crisis: i) assess learning to make it a serious goal; ii) act on evidence to make schools work for learners; and iii) align actors to make the system work for learning. This helped shape the World Bank’s Western and Central Africa Education Strategy (Savchenko et al., 2022), which provides the theoretical framework for this study. That education strategy delineates three critical domains essential for understanding and evaluating educational reforms: strengthening strategic leadership for long-term impact, investing in high-impact interventions, and enhancing implementation capacity. These domains offer a structured lens through which the EdoBEST program’s strategies, outcomes, and challenges can be Strengthening Strategic Leadership systematically assessed and understood. Using these domains for Long-term Impact is particularly suitable for the case of Edo since the vision of the state’s learning agenda aligns with the regional strategy: to ensure that all girls and boys arrive at school ready to learn, acquire real Investing in High-impact Interventions Across the Learning Lifecycle for Quick Wins learning, and become ready to enter the job market with the right skills to become productive and fulfilled citizens. Improving Teaching Expanding Building This report employs a mixed-methods approach to ensure a robust & Learning opportunities Job-relevant Skills and nuanced understanding of the EdoBEST program’s impact and the broader educational landscape in Edo State. The quantitative component of this study involves a detailed analysis of both Enhancing Implementation primary and secondary data sources. This includes administrative Capacity for Long-term Impact data, surveys, and information collected specifically for this study, such as through the Global Education Policy Dashboard (GEPD) Source: World Bank et al., 2022. 13 Improving educational outcomes for children in low-income countries I. Introduction and structured classroom observations. Overall, the data captures a wide range of indicators related to student learning outcomes, teacher performance, and school management practices. Secondary data were sourced from existing educational records, government databases, and previous studies, providing a contextual backdrop against which the EdoBEST program’s effects can be measured. Complementing the quantitative analysis, qualitative research methods enrich the study’s findings. Structured and unstructured interviews and in-depth technical discussions were conducted with a diverse array of key stakeholders, including education officials, school administrators, teachers, parents, and students. These interviews offered invaluable insights into the experiences, perceptions, and recommendations of those directly involved with or affected by the EdoBEST program, shedding light on the nuances that quantitative data alone cannot capture. Furthermore, multiple focus groups were organized with teachers, parents, and other key actors, facilitating in-depth discussions among stakeholders. These sessions provided a platform for participants to share their experiences, challenges, and aspirations regarding educational reform, thereby adding depth and perspective to the study’s findings. The qualitative aspects of the study are particularly useful for understanding the details of implementation at scale and extracting lessons for other settings. Lastly, the report draws upon the wealth of experiences accumulated throughout the years of the EdoBEST program’s implementation. Best practices and challenges encountered during the program’s journey are integral to understanding its impact and refining future initiatives. This experiential knowledge, combined with the data and insights gleaned from quantitative and qualitative research methods, forms the foundation of this comprehensive analysis of lessons learned from the EdoBEST program within the broader landscape of educational reform in developing contexts. Structure of the report This report is organized in six main sections. The first section provides an overview of the context in Edo. Section II summarizes the main interventions that are part of EdoBESST and how they are based on the available evidence. Section III assesses the impact of interventions, both in terms of outputs and learning outcomes. Section IV focuses on some of the political economy of the reforms, including the importance of commitment, leadership, and financing. Section VI reviews the implementation of EdoBEST, evaluating critical aspects such as the implementation modality and scalability of the program. Finally, Section V offers conclusions and identifies key lessons learned. 14 Early-Grade Literacy Instruction in Edo State II. The context in Edo State © SMARTEDGE N igeria’s education sector faces multiple challenges, including in terms of access and quality of learning. The country has the largest number of out-of-school children in the world, with a total of 15 million students out of basic education (UNESCO, 2024),1 which includes primary and lower secondary education, according to the 2004 Universal Basic Education Act. Learning levels are low, including for those in school. The latest Multiple Indicators Cluster Survey (MICS) shows that only 26.8 percent of children ages 7–14 demonstrate foundational reading skills, and only 25.3 percent demonstrate foundational numeracy. For those out-of-school, the values drop to 4.5 and 5.3, respectively. The low outcomes are caused by many factors, including supply-side constraints (such as the low availability of schools and the low-quality teachers), demand-side constraints (mainly economic factors and, sometimes, socio-cultural norms), and systemic challenges such as the low levels of public funding and the complex decision-making arrangements. Edo State, situated in the South-South geopolitical zone of Nigeria, is a highly diverse home to multiple ethnic groups. This includes the Edo (also known as Bini), Esan, Ora, Akoko-Edo, Owan, and Afemai communities, among others. The administrative structure of Edo State is divided into 1 Other sources estimate between 11.5 million and 15 million. 15 Improving educational outcomes for children in low-income countries II. The context in Edo State 18 Local Government Areas (LGAs). The 2023 population pyramid of Edo State shows that of the approximately 5 million people inhabiting the state, almost half of the population (48 percent) is below the age of 20 and 38 percent is below the age of 15. This makes education a fundamental sector within the state, and a key tool to create the skills to capture a demographic dividend in stronger economic growth. FIGURE 1. Population pyramid of Edo State, 2023 Population pyramid of Edo State 85+ 80-84 75-79 70-74 65-69 60-64 55-59 2023 female 50-54 45-49 2023 male 40-44 35-39 30-34 25-29 20-24 15-19 10-14 4-9 0-4 400000 300000 200000 100000 0 100000 200000 300000 400000 Source: Authors’ estimations based on the National Bureau of Statistics (NBS). Back in 2018, before the series of education reforms commenced, Edo was facing multiple challenges in its education sector, many of which are still present today. Teachers relied on antiquated materials and instructional techniques, lacking the contemporary training essential for fostering 21st-century learning. Both educators and school administrators were short of important resources and educational materials necessary for enhancing the teaching and learning experience. Additionally, there was widespread teacher absenteeism, exacerbated by the absence of effective accountability and oversight frameworks. At the school level, multiple infrastructure challenges were present, and the system lacked a focus on improving learning outcomes and suffered from weak governance and accountability measures. An analysis of the outcomes in Edo shows the status of the education sector in the state. According to the Multiple Indicator Cluster Survey (MICS) conducted in 2021, the adjusted net attendance rate for primary education in Edo is 87.5 percent, higher than the Nigerian average of 68.4 percent. The rate is slightly higher for girls (88.6 percent) than for boys (86.4 percent). For lower secondary school, the value goes down to 69 percent and shows a reversal of the gender gap seen in primary school, with a value of 71.4 percent for boys and 67.2 for girls. In terms of learning, among children ages 7–14, 48 percent demonstrate foundational reading skills (49.6 percent for females and 46.3 percent for males). The value is significantly higher than the 16 Improving educational outcomes for children in low-income countries II. The context in Edo State FIGURE 2. Foundational skills for children ages 7–14, 2021 Percentage of children ages 7-14 who demostrate Percentage of children ages 7-14 who demostrate foundational reading skills foundational numeracy skills Lagos Lagos 80 75 Anambra Anambra Enugi Ebonyi 60 Imo Edo 50 Edo Percentage Percentage 40 25 20 Zamfara Kebbi Jigawa 0 Jigawa 0 40 60 80 40 60 80 Monetary poverty Monetary poverty North Central North East North West South East South South South West Source: MICS 2021. average for Nigeria (26.8 percent) and for the South-South region of the country (36.8 percent). However, there is large room for improvement since the best-performing state (Lagos) reaches 87 percent of children with foundational reading skills. Similarly, 38.6 percent of children ages 7–14 demonstrate foundational numeracy skills in the state, much higher than the average for the country (25.3 percent) and for the South-South region (34.4 percent). However, nine states perform better than Edo, which suggests significant space for advancement. In short, the challenges in Edo’s education sector were (and indeed still are) many, varied, and interrelated—hence the response cannot be focused on one key element but rather needs to be reasonably comprehensive. The depth of problems—when students and families are being woefully let down by the education system—is indeed a clarion call to motivate dramatic, sustained reform. The state government responded by launching its Basic Education Sector Transformation (EdoBEST) project with the objective of improving teaching and learning practices in the state. This flagship initiative was originally structured around three main goals: i) enhance the quality of basic education provision anchored in new technologies; ii) ensure that all children of basic school age are enrolled in school and are actively participating and achieving the set learning benchmarks and outcomes; and iii) revamp basic education management and delivery to ensure that Edo State children are highly knowledgeable, competent, skillful, morally sound, and globally competitive. The EdoBEST program covers public schools only, which in 2023 served a total of 381,000 students. Of this total, some 293,000 students are in 1,002 primary schools, and a further 83,000 are in 287 junior secondary schools. 17 Early-Grade Literacy Instruction in Edo State III. Evidence-based interventions: the EdoBEST program © SMARTEDGE E doBEST represents a comprehensive set of education reforms aimed at enhancing teaching and learning in basic education. This section highlights the core interventions, noting that the program has encompassed numerous elements since its inception. It provides brief descriptions of each element, identifying key lessons learned from best practices and challenges encountered. At the outset there is an in-depth look at the structured pedagogy program, analyzing its key components. Next, the section examines Edo’s blended learning initiative, EdoBEST@Home, detailing its approach and impact. Finally, there is an exploration of how the program’s focus on learning outcomes has progressively intensified over the years, driving continuous improvement and adaptation. Through this analysis, the section aims to provide a comprehensive understanding of the interventions under EdoBEST and offer insights that can inform similar education reform efforts in other contexts. The structured pedagogy program The main component of EdoBEST is a technology-enabled structured pedagogy program, combined with a coherent package of investments that include lesson plans, learning materials, skills-based ongoing teacher coaching, continuous classroom observations, and teacher mentoring. These 18 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program are carefully coordinated to reinforce each other. The Global Education Advisory Panel2 identified structured pedagogy as a “smart buy,” one of the most cost-effective interventions to improve learning outcomes (GEEAP, 2023). It works particularly well in contexts of relatively low teacher capacity. Evidence of the effects of structured pedagogy in Sub-Saharan Africa has become available in the last few years. The basic approach has boosted education outcomes at a relatively affordable cost in countries including Kenya, Liberia, and South Africa,3 particularly in enhancing basic reading and numeracy skills. Several successful initiatives have involved providing teachers with guidance on the sequence of topics to be taught, without prescribing exact lesson scripts (noted in Kenya, Liberia, and South Africa). In some instances, the approach has gone a step further: a program in Kenya, for example, provided detailed lesson plans and scripts that proved beneficial (Gray-Lobe et al., 2022). Support for educators aims to ensure lessons build logically on each other, providing a clearer approach for students. Overall, the various elements of the structured teaching approach are designed to work together, enhancing program effectiveness.4 Despite this increase in the availability of evidence, it can be argued that lessons from the actual implementation of these programs are yet to be fully elucidated. The following subsections analyze each of the main components of the structured pedagogy intervention, namely the scripted lessons (including their utilization in the classroom), the measurement of learning, the coaching program, and the availability of other learning materials. For each, a brief description of the implementation in Edo State is followed by lessons learned, based on the identified best practices and the challenges that have been detected. The scripted lessons One of the main components of the structured pedagogy program is the provision of highly scripted and structured lesson plans for teachers. These lesson plans are delivered by teachers via tablets that track lesson progress in real-time. Lesson plans are for all subjects and all grades, starting in primary 1 up to junior secondary 3, a total of 9 years. The teachers receive the lessons in advance and are asked to review them ahead of their classes and then to follow them line by line. The plans include explanations, activities for the students, and other ways to lead the class with specific dialogue for teachers to use in class, as well as instructions to accompany the dialogue or guide interactive exercises. Different typographies identify the purpose of each line. Six years after the start of EdoBEST, the structured pedagogy program has been implemented widely across almost all of the state’s public primary and junior secondary schools. Teacher interviews and classroom observations have confirmed that the lesson plans are extensively used with high fidelity 2 The Global Education Evidence Advisory Panel (GEEAP) is an initiative designed to bridge the gap between education research and policy making. It brings together leading education experts from around the world to review and synthesize evidence on effective educational practices. The panel aims to provide policy makers with clear, actionable recommendations based on rigorous research to improve education outcomes, particularly in low- and middle-income countries. GEEAP is co-hosted by the Foreign, Commonwealth & Development Office, UNICEF, USAID, and the World Bank. Panelists are selected for their expertise in generating and using good evidence in education. 3 Kenya (Piper et al., 2014), Liberia (USAID, 2021b), and South Africa (Fleisch et al., 2016; Meiring, 2021). 4 (Piper et al., 2018). Examples include a roll out on a large scale, including nationwide in Kenya (Piper, Destefano, et al., 2018), extensively in Uganda (Brunette et al., 2019), and across several regions in South Africa (Ardington, 2023; Cilliers et al., 2022; Spaull, 2023). 19 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program in classrooms. Teacher feedback suggests that by and large, they adhere to—and do not skip—the components of the lessons, motivated by a feeling of professional obligation and believing that this is the best approach for students. As one teacher explained, “I make sure I do not skip any lesson. I take note of the time that I will use. I must complete the lesson, then save and exit, then go to the next lesson, no matter what. It is because if any lesson is not delivered, it’s going to count against my record, my ability. So as a teacher, I make sure that I must deliver that lesson. I will not leave anyone untouched. Even when we are tired, I still try.”5 Despite some initial resistance, teachers interviewed revealed that they highly value the lesson plans, stating that they simplify classroom management. They believe that the lesson plans enhance teaching by offering a clear structure and objectives, which were previously lacking. Additionally, lesson plans assist teachers in organizing the timing of activities, as they include estimated times and a timer indicating when to move on. Furthermore, lesson plans ensure uniformity in content delivery across the state, as all teachers follow the same curriculum and lesson guides. Several teachers also noted that the lesson plans include formative assessments to identify individual needs, including time for teachers to scan students’ work. Classroom observations show that teachers asked questions to students throughout the class and provided positive feedback. In interviews, teachers were fully able to describe how they use formative assessment in the classroom. Most formative assessments occur by listening to students read and asking them questions, which teachers appear to do continuously in class. To delve further into the inherent quality and appropriateness of the scripted lesson plan approach, this report conducted a more detailed analysis of the scripted lessons for English language for the early grades. The review of the lesson plans for early-grade literacy shows that, in general, they have a strong alignment with the science of reading.6 The lessons are highly scripted, especially when compared with other approaches in Nigeria. For instance, the Northern Education Initiative Plus (NEI+) follows the model of a typical lesson plan, where the objectives and some exercises might be outlined, but the teachers still have a large room to tailor the classes. In contrast, in Edo the lesson plans are very detailed. While some teachers might take some liberties—especially the most experienced and skilled ones—in general, they follow the lessons quite literally. This highly structured approach might not be ideal in places with experienced and highly qualified teachers, but it is considered a good tool for contexts with very low teacher quality. The lesson plans for early-grade literacy have an average of 1,205 words per lesson. This puts the lessons in Edo above the average of the 34 teacher guides across 19 projects reviewed by Piper et al. (2018). Since the lessons are set to last 30 minutes, the average is 40 words per minute, twice that of other countries. This reflects the high level of scripting the lessons have, even though there are still other countries that have even more words per minute, including in the English language. The teachers in Edo also receive a package of supplementary lessons, which are significantly longer (2,376 words on average) but do not have a pre-determined pace. 5 Direct references such as this one come from the analysis of lessons plans and its implementation conducted by the World Bank and the American Institute of Research. 6 The analysis of scripted lessons was conducted in the first half of 2024 and included a review of a sample of lesson plans, in-depth interviews with key stakeholders, and comprehensive classroom observations. The sample was not large and so the lessons should be interpreted with some caution. 20 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program FIGURE 3. Comparison of word counts for early-grade literacy lesson plans, 2018 Words per lesson Average words per minute Indonesia PRIORITAS Bahasa Indonesia G1 154 Nepal EGRP Nepali G2 2.61 Nepal EGRP Nepali G2 235 Nepal EGRP Nepali G1 3.00 Kenya Tusome Kiswahili G1 242 Uganda LARA/SHRP English G1 4.09 Jordan RAMP Arabic G1 258 Malawi EGRA Chichewa G2 5.53 Ethiopia READ TA Amharic G1 259 Jordan RAMP Arabic G1 6.45 Ethiopia READ TA Amharic G3 267 Ethiopia READ TA Amharic G1 6.48 Nepal EGRP Nepali G1 270 Ethiopia READ TA Amharic G3 6.68 Kenya Tusome English G2 282 Malawi MERIT Chichewa G1 7.27 Kenya DFID PRIMR Lubukusu G1-G2 288 Kenya Tusome Kiswahili G1 8.07 Kenya Tusome English G1 307 Uganda LARA/SHRP Runyankore-Rukiga G1 8.57 Kenya Tayari English/Kiswahili PP1 324 Kenya Tusome English G2 9.40 Malawi MERIT Chichewa G1 327 Kenya DFID PRIMR Lubukusu G1-G2 9.60 Malawi EGRA Chichewa G2 332 Uganda LARA/SHRP Luganda G3 10.07 Kenya Tayari English/Kiswahili PP2 348 Kenya Tusome English G1 10.23 Uganda LARA/SHRP English G1 368 Kenya Tayari English/Kiswahili PP1 10.80 Uganda LARA/SHRP English G2 379 Kenya Tayari English/Kiswahili PP2 11.60 Kenya DFID PRIMR Kikamba G1-G2 394 Ethiopia Afaan Oromo G3 12.03 Ethiopia Afaan Oromo G3 481 Uganda LARA/SHRP English G2 12.63 Ethiopia Afaan Oromo G1 610 Kenya DFID PRIMR Kikamba G1-G2 13.13 Uganda LARA/SHRP Runyankore-Rukiga G1 771 Ethiopia Afaan Oromo G1 15.25 Uganda LARA/SHRP Luganda G3 906 Malawi EGRA English G2 17.20 Malawi EGRA English G2 1032 Egypt GILO Arabic G1 23.09 Egypt GILO Arabic G1 1039 Kenya USAID PRIMR English G2 Haiti TO-TAL Creole G2 28.12 1203 Edo State Nigeria English G1-G3 Liberia EGRA Plus English G1-G3 30.36 1205 Liberia EGRA Plus English G1-G3 Liberia LTTP2 English G2 31.16 1366 Liberia LTTP2 English G2 1402 Malawi EGRA Chichewa G1 34.38 Nigeria RARA Hausa G2 1458 Nigeria RARA Hausa G2 36.45 Malawi EGRA Chichewa G1 1547 Kenya USAID PRIMR English G2 40.10 Kenya USAID PRIMR English G1 1549 Edo State Nigeria English G1-G3 40.17 Haiti TO-TAL Creole G2 1687 Liberia LTTP2 English G1 42.02 Liberia LTTP2 English G1 1891 Kenya USAID PRIMR English G1 51.63 South Africa IEP Sepedi G1 1944 Haiti TO-TAL French G2 61.32 Haiti TO-TAL French G2 3679 South Africa IEP Sepedi G1 64.80 Average 847 Average 20.43 0 10 20 30 40 50 60 70 80 00 00 0 0 00 00 00 00 00 50 10 15 20 25 30 35 40 Words per minute Words per lesson Non-English languages English English - Edo State Source: The authors’ elaboration is based on an analysis of Edo State lesson plans and information from other countries by Piper et al. (2018). Data for Edo is based on a sample of 15 lessons. Despite their strengths, the lesson plans and their actual implementation in the classroom have room for improvement. A review of the lesson plans focusing on early grade reading instruction identified strengths and weaknesses in how each of the subskills that contribute to reading comprehension are taught (table 1). The review follows the guidance of the Handbook for Literacy Lesson Planning (Crawford et al., 2022), which points out how learning to read involves acquiring multiple skills from the domains of word recognition and language comprehension. These skills 21 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program TABLE 1. Review of the Edo State scripted lesson plans focusing on early grade reading instruction Components of Lesson Sequence Lesson Strengths Identified Weakness Learning Activities Reading Covered Oral language Begins with simple Enhances oral Limited integration Repetitive speaking, phrases and builds language skills, which with reading and listening tasks, and up to more complex are critical for reading writing activities. question-answer structures. development. Could benefit from a sessions about familiar Interactive approach stronger link to specific objects and concepts. helps students apply reading texts or richer  Asking and answering language skills. content. “What” and “Who” questions Understanding Systematic approach Focuses on letter Does not connect Demonstrations, air Alphabetic Principle to writing individual recognition and letter-writing to reading writing, writing in letters both uppercase handwriting which is of words or texts. books, and teacher and lowercase. foundational for early immediate feedback. reading. Recognizing Words From identifying where Introduces Does not Writing on the board, commas are needed to punctuation, an integrate reading identifying comma applying this in writing. essential component comprehension or placement, writing of writing and reading higher-level literacy exercises in books. comprehension. skills. Focused on spelling and pronouncing vocabulary words, which are not robust vocabulary strategies Comprehend Text Follows progression Promotes Limited to literal Read aloud, repetitive from reading to comprehension by comprehension; and reading for fluency, recalling specific encouraging students recalling information; answering specific details. to retrieve information does not encourage questions from the from the text. inferential thinking or text. Builds on prior sessions deeper engagement by continuing with the with the text. same text, reinforcing Critical thinking and familiarity and problem-solving tasks understanding. are less available. Source: Based on analysis conducted by the American Institute of Research, commissioned by the World Bank. are acquired in an identifiable sequence but also occur simultaneously through multiple feedback loops. For example, letter names and sound knowledge permit decoding and word reading, which then promotes automaticity and growth of fluency in reading comprehension. Likewise, decoding permits word reading, which promotes vocabulary growth, which in turn promotes listening and reading comprehension skills. So, lessons plans should respect both the sequence of skills and the feedback loops. They also need variation in activity and changes to content so students do not become bored. 22 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program Preliminary evidence suggests that the current scripted lessons may move too quickly from basic subskills to more complex ones. This might explain why students perform relatively well—compared to the Nigerian standards—in basic subtasks such as letter recognition but not as well for tasks such as reading comprehension. A strong focus on memorization in the education system might also be contributing to this. Previous research suggests that, in Nigeria, curriculum standards, national exams, and classroom instructional content place a very high emphasis on the low cognitive demand processes of ‘memorize’ ‘perform’, and ‘demonstrate’, and very low emphasis on the more demanding cognitive processes of ‘analyze’ and ‘apply to non-routine situations’ (Adeniran et al., 2023). A recurring theme throughout the interviews with teachers is that lesson plans contain too many topics and activities that can be covered in the allotted time. Particularly there is not enough room for students to ask questions or for teachers to spend extra time on areas that students may be struggling with. This concern aligns with survey data that shows that about half of the lessons are not finished on time. That said, some other lessons are actually finished ahead of time—so there may be an opportunity to customize the timing for each lesson further, without increasing the total time allocated for classroom instruction. Finally, survey respondents had views on the relevance of the specific content and settings within scripted lessons to Edo State pupils. Some scripted lessons were based on “foreign content.” Overall, respondents reported that the foreign content made the lessons less understandable to students, although some teachers noted that exposing children to other parts of the world was positive. Respondents suggested that the setting, plot, and characters in lesson plans were unfamiliar to students and made the content less relatable or understandable. Similarly, some teachers feel that the lesson plans were not tailored to students’ needs and ability levels (such as reading passages that are too lengthy and above students’ reading levels). FIGURE 4. Deviation of lesson times from expected duration, by subject area (2023) Distribution of lesson duration deviations from exprected duration, by subject area Mathematics Above expected duration Below expected duration English -50 -25 0 25 50 Deviation from the expected duration of the lesson, in percentage points Source: Based on EdoBEST data. 23 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program The analysis of the lesson plans and their implementation in the classroom provides several insights for the implementation of similar programs. First, it shows that with continuous efforts and strong monitoring and evaluation, a structured pedagogy program can be implemented at scale, even when teachers are not accustomed to using lesson plans and provide resistance at first. A significant behavioral change in teachers in Edo was possible so that using scripted lessons became part of the routine. Second, standardized lesson plans offer an excellent way to scale up a structured pedagogy program, but they must be tailored. In contexts with no experience producing lesson plans, standardized materials can save time and provide a strong baseline of quality. While developing lesson plans from scratch is an option, it is labor-intensive and time-consuming. Utilizing ready-made materials provides a good starting point. However, to maximize impact, these scripted lessons should be adapted to the specific context. This includes making them culturally relevant, adjusting the time allocated for each session to match the students’ abilities, and ensuring that the relevant materials provide appropriate coverage to meet the desired learning objectives and outcomes. To save time, this customization can be done as the program’s implementation moves along. The third lesson builds on this last point—it is important to create iterative loops for lesson plan adaptations. Particularly when the initial set of plans is standardized, it is useful to receive feedback from teachers and other stakeholders and to consider adjusting the lesson plans periodically. Similarly, organizing a thorough review by independent experts can provide useful insights, for instance, to ensure a strong alignment between the content of the lesson plans—including the sequence in which the different subskills are taught—and the science of learning. Finally, having well-designed lesson plans does not guarantee perfect implementation in the classroom. Effective lesson plans must be complemented by continuous classroom observations and resources to support teachers who may encounter challenges and students who may need additional resources. This is especially crucial for addressing highly technical issues, such as ensuring the proper sequence of subskills necessary for effective learning for all students. Coaching program and support for teachers Teachers play a pivotal role in enhancing student achievement. There is a wealth of evidence demonstrating how effective teaching significantly influences educational outcomes. For instance, two students beginning the year in identical circumstances but with teachers of differing abilities—one highly effective, the other less so—will end up at vastly different levels of learning by year’s end (Hanushek, 2011). Students taught by adept teachers not only gain more knowledge and pursue further education but also enjoy higher earnings in adulthood. Moreover, female students supervised by good educators are less likely to experience teenage pregnancy (Chetty, Friedman, and Rockoff, 2014; Hanushek, 2011). Proficient teachers play a crucial role in developing not only their students’ academic knowledge but also their socioemotional abilities (Villaseñor, 2017). The transition from a teacher with lower performance to one with higher performance can significantly boost students– learning. This impact ranges from an increase of over 0.2 standard deviations in Ecuador to more than 0.9 in India, equating to the educational gains of several years under typical schooling conditions (World Bank et al., 2022). 24 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program © SMARTEDGE Given the centrality of teachers, the EdoBEST program made significant efforts to support them, with a focus on in-service professional development. Before a school joins the EdoBEST program, teachers go through a two-week program of face-to-face training.7 The training covers pedagogy, classroom management, school and child safety, anti-harassment, anti-corporal punishment, basic digital skills, and the use of technology. All the training materials were developed in English (see Box 1 on language of instruction). Beyond this initial program, even more substantial support to teachers is provided through a robust system professional coaches—in Edo, known as learning and development officers (LDOs). On average, there is one LDO for every 11 primary schools, and they are managed by regional supervisors. LDOs undertake regular school visits and classroom observations, provide feedback to teachers and headteachers, and coach and mentor teachers. The LDOs have access to teacher attendance and time-on-task data to assist them and provide targeted teacher support. During the academic year ending in 2023, LDOs visited 96 percent of primary schools at least three times. LDOs visited 98 percent of junior secondary schools (284 schools in total) three or more times and gave personalized feedback to at least 80 percent of their teachers. 7 Many of the reforms under the EdoBEST programs were implemented through a technical implementing partner hired by the government. The in-service training is one of them. 25 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program LDOs conduct structured classroom observations using a short classroom observation tovol, and a long observation tool that is deployed less frequently. The observation focuses on how well teachers are: i) motivating students to behave and work hard; ii) following the teacher guides; iii) checking on the performance of all students; and iv) responding to the performance of students. Teachers are ranked on a scale of 1 to 10 for each variable. During the focus group discussions, all stakeholders emphasized the critical role played by LDOs in providing timely and quality supportive supervision to teachers, headteachers, and project management teams, to improve the quality of teaching and learning in schools. The LDOs provide quality supportive mentoring for teachers with tailored guidance for improvement, a strength that was identified and commended by participants. The design of the coaching program is aligned with analytical findings that suggest practicality, specificity, and continuity are key to effective teacher professional development (Popova, Evans, and Arancibia, 2016). Practicality means teachers are trained using concrete methods as opposed to theoretical constructs, and the training is classroom-based (Walter and Briggs, 2012). This is very much the case in Edo, given the efforts to provide in-classroom direct feedback. Specificity means teacher training programs are most effective when they teach pedagogy specific to a subject area (for instance, how to effectively teach a mathematics class). While training programs in Edo embrace specificity, some aspects could be improved. For instance, much of the classroom observation focuses on classroom management and is common across subject areas. This was a needed prioritization during the first phase of the program, but it could be adjusted towards a more subject-specific tool now that the program has matured. In particular, for English classes in early grades, there could be a stronger focus on ensuring that the phonemic awareness approach and the science of reading are followed. Continuity means teachers receive significant continual support—not one-off workshops (Darling-Hammond et al., 2009; Yoon et al., 2007). Coaches are complemented with quality assurance officers. In the academic year 2023/24, there was, on average, one officer for every 15 primary schools and one officer for every 10 junior secondary schools. Quality assurance officers (69 of them) visited at least 783 primary schools (79 percent of the total) at least three times during the academic year ending in 2024. . Quality assurance officers play a more supervision or inspection-oriented role during their school visits. However, the stronger focus of the program seems to be on coaching rather than supervision, which is also aligned with the available evidence.8 Headteachers play an important role in supporting teachers. According to the GEPD, school management tends to be relatively strong, with most headteachers observing the teachers (99.7 percent) and discussing results with them (85.2 percent). However, there is also significant room for improvement, with only 68.2 percent of principals being familiar with the teachers’ content knowledge. To measure what teachers do in the classroom more systematically, the government developed an index of the quality of teaching practices (table 2). It set specific targets to improve teaching 8 See, for example, RTI International (2023a). 26 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program TABLE 2. Index of the quality of teaching practices Indicator Baseline Year 2 Change Alignment with the TEACH Framework 2018 Jan. 30–Mar. 22 2023 % of Teachers Who DID NOT Scold, Classroom culture (supportive learning 56.0% 93% 37% Belittle, or Use Corporal Punishment environment) % of Teachers Who Praised or Positively Classroom culture (positive behavioral 48.0% 92% 44% Narrated Behavior of Pupils expectation) Instruction (checks for understanding; feedback) % of Pupils Focused for Most Minutes Instruction (lesson facilitation) 73.0% 59% -14% % of Pupils Followed Directions Instruction (lesson facilitation) 82.0% 64% -18% % of Classrooms with Independent Instruction (checks for understanding) 43.0% 88% 45% Practice % of Pupils Working Hard during Socioemotional skills (autonomy & 63.0% 58% -5% Independent Practice perseverance) % Lessons Completed Using edtech 0.0% 54% Composite Score – Quality of Teaching 52.1% 72.6% 20.4% Practices Source: EdoBESST results framework. practices during project implementation, as well as to monitor teachers’ attendance. The design of the index is aligned with the World Bank’s TEACH, a free classroom observation tool that provides a window into one of the less explored and more important aspects of a child’s education: what goes on in the classroom. TEACH was piloted in over 1,000 classrooms across Mozambique, Pakistan, the Philippines, and Uruguay, and tested with global video footage from 12 low- and middle-income countries (World Bank, 2021b). Even with these initiatives to support teachers, observations in the classroom and discussions with key stakeholders suggest there is still considerable scope to improve the quality of teaching across the board, and this remains a big challenge in Edo. The heterogeneity in the quality of teachers affects the fidelity of implementation. Those areas with low quality teachers—particularly hard- to-reach areas—present more challenges to implement the program properly. Some schools have teachers who have serious difficulties reading the scripted lessons, creating obstacles to delivering good results. 27 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program The existence of teachers with low skills in Edo is supported by a recent assessment with data from a random sample of teachers in 200 primary schools conducted as part of the implementation of the GEPD in late 2023. This revealed that only 19.2 percent of teachers are proficient in the content they teach, including 15.5 percent in math and 23 percent in language (World Bank, 2024).9 Teachers also score particularly low in socioemotional skills. The assessment is aligned with the Service Delivery Indicators (SDI)10 survey. These challenges are not unique to Edo State. Data from the SDI conducted a decade ago found that only 3.7 percent of public-school teachers in Nigeria11 met the minimum competency benchmark. The contrast with other countries is enormous. For instance, 40 percent of teachers reached the same competency benchmark in Kenya (Pimhidzai, 2015). It is important to note that the training the teachers receive as part of EdoBEST does not focus on content knowledge, but rather on the utilization of scripted lessons. The professional weakness of teachers derives from a stock of teachers who were part of the workforce before the launch of the program. They were not always recruited meritocratically and, in many cases, are not qualified. The low quality of the pre-service teacher training—which tends to be theoretical instead of practice-based and generally weak and misaligned—also accounts for teachers’ shortcomings. Reforms of the pre-service training have not been a priority, the only key change being in the institutional structure of the system through the merging of the previous three colleges of education into one. Some measures are being pursued through EdoBEST to address these challenges. First, the government launched the Edo Supporting Teachers to Achieve Results (EdoSTAR) Teaching Fellows Programme. EdoSTAR selects individuals to teach in public schools through a competitive process. The candidates do not need to have a teaching degree; they can have a BA degree in any other discipline. The selected candidates become “fellows” for three years, during which they are placed in public schools to teach and receive an induction program. After the three years, they have the possibility of becoming civil servants like regular teachers. Thus, EdoSTAR is an alternative way to recruit and train teachers, and it prioritizes those who apply to teach in remote areas. Given the challenges with pre-service teachers’ training, EdoSTAR can provide a viable and promising alternative. Its design is similar to programs such as Teach for America, but it is fully organized by the government. Evaluations of similar programs in other settings have either demonstrated positive results or, at the very least, have not shown any negative impacts on student learning (Duflo et al., 2015; Glazerman et al., 2006). While the effects of EdoSTAR are yet to be seen, and a lot will depend on the implementation arrangements. Countries facing similar challenges could consider the design of programs that skip regular pre-service training and instead invest significantly in induction. 9 To be considered proficient, teachers should score above 80 percent of the items correctly. 10 The Service Delivery Indicators (SDI) initiative collects data on service delivery in schools and health facilities. It helps countries identify areas of progress and areas for improvement with potential lessons for progress within and between countries. 11 Defined as teachers scoring 80 percent in the SDI teacher assessment module. 28 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program Second, the government designed a performance-based management system for teachers to shape their career progressions and determine monetary and non-monetary incentives. The system describes different levels for the teachers’ careers and determines promotion based on a combination of seniority and performance. The process for designing the system included discussions with multiple stakeholders, such as teacher unions, local government authorities, and the head of the civil service. The system, however, has not been implemented as of 2024. It is worth noting that in Edo, teachers seem to be supported by relatively solid management capacity from school leaders. Based on the analysis of the GEPD, school management shows strong capabilities in providing instructional leadership and managing core functions effectively. This positions school management as a valuable ingredient in tackling educational challenges and implementing further reforms. This is particularly the case for operational management functions, where performs relatively well. To summarize, EdoBEST illustrates that even very well-designed programs can have challenges with the fidelity of implementation in contexts of very low teacher capabilities. In these settings, the program shows the importance of combining teacher support with mechanisms to ensure accountability. Measuring and incentivizing teachers’ attendance (at a minimum) and performance is critical and possible, even in contexts not used to these arrangements. Besides the focus of structured pedagogy programs on coaching and practical training, governments can also take advantage of windows of opportunity to invest in alternative recruitment mechanisms for new teachers and pre-service training modalities. Learning materials The availability of textbooks and homework books is a necessary input in the education production function. The program has explicitly recognized the shortage of these learning materials and set a minimum of one homework book for every pupil and one textbook for every three students in at least 75 percent of schools. A book count exercise conducted to quantify the shortage in learning materials in 2022 yielded about 715,000 learning materials in Early Childhood Care, Development and Education (ECCDE) and primary (grades 1 to 6). Against this background, before the beginning of the academic year 2023, almost 1.5 million learning materials were distributed to meet the targets across Edo State, including about 640,000 new textbooks. In addition, up to 475,000 homework books for primary grades and approximately 376,000 books for ECCDE grades were delivered. For junior secondary schools, delivery was made of 351,000 textbooks across five subjects and 56,000 homework books to JSS 1, 2, and 3 in 2023. 29 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program BOX 1. Language of instruction in Edo State Children learn much better and faster when they are taught in their own language for at least the first six years of primary education (World Bank, 2021). For instance, a study across 48 countries shows the significant positive impacts of learning in the first language on literacy skills (Mullis et al., 2012). The effects have also been found for mathematics (Martin et al., 2008). Furthermore, learning in the first language is also linked to better attendance and school attainment (Ramachandran, 2012), better outcomes in more specific subjects later in life, and higher development of cognitive abilities (Trudell and Piper, 2014). Importantly, the evidence also shows that teaching in the language children speak at home is highly cost-effective (World Bank, 2021). Nigeria is one of the most linguistically diverse countries in the world, and the most diverse in Africa, with approximately 515 living languages (USAID, 2021). Edo State is not the exception, with at least 31 languages are spoken in the state, including at least 17 spoken as first languages. This includes major languages such as Edo, Etuno, Etsako, Esan, Ake-Ievbu, and Okpamheri. In Edo (and most of Nigeria), English is more commonly spoken and strongly endorsed for broader use by individuals in higher-income groups and urban settings, whereas in rural regions and among lower-income groups, the use of English is significantly less prevalent (Trudell, 2018). In interviews, teachers described several ways in which they handled language issues in the classroom. When students appeared not to understand the teacher in English, teachers explained or defined words primarily using pidgin and secondarily using a local language. Some teachers also indicated using gestures or pictures if needed. Teachers explained that pidgin English worked better in cases in which students in a classroom did not speak the same local language or the teacher did not speak the language of the students. A few teachers also reported asking students to serve as translators if needed. Despite this language heterogeneity, the state has not focused on expanding the languages of instruction. All the instruction in the education system is in English, at least formally. In practice, many teachers are also not highly proficient in English and, therefore, practice codeswitching between the mother tongue and other languages (Trudell, 2018). The lack of focus on expanding the languages of instruction is mostly due to the need to show results quickly and the lack of learning materials in other languages. The Literacy Policy Package is a set of elements identified by the World Bank to ensure every child can read (Crawford et al., 2021). It has five core elements for helping students become literate, and start on the path to becoming advanced readers: (i) a political commitment to clear goals for making all students literate; (ii) effective instruction by supported teachers; (iii) adequate teaching and learning materials, including at least one teacher’s guide per teacher, one textbook per child, and additional reading books for practice; (iv) promotion of love of reading; and (v) instruction in a language that the student speaks and understands. While Edo has made significant strides in the first four elements, it is yet to focus on the fifth. Going forward, there are some opportunities to focus on expanding the languages of instruction. In November 2022, the Federal Government of Nigeria approved a new policy that stipulates the 30 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program first six years of primary education should be in the language of the “immediate environment” of the students. While the implementation of the policy has been slow, and in many cases resisted because of the status associated with English, the new regulation might provide an opportunity for Edo State to explore potential reforms. The preparation of a language mapping of the sociolinguistic context could be a good first step in the right direction. Even if expanding the languages of instruction is not feasible in the short term, it is useful to remember that whether or not students learn to read in their first language, they do best in learning to read in a second language when their first language is treated with respect (Shanahan, 2022). Finally, if the state chooses not to transition to another language of instruction, it will be crucial to address the needs of students learning English as a second or third language. This should be a key consideration in decisions related to the curriculum, learning materials, and the establishment of specific language proficiency goals for each grade. According to the inventory count and new learning materials distribution per school, the homework books and textbooks targets of 1 to 1 and 1 to 3, respectively, should be met in 98 percent of primary schools and 95 percent of junior secondary schools. A field survey conducted in 2023 found that 90 percent of primary school pupils had access to textbooks, and 67 percent stated that they were provided with homework books. In junior secondary, 88 percent claimed to have access to textbooks and 93 percent to homework books.12 Besides their availability, the program has not included, so far, a comprehensive review of the quality of textbooks and their alignment with the science of learning. A review of the alignment between the textbooks and other learning materials has not been done either. The GEPD also includes some findings on the availability of textbooks and other learning materials. It shows that only 45 percent of classrooms meet a standard of at least 90 percent of students with a textbook. This is related to the specific goals of the program during these years. Given the almost absolute lack of textbooks when the program was launched in 2018, the initial objective was to reach at least 1 textbook for every three students. As the program continues, this target might need to be adjusted. However, the availability of other learning materials seems adequate. For instance, 93.4 percent of schools have adequate furniture, 94 percent of classrooms have a blackboard, and 94 percent of students have pencils (World Bank, 2024). The program presents an opportunity to enhance the alignment between scripted lessons and other learning materials. Strengthening this connection would ensure a more cohesive and integrated learning experience, allowing students to effectively engage with and apply the content across different resources. 12 This field survey was conducted as part of the verification process of the EdoBESST World Bank-supported program. 31 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program © SMARTEDGE Measurement of learning A focus on assessments has been a critical component of the reforms in Edo State, and the drive to build a solid system of learning assessments has intensified with recent efforts to improve learning outcomes.13 Currently, Edo State utilizes two different types of assessments that have been incorporated as part of the education reforms started in 2018. The first type is classroom assessments, which are assessments for learning and, therefore, formative in nature. They provide teachers with real-time information to support teaching and learning in individual classrooms. The second type is large-scale assessments, which are an assessment “of learning” and of a summative nature. EdoBEST has been implementing classroom assessments for all subjects twice a term. For instance, for reading skills, the classroom assessments conducted rely on the use of selected subtests from the Dynamic Indicators of Basic Early Literacy Skills (DIBELS) collection of tests (University of Oregon, 2020). DIBELS 8th Edition includes six subtests that collectively indicate whether a student is on track to become an independent reader. Different subtests target different grades, and specific benchmarks are available at the beginning, middle, and end of each grade. Overall, the DIBELS battery of subtests indicates a four-year trajectory at the end of which students above the risk cut-off points can consistently read at least 114 words per minute and understand what they have read. This could be called a “gold standard” for reading performance as students meeting such 13 Assessment is the process of gathering and evaluating information on what students know, understand, and can do (Clarke, 2012). 32 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program benchmarks would likely have all the abilities they need to “read-to-learn.” DIBELS uses six subtests to look at the earlier subskills (such as letter naming fluency) that contribute to the end goal of fluent reading with understanding.14 Despite the availability of classroom assessments, it became evident that Edo lacked an assessment of learning at the systemic level. Given the new enhanced focus of the state on improving learning outcomes, the government has invested in the development of a large-scale assessment system that can provide information on overall performance levels and trends in the education system as an aid to policy decision-making (Clarke and Bazaldua, 2021). In Edo, the system has been developed with the support of technical assistance provided by the World Bank (through the Edo Basic Education Sector and Skills Transformation Project, EdoBESST) and the Accelerator Program, with a strong focus on developing local capacity. First, a document outlining the Edo State Learning Assessment System (ELAS) was developed as a foundation and background for learning assessments and learning assessment frameworks and standards were established. The document covers the purpose and components of learning assessments, highlights critical decision points for the government, and delves into the elements of learning assessment, such as the target population, sampling methods, types of tests, frequency, test administration, analysis, and reporting. Following the preparation of the guidance note, assessment frameworks and learning standards were developed for specific grades and topics—Mathematics and Reading Comprehension for grades primary 3 (P3), P6, and junior secondary 3 (JSS3). Further, a comparison was made between the Edo Learning Standards and the Global Proficiency Framework (GPF) to derive the main structure that reflects both the Edo standard objectives and the GPF objectives. Multiple workshops were conducted with teachers selected by the government (item writers of the learning assessments) with the objective of building the government’s capacity to create assessment tools. In addition to workshops, training materials for the item writers were created to strengthen and sustain the government’s capacity to develop learning assessments. A learning assessment tool was then developed, and an implementation strategy was prepared for the state. The assessment tool includes test items aligned with the established learning standards, while the implementation strategy includes a plan and budget for a learning assessment. With the learning assessment frameworks, strategy, and tools in place, the government successfully conducted a pilot of the learning assessment in March 2022. The pilot featured 3,500 students in 40 schools across the state (20 primary schools and 20 junior secondary schools). The results have already been analyzed with the objective of adjusting the assessment tools and preparing the implementation of a full learning assessment in early 2023. 14 Note that the DIBELS benchmarks are from the United States, a developed, largely monolingual, and high-literacy context, and the benchmarks may vary for other contexts. In any case, the use of this instrument has provided a very methodical approach to measuring reading skills in Edo periodically and systematically. DIBELS also served as one of the inspirations for the Early Grade Reading Assessment (EGRA), which is more commonly used across Sub-Saharan Africa. Looking ahead, Edo State may want to reassess which instrument is best suited to accurately measure and track its early literacy levels, considering its unique context and objectives. 33 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program FIGURE 5. Percentage of students above average performance in reading and mathematics, by LGA, 2024 Percentage above average performance in Reading and Mathematics, by LGA Primary 3 Primary 6 Junior Secondary 3 75 p = 0.80 75 p = 0.86 75 p = 0.74 % Students above state average, Reading % Students above state average, Reading % Students above state average, Reading Akoko Edo Owan East Ovia North East 65 Ovia North East 65 65 Oredo Ikpobaokha Ovia South West Esan South East Orhiomwon Igueben Akoko Edo Uhumwonde 55 Esan West Akoko Edo 55 55 Oredo Igueben Egor Orhiomwon Owan West Esan South Esan West Ovia South West Esan West Ovia North East East Etsako East Egor 45 Uhumwonde 45 Esan Central 45 Igueben Ikpobaokha Uhumwonde Ikpobaokha Egor Owan East Ovia South West Owan East Orhiomwon 35 Orldo 35 Etsako Central 35 Esan Central Esan Central Etsako Esan North East Central Owan West Esan North East Owan East 25 25 25 Esan North East Etsako Central Esan South East Etsako East Etsako East 15 15 15 15 35 55 75 15 35 55 75 15 35 55 75 % Students above state average, % Students above state average, % Students above state average, Mathematics Mathematics Mathematics Source: Edo State Government, 2024. After the lessons learned from the pilot, the first round of the assessment was conducted in March 2023. The decision was to conduct a census-based learning assessment, the first of its kind in Nigeria. The advantage of that approach is that in addition to providing a picture of the entire system, the results could be used for other purposes, such as tracking students’ learning trajectories and providing feedback loops to the schools. The results were studied by a team of analysts composed of civil servants from the State Universal Basic Education Board (SUBEB) and the Ministry of Education. The analysts were previously trained in item response theory and statistical analysis. The standard setting to determine proficiency thresholds was organized with a panel of Edo State representatives from the education sector following the bookmark method. The individuals involved in the July 2023 Standard Setting procedure included only a few teachers and an over-representation of educational authorities and most judges came from Edo South, with only a few from Edo Central and Edo North. The second round of the assessment was recently concluded in June 2024, and it included several sets of new items. The results from the first round, to be read with caution since this was the first time the state implemented a large-scale assessment, show relatively low proficiency levels in early years with some improvements in later years and suggest large room for progress going forward. Test items were generally difficult for students, and since this was the first time this exercise has been undertaken, it remains an open question as to whether the tests were too stringent or whether student capacity is low. That said, the tests were nonetheless very consistent in differentiating between students at the high and low ends of the proficiency scale. 34 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program Across the geography of the state, some Local Government Areas (LGAs) performed much better than others, and the correlation in the performance across subjects using the LGAs as units of analysis is high (Edo State Government, 2024). Although there are clear differences in outcomes between LGAs, students within LGAs remain very diverse in terms of performance. However, the variation within LGAs tends to decrease for upper primary with respect to early primary and even more for lower secondary with respect to primary. LGAs with higher overall performance tend to also have greater variation in performance, a pattern that has also been found in international studies such as PISA. In general, students attending schools in areas with dense populations tend to perform better than those attending schools in less dense areas, although unexpectedly good results were achieved in certain remote areas. Across all grade levels and subjects, female students tend to perform better than males. Several key lessons have emerged from Edo’s experience in implementing assessments. First, capacity building for complex initiatives can happen speedily but must be conceived as an ongoing (rather than one-off) process. Edo State government designed and executed integrally the large-scale assessment. The support of two international experts focused on building the technical capacity of civil servants and teachers to continue this process in the future. Another notable aspect of the ELAS was the active involvement of a broad range of stakeholders throughout the process. Effective communication strategies further enhanced this inclusive approach, including the use of video materials to disseminate information and engage with various participants in the education sector. For instance, the week before the actual assessment was conducted, multiple videos were produced to communicate the importance of the assessment; highlight the relevance of ensuring that every student takes it; and, importantly, inform teachers, students, families, and the public that the sole purpose of the assessment was to make decisions to improve the system and support them, rather than to punish the worst performers or reward the best ones. This was critical to avoid cheating or pervasive practices such as exclusion of low-performing students. These communication materials were prepared in a friendly format and disseminated widely, and they always included real teachers and students. They facilitated a deeper understanding and buy-in from all involved, fostering a collaborative environment conducive to the successful implementation of the assessment system. Two challenges highlight some of the lessons learned for future and similar initiatives. Despite the progress made, no formal policy framework currently governs large-scale learning assessments in Edo State. While this did not create major gaps in the implementation, it poses risks for the sustainability of the assessment in a future administration, and it makes the distribution of roles and responsibilities unclear. Producing a document highlighting the learning assessment strategy and the intentions of the government to establish a center specialized in learning assessment would help bridge this gap. A structured policy environment is critical to provide clear guidelines and support the continuous and systematic conduct of such assessments, ensuring sustainability and alignment with broader educational goals. Another significant challenge was the limited collection of information on noncognitive factors, such as sociodemographic, family, and school contexts, which play a crucial role in student achievement. Future rounds could incorporate a simple questionnaire to create a more comprehensive database and understand drivers of learning that might be outside of the school system. By integrating 35 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program various tools and methods, the assessment system can provide a more comprehensive and nuanced understanding of student learning, encompassing both cognitive and noncognitive dimensions. This holistic approach enables a deeper analysis of the factors affecting education quality and equity, facilitating more informed decision-making and targeted interventions. Edo highlights the importance of using multiple assessment mechanisms simultaneously. Combining continuous classroom assessments with large-scale assessments is crucial for a holistic understanding of educational outcomes and for fostering a supportive learning environment. Continuous classroom assessments offer real-time insights into students’ understanding, allowing teachers to tailor instruction to meet individual learning needs. In Edo, these assessments have been systematic and, for key subjects, have used tools that allow us to see Edo students in a comparative perspective with other education systems. These assessments promote a culture of continuous feedback and improvement, enabling educators to promptly identify and address learning gaps. On the other hand, large-scale assessments provide a macro-level view of educational performance. Edo decided to go one step further and use a census-based approach. On the one hand such an approach is ambitious and rather stretching (especially when done for the first time where capacity and knowledge is limited). On the other hand, it can have multiple benefits to track every student and create incentive mechanisms for schools, as it has been done in other countries.15 By integrating both assessment types, education systems can ensure that individual student needs are addressed while also working towards overarching educational goals, ultimately leading to improved learning outcomes and equity across the educational landscape. A clear next step for ELAS is to start using the data for better decision-making, which is another important challenge for similar initiatives to consider. Edo shows that a gradual approach might work effectively. Before using the data to make decisions or to create systems that incentivize better performance, it is crucial to ensure that the measurement is accurate and consistent across time. Having more than one round of large-scale assessments before the information is used, for example to set targets for each school, would be helpful. Otherwise, the risk is that the initial adjustments that any learning assessment system needs might jeopardize its credibility among key stakeholders. A key lesson is that well-structured, system-wide learning assessments are essential for providing a reliable benchmark to measure educational progress. However, implementing these assessments is more complex and resource-intensive than initially anticipated. Their true value only becomes evident over several years, as multiple assessments across different periods are necessary to accurately measure improvements. Additionally, these 15 For example, in the Brazilian state of Ceará, 72 percent of discretionary funds are distributed to municipalities based on their educational outcomes. These funds are crucial for municipalities, especially the less affluent ones in Ceará, where they can constitute up to one-third of their total income (Loureiro and Cruz, 2020). The allocation of these funds is guided by a key “education quality index,” aimed at enhancing educational standards and promoting fairness among students within each municipality. This index is derived from an extensive census- based evaluation of learning (Loureiro et al., 2021). 36 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program assessments do not always resonate with all stakeholders, as they can seem detached from the daily operations of schools and students, unlike regular classroom-based test scores. The data from system-wide assessments can also lead to challenging debates about how to evaluate progress, as they may not always align with data from other sources at certain times. Furthermore, these considerations raise the question of whether smaller-scale, sample-based learning assessments might be more suitable in the early stages of a major educational reform program. The edtech components of the program The program in Edo is strongly supported by educational technologies, which is a significant difference from other structured pedagogy programs that are paper based. Both administrative data from the program and information collected through the GEPD confirm that the use of edtech is highly prevalent in schools. The GEPD showed that at least 99.7 percent of the schools use edtech (World Bank, 2024). The program relies on an “Integrated Technology Platform,” which comprises the following elements: ● Tablets for teachers, which are used to receive lesson guides (scripted lessons), take student attendance, record assessment results, and receive messages/notices. The teacher’s tablets connect to the headteacher’s smartphone to access the internet. Each teacher in the school is equipped with a tablet. The tablets in use in Edo are custom-made, and they run a customized Android version and the technical partner’s proprietary software. They are customized so they cannot be used for other purposes, which reduces their resale value and reduces the risk of theft. As of 2024, 97 percent of teachers in primary and junior secondary schools have a tablet. ● Smartphones with internet connectivity for headteachers, which are used as an internet access point (“hot spot”) for teacher tablets to download lesson guides and messages and upload usage statistics for student registration and management of teacher attendance. The program covers internet access costs. Current devices used are Nokia smartphones running Android OS. ● Cloud-based backend platform that stores all teacher guides, lesson guides, and data collected from schools (teacher and student IDs, biodata, attendance, teacher lesson completion, student grades, QA reports, and messages to teachers). This backend also pushes new teacher content and messages to the teacher tablets. The cloud system is reportedly run off data centers in the United States, India, and Nigeria, offering automatic backups and failover. ● Smartphones for field Quality Assurance teams. Each morning, the headteacher turns on his/her smartphone and launches the application that synchronizes with the cloud-based server to download lesson guides, assessments, and any messages received. Headteachers also create a “hot spot” for teacher tablets to access the cloud. The headteacher takes teacher attendance (although the teacher device also logs teacher attendance and GPS coordinates to ensure the teacher is at school—independently, and the two attendance data points are compared). 37 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program TABLE 3. Edtech in Edo schools Tablet distribution Primary schools (2024) Junior secondary schools (2024) Number of schools % Number of schools % Schools with at least 80 percent of teachers with a tablet 983 98.10% 283 99% Schools with less than 80 percent of teachers with a tablet 19 1.90% 3 1% Total 1002 100.00% 286 100% Source: EdoBESST Results Framework At the start of each class, the teacher logs into their tablets, starts the class, and takes attendance of students. At the end of the class, the teacher marks the lesson as completed. The teacher is expected to follow the lesson guide (script). The tablet tracks “time on task” by the rate at which the reader turns the pages. The program expects teachers to spend a pre-determined time on each “page” of the guide. The system also tracks progress in the syllabus coverage. The system can thus determine if the teacher attended class and if the teacher is spending an “adequate time” of instruction per class. If the lesson needs an assessment, the teacher marks the assessment and enters the data into the tablet. This data is synched with the headteacher’s smartphone and sent to the cloud platform. The headteacher’s smartphone is internet-enabled and uses mobile broadband. It was reported that almost the whole state is covered by mobile broadband by one or more providers. Specific providers are selected for each school depending on the availability and the providers’ quality of service for those specific areas. As such, access is not an issue. The sync between the teacher and the headteacher smartphone and with the server should happen at least once a week (most lesson guides are pushed down weekly). Furthermore, the headteacher’s smartphone is used to capture student enrollment data. During enrollment, the headteacher takes a photo of the student, basic biodata, and parents’ contacts. Parent contact data is used to communicate with the parents if the student is absent from school or to notify parents of important activities and events. Headteachers and teachers are responsible for charging their own devices (presumably at home for schools with no electricity). When asked about potential challenges, teachers usually respond that they do everything they can to charge their devices since it is their responsibility. Note that the tablets are reported to be designed to consume minimal battery power in an attempt to extend the battery life of the tablets from each charge. For example, the tables are dimly lit, which reduces battery consumption during use. This, along with other features, adapts the technology to the needs of the local context and makes it more appropriate for use. 38 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program Regarding basic digital skills, most teachers had no digital skills at the start of the program but through training and daily use, they become comfortable and familiar with the devices. The skills program is focused on device usage and not general digital skills. The analysis of the edtech components of the project provides key insights for the implementation of similar programs. First, the use of technology and the data-rich aspects of the implementation go hand-in-hand. The deployment of this technology-enabled ecosystem allows for real-time collection of data and a very close monitoring of the implementation, including its dosages and fidelity. The large amounts of data collected are, in fact, a direct consequence of the tech components of the project, even if the actual utilization of the data could be strengthened. The availability of data collected through the tablets also helps monitor the program closely, including in remote and hard-to-reach areas, partly thanks to the geo-referenced information that the tablets collect. For instance, the very detailed information regarding the time on task for the teachers is key to ensuring the success of the reforms and could not be collected without the technology that is part of the program. Second, technology should not be an end in itself. Incorporating technology into the program holds significant potential for enhancing its impact, yet its effectiveness is contingent upon being integrated with appropriate complementary interventions. Specifically, the deployment of tablets in Edo would have yielded minimal benefits had the lesson plans lacked quality and if the structured pedagogy program failed to cohesively align its various components towards promoting learning outcomes. The true value of technology in educational settings emerges only when it is part of a well-coordinated strategy that includes high-quality educational content and methodologies tailored to facilitate learning. Third, the technology associated with the program has important behavioral impacts. In a context like Edo, where the education sector was in a dire state and suffered a very low reputation, the technology associated with the program helped increase the reforms’ attractiveness and elevate the status of the teaching profession, at least based on anecdotal evidence. The label of “digital teachers” for those who initially joined the program voluntarily helped create a layer of prestige around the profession. Fourth, a program like this can be implemented in settings where teachers have very low digital skills. Data from MICS 2021 shows that in Edo, only 9.5 percent of the population sent an email with an attached file in the last three months, only 10.4 percent copied or moved a file, and only 1.1 percent used a basic arithmetic formula in a spreadsheet. The values specific to teachers are likely not to be very different. In that context, a simple training course of two weeks enabled all teachers to learn how to use the devices for their specific tasks. In other words, the low digital literacy of teachers in certain areas should not be used as a reason to avoid the implementation of programs like EdoBEST. Fifth, this program can also be implemented in a context of relatively low connectivity. Even though mobile networks cover most areas of the state, the connectivity is neither stable nor strong. However, only a few specific issues require connectivity, such as the download of the lesson plans. Other functionalities do not require connectivity or at least immediate access to it. For instance, the geo-referenced information can be captured offline and then uploaded to the cloud when network signal becomes available. 39 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program BOX 2. Infrastructure in Edo schools School infrastructure plays a critical role in shaping learning outcomes, serving as the foundational environment where education takes place. Well-designed and maintained facilities provide a safe, comfortable, and conducive atmosphere for both teaching and learning. Adequate classrooms, functional restrooms, reliable electricity, and proper ventilation are essential for reducing distractions and promoting student concentration. Moreover, good infrastructure can positively impact student attendance and teacher retention, creating a stable and motivating environment that supports academic achievement and overall well-being. Investing in school infrastructure is thus a pivotal step towards enhancing the quality of education and ensuring equitable learning opportunities for all students (Barrett et al., 2018). Investing in infrastructure, however, has not been the focus of the reforms in Edo State. An opportunity to generate support for infrastructure reforms among some audiences may have been missed, since infrastructure tends to be the most visible reform that can capture the imagination. Investments in new pedagogical approaches, support to teachers, and innovative methodologies, on the other hand, tend to be less noticeable. The analysis from the GEPD shows that access to water and sanitation facilities in schools is limited. In fact, only 19.1 percent of schools have drinking water and 12.8 percent have functioning toilets (World Bank, 2024). Similarly, connectivity in schools tends to be limited, with 13.5 percent having access to electricity and 23.9 percent to the internet. Schools in rural areas are less likely to have access to drinking water (15.7 percent compared to 28.6 percent), electricity (11.2 percent compared to 19.9 percent), and functioning toilets (9.7 percent compared to 21.5 percent). On the other hand, schools in Edo tend to be much better in terms of the availability of critical inputs such as furniture (93.4 percent of schools), blackboards (94 percent), and an appropriate number of pencils (94 percent) (World Bank, 2024). In recent times, however, the government has made efforts to address these gaps. According to the government’s information, there is an ongoing infrastructure plan that is targeting 72,000 pupils in 336 schools through the construction and renovation of classrooms, fences, toilet blocks, and classroom furniture for at least 30,000 pupils and teachers. These projects are located across all 18 local government areas and split 42 percent in urban communities compared to 58 percent in rural communities. The limited focus of the program on school infrastructure reflects the need to prioritize certain investments in resource-constrained settings. However, the limited infrastructure might be one of the drivers of the relatively low learning outcomes that are still seen in Edo, despite the improvements since 2018. 40 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program If a program decides to utilize tablets as has been done in Edo, a few points are worth following. First, restricting the functionality of the tablets in a way that they cannot run other software programs is critical to avoid leakages and reduce the incentives to deviate the hardware into the market. Similarly, the program should plan to have a stock of tablets to replace those that fail or need repair. Also, while access to electricity is a challenge, EdoBEST shows that given the right incentives, the teachers can find a way to charge tablets either at the school, their homes, or other community spaces. Finally, while the use of tablets can be beneficial there are some risks. Since the software utilized is proprietary, the transition out of the specific vendor can be challenging. This should be carefully considered at the time agreements with technical partners are signed. EdoBEST@Home: complementing learning in school with learning at home In early 2020, the COVID-19 pandemic profoundly disrupted the educational landscape in Edo State, as it did in Nigeria as a whole and globally. Schools were forced to close their doors to prevent the spread of the virus in March 2020 and remained closed until September of the same year, leaving students without the traditional means of classroom learning. Evidence at the country level shows that the disruption had important effects on learning outcomes, school engagement, and the financial resilience of schools (Adeniran et al., 2022; Ogenyi, 2022). In April 2020, amidst the pandemic, Edo State responded by adjusting its EdoBEST initiative into EdoBEST@Home, a distance learning initiative designed to continue the education of its students. This adaptation included the distribution of digital learning packages for students and their caregivers, accessible online without data charges and through platforms like WhatsApp. The adapted program featured mobile quizzes, audio lessons, and virtual teacher interactions, leveraging widespread mobile phone access in households and utilizing WhatsApp for communication, alongside other channels (De Simone et al., 2020). The remote learning strategy adhered to the original program’s core tenets: curriculum-aligned, evidence-based instruction; extensive practice opportunities for each lesson; evaluations; a focus on enhancing reading skills through data analysis; and tailored attention to students, either individually or in small groups. In 2024, the program also started utilizing a digital learning platform. EdoBEST@Home successfully utilized available resources across four key areas: the skills of over 11,000 previously trained teachers, pre-existing digital content and educational activities, established technological infrastructures for delivery, and robust databases (Munoz-Najar and Oviawe, 2020). This readiness, fostered by the EdoBEST initiative, enabled a swift transition to remote education, supported by the groundwork laid by the Edo SUBEB over the years in building educational resilience. Information from focus groups shows that teachers praised the program as a valuable educational resource, yet highlighted the critical role of reliable internet, electricity, suitable devices, and supportive home environments in leveraging its full benefits. These discussions revealed not just the appreciation 41 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program for the program’s ability to facilitate ongoing professional FIGURE 6. growth for educators but also a call for more varied Example of learning guide 1 resources to cater to the diverse learning paces and styles of their students. Parents, too, lauded the program for kindling their children’s enthusiasm for learning and enabling a more active parental involvement in the educational journey. They observed noticeable improvements in their children’s engagement and performance, although concerns were raised about the challenges of limited access to technology and the potential risks of unsupervised internet use. One teacher captured the essence of these discussions in the following way: “the initiative is promising but its effectiveness depends on various factors like internet connectivity, electricity, and students’ home environment.” These conversations collectively painted a picture of a program with significant potential, underscored by the necessity of addressing infrastructural and socioeconomic barriers to unlock the full spectrum of benefits for remote or blended learning models. Data shows that, as of early 2024, more than 6,500 primary school teachers, representing 71 percent of all teachers, took and completed the three online remote learning courses, a significant increase with respect to the 4,700 teachers who had taken it as of 2023.16 To increase access to the materials, the EdoBEST team shifted the focus from web-based interactions to WhatsApp groups and a dedicated WhatsApp number with interactive mobile quizzes. Students’ participation in EdoBEST@Home: In 2023, 48 percent of students surveyed confirmed receiving the Digital self-study Pack and storybooks. However, sampled data analysis showed that many WhatsApp groups were duplicated and had an average of less than 10 members, indicating that the outreach was smaller than desired, and that implementation presented several challenges at the coordination, technological, and operational levels. The experience of EdoBEST@Home can provide some valuable lessons for the implementation of remote or blended learning approaches as part of comprehensive education reforms to improve teaching and learning. First, investing in educational technologies and even basic digital skills for teachers in times of calm can pay off in times of disruption. The investments between 2018 and 2020 made the transition to EdoBEST@Home much smoother than it would have been without that experience. 16 This information comes from the EdoBESST M&E system and has been corroborated by an independent verification agency. 42 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program Second, in the design of blended learning programs, context specificity is key. At the time of the intervention design, the latest available survey in the state showed that only 46 percent of households possessed a radio, 69 percent a television, and 91 percent a mobile phone (National Population Commission, 2019). This information was taken into consideration for the design of the program. Initially, EdoBEST@Home planned to leverage the radio for remote instruction. However, the data on device availability changed this strategy and prioritized the use of mobile phones. Nevertheless, there are always unexpected challenges. Even though households have access to mobile phones, many children have trouble utilizing them because those available are used by parents for work. Moreover, if parents have more than one child, it becomes more complicated to share one device among many children (Munoz- Najar and Oviawe, 2020). Similarly, parental involvement was more challenging than expected. The main lesson is the importance of considering the local reality while also keeping in mind that infrastructure and economic challenges make it very difficult to replace in-person schooling. Thus, many programs in similar contexts should convey a clear and realistic aim, which is to minimize the disruption in learning rather than completely compensate for the loss of traditional in-classroom services. Third, in contexts of low device accessibility, combining virtual with in-person support is critical. Training programs were established to equip learning and development supervisors and quality assurance officers with the skills needed to provide virtual coaching and support to teachers utilizing the EdoBEST@Home platform. This initiative aimed to ensure that educators receive the guidance and assistance necessary for effective remote teaching. The virtual coaching program was made accessible to all teachers, enhancing their ability to navigate and maximize the benefits of the platform. Acknowledging the digital divide, the state took measures to include teachers without smartphones. For these educators, additional in-person support was arranged, ensuring that no teacher was left behind in the transition to digital education, thereby maintaining a comprehensive and inclusive approach to teacher development and support. Fourth, monitoring mechanisms for this type of intervention are sometimes more challenging than expected and should be well incorporated from the outset. The EdoBEST@Home has some built-in analytics showing access to learning and some learning achievement data (the data is not public). However, note that most students, especially after the pandemic, engaged more with the program via WhatsApp, which is not linked to analytics to track use and engagement. Finally, blended learning programs should be designed to address multiple types of disruptions and complement learning in school with learning at home. After the pandemic, the state faced some challenges maintaining the engagement with the EdoBEST@Home initiative and even justifying its rationale in a context where children were back to school. However, Nigeria, like many other developing countries, suffers from multiple sources of disruption to its school continuity, including teacher strikes, insecurity that forces schools to close, and climate-induced events, such as flooding, that have been affecting school attendance strongly in the last couple of years in the state. After a transition period, the program was reactivated and, in fact, proved very useful in 2023. In June of that year, given the sharp increase in the price of fuel, the state implemented a policy for public servants that required only three days a week of in-person work and two days of remote work. This included teachers and therefore, the school program became hybrid. The two days of remote learning used EdoBEST@Home and were further strengthened using individual homework books, made available to over 384,000 basic education students. EdoBEST@Home has also been regularly used by teachers and students during the holidays. 43 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program © SMARTEDGE The stronger focus on foundational learning  in the last few years of the program Starting in 2022, the EdoBEST program started having a stronger focus on improving foundational learning outcomes. These renewed efforts were shaped by both internal and external forces. On the internal front, coming towards the end of the second term of the political administration, showing actual results became one of the government’s main priorities under the governor’s direct leadership. The new objective was to ensure “every child is a reading champion by the age of 10.” On the external front, at least two factors were important. First, the World Bank supported reform efforts through a Program for Results, which will be discussed later. Under this program, the World Bank’s policy dialogue focused on improving learning outcomes. Second, thanks to its high-level of political commitment, Edo State was selected to join the inaugural cohort of the Accelerator Program, an initiative supported by several partners to provide technical assistance that can show rapid improvements in foundational learning outcomes. The reform agenda was originally structured around three questions: i) are the teachers in school? ii) are the learners in schools? iii) are the teachers teaching? The successful implementation of EdoBEST translated into significant improvements in the answers to those three questions. Thus, the success and the influence of the internal and external forces opened the window to focus on the ultimate question: are the learners learning? 44 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program BOX 3. The Accelerator Program Due to the effectiveness and positive educational results of the EdoBEST initiative, Edo State has been chosen to be part of the first group of the Accelerator Program. This program identifies and aids groups of governments that possess the essential elements required to combat learning poverty. It was launched in late 2020 by the World Bank and the United Nations Children’s Fund (UNICEF) in partnership with the Bill & Melinda Gates Foundation; the United Kingdom Foreign, Commonwealth & Development Office (FCDO); the United Nations Educational, Scientific, and Cultural Organization (UNESCO) Institute of Statistics; and the United States Agency for International Development (USAID). The participating governments demonstrate strong political and financial commitment to reducing learning poverty; willingness to measure and monitor learning outcomes; and readiness to implement large-scale, evidence-based reform programs to improve foundational literacy and skills. In addition to Edo State, the initial cohort of Accelerators includes Brazil (state of Ceará), Ecuador, Kenya, Morocco, Mozambique, Niger, Pakistan, Rwanda, and Sierra Leone. In pursuit of Sustainable Development Goal 4 (SDG4), the Accelerator Program aims to demonstrate that with adequate political and financial support, governments that are dedicated to improving their foundational learning outcomes can achieve results within a few years through focused, evidence-based action. The renewed focus was reflected in a new high-level policy document: the Edo State Learning Agenda. The main document defines the agenda as a comprehensive plan aimed at creating a conducive learning environment for all children in the state.17 The learning agenda represents a step toward learning rather than mere access to education and has a strong focus on sustainability. It prioritizes actions that can be taken in the short term and sustained in the long term. The agenda does not claim to encompass all the ongoing reforms in the state but does aim to emphasize the focus on learning. While the agenda covers all levels of education, its main document, “Learning to Read,” focuses on foundational learning (Edo State Government, n.d.). The agenda defined a vision, identified specific targets, and delineated some short-term actions to achieve them. The targets were defined in terms of reading fluency in grades 2 and 3. The reading skills included in the targets below are aligned to specific reading constructs, subconstructs, and skills covered in the Global Proficiency Framework (GPF) for Reading. Under these international learning standards, it is expected that students in grade 1 will be able to understand oral language and identify letters and words but will not read fluently yet. Students in grade 2 are expected to reach a certain level of reading fluency and retrieve information from text appropriate for their school grade. Finally, students in grade 3 are expected to read fluently and to retrieve and interpret information from a grade-level text. The targets were constructed based on the improvements seen over the years of implementation but with a high level of ambition. This represented the first time Edo State committed to achieving specific reading targets for its public schools. 17 The agenda’s vision is “to ensure that all girls and boys arrive at school ready to learn, acquire real learning, and become ready to enter the job market with the right skills to become productive and fulfilled citizens” (Edo State Government, n.d.). 45 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program TABLE 4. Reading skill and expected performance (Targets) Reading skill and expected performance Baseline 2023 2024 2025 2026 2027 2021 (%) Reading fluency in Grade 2: Students able to read fluently at least nine words in a short text 40.8 43 45 50 60 70 in one minute (Global Proficiency Framework [GPF] D.2.1) *Measured as scoring correctly nine or above in REA task. Reading fluency in Grade 3: Students able to read fluently at least 58 words in a short text in 8.4 10 15 20 30 40 one minute (GPF D.2.1) Measured as scoring correctly 58 or above in REA task. Source: Edo State Learning Agenda: Learning to Read The targets were accompanied by a series of activities to be implemented in the short term classified under three categories: measuring outcomes; using real-time data; and a razor-sharp focus on early reading. In terms of measuring learning, the state committed to continue the implementation of the regular classroom assessments that are part of EdoBEST but increasing the utilization of the data to improve teaching practices, aligned with the index of quality of teaching practices developed by the state. The state also committed to sustaining the large-scale learning assessment for English and Math that started in 2023. FIGURE 7. Edo State learning agenda Vision: Everychild to be a reading champion by the age of 10 Medium-term targets: Increasing the percentage of children that can read fluently at least nine words in one minute for grade 2 (at least 70%) and at least 58 words in one minute for grade 3 (at least 40%) Measuring outcomes Using real-time data A razor-sharp focus on early reading • Large-scale regular learning assessments • EMIS • Alignment of instructional materials for English and Math • GEPD • Classroom observation • Regular classroom assessment • Incentives tied to learning outcomes • Teacher training • Index of quality of teaching practices • Innovations for learning Institutional Sustainability Financial Community engagement Source: Edo State Learning Agenda: Learning to Read 46 Improving educational outcomes for children in low-income countries III. Evidence-based interventions: the EdoBEST program Regarding the use of real-time data, the state committed to developing an education management information system (EMIS) that goes well beyond the typical system implemented in other states, in order to include the real-time data collected by the EdoBEST program and, more importantly, ensure the data is utilized to improve teaching and learning. Thus, the EMIS—which was launched in 2024—follows the EMIS 2.0 model proposed by the World Bank. Finally, the focus on early reading was reflected in three activities. Firstly, the government committed to conducting an evaluation of the teaching materials for reading for the initial school years, with the assistance of global specialists. This assessment aims to verify that these resources are consistent with reading science principles and adopt a phonemic strategy for teaching reading. The goal was to enhance consistency among various educational tools, including lesson plans, workbooks, textbooks, and instructional materials. Secondly, the government paid attention to enhancing teacher training, adjusting its classroom observation tool to include specific questions on how reading is taught, and exploring innovations for learning that and boost outcomes using technology. Since the launch of the agenda, many of its activities have already been implemented, while others are under implementation. The launch of the learning agenda can provide multiple lessons. First, maintaining high levels of political commitment throughout the implementation of the reforms is critical. The learning agenda was launched during an event hosted in April 2023 called “Edo Education Week,” which brought together more than 2,000 stakeholders. The agenda itself, the vision, targets, and activities were presented by the governor himself, showing a very high level of commitment to improving education outcomes. This creates an alignment of expectation across the education system and sends the signal that the priority is the actual improvement of learning, while other measures are just means to achieve that end. Second, the renewed focus on learning shows the importance of conceiving reforms as a continuous and iterative process. The initial focus on improving inputs and processes was necessary given the state of the education sector in 2017. However, once the improvements in teacher and learners’ attendance were evident and the EdoBEST reforms naturalized the main interventions, the next step was to shift the focus toward learning. An initial focus on learning might have been too abrupt without ensuring the basic ingredients of a functional education system and a clear evidence-based intervention was in place. For results to be sustainable, leapfrogging is not always an option. Finally, setting ambitious, realistic, and public targets is crucial. It shows that the commitment is concrete instead of abstract and creates accountability mechanisms to measure progress and aligns the incentives of all the actors of the education sector under a clear objective. Besides, the fact that the targets cut across political cycles creates incentives to sustain the reforms under the new administration and augments the probability of policy continuity, especially since the targets were developed following technical criteria. 47 Early-Grade Literacy Instruction in Edo State IV. Is EdoBEST working? © SMARTEDGE T he EdoBEST initiative has implemented its core interventions on a large scale and with sustained consistency, thereby introducing a structured routine into the education sector that was previously lacking. However, the critical question remains: Is EdoBEST truly making a difference? To address this query comprehensively, this section explores the program’s effectiveness from three distinct perspectives: its influence on educational inputs, the tangible outputs it generates, and the outcomes it achieves. To underpin the analysis, this section draws upon a diverse array of sources and employs various methodologies, incorporating original quantitative research to provide a well- rounded assessment of EdoBEST’s impact. EdoBEST is improving inputs and outputs As of early 2024, 1,282 schools and 381,012 students are benefiting from EdoBEST. More than 16,289 teachers have been trained and the number of field officers has reached 195. In other words, virtually all primary schools and junior secondary schools (JSS) are implementing EdoBEST. The program has been successful at establishing a routine in which teachers are in the classroom, teach their lesson plans, and receive constant feedback from LDOs that use classroom observation tools to gather information on teaching practices. 48 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? One of the early pieces of evidence suggesting a positive impact of FIGURE 8. the program was the increase in the number of students enrolled Number of students enrolled in public basic in public schools (figure 8). Before the launch of EdoBEST, the education in Edo State, 2013–23 crisis in the public education system was reflected in a decrease in Number of students enrolled in public basic education in Edo the number of pupils enrolled. The value, however, picked up after 500,000 EdoBest COVID-19 the program’s launch, grew at higher rates than the population commences pandemic 450,000 growth, and recovered from the effect of the COVID-19 pandemic. 400,000 350,000 A significant achievement has been the rapid improvement in 300,000 teachers’ attendance. According to the real-time data collected 250,000 through teachers’ tablets, attendance is at its highest level in primary schools (83 percent) and junior secondary schools (88 200,000 percent). This is a significant improvement considering that it 150,000 End of previous was 77 percent for both levels in 2018, when the program was 100,000 administration already implemented, and presumably even lower in 2018, before 50,000 the program was designed, although there is no official data for 0 14 20 15 20 16 20 17 20 018 20 019 20 20 20 21 20 022 23 that year. A recent survey of 200 schools that included surprise -20 -20 -20 -20 -20 -20 -20 -2 -2 -2 16 20 14 17 13 15 18 21 22 19 20 20 visits by enumerators found even higher attendance rates: 93.8 percent of teachers were in the school during an unannounced Source: Administrative data from Edo State Government. visit, with high levels in both urban schools (93.9 percent) and rural schools (93.7 percent) (World Bank, 2024). TABLE 5. Teacher attendance in Edo State, 2017–24 Indicator 2017/2018 2018/2019 2019/2020 2020/2021 2021/2022 2022/2023 2023/2024 Teacher attendance rate in No data (before 63% 66% 85% 70.7% 80% 82% primary schools (% of school program days teacher is in class) implementation) Teacher attendance rate in No data (before - - - 66% 79.60% 84% junior secondary schools (% program of school days teacher is in implementation) class) Number of students 21,731 177,329 241,073 222,474 286,629 384,465 381,012 befitting from the structured pedagogy program (and percentage of total) Number of teachers trained 6599 11,462 11,462 11,462 15,152 16,289 16,289 in the structured pedagogy program Number of field officers 71 71 112 103 195 195 195 participating in EdoBEST Source: Based on EdoBEST data. 49 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? EdoBEST appears to be improving learning A report titled “The EdoBEST Effect” was produced by the Edo State Government and its technical partner (providing support in implementing the program) in 2019 that tried to identify the impact of the program on learning outcomes using a difference-in-difference methodology. The report concluded that the difference in gains achieved by the primary 3 students in EdoBEST compared to control schools was 7 percent in math and 6 percent in literacy, which equated to nearly three- quarters of a year more math instruction and two-thirds of a year more literacy instruction. The impacts were almost entirely driven by girls (Cantrell et al., 2019).18 However, the study’s conclusions cannot be extrapolated to the entire program. In education, it is typically the case that the effect sizes of small studies are much larger than the ones for the same programs implemented at scale. For instance, a recent review of studies in low- and middle-income countries found that the median impact on learning outcomes for small studies with under 500 participating students was about 0.10 standard deviations (SDs). For large studies with more than 5,000 students, the median impact is half that, at 0.05 standard deviations (Evans and Yuan, 2022). Implementing programs at scale brings challenges that are not present in small pilots, including the impossibility of keeping the same quality of instructors, ensuring a controlled environment, and dealing with systemic issues affecting the intervention. To address these constraints, this report includes a novel analysis, leveraging the ample availability of data in the program. While it is not an impact evaluation, the analysis provides valuable information about the impacts of EdoBEST at scale. To analyze the correlates of students’ performance in evaluations, it draws from different pieces of data collected as part of the program. First, this report uses data on evaluations themselves, including the subject, the date of examination, the school, and the pupil. Second, this report uses data on the school where the evaluation was conducted, which includes its geographic location and the degree to which the intervention reached it. Third, this report uses data on pupils, which includes gender and enrollment status.  The analysis also incorporates data from a high-resolution poverty map developed by Blumenstock et al. (2021) from satellite imagery and other sources of geospatial Big Data. The maps were constructed using a machine learning algorithm exposed to a large dataset that matches ground-truth labels of poverty (from geo-located household surveys) to imagery and other geospatial data. The algorithms learn the visible features that are predictive of poverty, such as road quality, building density, and land topology. These maps establish an estimate of relative wealth deprivation and the micro-local level, which was matched to the schools’ location as a proxy of the socioeconomic conditions in the proximity of each school. 18 Even though the report provided the first quasi-experimental evidence that the project was having an impact, it suffered from many weaknesses. First, the study did not take appropriate measures to minimize selection bias. EdoBEST schools were selected by SUBEB based on observable characteristics and teacher participation was voluntary. Even though the study shows that the control and treatment groups were relatively similar in observable characteristics, it is very likely that some unobservable characteristics associated with learning and voluntary participation might have driven the effects. In fact, some of the observable characteristics also presented important differences, such as the incidence of corporal punishment, which was much lower in EdoBEST schools. Second, the study was based on only 30 schools implementing the program and 30 schools in the comparison group. This small sample size creates many challenges. To start, only large effects could have been detected with such a limited size. Furthermore, since the schools were not selected randomly, the conclusions are not generalizable to the entire state. 50 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? The analysis was performed on the resulting dataset by estimating the effect of the intervention, as measured by different indicators, on students’ performance on a set of subjects. The statistical strategy consisted of the estimation of variations of the following model:  Yi=β0+β1(Indicator_i)+β2(Closest Tile RWIi)+β3(Femalei)+β4(Term Numberi)+ϕi+ξi+εii. In this model, Yi is the outcome of the evaluation for student i, Indicator is a measure of the progress of the program, closest tile RWI is the estimated relative wealth indicator of the 1-kilometer grid cell tile closest to the school’s location, Female is an indicator variable of the pupil’s gender, term number is a time trend that captures time measured in school terms, ϕ is a fixed effect by local government authority (LGA), and ξ is a fixed effect by school grade. For this report’s purpose, the results shown cover two fundamental subjects: verbal and quantitative reasoning19, two of the most frequently evaluated subjects in primary schools. Three indicators of the progress of the program at the school level are employed as explanatory (independent) variables: number of teacher tablets distributed to the school as a share of the number of teachers in that school, number of visits by Learning Development Officers (LDOs), and number of Quality Assurance (QA) Visits. The main results are presented in tables 6 and 7 below.  TABLE 6. Score in verbal reasoning Dependent variable: Score in Verbal Reasoning (in % of total possible score) (1) (2) (3) 0.162*** Number of LDO Visits (0.007) 0.116*** Number of QA Visits (0.006) 0.337*** Tablets per Teacher (0.073) 1.157*** 1.270*** 1.576*** Closest Tile RWI (0.049) (0.049) (0.047) 2.929*** 2.923*** 2.926*** Female (0.037) (0.037) (0.037) 1.432** Time Trend (0.010) (0.010) (0.010) 47.443*** Constant (0.109) (0.110) (0.131) LGA fixed effects f f f Grade fixed effects f f f Observations 1,508,070 1,508,070 1,508,070 Adjusted R2 0.036 0.035 0.035 Note: Robust standard errors clustered at the school level in parentheses. *p<0.1; **p<0.05; ***p<0.01. 19 The choice of subjects was made based on the availability of data shared with the team. Verbal reasoning is a component of reading comprehension. 51 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? TABLE 7. Score in quantitative reasoning Dependent variable: Score in Verbal Reasoning (in % of total possible score) (1) (2) (3) Number of LDO Visits 0.162*** (0.007) Number of QA Visits 0.1147*** (0.006) Tablets per Teacher 0.513*** (0.079) Closest Tile RWI 0.633*** 0.676*** 0.991*** (0.053) (0.053) (0.050) Female 2.125*** 2.119*** 2.122*** (0.039) (0.039) (0.039) Time Trend 1.536*** 1.536*** 1.534*** (0.011) (0.011) (0.011) Constant 48.572*** 48.611 *** 47.920*** (0.117) (0.118) (0.141) LGA fixed effects f f f Grade fixed effects f f f Observations 1,496,731 1,496,731 1,496,731 Adjusted R2 0.031 0.031 0.031 Note: Robust standard errors clustered at the school level in parentheses. *p<0.1; **p<0.05; ***p<0.01. As the tables show, all three measures of implementation of the program are robustly associated with better performance in evaluations. The results are consistent across different models and robust to the inclusion of different sets of control variables, including locality and grade fixed effects. Moreover, they are similar in both scores in quantitative reasoning and in verbal reasoning assessments. They are also not insignificant in substantive terms. Across the entire sample, students in schools with the highest observed number of LDO visits performed almost three points higher (more than 10 percent of a standard deviation of the outcome) than schools with the lowest observed number of visits. This is the same as the median effect that Evans and Yuan (2022) find in their review of education studies for interventions in low- and middle-income countries on learning. The analysis also shows interesting variations when estimating heterogeneous effects by different values of covariates. The table below shows a sample of those additional results. Interaction terms show statistically significant conditional effects of the program on both gender and relative wealth of the school’s area (in both cases, in a direction that mitigates the baseline differences across groups), 52 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? though coefficient sizes are small. More meaningful are the results when the indicator of progress is interacted with a time indicator. The positive coefficient shows that the program’s positive effect on the outcome increases in magnitude over time.  TABLE 8. Score in quantitative reasoning, additional results Dependent variable: Score in Verbal Reasoning (in % of total possible score) (1) (2) (3) Number of LDO Visits 0.176*** 0.164*** -0.072*** (0.009) (0.009) (0.014) Closest Tile RWI 1.316*** 0.631*** 0.633*** (0.110) (0.053) (0.053) Female 2.121*** 2.697*** 2.124*** (0.039) (0.105) (0.039) Term Number 1.535*** 1.536*** 1.086*** (0.011) (0.011) (0.027) Number of LDO Visits * -0.081*** Closest Tile RWI (0.011) Number of LDO Visits * -0.062*** Female (0.011) Number of LDO Visits * 0.050*** Term Number (0.003) Constant 48.279*** 48.287*** 50.437*** (0.123) (0.126) (0.156) LGA fixed effects f f f Grade fixed effects f f f Observations 1,496,731 1,496,731 1,496,731 Adjusted R2 0.031 0.031 0.031 Note: Robust standard errors clustered at the school level in parentheses. *p<0.1; **p<0.05; ***p<0.01. This increase in the magnitude of the program’s effectiveness on the outcome over time can be illustrated with a predicted outcome plot based on model 3 from table 8 above.  53 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? FIGURE 9. Predicted outcome plot based on model 3 from table 8 58 54 Predicte test score 50 46 1 2 3 4 5 6 7 Term number (1 = 2021T2, 7 = 2023T3 Number of LDO visits 3 6 9 12 15 18 The plot shows that the difference in student test score outcome between schools with a higher level of progress in the program is negligible at the early stages of EdoBEST but widens considerably and is substantially large in the latest school terms. What this means in practice is that for schools that stick with the EdoBEST program, the improvement in their pupils’ performance is not one-off, but rather they are sustained and indeed can even grow over time. Overall, the quantitative analysis presented in this report shows promising insights into the effect that the program is having on students’ performance in evaluations. All measures of progress in the implementation of the program correlate with better outcomes in student assessments across different subjects. The results are robust to many different statistical specifications and control variables. Encouragingly, the effects appear to increase in strength over time, which suggests that gains from this type of intervention are not only successful in the short term but also keep improving outcomes over time as teaching and learning practices keep benefitting from them.  54 Improving educational outcomes for children in low-income countries IV. Is EdoBEST working? TABLE 9. Student performance outcomes in EdoBEST supported schools, 2021–23 Grade Indicator Quantitative reasoning Verbal reasoning 2021 2022 2023 Change 2021 2022 2023 Change 2021-2023 2021-2023 Primary 1 Mean 52.5 51.9 55.5 +3 48 53.4 54 +6 Median 53 50 57 +4 47 51 53 +6 Primary 2 Mean 45.7 53.1 54 +8.3 50.7 55.7 56.3 +5.6 Median 45 50 54 +9 50 57 57 +7 Primary 3 Mean 42.6 48.2 51.9 +9.3 48.8 53.1 55 +6.2 Median 42 50 50 +8 47 52 57 +10 Primary 4 Mean 46.1 48.2 52.2 +6.1 49.1 54.4 55.8 +6.7 Median 45 48 50 +5 48 53 57 +9 Primary 5 Mean 49.1 49.8 52.1 +3 46.8 52.8 52.6 +5.8 Median 50 50 50 0 45 51 53 +8 Primary 6 Mean 47.5 51.9     46.6 57.7     Median 46 50     46 60     Primary 1–6 Mean 47.1 50.2 53 +5.9 48.3 54.2 54.7 +6.4 combined Median 47 50 51 +4 48 53 55 +7 % within 51.4 46.8 40.6 52.4 44.8 41.1 one SD 55 Early-Grade Literacy Instruction in Edo State V. Political commitment, leadership, and financing © SMARTEDGE P olitical leadership is essential to catalyze education reforms at scale. Political leaders can bring salience to education challenges, create a narrative to mobilize support, and build coalitions that can defend, implement, and sustain the reforms. In the case of Edo State, this leadership was provided directly by the elected governor, the highest authority at the subnational level, and a key player at the national level, given the highly decentralized nature of Nigeria’s federal system. In 2017, His Excellency Godwin Obaseki, Executive Governor of Edo State, started creating a narrative to prioritize education reforms. Interestingly, the initial justification for the reforms was anchored in the state’s suffering migration crisis. In the words of the governor: “The extent of human trafficking and irregular migration that peaked in 2016–17 was our call to action” (Obaseki, 2021). The diagnostic of the governor was that the collapse of the education system, in particular at the basic education level, created a significant learning deficit that, in turn, made the youth unemployable and pushed them to find a future somewhere else, including illegal migration and in some cases even falling victim to trafficking organizations. In fact, Edo State has one of the highest rates of persons that are trafficked among states in Nigeria (UKAid, 2020). This situation created an opportunity to pursue a comprehensive education reform. In many cases, technical solutions might be available, but they cannot be implemented until a window of opportunity opens up. The presence of a new leader and the increased prominence of a migration crisis created the right political environment to advance the reforms as a policy priority and eventually implement them. 56 Improving educational outcomes for children in low-income countries V. Political commitment, leadership, and financing Mobilizing stakeholders to create consensus around education reform can be daunting, especially since improved educational outcomes take time to emerge and are less immediately visible than tangible projects like infrastructure development. The case in Edo illustrates that political leadership can leverage crises in other sectors to bring attention and create support for education reforms. The governor could have decided to take measures more directly related to the migration crisis but instead connected the crisis to the education challenges in the state. Political leadership is also needed to build a coalition of supporters. In Edo, this coalition brought together communities, parents, part of the private sector, and international donors and agencies. However, the initiative did not lack opponents in its early stages: a sizeable contingent of teachers and many academics and educationists opposed the reforms initially. Some of the main interventions implemented (and discussed below) included the use of scripted lessons by teachers, and many considered this as an element that would reduce the autonomy of teachers and degrade their status. Some opposition leaders also voiced this concern. One key lesson is that electoral and political cycles are important influences on the implementation of comprehensive reforms. Governor Obaseki pushed for these reforms in the initial phase of his first term, when political support was strong enough to continue the efforts in the face of the strong resistance of teachers and part of the education community. This would have likely been much more difficult towards the end of his term, when an election was upcoming. It would have also been more difficult during his second term, without the support that typically characterizes the initial phases of a political cycle. In addition, showing quick wins, particularly those that can benefit those resistant to change, helps sustain reform momentum. Much of the initial resistance shown by teachers declined when many of them realized that the scripted lessons were helping them save time and prepare better for their classes. Furthermore, the initial training for teachers was on a volunteer basis. Despite the relatively low early take-up rate, those who participated were publicly described as innovative teachers and classified as “digital teachers.” This narrative promptly increased the take-up rate, and soon, most teachers were enrolled in the training. Publicizing the early positive results of a pilot of the main interventions also diluted some of the support for the status quo. Bundling reforms was useful since the new interventions were accompanied by other measures that clearly benefited teachers, so opposing the whole reform package was not politically savvy. Thus, the initially strong and relatively concentrated resistance of certain groups became weaker and dispersed when the main interventions of the programs started to be seen as a “win-win” situation. More actively included the teachers in the program’s design at the outset, however, perhaps could have reduced the initial resistance—a useful lessons for other countries aiming to implement similar programs. The high-level political commitment helped secure the financial resources needed for the reform. The EdoBEST program required prioritizing the education budget, particularly given that a significant part of the interventions was implemented through a private sector technical partner. The prioritization of basic education within the budget of the new administration was critical, and it involved trade-offs. 57 Improving educational outcomes for children in low-income countries V. Political commitment, leadership, and financing A focus on the efficiency of expenditures was as important as the increased prioritization of education. One of the common critiques of the EdoBEST program is its relatively high cost, which would create obstacles to its potential scale-up to other states. However, Edo State’s proportion of resources assigned to education is not particularly high. For instance, in 2022, Edo State assigned 10.3 percent of its budget to the education sector and only 4.5 percent to basic education (figure 10).20 This is lower than commonly cited international benchmarks,21 which propose assigning between 15 and 20 percent of budgets to education. Importantly, it is also lower than the average Nigerian state, which dedicates 17 percent of its budget to education. In other words, while the volume of resources is critical, the efficiency of expenditures and the prioritization of interventions that can have an important impact is even more relevant. This, of course, requires some tough decisions. In particular, the government in Edo has been criticized for what some have claimed is a relatively weak focus on investments in education infrastructure compared to other less visible investments. FIGURE 10. Education as a percentage of total budget in Nigeria, by state, 2022 Education as a percentage of the total budget by state in 2022 40 35 30 25 Percentage 20 15 10.28 10 5 0 Jigawa Kaduna Kano Osun Enugu Kogi Kwara Adamawa Kebbi Ekiti Oyo Yobe Sokoto Taraba Rivers Plateau Ebonyl Niger Ogun Ondo Borno Benue Cross Rvier Katsina Abai Bauchi Gombe Delta Edo Bayelsa Zamfara Lagos Anambra Akwa lbom Imo Source: Nigeria Governors Forum. Public Finance Database. https://www.publicfinance.ngf.org.ng. Institutional and political economy challenges Emphasizing systemic approaches is crucial, as merely deploying evidence-based education interventions is insufficient in environments where the overarching system is flawed. As highlighted by Pritchett (2015), a thorough examination of research on learning determinants reveals wide variations in impact across different settings. While factors such as reduced class sizes, increased teacher salaries, additional textbooks, and enhanced teacher training are generally linked to improved learning outcomes, there are numerous instances where these interventions have had no significant effect. Often, the effectiveness of these measures is diminished by systemic inefficiencies, 20 From this budget, Edo State assigned N24.2 billion to education, which is equivalent to 10.3 percent of the Edo State approved budget for 2022. Expenditures in basic education, which include students from grades 0–9, account for 4.5 percent of the state’s approved budget. 21 For instance, the Education 2030 Incheon Declaration: https://uis.unesco.org/sites/default/files/documents/education-2030-incheon-framework-for- action-implementation-of-sdg4-2016-en_2.pdf. 58 Improving educational outcomes for children in low-income countries V. Political commitment, leadership, and financing a situation exemplified in many interventions in Nigeria, where, despite numerous efforts over the years, the desired educational outcomes remain elusive. Empirical evidence may challenge traditional assumptions in environments where the system is marked by disarray and dysfunction. That is why understanding how the systems in Edo contributed to the success of the reforms and, in certain instances, prevented further progress, is essential. A significant facet of the success of educational reforms in Edo can be attributed to political economy factors, particularly the stability and continuity of educational leadership. Over the six years of reform, Edo State has seen only two commissioners of education, with one having previously served as the Chair of the SUBEB. Moreover, there have been only two SUBEB chairs during this period, further contributing to a consistent and focused implementation of educational strategies. This stability at the helm of education governance has fostered an environment conducive to sustained reform efforts, allowing for the development and execution of plans without the disruptions often caused by frequent changes in leadership. Such continuity has facilitated the accumulation of institutional knowledge and expertise, crucial for the effective implementation and scaling of initiatives like EdoBEST. But systemic issues also created challenges. The unclear division of responsibilities between the State Ministry of Education and the State Universal Basic Education Board (SUBEB), which is common across most states in Nigeria, often leads to significant operational inefficiencies within the education sector. This ambiguity in roles can result in overlapping functions and duplicated efforts, which wastes valuable resources and creates conflicts between these two critical educational bodies. For instance, the production of an education management information system and the development of a comprehensive learning assessment in many cases suffered delays because duplication of efforts was identified, with similar initiatives pursued by the Ministry and SUBEB, which also created inefficiencies in the utilization of resources. The leadership of the governor was critical in many situations to overcome these challenges. There are also indicators that different levels of government have different views on some aspects of how students should be served and prioritized. Survey evidence shows that authorities at the local level tend to believe that motivated or well-prepared students deserve more attention than others, whereas this is a less sharply held view among state level practitioners. The key here is to ensure that such different perspectives are carefully managed, not least to avoid sending confusing signals to school leaders and teachers on the ground. 59 Improving educational outcomes for children in low-income countries V. Political commitment, leadership, and financing FIGURE 11. Percentage of respondents who agree with the following statement: “Students deserve more attention if... 70.00 65.10 60.00 49.55 49.27 50.95 50.00 2018 USD 40.00 30.00 20.00 15.51 16.22 10.00 - 2018 2019 2020 2021 2022 2023 Source: World Bank (2024). EdoBEST shows that, when possible, leaders should attempt to establish clear responsibilities and roles and delineate specific delivery processes. However, that is not always possible, given the institutional scaffolding and the timeline of the reforms. In Edo, for example, many of the overlapping responsibilities are determined by a federal law enacted in 2004, the Universal Basic Education Act. While the act made great progress in establishing the principle of free basic education and creating a basic education fund, it also created conflicting responsibilities across and within levels of government. Changes to this legislation are not possible for a single state. Even reforms that are possible at the state level might take a long time to create the necessary consensus. Thus, when changing the institutional incentives of the main organizations in charge of education the presence of clear leadership that identifies a vision and sets a limited number of goals is critical. In contrast to what has been observed in some other states, the appointed Commissioner of Education has ample experience in the education sector and receives the necessary support from the governor to design and implement the reforms, first as Chair of SUBEB and then as Commissioner of Education. 60 Improving educational outcomes for children in low-income countries V. Political commitment, leadership, and financing BOX 4. The World Bank’s support While the reforms from the Edo State Government commenced in 2018, since 2020, they have been supported by a World Bank program that utilized a Program for Results (PforR) Instrument for a total of US$70 million, plus an additional US$5 million for technical assistance. This program was instrumental in two main areas: firstly, it aimed to support the scale-up of EdoBEST to all the schools in Edo State. Second, it incentivized the development and implementation of additional interventions beyond the original scope of EdoBEST but still critical to advance learning outcomes, notably the establishment of the learning assessment system previously mentioned. In addition, the program supported investments in digital and entrepreneurship skills for youth (not the focus of this report) and was therefore called Edo State Basic Education and Skills Sector Transformation (EdoBESST) (World Bank, 2020). The World Bank’s results-based financing mechanisms proved particularly effective in this context, as they were designed to release funds based on achieving specific, pre-agreed educational outcomes. This approach not only ensured accountability but also motivated the various stakeholders involved to meet their targets, thereby maximizing the impact of the interventions. More specifically, the PforR has been supporting the following reforms: ● Expansion of EdoBEST to all primary schools and inclusion of all junior secondary schools. ● Design and roll-out of EdoBEST@Home ● Design and implementation of Education Management Information System ● Design and implementation of the ELAS ● Reforms to pre-service teacher training. The experience of using results-based financing provides several valuable lessons for the use of this instrument in similar contexts: ● Selection of indicators: Several factors come into play in defining results for a finance-based operation (O’Brien and Kanbur, 2014). For interventions in their nascent stages, focusing on inputs or outputs can help set things in motion. Focusing on outcomes is beneficial for more mature interventions as it allows for the customization of interventions to fit specific contexts, enhancing the effectiveness and use of government systems. The decision to focus on inputs, outputs, or outcomes should also consider the government’s capacity and the available evidence linking interventions to desired outcomes. In any case, the establishment of appropriate results must be driven as much as possible by the objective measurement of data in order to offset the information asymmetries between principals and agents (Terway et al., 2021). Sometimes, this implies having suboptimal indicators that are, however, more feasible to verify. Results-based indicators must strike a balance between cost, effort, feasibility, and ambition. 61 Improving educational outcomes for children in low-income countries V. Political commitment, leadership, and financing ● Value of incentives: Incentives must be internally coherent, with resources allocated in a way that reflects the effort required for each objective. Efforts should be evaluated not just in monetary terms but also considering time commitment, stakeholder involvement, and required technical capacities. Furthermore, the incentives should exceed the cost of the intervention to motivate the achievement of results. ● Verification processes: Verification protocols need to be established in advance and detailed to avoid bottlenecks. The actual verification needs to follow a very rigorous process. In Edo, verification conducted by private firms was found to be more effective than verification by other government organizations. ● Operational and technical assistance funds: Allocating a portion of the overall funding for technical assistance is crucial, especially for complex interventions, to ensure the achievement of desired results. The success of this initiative in Edo State serves as a valuable model for other states and countries in the region, demonstrating the potential of well-designed and supported educational reforms to achieve significant improvements in learning outcomes. The World Bank’s involvement in Edo’s educational reforms underscores the value of international partnerships in advancing educational quality and equity, showcasing a collaborative path toward enhancing education systems globally. 62 Early-Grade Literacy Instruction in Edo State VI. A focus on implementation capacity © SMARTEDGE W hile the right interventions are essential, their outcomes could be affected or even annulled in the absence of solid implementation capabilities. A recent report shows that, in low- and middle-income countries, policy makers identify implementation challenges (including limited implementation capacity) as one of the main reasons behind reform failures (Bryant et al., 2024). Leaders must not only be aware of the “what” in terms of interventions but also possess a deep understanding of the “how” to effectively implement these strategies on a large scale and in their specific contexts (Bryant et al., 2024). This section reviews the importance of using data to guide reforms, the scalability model followed, the implementation modality, and the role of communities. It ends with an analysis of the cost of EdoBEST and its sustainability. Data as a key element There is a critical need for robust information systems within the educational sector. The establishment and strengthening of reliable data systems are crucial for both data producers and users, serving as a cornerstone for achieving impactful educational outcomes. By enhancing data systems, policy makers can foster an environment where decisions are informed by data, leading to a more strategic allocation of resources that aligns closely with educational priorities. 63 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity The collection of data is one of the strongest points of the EdoBEST programs. Over the years, the program has been able to collect millions of data points, including many variables that are collected daily. EdoBEST has among the most comprehensive information on any education program in Nigeria. More precisely, the following data are collected from each school, stored, and processed on the cloud platform: ● School data (including a unique ID for each school, and data related to the facilities and the teachers) ● Student photo, biodata, and a unique ID for each student ● Parent’s data and contact information ● Teacher attendance data (Teachers identified by their “Oracle ID” which is their payroll ID, with daily attendance information) ● Device ID and system health for the tablets used by teachers ● Student attendance data ● Student assessment grades for the classroom assessments ● Teacher time on task for each of the lessons ● Quality assurance reports ● Data from the classroom observation tools collected by LDOs. Most of the data is processed and used as follows: ● Create management reports and dashboards for government officials, which include student attendance, teacher attendance, and teacher time on task. ● Dashboard data is also used by LDOs for coaching purposes. ● Headteachers use data to communicate with parents when students are absent. ● Assessment data and time on task used by the instructional design team to modify teacher lesson guides. The collected data consists of a relational database, which in turn comprises six tabular datasets: (1) daily logs of pupil attendance to class, (2) daily logs of teacher attendance to schools, (3) lesson- level logs of completion and duration, (4) visit-level information of Learning Development Officers visits, (5) visit-level information of Quality Assurance visits and (6) exam-level information on student evaluations. These are large datasets, with a long-time scope of coverage and high granularity. For instance, they have detailed information on 17,570,039 individual lessons, 2,713,194 daily attendance logs for individual teachers, 1,048,575 individual student attendance logs, and 35,208,857 individual student evaluation results. A data warehouse owned by the government houses all the data. Despite the large amounts of data, there were challenges during the first years of program implementation that are worth highlighting. First, in many cases, the ample availability of data has not been leveraged enough for decision-making, and its utilization by public officials has been limited (Saavedra and De Simone, 2023). Moreover, the feedback loops with schools and teachers have been limited. The utilization has been constrained to the production of reports for SUBEB management and, in some limited cases, for headteachers to use data to contact families of those who are absent from school. However, a rich data environment like the one in Edo provides many more opportunities 64 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity to harness the potential of data. In addition, the data has not been cross-referenced with other important data sources, including demographic and administrative information. Second the large availability of data has not been supported by the appropriate skilled personnel. SUBEB and the Ministry of Education have had limited investment in human capacity to exploit the availability of data, and, as such, most of the data utilization is simply done by reports produced by the technical partner and sent to the political authorities. Third, there has been a lack of institutionalization of the data systems in the state. It is not clear what organization has the primary responsibility for collecting and analyzing data, and, in some cases, there have been overlapping responsibilities and duplication of functions between SUBEB and the Ministry. The contract with the technical partner determines that the data belongs to the government, but it does not define the data precisely. Many of these challenges started to be addressed during the last year of the program implementation. In particular, the government, partly incentivized by the World Bank’s PforR, invested in the development of an education management information system (EMIS). This process defined a long list of indicators and identified the sources, frequency, and responsibilities for data collection. This EMIS, hosted by the government, now contains all the historical data that, until recently, was on the technical partner’s cloud. This EMIS is following the EMIS 2.0 model proposed by the World Bank (EdTech Team, 2022). The utilization of data for decision-making purposes has also taken off. For instance, the data has recently been merged with a micro-poverty map and administrative data from the National Personnel Audit conducted by UBEC to develop a machine-learning model to predict dropouts and target interventions to prevent them.22 It is also now being used for the design of a teacher performance- based management system. The capacity to utilize data is also growing thanks to these experiences and to the capacity-building efforts that were part of the development of the learning assessment system, which had a strong focus on data management. The experience of EdoBEST shows that collecting data is crucial for effective decision-making but is not enough. Improving education outcomes based on data requires a conscious effort and investments in the right human and institutional capabilities. Pritchett (2018) posits that the collection of real-time data should be grounded in a cohesive theory of change framework to avoid being overwhelmed with unstructured data and potentially losing sight of the larger picture of child well-being. Furthermore, understanding causal connections between different inputs is essential for making sense of real-time data in education systems, as they tend to be non-linear and interactive. In more advanced stages, making the data public can also increase accountability and encourage think tanks and civil society to contribute to the knowledge of the education sector, create original 22 This analytical activity was led by the World Bank, which also shows the potential of making the data publicly available for researchers, partners, think tanks, and civil society to contribute to policy making. 65 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity © SMARTEDGE research, and open avenues for new interventions. That has not yet been done in Edo State and is something for consideration as the program moves forward. Integrating multiple data sources and ensuring the existence of clear feedback loops that incentivize actors at several levels to use data to improve outcomes is fundamental. Another lesson points to the importance of having clear agreements between parties when programs are implemented through public-private partnerships. While data can be beneficial in many settings, the government’s ownership of the data and oversight of its use must be defined clearly. Finally, data privacy and data security are critical. In the case of Edo, for many years, data was secured in the backend, backed up regularly, and multiple data centers were used for redundancy and failover. Access to dashboard reports is password protected. In 2024, Edo moved toward a new EMIS that hosts the data in government servers. Scalability of the program The educational reforms in Edo are predominantly aligned with what have been termed as “implementation-intensive services” or “high-discretion” interventions (Andrews, Pritchett, and Woodcock, 2017). These approaches necessitate the involvement of highly skilled professionals, a resource often in short supply. They rely heavily on educators, school leaders, and district administrators to embrace and consistently execute new methodologies, posing significant challenges in terms of supervision and adherence. 66 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity Contrastingly, these interventions are distinct from “logistical” interventions as defined by the same authors. The latter category entails changes that do not demand advanced technical skills, intricate monitoring systems, or substantial behavioral shifts among educators, such as the removal of school fees or infrastructure developments like constructing new schools. In Edo, the structured pedagogy program demands considerable behavioral adjustments from educators, necessitating a transition to scripted lessons and real-time monitoring. Teachers are expected to adapt to regular feedback from coaching staff, who may not always possess more experience. Additionally, LDOs, along with quality assurance officers, are required to establish a consistent schedule for school visits and adhere to a standardized protocol for classroom observations. Research by Crawfurd et al. (2022) indicates that skill-intensive policies can yield significant benefits on a smaller scale. However, these benefits tend to diminish as the scale expands and are rarely assessed at larger scales. In contrast, less skill-intensive policies may have more modest effects initially but maintain consistency as they are scaled up. The scale-up of EdoBEST was fast (table 10). The pilot in 2018 included only 30 schools but was rapidly scaled up to include 237 schools at the primary level. Soon, all the schools were covered in primary education, and the junior secondary schools were incorporated in 2021 and 2022. The rapid scale-up of EdoBEST offers valuable lessons. First, it demonstrates that even implementation- intensive interventions can be scaled up swiftly in low-capacity environments, albeit with pragmatic choices, such as outsourcing certain capacities during the initial stages. This strategy requires careful execution to facilitate a potential transition to full government implementation. Unlike incremental approaches that allow for fine-tuning, rapid expansion demands a flexible implementation strategy TABLE 10. Pace of rollout of EdoBEST   2017/2018 2018/2019 2019/2020 2020/2021 2021/2022 2022/2023 2023/2024 Number of primary schools 237 848 848 848 992 1002 997 enrolled in program % of all schools covered 24% 85% 85% 85% 99% 100% 100% Number of junior secondary 0 0 0 0 236 287 285 schools (JSS) enrolled in program % of all schools covered 0% 0% 0% 0% 83% 100.0% 100.0% Total primary schools 1002 1002 1002 1002 1002 1002 997 Total JSS schools 286 286 286 286 286 287 285 Source: Based on EdoBEST data. 67 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity complemented by a continuous, iterative approach to accommodate necessary adjustments. Rapid expansion does build a sense of real purpose and momentum, which arguably creates incentives for change and consolidates support among the public and other stakeholders. Second, the impact of structured pedagogy programs that expand quickly cannot be assumed to be uniform across schools nor the same from year to year (indeed impacts could compound over time). Patience is required until significant impacts emerge, as evidenced in Edo, where the program’s effectiveness increased over time. This suggests a virtuous cycle of learning and adaptation among implementers, leading to more effective teaching practices as routines are established and materials become more widely available. Once the multiple actors get familiar with the routine of the program, a high-discretion intervention might gradually become more expeditious as the implementers naturalize their new behaviors into a new routine. Lastly, while less skill-intensive policies are more scalable, they pose two significant challenges: their impacts are typically smaller and may not meet the transformative aspirations of some leaders, and they are more common in access-increasing programs rather than learning-focused interventions. For regions with sufficient access but needing to enhance learning outcomes, high-discretion interventions may be necessary despite scalability challenges. These can be supplemented with less demanding initiatives to optimize impact, such as behavioral campaigns in Edo to reduce teacher absenteeism, thereby enhancing overall educational outcomes. A public-private partnership for implementation A public-private partnership (PPP) in education is an arrangement between a government and a private body—whether that body is for profit or not—whereby the government is providing financing and guiding policy decisions, but the private entity (such as a nongovernmental organization) is delivering education services (Akmal et al., 2022). This is the model followed in Edo. Understanding what worked and what did not work in the context of Edo is critical because the evidence on the impact of PPPs in low- and middle-income countries is limited (Ngware, 2022) and the studies that exist do not show a common pattern. In fact, the picture is rather mixed, with some cases of improvements in learning and access and some other cases of null impacts. Similarly, the cost- effectiveness of these models is disputed. Most of the implementation of the interventions that are part of EdoBEST was led by a technical partner hired by the government.23 The technical partner is a for-profit firm that has a presence in several countries in the region but for which the Edo program represents its largest intervention. The firm deployed a team of local staff to implement EdoBEST. In practice, the firm brings a package that includes the scripted lessons for all subjects, the proprietary software described above, and the training programs for teachers. They also organize the system of LDOs and the monitoring system. They store the information on their own servers. 23 In the case of EdoBEST, the government’s technical partner implementing the program was New Globe. 68 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity © SMARTEDGE This delivery model was conducive to the implementation of EdoBEST for at least two reasons. First, the government was looking for a rapid solution that could be scaled up quickly and, therefore, did not have the time to develop its own structured pedagogy program. Second, and probably more important, the government had limited capacity to deploy a program like EdoBEST at scale. In particular, the government lacked public servants with the appropriate skills to develop the program materials, supervise the implementation, and ensure rapid deployment across all the schools. However, the program in Edo State differed significantly from technical partner-supported programs in other countries because the intervention was in public schools and, critically, used public teachers. These were not schools owned by the technical partner, as has been the case in other countries. Thus, the technical partner managed the program and brought many of the technical resources but was under the supervision of the SUBEB and used the public school system. The technical partner and the government constituted a public-private partnership that can offer valuable insights for the design of similar programs. The rapid scalability of the program was a direct consequence of the involvement of the technical partner. This underscores the potential of private partners to accelerate the rollout of educational initiatives, especially when time constraints and the need for quick wins are critical factors. Rapid scalability also comes with risks. One of them is the potential lack of customization of some of the program materials and interventions to the local context. Even though the partner claims to align the scripted lessons with the Nigerian curriculum, alignment does not mean customization. As noted above, the scripted lessons could benefit from stronger customization to the context of Southern Nigeria. 69 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity Other potential issues that come with PPPs were well mitigated, given the design of the program in Edo. For instance, a very similar program in Liberia, where public schools were managed by private contractors, raised student learning by 60 percent, compared to standard public schools (Romero et al., 2017). However, costs were high, performance varied across contractors, and contracts authorized the largest contractor to push excess pupils and underperforming teachers onto other government schools. This did not happen in Edo because there was no competition among different contractors and, therefore, fewer incentives to push low-performing students to other schools. Additionally, since the program rapidly scaled to cover all the public schools in the state, there was no way to push students to public schools not managed by the private entity. A clear government mandate on including all students, even in remote areas, and avoiding dropouts mitigated the risk of the contractor gaming the system. A strong monitoring system can help avoid these risks. In Edo, the fact that the quality assurance officers and the LDOs were quickly transferred to the government’s payroll is likely to have helped. While PPPs can provide immediate benefits in terms of resources and expertise, it makes sense to consider the long-term sustainability of such programs. Ensuring that the government can eventually take over and maintain the initiatives without compromising quality is crucial for lasting impact. The development of institutional and human capabilities within the government structures is critical to ensure that the reforms can be sustained in the long term regardless of the political cycles and the financial ups and downs. In the case of Edo, the governor made clear his intention to transfer all the knowledge and functions before the end of the current administration, but, in practice, the transfer has been a mixed experience. On the bright side, the LDOs, who were originally paid by the technical partner, have been transferred to the government payroll after a few years of implementation. Furthermore, the integration of the local staff of the technical partner within the government structure was seamless, which helped with coordination and knowledge transfer. On the other hand, the transfer of specific knowledge and skills was very weak. For instance, the technical partner was not always cooperative in transferring its capacity to produce lesson plans. When, in early 2022, the government conducted an independent evaluation of the lesson plans to increase their alignment with the science of learning, getting the scripted lessons from the partner was challenging. Partly, the partner relied on a clause in the contract which specifies that the lesson plans are proprietary. Similarly, the process of transferring the data from the technical partner to the government only started during the last year of the program implementation, partly triggered by the deployment of EMIS. This transfer of knowledge is not “a given” when the implementers are private sector organizations and, therefore, it must be carefully planned and follow a gradual process. From the initial phase, the contract must be very clear about certain aspects of the program. In particular, the Edo experience shows that several items must be properly incorporated into the legal agreement: ● A clear understanding that all the data collected through the program belong exclusively to the government and an explicit definition of what the data is, including the multiple indicators that the program collects. 70 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity ● An explicit mention to the need to transfer knowledge, which was the case in Edo. ● An explicit mention to how the lesson plans can be transferred to the government and, if that is not possible, how the technical partner will support the development of in-house capacity. Experience from other countries shows that this is possible. For instance, with the Tusome project in Kenya, copyright ownership of all instructional materials, as well as the digital infrastructure (such as the dashboard), were handed over to the government to strengthen government systems (RTI International, 2023b). ● If the structured pedagogy program includes edtech components, a clear understanding of how the transfer to the government will occur, including when the hardware and the software might be proprietary. ● An explicit design of accountability mechanisms to ensure alignment with the government’s educational objectives and maintaining transparency were essential for mutual trust and the program’s success. ● In some cases, contracts can incorporate performance-based clauses that go beyond the achievement of specific outputs to include the improvement of outcomes, which could be attached to specific rewards. ● There needs to be clear stipulations to avoid the possibility for the contractors to game the system. More specifically, contractors should not be allowed to push students out of the system to show improvements in learning, and clear monitoring mechanisms need to be established. Even beyond the contract, the transfer of knowledge needs to be intentional and driven by the government. The government must ensure its staff has enough time and resources to receive knowledge from the partners. During the project implementation, there should be a clear roadmap for the transition of responsibilities, including identifying key stakeholders, developing a timeline, identifying and assessing risks, and establishing a communications and training plan. The roadmap should identify the steps and actions needed to transfer responsibilities from the current entity to the new one, and to ensure a smooth and efficient transfer of responsibilities, minimizing disruptions to operations and services. The roadmap should also include the identification of short-, medium-, and long-term phases, milestones, and goals for the transition process. 71 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity The role of communities Since the early stages of the reform, community engagement has been a critical element of the government’s agenda. The role of communities has been twofold. On the one hand, they have been an instrument to ensure accountability of the interventions. On the other hand, more recently, their role has been leveraged to try to ensure the sustainability of the reforms. The community’s first task is to monitor the program and ensure it achieves results. This is usually referred to as the “third long route of the accountability triangle” in education systems. In Nigeria, this task is mostly performed by School Based Monitoring Committees (SBMCs). They represent a crucial aspect of the country’s educational reform efforts, aiming to decentralize decision-making and enhance community involvement in school governance. These committees are typically composed of parents, teachers, students, and community leaders who work collaboratively to support the improvement of school infrastructure, enhance the quality of education, and ensure that schools are responsive to the needs of their local communities. In Edo State, they usually have a total of 17 members. SBMCs play a pivotal role in mobilizing resources, monitoring school activities, and fostering partnerships that can lead to better educational outcomes. Their establishment is part of a broader strategy to promote transparency, accountability, and community ownership of educational initiatives. They also play a critical role in ensuring safety in and around schools (see Box 5). By empowering local stakeholders to take an active role in the management of schools, Nigeria aims to create a more inclusive, equitable, and effective educational system that can address the diverse needs of its student population. Even during the program’s initial phases, when only 230 schools were part of EdoBEST, the government emphasized the importance of SBMCs in the reform. In the words of the governor, “I will rely on the SBMCs to help the state government monitor all the contractors to ensure good service delivery and, where necessary, draw government’s attention to poor jobs.” In focus groups with multiple members of SBMCs, members usually highlight the achievement of the program and focus on the improvements in children’s learning. In the most recent years of EdoBEST, communities have started to play a key role in trying to ensure the sustainability of the program. In earlier periods, the government organized multiple stakeholder engagement sessions to sensitize the communities around the main components of the program. More recently, the government arranged a series of large-scale Parents’ Summits with the explicit intention of gathering support for the sustainability of EdoBEST. During these events, parents endorsed specific aspects of the program with the hope that their voices might help maintain the initiative in the years to come. For instance, during a Parents’ Summit organized in April 2023, the President of the Edo Market Women Association announced the group’s adoption of the “School O’Clock” program, which seeks to ensure that no child of school age is found outside the school during school hours (Agbonigiarhuoyi, 2023). The case of Edo illustrates the importance of engaging communities to gather support for large- scale reforms, ensure implementation is adequate where the government cannot monitor directly, and create local coalitions to sustain the reforms. 72 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity BOX 5. Ensuring safety in Edo schools: the role of communities Safety in and around schools is paramount for fostering a secure and nurturing educational environment. When students feel safe, they are more likely to engage fully in their studies, participate in extracurricular activities, and develop positive relationships with peers and teachers. Effective safety measures, including secure buildings, well-trained staff, clear emergency protocols, and anti-bullying programs, contribute to reducing incidents of violence, accidents, and other threats. A safe school environment not only protects students from physical harm but also supports their emotional and mental well-being, creating a sense of stability and trust. Additionally, parents and the community are more confident in the school system when safety is prioritized, leading to increased involvement and support. Ultimately, ensuring safety in and around schools is crucial for promoting a positive educational experience and fostering a conducive academic and personal growth atmosphere. In Edo, one of the most significant efforts to promote a safe learning environment is the creation of a grievance redress mechanism (GRM) with strong community participation. The GRM is a locally based, formalized system for accepting, assessing, and resolving community feedback or complaints. It provides accessible channels for communities to express concerns or submit complaints and establishes procedures to address them. When complaints involve victims, GRMs determine pathways to provide support and ensure perpetrators are prosecuted accordingly. Through the EdoBESST GRM, communities have been able to report multiple cases of gender-based violence (GBV) and sexual harassment. In most cases, the perpetrators are not part of the project but are mostly family or community members of students. However, students have consistently used the GRM to report these cases, which would likely have gone unreported without this system. Thus, the project has created a trusted system, opening a path for survivors to receive appropriate support and ensuring legal actions are taken against perpetrators. ● Institutional provision of the Social Mobilization Department: By law, the Ministry of Education established the Department of Social Mobilization, which is responsible for mass mobilization, advocacy, and sensitization of the general public, forging partnerships with stakeholders in basic education, and disseminating messages of the Universal Basic Education program. The department also engages with stakeholders and community members to ensure various government initiatives aimed at educating children are effective. Social Mobilization Officers (SMOs) from the department serve as grievance redress officers (GROs) in local government areas (LGAs), each overseeing 10 schools. ● Government initiatives: The Edo State Government has initiatives like School O’clock, encouraging children to be at school during school hours. SMOs monitor their areas to ensure children are not loitering during school hours. 73 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity ● Experience of the delivery team: The EdoBESST delivery team has a well-experienced GRM Desk Officer who has previously worked on social welfare issues, including childcare development. This experience has been transferred to her current role. ● Sensitization and awareness: The program conducts sensitization activities on issues of sexual exploitation and abuse/sexual harassment (SEA/SH) and the reporting process. Students are encouraged to report SEA/SH cases to their headteachers and guidance counselors. ● Grievance redress reporting structure: Each SMO/GRO is allocated ten schools per LGA for monitoring. Once an issue is reported to a headteacher or guidance counselor, the SMO/ GRO is notified. For GBV cases, the survivor is referred to the nearest GBV service provider or government health facility, and the case is lodged at the nearest police station. The GRM Desk Officer documents cases using a survivor-centered approach and ensures follow-up to stabilize the survivor and prosecute perpetrators. The GRM system demonstrates that assigning a specific team to ensure children’s protection is critical for providing adequate support and maintaining a safe learning environment. This community- driven approach is essential for safeguarding students’ well-being and fostering a culture of safety and trust in schools. The delivery team A central tenet of the World Bank’s support was its emphasis on building local capacity to ensure the sustainability of these reforms. This approach was grounded in the belief that empowering civil servants, administrators, and policy makers with the necessary skills and knowledge would lead to more enduring and self-sufficient educational improvements. A critical aspect of the capacity-building efforts undertaken was the strategic enhancement of the abilities of the Project Implementation Unit (PIU). Recognizing this unit’s pivotal role in the successful rollout and sustainability of educational reforms, the PIU was renamed the “Delivery Team.” This rebranding was a deliberate behavioral nudge to underscore the unit’s essential role in the reform process and to infuse a sense of mission and purpose into its members. By reframing the PIU as the Delivery Team, the initiative aimed to shift the team members’ perception, encouraging them to see themselves not as mere components in a bureaucratic machine but as integral players in a mission-driven endeavor to transform education in Edo State. This repositioning was instrumental in motivating the team and fostering a culture of ownership, accountability, and proactive engagement with the project’s objectives. One of the lessons learned from the Delivery Team’s operations was the challenge posed by the team’s bureaucratic structure and limited autonomy from political authorities. This situation inadvertently affected the team’s operational speed and efficiency, highlighting a critical area for 74 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity © SMARTEDGE improvement in the management and execution of educational reform projects. While motivated and aligned with the mission to transform education, the Delivery Team found its effectiveness somewhat constrained by the traditional bureaucratic frameworks within which it operated. The close ties and dependencies on political structures meant that decisions often required multiple layers of approval, leading to delays in implementation and a reduction in the team’s ability to respond swiftly to emerging challenges or adapt to changing circumstances. This experience underscores the importance of ensuring that key operational teams in large-scale reform projects are endowed with sufficient autonomy to make timely decisions and implement strategies effectively. A balance must be struck between oversight and operational freedom to allow teams like the Delivery Team to execute their roles with agility and innovation. As a lesson for future initiatives, this highlights the need for a more flexible and responsive operational structure that empowers project teams to act efficiently. Ensuring that such teams have the autonomy to navigate day-to-day decisions can significantly enhance the speed and effectiveness of project implementation. 75 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity The cost of EdoBEST and its sustainability Understanding the cost-effectiveness of educational programs like EdoBEST is essential for several reasons. First, it ensures that the investments made in education yield the highest possible returns in terms of improved learning outcomes and societal benefits. Stakeholders can make informed decisions about scaling, replicating, or modifying the initiative by evaluating the costs relative to the program’s impact. Second, a thorough analysis of costs helps identify efficient resource allocation strategies, ensuring that limited educational budgets are utilized in the most effective manner. Moreover, insights into the cost structures of successful programs provide valuable benchmarks for other regions or countries contemplating similar reforms, facilitating a more pragmatic approach to educational investment and policymaking. Finally understanding the costs of EdoBEST is fundamental to assessing its potential sustainability in the medium and long term. Calculating the cost per student in Edo State presents quite a challenge due to the intertwined nature of expenditures for teaching and non-teaching staff across different educational levels, coupled with ambiguity regarding the financial contributions from state and local governments towards personnel costs. The federal governance system in Nigeria, combined with a lack of clear accountability for spending and a fragmented funding landscape, further complicates this task. The complexities of consolidating budgets from federal, state, and local entities hinders the attempt to ascertain the per-student cost using a top-down approach. A more reliable method involves constructing the total expenditure from the ground up. This approach relies on more verifiable data points, such as teacher salaries, the total number of educators, and the average class sizes in primary and junior secondary schools, to achieve a more accurate financial assessment. The first building block in the construction of a prototype school is the calculation of teaching and non-teaching salaries. Total monthly expenses in primary school salaries amounted to approximately ₦ 1.1 billion per month. Dividing this amount by the number of teachers in primary yields an average monthly gross salary of ₦ 144,917 before taxes, and a monthly take home pay of ₦ 100,692 after taxes. This salary is fairly close to the average salary of a teacher at the grade 7 level in the pay scale. Thus, the average monthly salary of a teacher in primary or JSS is about US$195 at the exchange rate of 2023. This salary will be the representative salary for teachers in the school prototype calculations. For non-teachers, such as school administrative staff, custodians, and laborers, the average monthly salary is about ₦ 52,000 or US$69. Given the profile of a typical school, one can derive its average recurrent costs under the conservative assumption that salaries represent about 80 percent of the total operating costs of a school. Table 11 below shows that the average annual operating cost of a typical primary school in Edo State would be about ₦ 13.6 million, or about US$18,466. For a typical JSS the annual operating costs would be ₦ 25.7 million or about US$34,794. The resulting annual per-student costs are ₦ 52,966 (US$72) for primary and ₦ 72,721 (US$97) for JSS. In other words, the total cost of running the basic education system without EdoBEST is US$72 per student for primary education and US$97 per student in JSS. Investments in infrastructure are not included in the calculation because they exceed the objective of this report and available data on the costs of building new schools or expanding the number of classrooms is highly volatile. 76 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity TABLE 11. Annual recurrent cost per primary and junior secondary school Type of recurrent cost Primary Junior Secondary (₦) US$ (₦) US$ Personnel: 10,932,082 14,773 18,023,423 24,356 Teacher salaries 9,666,458 13,063 16,124,987 21,791 Non-teaching staff salaries 1,265,624 1,710 1,898,436 2,565 Operation and Management (O&M) 2,733,020 3,693 7,724,324 10,438 Total  13,665,102 18,466 25,747,747 34,794 Cost per Student 52,966 72 71,721 97 Source: Own estimations based on Edo SUBEB (2023). To understand the additional cost of adding EdoBEST, the analysis relies on the actual cost of the intervention based on the support provided by the technical partner. The cost covers mainly the structured pedagogy program including the scripted lessons, the accompanying edtech software and hardware, the monitoring and coaching systems, other teaching and learning materials, and the overall support for data collection and analysis. In addition, the cost estimates include the delivery of EdoBEST@Home. Figure 12 below shows that the cost of EdoBEST has changed significantly over the years. While it started with an annual cost of US$65.10 per student, the cost today is US$16 per student. Why has the cost fallen over time? Well, the first year of the program involved a series of fixed costs that are not present for the following years. Specifically, it included approximately US$49 variable cost per student and a fixed cost that, when divided by the number of students participating in the program, was about US$16 per student. Subsequently, the program grew significantly in its number of beneficiaries and was able to achieve economies of scale. Currently, the program covers almost all the students in basic education. The expansion to JSSs included many schools that are adjacent to a primary school and, therefore, already have taken on board the fixed investments needed. Then, in 2022 a new contract started with the technical partner, which included a significant reduction of the costs and the transfer of some of the costs to the government. This latter point involves, for instance, the LDOs, who are currently paid by the government and therefore included in the abovementioned calculation of the cost of running the education system in Edo as non-teaching staff. The depreciation of the naira with respect to the US dollar also helped reduce the cost in dollars, since almost all the EdoBEST staff is local. Assuming that the annual cost per student going forward could be close to the average of the last three years, that would translate to US$27.56 per student. A more optimistic scenario would be that the cost of the most recent year (2023) is maintained—at US$16.22. Finally, the large-scale learning assessment has an estimated cost of US$1.75 per student per year. This means that the EdoBEST program adds about 24–40 percent to the per pupil cost of education provision in primary education and 18–30 percent for secondary education. 77 Improving educational outcomes for children in low-income countries VI. A focus on implementation capacity FIGURE 12. Total cost of EdoBEST per student 70.00 65.10 60.00 49.55 49.27 50.95 50.00 2018 USD 40.00 30.00 20.00 15.51 16.22 10.00 - 2018 2019 2020 2021 2022 2023 Source: Own estimations. The cost of EdoBEST is significantly lower than the cost of similar programs implemented by the same technical partner in other countries. For instance, in Liberia, the first year of the program cost more than US$1,052 (Romero et al., 2017) per pupil, and the following years, it went down to more than US$300 (Romero and Sandefur, 2019). It is also lower than the cost of similar interventions supported by other providers. For instance, again in the case of Liberia, the average expenditure was around US$300 for the first year and US$119 for the subsequent years (Akmal et al., 2022). However, EdoBEST is still more expensive than what a recent comparative analysis shows is the cost per student for one year of a structured pedagogy program, which is US$ 7.75, for low- and middle- income countries (Angrist et al., 2023). However, a higher cost is understandable, given the larger scope of the program in Edo, which covers all subjects. In any case, EdoBEST may be able to lower its costs further. If over the next years, the government builds its capacity to implement the program without a technical partner and ensures enough knowledge transfer for that to happen, the cost might go down significantly, helping make the program more sustainable. The main takeaway of this analysis is positive. EdoBEST seems to be reaching a virtuous cycle in which the costs have gone down significantly while the impacts are growing. Structured pedagogy programs have been identified as one of the most cost-effective, given their large effects in increasing the learning-adjusted years of schooling for every dollar invested (GEEAP, 2023). A recent analysis shows a benefit-to-cost ratio of 105, which makes it one of the most cost-effective interventions to improve learning outcomes (Angrist et al., 2023). 78 Early-Grade Literacy Instruction in Edo State VII. Conclusion and key lessons learned © SMARTEDGE T he EdoBEST program in Edo State, Nigeria, offers valuable insights for other low-income countries pursuing education reforms. This is particularly true for those implementing structured pedagogy programs to enhance foundational learning outcomes in challenging environments, which are considered one of the most cost-effective interventions to improve learning. Edo State’s education system faces many of the problems encountered across Sub-Saharan Africa, making it a highly relevant case study for similar reform initiatives. Although EdoBEST is still an ongoing reform program, its six years of implementation provide a solid foundation from which to draw several critical lessons. Learning outcomes in the state are still low, but three findings suggest there are noteworthy features: i) learning outcomes are improving as a result of the program interventions, ii) the effect of the program is increasing over time, and iii) the unit cost of the interventions is going down. This trajectory of improvement underscores the importance of sustained efforts and long-term commitment to education reforms. Central to EdoBEST’s success has been the political leadership and entrepreneurship that identified opportunities to implement these reforms and build consensus around them. Rather than merely increasing funding, the program focused on strategic investments, particularly in “start-up” costs, which remained within feasible budgetary limits. 79 Improving educational outcomes for children in low-income countries VII. Conclusion and key lessons learned Placing learning at the forefront of the agenda, supported by publicly available specific targets, has been crucial. Measuring learning outcomes is essential, especially in systems with little prior experience of doing that in a rigorous way. Careful communication of these outcomes is fundamental to maintaining transparency and building public trust. This is particularly important to prevent backlash in systems that did not measure learning systematically ahead of the reforms and, therefore, might bring light to low learning outcomes for the first time. The structured pedagogy program, grounded in evidence-based practices, underscores the importance of continuous improvement and policy refinement. For instance, while the lesson plans used in Edo are based on the science of reading, there is room for further enhancement to make the program more effective and responsive to the needs of students. The experience shows that education reforms that focus on learning outcomes are complex. It is not possible to tackle all challenges simultaneously, so priorities must be regularly monitored and adjusted as needed. In Edo, for instance, language of instruction was not among the main priorities initially, but it might need to be in the future so even larger positive effects can be achieved. Teachers play a pivotal role in the success of major reform programs. Despite the support provided through coaching and scripted lessons, teacher capacity remains a challenge. Nonetheless, there has been a noticeable improvement in teaching practices, highlighting the need for systemic reforms to ensure high-quality teacher recruitment and development from the outset. This dual focus on immediate support and long-term capacity building is essential for sustainable success. Education technology has proven to be a powerful tool when integrated coherently with other interventions, such as teacher training and school modernization. It should not be viewed as a standalone solution, but rather as a complement to a broader, integrated approach. This integrated approach ensures that technology enhances rather than distracts from the core educational objectives. Importantly, education technology also helps improve the status and reputation of frontline providers, such as teachers and headteachers, facilitating the implementation and removing sources of resistance to the reforms. Edtech, however, should always be adapted to the specific context of low-income settings and designed to avoid potential misuse. Furthermore, technology for learning outside the classroom, or @Home technology, shows promise, especially in contexts where education disruptions are frequent, such as due to insecurity or climate- induced shocks. However, the successful implementation of these technologies requires overcoming substantial challenges, as evidenced by the experience in Edo. Having a robust alternative delivery mechanism can be invaluable in maintaining the continuity of learning. While the design of an educational reform programs like EdoBEST is crucial, the implementation process is equally or perhaps even more significant. Drawing from the principles of the science of implementation, it is evident that the success of such reforms hinges on the fidelity and quality of their execution. For EdoBEST, this involves not only the initial rollout but also continuous monitoring, adaptation, and support to address emerging challenges and contextual nuances. Effective implementation requires building strong local capacity, fostering stakeholder collaboration, and ensuring that the program’s components are delivered as intended. Additionally, it necessitates a 80 Improving educational outcomes for children in low-income countries VII. Conclusion and key lessons learned robust feedback mechanism to inform iterative improvements, thereby enhancing the program’s impact over time. Edo’s experience shows that even implementation-intensive interventions can be scaled up swiftly in low-capacity environments, albeit with necessary compromises, such as outsourcing certain capacities during the initial stages. At the same time, Edo also shows that it is possible to build institutional capacity within governments, even when starting from a very low baseline. The learning assessment system, for instance, is a testimony that highly complex and technical tasks can be done with government resources when the right support is available. A focus on sustainability must be embedded from the beginning of the reforms. Public-private partnerships can be effective in contexts of low institutional capacity and where rapid results are needed, but these should be coupled with measures to ensure the effective transfer of knowledge and capacity to the state bureaucracy. This approach ensures that the benefits of the reform are not only immediate but enduring. Data is fundamental to the success of education reforms. For data to be truly helpful, it needs to be actively used. In more advanced stages, making data public can enhance accountability and encourage contributions from think tanks and civil society, fostering original research and new intervention avenues. However, data privacy and security must be diligently respected to protect the integrity of the information and the individuals to whom it relates. The education challenges facing so many low-income countries in Africa are daunting. Yet the need to do better—much better—for children across the continent is more important than ever. EdoBEST is still an ongoing, work-in-progress reform program, so this is not the point at which a final evaluation can be made. That said, its six years of implementation are showing promise and shining a light on ingredients that can help drive meaningful and lasting improvements in educational outcomes. 81 Improving educational outcomes for children in low-income countries References References Adeniran, A., D. Okoye, M. P. Yedomiffi, and L. Wantchekon. 2022. “COVID-19 Learning Losses, Parental Investments, and Recovery: Evidence from Low-Cost Private Schools in Nigeria.” RISE Working Paper Series, 22/120. Research on Improving Systems of Education (RISE) Programme. https:// doi.org/10.35489/BSG-RISEWP_2022/120. Adeniran, A., S. Onyekwere, A. Okon, J. Atuhurra, R. Chaudhry, and M. Kaffenberger. 2023. “Instructional Alignment in Nigeria using the Surveys of Enacted Curriculum.” RISE Working Paper 23/143. Research on Improving Systems of Education (RISE) Programme. Agbonigiarhuoyi, Nosakhare. 2023. “How do we make sure that all of these reforms will last?”, https:// edostate.gov.ng/stakeholders-in-edo-state-pledge-commitment-to-governor-obasekis-educa- tion-reforms-at-parents-summit/. Akmal, M., S. Hares, and R. Perakis. 2022. “The Evidence on Outsourcing Education to the Private Sector is Mixed. Results are Better for Increasing Enrollment than for Raising Learning Outcomes.” In J. Sandefur (Ed.), Schooling for All: Feasible Strategies to Achieve Universal Education (pp. 103–121). Washington, DC: Center for Global Development. Andrews, M., L. Pritchett, and M. Woolcock. 2017. Building State Capability: Evidence, Analysis, Action. Oxford University Press. Angrist, N., E. Aurino, H. Patrinos, G. Psacharopoulos, E. Vegas, R. Nordio, and B. Wong. 2023. “Improving Learning in Low- and Lower-Middle-Income Countries.” Journal of Benefit-Cost Analysis 1–26. http//:doi.org/10.1017/bca.2023.26. Ardington, C. 2023. “Funda Wande Limpopo Evaluation, Second Midline Report.” Southern Africa Labour and Development Research Unit. Barrett, P., A. Treves, T. Shmis, D. Ambasz, and M. Ustinova. 2018. “The Impact of School Infrastructure on Learning: A Synthesis of the Evidence (English).” International Development in Focus. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/ en/853821543501252792/The-Impact-of-School-Infrastructure-on-Learning-A-Synthesis-of- the-Evidence. Behrman, J. R., S. W. Parker, P. E. Todd, and K. I. Wolpin. 2015. “Aligning Learning Incentives of Students and Teachers: Results from a Social Experiment in Mexican High Schools.” Journal of Political Economy 123(2): 325–64. 82 Improving educational outcomes for children in low-income countries References Belay, T., M. E. De Simone, D. Parajuli, O. O. Okunola, M. Dhal, A. Gafar, E. Pradhan, D. Chugunov, D. Isser, and I. J. Nweje. 2024, forthcoming. “Nigeria Human Capital Public Expenditure and Institutional Review.” World Bank, Washington, DC. Blumenstock, J. et al. 2021. “Using Big Data and machine learning to locate the poor in Nigeria” Blog post. World Bank, Washington, DC. https://blogs.worldbank.org/opendata/using-big-data-and- machine-learning-locate-poor-nigeria. Brunette, T., B. Piper, R. Jordan, S. King, and R. Nabacwa. 2019. “The Impact of Mother Tongue Reading Instruction in Twelve Ugandan Languages and the Role of Language Complexity, Socioeconomic Factors, and Program Implementation.” Comparative Education Review 63(4). Bryant, A. 2024. “Spark & Sustain: How All the World’s School Systems Can Improve Learning at Scale.” McKinsey & Co. [White paper]. https://www.mckinsey.com/industries/education/our-insights/ spark-and-sustain-how-school-systems-can-improve-learning-at-scale. Cantrell, S., J. Oviawe, L. Chen, M. Buttweiller, and S. Nadh. 2019. “The EdoBEST Effect.” Edo State Government. https://issuu.com/newglobe/docs/1._the_edobest_effect_-_study_of_gains_in_ edo_nig. Chetty, R., J. N. Friedman, and J. E. Rockoff. 2014. “Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood.” American Economic Review 104(9): 2633–79. Cilliers, J., B. Fleisch, J. Kotze, M. Mohohlwane, and S. Taylor. 2022. “The Challenge of Sustaining Effective Teaching: Spillovers, Fade-out, and the Cost-Effectiveness of Teacher Development Programs.” Economics of Education Review 87(2): 102215. Clarke, M. 2012. “What Matters Most for Student Assessment Systems: A Framework Paper.” SABER—Systems Approach for Better Education Results series, Working Paper No. 1. World Bank, Washington, DC. Clarke, M., and D. Luna-Bazaldua. 2021. “Primer on Large-Scale Assessments of Educational Achievement. National Assessments of Educational Achievement.” World Bank, Washington, DC. Crawford, M. F., M. R. Barron Rodriguez, E. Y. Z. Ding, M. Gutierrez Bernal, J. Saavedra Chanduvi, and O. S. Arias Diaz. 2021. “Attaining the Learning Target: A Policy Package to Promote Literacy for All Children.” World Bank, Washington, DC. Crawford, M.; M. Oviedo Buitrago, N. Faquir, and N. Raheel. 2022. Handbook for Literacy Lesson Planning: Early-grade Classroom Activities for Teaching Children to Read (English). Tools for Improving Reading. Washington, DC: World Bank Group. http://documents.worldbank.org/ curated/en/099010004212217887/P17425203005530e5087f50fa4754b643d3 83 Improving educational outcomes for children in low-income countries References Crawfurd, L., S. Hares, and L. Sandefur. 2022. Susannah Hares, and Justin Sandefur. In J. Sandefur (Ed.), Schooling for All: Feasible Strategies to Achieve Universal Education (pp. 103-121). Center for Global Development. https://www.cgdev.org/publication/schooling-all-feasible-strate- gies-achieve-universal-education Darling-Hammond, L., R. C. Wei, A. Andree, N. Richardson, and S. Orphanos. 2009. “Professional Learning in the Learning Profession: A Status Report on Teacher Development in the United States and Abroad.” National Staff Development Council, Washington, DC. De Simone, M. E., A. Garba, G. Joseph-Raji, J. O. Oviawe, A. Ragatz, and A. Twinomugisha. 2020. “Learning Despite the Crisis: The Case of Edo State in Nigeria.” World Bank, Washington, DC. https://blogs.worldbank.org/education/learning-despite-crisis-case-edo-state-nigeria. Dee, T. S., and J. Wyckoff. 2015. “Incentives, Selection, and Teacher Performance: Evidence from Impact.” Journal of Policy Analysis and Management 34(2): 267–97. Duflo, E., P. Dupas, and M. R. Kremer. 2015. “School Governance, Teacher Incentives, and Pupil-Teacher Ratios: Experimental Evidence from Kenyan Primary Schools.” Journal of Public Economics 123: 92–110. Duflo, E., R. Hanna, and S. P. Ryan. 2012. “Incentives Work: Getting Teachers to Come to School.” American Economic Review 102(4): 1241–78. Edo State Government. 2024. “Edo State Learning Assessment Systems and Strategy. Mathematics and Reading Comprehension Primary 3, Primary 6, and Junior Secondary 3. First Cycle (2021– 2023).” Benin City, Edo State, Nigeria. Edo State Government. n.d. “Edo State Learning Agenda: Learning to Read.” https://subeb.edostate. gov.ng/edo-state-learning-agenda/. Benin City, Edo State, Nigeria. EdTech Team. 2022. “Knowledge Pack: EMIS 2.0 Learning and Accountability System Architecture (LASA).” World Bank Group, Washington, DC. Evans, D. K., and F. Yuan. 2022. “How Big are Effect Sizes in International Education Studies?” Educational Evaluation and Policy Analysis 44(3): 532–40. https://doi.org/10.3102/01623737221079646. Fleisch, B., V. Schoer, G. Roberts, and A. Thornton. 2016. “System-Wide Improvement of Early-Grade Mathematics: New Evidence from the Gauteng Primary Language and Mathematics Strategy.” International Journal of Educational Development 49: 157–174. Fryer, R. G., Jr. 2013. “Teacher Incentives and Student Achievement: Evidence from New York City Public Schools.” Journal of Labor Economics 31(2): 373–407. 84 Improving educational outcomes for children in low-income countries References Glazerman, S., D. Mayer, and P. Decker. 2006. “Alternative Routes to Teaching: The Impacts of Teach for America on Student Achievement and Other Outcomes.” Journal of Policy Analysis and Management 25(1): 75–96. Global Education Evidence Advisory Panel (GEEAP). 2023. “2023: Cost-Effective Approaches to Improve Global Learning. London, Washington D.C., New York. FCDO, the World Bank, UNICEF, and USAID. Gray-Lobe, G., A. Keats, M. Kremer, I. Mbiti, and O. W. Ozier. 2022. “Can Education be Standardized? Evidence from Kenya.” University of Chicago, Becker Friedman Institute for Economics Working Paper No. 2022-68. https://doi.org/10.2139/ssrn.4129184. Hanushek, E. A. 2011. “The Economic Value of Higher Teacher Quality.” Economics of Education Review 30(3): 466–479. Kosack, S. 2012. The Education of Nations: How the Political Organization of the Poor, Not Democracy, Led Governments to Invest in Mass Education. Oxford University Press. Le Nestour, A., L. Moscoviz, and J. Sandefur. 2022. “The Long-Run Decline of Education Quality in the Developing World.” Working Paper 608. Center for Global Development, Washington, DC. Loureiro, A., and L. Cruz. 2020. “Achieving World-Class Education in Adverse Socioeconomic Conditions: The Case of Sobral in Brazil.” Washington, DC: World Bank. Loureiro, A., L. Cruz, and U. Mello. 2021. “Chapter 8: Brazil Case Study.” In S. Al-Samarrai and B. Lewis, eds. 2021. The Role of Intergovernmental Fiscal Transfers in Improving Education Outcomes. International Development in Focus. Washington, DC: World Bank. Martin, M., I. V. S. Mullis, and P. Foy. 2008. “Students’ Backgrounds and Attitudes Towards Science.” In TIMSS 2007 International Science Report: Findings from IEA’s Trends in International Mathematics and Science Study at the Fourth and Eighth Grades (Chapter 4). TIMSS & PIRLS International Study Center, Boston College. Mbiti, I. M., K. Muralidharan, and Y. Schipper. 2016. “Inputs, Incentives, and Complementarities in Primary Education: Experimental Evidence from Tanzania.” University of California at San Diego. Meiring, T. 2021. “Impact Evaluation of Funda Wande In-Service Teacher Coaching Intervention: Findings from the First Year.” Southern Africa—Towards Inclusive Economic Development. Mullis, I. V. S., M. O. Martin, P. Foy, and K. T. Drucker. 2012. The PIRLS 2011 International Results in Reading. TIMSS and PIRLS International Study Center, Boston College. Munoz-Najar, A., and J. O. Oviawe. 2020. Nigeria: Edo-BEST@Home. OECD Publishing. 85 Improving educational outcomes for children in low-income countries References Muralidharan, K. 2012. “Long-Term Effects of Teacher Performance Pay: Experimental Evidence from India.” University of California at San Diego. Ngware, M. 2022. “Comment: Be Cautious about Public-Private Partnerships, and Fund Students Not Schools.” In J. Sandefur (Ed.), Schooling for All: Feasible Strategies to Achieve Universal Education (pp. 130–131). Washington, DC: Center for Global Development. National Population Commission - NPC/Nigeria and ICF. 2019. Nigeria Demographic and Health Survey 2018. Abuja, Nigeria, and Rockville, Maryland, USA: NPC and ICF. Obaseki, G. 2021. “How an Exodus of Young Nigerians Spurred a Rethink on Schools. Financial Times, June 10. https://www.ft.com/content/6f634094-686a-4123-85dc-3be43d35e1e4. O’Brien, T., and R. Kanbur. 2014. The Operational Dimensions of Results-based Financing. Public Administration and Development. Wiley. https://onlinelibrary.wiley.com/doi/10.1002/pad.1698. Ogenyi, M. 2022. “Looking Back on Nigeria’s COVID-19 School Closures: Effects of Parental Investments on Learning Outcomes and Avoidance of Hysteresis in Education.” RISE Insight Note, 2022/040. Research on Improving Systems of Education (RISE) Programme. https://doi. org/10.35489/BSG-RISE-RI_2022/040. Pimhidzai, O., and G. Martin. 2015. “Africa Region—Education Service Delivery in Nigeria: Results of 2013 Service Delivery Indicator Survey.” World Bank Group, Washington, DC. http://documents. worldbank.org/curated/en/099511510072218372/IDU03d762d210e215040e70887c04b10c4a67a32. Piper, B., J. Destefano, E. Kinyanjui, and S. Ong’ele. 2018. “Scaling Up Successfully: Lessons from Kenya’s Tusome National Literacy Program.” Journal of Education Change 19: 293–321. https:// doi.org/10.1007/s10833-018-9325-4. Piper, B., S. Simmons Zuilkowski, and A. Mugenda. 2014. “Improving Reading Outcomes in Kenya: First- Year Effects of the PRIMR Initiative.” International Journal of Educational Development 37: 11–21. Piper, B., S. Simmons Zuilkowski, M. Dubeck, E. Jepkemei, and S. King. 2018. “Identifying the Essential Ingredients to Literacy and Numeracy Improvement: Teacher Professional Development and Coaching, Student Textbooks, And Structured Teachers’ Guides.” World Development 106: 324–36. Piper, B., Y. Sitabkhan, J. Mejia, and K. Betts. 2018. “Effectiveness of Teachers’ Guides in the Global South: Scripting, Learning Outcomes, and Classroom Utilization.” RTI Occasional Paper, May 2018. RTI International. Popova, A., D. K. Evans, and V. Arancibia. 2016. “Training Teachers on the Job: What Works and How to Measure it.” Policy Research Working Paper 7834. World Bank, Washington, DC. 86 Improving educational outcomes for children in low-income countries References Pritchett, L. 2015. “Creating Education Systems Coherent for Learning Outcomes.” Research on Improving Systems of Education. https://riseprogramme.org/publications/creating-educa- tion-systems-coherent-learning-outcomes.html Pritchett, L. 2018. “The Risks of Dangerous Dashboards in Basic Education.” Center for Global Development, Washington, DC. https://www.cgdev.org/publication/risks-dangerous-dash- boards-in-basic-education Ramachandran, R. 2012. “Language Use in Education and Primary Schooling Attainment: Evidence from a Natural Experiment in Ethiopia.” Working Papers 2012/34. Institut d’Economia de Barcelona (IEB). Romero, M., J. Sandefur, and W. Sandholtz. 2017. “Can Outsourcing Improve Liberia’s Schools? Preliminary Results from Year One of a Three-Year Randomized Evaluation of Partnership Schools for Liberia.” CGD Working Paper 462. Washington, DC: Center for Global Development. Romero, M., and J. Sandefur. 2019. “Beyond Short-term Learning Gains: The Impact of Outsourcing Schools in Liberia after Three Years.” CGD Working Paper 521. Washington, DC: Center for Global Development. RTI International. 2023a. “Let’s Read: Understanding Kenya’s Success in Improving Foundational Literacy at Scale.” RTI International. https://www.rti.org/impact/lets-read-understanding-ken- yas-success-improving-foundational-literacy-scale. RTI International. 2023b. “Learning at Scale: Time and Money.” RTI International. https://learnin- gatscale.net/wp-content/uploads/2023/10/Learning-at-Scale-Cost-Analysis-Report_Final87.pdf. Saavedra, J., and M. De Simone. 2023. “Improving Education in Edo State, Nigeria: Time to Focus on Learning and Sustainability.” Blog post, Education for Global Development. World Bank, Washington, DC. https://blogs.worldbank.org/education/improving-education-edo-state-nige- ria-time-focus-learning-and-sustainability. Savchenko, Y., M. E. De Simone, J. A. Weaver, H. D. Rasolonjatovo Andriamihamina, E. N. Bentil, J.-P. Tan, E. Baranshamaje, M. Dahal, W. O. Mosuro, K. T. Silva, and E. Y. Yukseker. 2022. “Western and Central Africa Education Strategy from School to Jobs: A Journey for the Young People of Western and Central Africa.” World Bank Group, Washington, DC. Shanahan, T. E. 2022. “How to Provide Effective Reading Instruction.” World Bank Group, Washington, DC. http://documents.worldbank.org/curated/en/099620103312223967/ P17425203c5a110520a5c3004086a91b687 Spaull, N. 2023. “Background Report for the 2030 Reading Panel.” Cape Town. https://groundup. org.za/media/uploads/documents/embargoed_2023_reading_panel_background_report_7_ feb_2023.pdf. 87 Improving educational outcomes for children in low-income countries References Terway, A. et al. 2021. “Results-Based Financing in Education for Sub-National Government and School Administrators: A Conceptual Framework and Practical Recommendations.” World Bank, Washington, DC. https://documents1.worldbank.org/curated/en/182581644936918794/ pdf/Results-Based-Financing-in-Education-for-Sub-national-Government-and-School- Administrators-A-Conceptual-Framework-and-Practical-Recommendations.pdf. The Nation, 2018. “School-based management committee to monitor renovation of 230 schools”, https://thenationonlineng.net/school-based-management-committee-to-monitor-renova- tion-of-230-schools/ Trudell, B. 2018. “Language and Education in Nigeria: A Review of Policy and Practice.” British Council. https://www.britishcouncil.org.ng/sites/default/files/language_and_education_nigeria.pdf. Trudell, B., and B. Piper. 2014. “Whatever the Law Says: Language Policy Implementation and Early Grade Literacy Achievement in Kenya.” Current Issues in Language Planning 15(1): 4–21. UKAid. (2020). “How Migration and Trafficking from Edo State Happens: Stamping Out Trafficking in Nigeria.” UKAid. https://www.alignplatform.org/sites/default/files/2020-12/how_migration_and_ trafficking_happens_research_final_report.pdf. United Nations Educational, Scientific and Cultural Organization (UNESCO). 2024. Global Education Monitoring Report. Paris: UNESCO. https://education-estimates.org/out-of-school/data/. University of Oregon. 2020. 8th Edition of Dynamic Indicators of Basic Early Literacy Skills (DIBELS): Administration and Scoring Guide. University of Oregon. https://dibels.uoregon.edu/sites/dibels1. uoregon.edu/files/2021-05/dibels_8_admin_and_scoring_guide_05_2020.pdf. United States Agency for International Development (USAID). 2021a. Language of Instruction Country Profile: Nigeria.” USAID, Washington, DC. https://pdf.usaid.gov/pdf_docs/PA00XH27.pdf. United States Agency for International Development (USAID). 2021b. “READ Liberia Endline Report.” USAID, Washington, DC. https://pdf.usaid.gov/pdf_docs/PA00Z92Q.pdf. Villaseñor, P. 2017. “The Different Ways that Teachers can Influence the Socio-emotional Development of their Students: A Literature Review.” World Bank, Washington, DC. Walter, C., and J. Briggs. 2012. “What Professional Development Makes the Most Difference to Teachers?” Department of Education, University of Oxford. World Bank, UNESCO, UNICEF, USAID, FCDO, Bill & Melinda Gates Foundation. 2022. “The State of Global Learning Poverty: 2022 Update.” World Bank, Washington, DC. World Bank. 2018. World Development Report 2018: Learning to Realize Education’s Promise. Washington, DC: World Bank. 88 Improving educational outcomes for children in low-income countries References World Bank. 2020. “Edo Basic Education Sector and Skills Transformation Operation.” International Development Association, Program Appraisal Document, Report No. PAD3600. World Bank, Washington, DC. World Bank. 2021a. “Loud and Clear: Effective Language of Instruction Policies for Learning.” World Bank Policy Approach Paper. World Bank, Washington, DC. World Bank. 2021b. “Teach Primary.” Brief. World Bank, Washington, DC. https://thedocs.worldbank. org/en/doc/bd7b15d459fe78fab0201297b9f4a45d-0200022022/related/Teach-Implementation- Brief-english-04012022.pdf World Bank. 2022. “AFW Education Strategy—Technical Report.” World Bank, Washington, DC. World Bank. 2024, forthcoming. Global Education Policy Dashboard in Edo State. World Bank, Washington, DC. Yoon, K. S., T. Duncan, S. Lee, B. Scarloss, and K. L. Shapley. 2007. “Reviewing the Evidence on How Teacher Professional Development Affects Student Achievement.” Issues and Answers Report REL 2007–033. Regional Educational Laboratory Southwest, Institute of Education Sciences, U.S. Department of Education. 89