Publication: From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria
Loading...
Date
2025-05-20
ISSN
Published
2025-05-20
Editor(s)
Abstract
This study evaluates the impact of a program leveraging large language models for virtual tutoring in secondary education in Nigeria. Using a randomized controlled trial, the program deployed Microsoft Copilot (powered by GPT-4) to support first-year senior secondary students in English language learning over six weeks. The intervention demonstrated a significant improvement of 0.31 standard deviation on an assessment that included English topics aligned with the Nigerian curriculum, knowledge of artificial intelligence and digital skills. The effect on English, the main outcome of interest, was of 0.23 standard deviations. Cost-effectiveness analysis revealed substantial learning gains, equating to 1.5 to 2 years of ’business-as-usual’ schooling, situating the intervention among some of the most cost-effective programs to improve learning outcomes. An analysis of heterogeneous effects shows that while the program benefits students across the baseline ability distribution, the largest effects are for female students, and those with higher initial academic performance. The findings highlight that artificial intelligence-powered tutoring, when designed and used properly, can have transformative impacts in the education sector in low-resource settings.
Link to Data Set
Citation
“De Simone, Martin; Tiberti, Federico; Barron Rodriguez, Maria; Manolio, Federico; Mosuro, Wuraola; Dikoru, Eliot Jolomi. 2025. From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria. Policy Research Working Paper; 11125. © World Bank. http://hdl.handle.net/10986/43212 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Future of Poverty(Washington, DC: World Bank, 2025-07-15)Climate change is increasingly acknowledged as a critical issue with far-reaching socioeconomic implications that extend well beyond environmental concerns. Among the most pressing challenges is its impact on global poverty. This paper projects the potential impacts of unmitigated climate change on global poverty rates between 2023 and 2050. Building on a study that provided a detailed analysis of how temperature changes affect economic productivity, this paper integrates those findings with binned data from 217 countries, sourced from the World Bank’s Poverty and Inequality Platform. By simulating poverty rates and the number of poor under two climate change scenarios, the paper uncovers some alarming trends. One of the primary findings is that the number of people living in extreme poverty worldwide could be nearly doubled due to climate change. In all scenarios, Sub-Saharan Africa is projected to bear the brunt, contributing the largest number of poor people, with estimates ranging between 40.5 million and 73.5 million by 2050. Another significant finding is the disproportionate impact of inequality on poverty. Even small increases in inequality can lead to substantial rises in poverty levels. For instance, if every country’s Gini coefficient increases by just 1 percent between 2022 and 2050, an additional 8.8 million people could be pushed below the international poverty line by 2050. In a more extreme scenario, where every country’s Gini coefficient increases by 10 percent between 2022 and 2050, the number of people falling into poverty could rise by an additional 148.8 million relative to the baseline scenario. These findings underscore the urgent need for comprehensive climate policies that not only mitigate environmental impacts but also address socioeconomic vulnerabilities.Publication Central Bank Independence and Sovereign Borrowing(Washington, DC: World Bank, 2025-07-25)This paper studies the impact of central bank independence on sovereign borrowing, using an index that captures institutional constraints on central bank lending to the government across 155 countries from 1972 to 2023. The findings show that tighter lending to the executive significantly reduces sovereign interest rates and raises the debt-to-gross domestic product ratio in developing countries. These effects reflect the executive’s improved ability to borrow at lower costs under greater central bank independence. The results are robust to multiple tests, but there are no significant effects in advanced economies. From a policy perspective, the results highlight the key role of independent central banks as catalysts for reducing governments’ borrowing costs and enhancing the government’s borrowing capacity.Publication Disentangling the Key Economic Channels through Which Infrastructure Affects Jobs(Washington, DC: World Bank, 2025-04-03)This paper takes stock of the literature on infrastructure and jobs published since the early 2000s, using a conceptual framework to identify the key channels through which different types of infrastructure impact jobs. Where relevant, it highlights the different approaches and findings in the cases of energy, digital, and transport infrastructure. Overall, the literature review provides strong evidence of infrastructure’s positive impact on employment, particularly for women. In the case of electricity, this impact arises from freeing time that would otherwise be spent on household tasks. Similarly, digital infrastructure, particularly mobile phone coverage, has demonstrated positive labor market effects, often driven by private sector investments rather than large public expenditures, which are typically required for other large-scale infrastructure projects. The evidence on structural transformation is also positive, with some notable exceptions, such as studies that find no significant impact on structural transformation in rural India in the cases of electricity and roads. Even with better market connections, remote areas may continue to lack economic opportunities, due to the absence of agglomeration economies and complementary inputs such as human capital. Accordingly, reducing transport costs alone may not be sufficient to drive economic transformation in rural areas. The spatial dimension of transformation is particularly relevant for transport, both internationally—by enhancing trade integration—and within countries, where economic development tends to drive firms and jobs toward urban centers, benefitting from economies scale and network effects. Turning to organizational transformation, evidence on skill bias in developing countries is more mixed than in developed countries and may vary considerably by context. Further research, especially on the possible reasons explaining the differences between developed and developing economies, is needed.Publication Crowding Out and Banking Crises(Washington, DC: World Bank, 2025-07-22)This paper studies the effect of government issuance on firm issuance during banking crises using transaction-level bond and loan data from 66 countries between 1991 and 2017. Governments rarely issue loans, preferring to issue in bond markets. In contrast, firms receive most of their financing from banks. During banking crises, as the supply of domestic loans decreases, firms switch to issuing bonds in domestic markets. The paper uses a novel instrument based on maturing debt to overcome the potential endogeneity of government issuance. The findings show that firms must compete with the government for funds in the domestic bond market and are crowded out from this market as a result. This happens not only in developing countries, but in advanced countries as well. The paper also shows that firms with the ability to tap international debt markets switch to these markets when crowding out occurs in domestic bond markets. Lastly, the paper shows that more developed domestic bond markets mitigate, but do not eliminate, the degree to which crowding out occurs.Publication Designing and Analyzing Powerful Experiments(Washington, DC: World Bank, 2025-07-22)This paper offers practical advice on how to improve statistical power in randomized experiments through choices and actions researchers can take at the design, implementation, and analysis stages. At the design stage, the choice of estimand, choice of treatment, and decisions that affect the residual variance and intra-cluster correlation can all affect power for a given sample size. At the implementation stage, researchers can boost power through increasing compliance with treatment, reducing attrition, and improving outcome measurement. At the analysis stage, power can be increased through using different test statistics or estimands, through the choice of control variables, and through incorporating informative priors in a Bayesian analysis. A key message is that it does not make sense to talk of “the” power of an experiment. A study can be well-powered for one outcome or estimand, but not others, and a fixed sample size can yield very different levels of power depending on researcher decisions.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Efficient Learning for the Poor : Insights from the Frontier of Cognitive Neuroscience(Washington, DC : World Bank, 2006)This book integrates research into applications that extend from preschool brain development to the memory of adult educators. In layman's terms, it provides explanations and answers to questions such as: Why do children have to read fast before they can understand what they read? How do health, nutrition, and stimulation influence brain development? Why should students learn basic skills in their maternal language? Is there such a thing as an untrained teacher? What signs in a classroom show whether students are getting a quality education? How must information be presented in class so that students can retain it and use it? What training techniques are most likely to help staff put their learning into use? This book is intended for use by policymakers, donor agency staff, teacher trainers, supervisors, and inspectors, as well as university professors and students.Publication Developing Cross-Language Metrics for Reading Fluency Measurement(World Bank, Washington, DC, 2012-07-10)Since 2005, over 70 oral reading fluency tests have been given in many languages and scripts, either as part of the Early Grade Reading Assessment (EGRA) or as individual one-minute tests. Particularly in multilingual countries, reading speed and comprehension measures have been taken in multiple languages and also in multiple scripts. The development of language has a significant genetic component, which tends to create common grammatical structures. Then languages must conform to information processing limitations, notably to working memory capacity. On the basis of such features, it may be possible to develop common standards for performance improvement compare findings cross linguistically. Languages are most comparable when large chunks are used rather than single words. To arrive at some comparisons, several methods may be tried. These include: a) counting actual words in connected texts or in lists, using some conventions if needed; b) using computational solutions to arrive at coefficients of certain languages vis a vis others, such as 1 Swahili word being equivalent roughly to 1.3 English words; c) using in multiple languages lists of words of a defined length, e.g. 4 letters; d) measuring phonemes or syllables per minute, possibly dividing by average word length; and e) rapid serial visual presentation, potentially also measuring perception at the letter feature level. Overall, reading rate as words per minute seems to be a valid and reliable indicator of achievement, with 45-60 words being a range that is usable as a benchmark.Publication Vocational Education in the New EU Member States : Enhancing Labor Market Outcomes and Fiscal Efficiency(Washington, DC: World Bank, 2007)This report explores the fiscal aspects of vocational education reform in the context of secondary education as a whole and considers the implications of any changes in the vocational education (VE) system for post-secondary and other modes of skill development. The report begins by describing the inherited system of vocational education in the former socialist countries of Central and Eastern Europe which was based on the assumption that everyone had to be trained for a specific occupation before starting work and that it was the function of vocational schools to provide such training. The report explores the scope for improvements in fiscal efficiency via a number of propositions about VE in the EU8 countries today: a) It would not be possible or advisable to fund adequately a traditional VE system which would provide ready-to-work recruits with narrowly specialized skills for the economy's enterprises; b) One way to reduce costs to government would be to locate practical training entirely in-plant but this is increasingly difficult; c) EU8 employers' traditional expectations of a fully-subsidized VE system delivering ready-to-work, specifically-skilled recruits are unreasonable; d) Traditional VE was the traditional answer to the question "What to do with those who have performed less well in basic education?" but this answer no longer convinces; and e) Parents and students are showing an increasing preference for general education (GE) over VE. Each of these propositions was discussed in this report not with a view to prescribing a detailed "one-size-fits-all" strategy for all the EU8 countries, but rather to deriving some principles that continued reform of VE could take into account, to the benefit of fiscal efficiency.Publication Developing Social-Emotional Skills for the Labor Market : The PRACTICE Model(World Bank Group, Washington, DC, 2014-11)Although there is a general agreement in the literature of the importance of social-emotional skills for labor market success, there is little consensus on the specific skills that should be acquired or how and when to teach them. The psychology, economics, policy research, and program implementation literatures all touch on these issues, but they are not sufficiently integrated to provide policy direction. The objective of this paper is to provide a coherent framework and related policies and programs that bridge the psychology, economics, and education literature, specifically that related to skills employers value, non-cognitive skills that predict positive labor market outcomes, and skills targeted by psycho-educational prevention and intervention programs. The paper uses as its base a list of social-emotional skills that employers value, classifies these into eight subgroups (summarized by PRACTICE), then uses the psychology literature -- drawing from the concepts of psycho-social and neuro-biological readiness and age-appropriate contexts -- to map the age and context in which each skill subset is developed. The paper uses examples of successful interventions to illustrate the pedagogical process. The paper concludes that the social-emotional skills employers value can be effectively taught when aligned with the optimal stage for each skill development, middle childhood is the optimal stage for development of PRACTICE skills, and a broad international evidence base on effective program interventions at the right stage can guide policy makers to incorporate social-emotional learning into their school curriculum.Publication Exports, University-Industry Linkages, and Innovation Challenges in Bangalore, India(World Bank, Washington, DC, 2006-04)The success of the Indian software industry is now internationally recognized. Consequently, scholars, policymakers, and industry officials everywhere generally anticipate the increasing competitiveness of India in high technology activities. Using a structural framework, the author argues that Bangalore's (and India's) information technology (IT) industry is predicated on an Indian business model which does not encourage thick institutional linkages such as those encapsulated by the triple helix model. Under this institutional arrangement there is cross-fertilization of new ideas and new modes of institutional interaction between industry, academia, and government. Though there are several hundred IT businesses in a milieu of numerous engineering and science colleges and high-end public sector research institutes, the supposed thick institutional architecture is in reality quite thin. This is due to a particular type of an export-oriented model which is based on off-shore development of software services, targeted mainly to the United States. Neither domestic market nor non-U.S. markets such as East Asia are pursued aggressively by Indian firms, which offer alternative forms of learning. Consequently, Bangalore's dynamism in the IT industry stems from linear and extensive growth rather than nonlinear and intensive growth. The author argues that Bangalore has serious innovation challenges with weak university-industry linkages, lack of inter-firm collaboration, and the absence of cross-fertilization between the knowledge-intensive defense/public sector and the commercial IT industry. To strengthen Bangalore's and India's innovation system, the Indian business model must be reformed by diversifying geographical and product markets, stemming international and internal brain drain, and contributing to urban infrastructure.
Users also downloaded
Showing related downloaded files
Publication Beyond Aggregates(Washington, DC: World Bank, 2025-05-21)This paper develops a bottom-up, sector-specific approach to modeling potential output that overcomes limitations of traditional top-down estimates for long-term projections and policy analysis. The model disaggregates total-factor productivity (TFP) growth into within-sector productivity effects and between-sector reallocations. Such endogenous between effects capture structural transformation, notably the shift from low-productivity sectors like agriculture to higher-productivity industrial and service sectors—a key driver of growth in developing countries. At the heart of the framework, wedges in sectoral factor prices, substitution elasticities, and productivity differentials describe the contribution of between-effects to aggregate productivity. Although the approach here can be applied to any macro-structural model, its benefits are illustrated by introducing it into the World Bank’s semi-structural models for Ghana and the Kyrgyz Republic to showcase its potential to enhance the analysis of long-run growth dynamics through structural change.Publication Letter of Authorization and Acknowledgement(Washington, DC: World Bank, 2025-05-22)This Letter of Authorization provides a common template to be used with schedules that may be specified by each Member Country. This template is intended to simplify the process for authorization, reduce transaction costs, and allow flexibility for bilateral arrangements. It is intended to be used for all authorizations to be granted under Article 6 of the Paris Agreement (6.2 and 6.4). It includes an illustrative schedule of terms that is most likely to maximize investment and value for the Member Country. The Guidance document allows each Member Country to actively choose whether the project is intended to be authorized (subject to corresponding adjustments and a Letter of Authorization) or outside the scope of authorization (ideally subject to a Letter of Acknowledgement in order to increase certainty and corresponding investment value). This version incorporates changes related to decisions made at COP29.Publication Global Socio-economic Resilience to Natural Disasters(Washington, DC: World Bank, 2025-05-22)Most disaster risk assessments use damages to physical assets as their central metric, often neglecting distributional impacts and the coping and recovery capacity of affected people. To address this shortcoming, the concepts of well-being losses and socio-economic resilience—the ability to experience asset losses without a decline in well-being—have been proposed. This paper uses microsimulations to produce a global estimate of well-being losses from, and socio-economic resilience to, natural disasters, covering 132 countries. On average, each $1 in disaster-related asset losses results in well-being losses equivalent to a $2 uniform national drop in consumption, with significant variation within and across countries. The poorest income quintile within each country incurs only 9% of national asset losses but accounts for 33% of well-being losses. Compared to high-income countries, low-income countries experience 67% greater well-being losses per dollar of asset losses and require 56% more time to recover. Socio-economic resilience is uncorrelated with exposure or vulnerability to natural hazards. However, a 10 percent increase in GDP per capita is associated with a 0.9 percentage point gain in resilience, but this benefit arises indirectly—such as through higher rate of formal employment, better financial inclusion, and broader social protection coverage—rather than from higher income itself. This paper assess ten policy options and finds that socio-economic and financial interventions (such as insurance and social protection) can effectively complement asset-focused measures (e.g., construction standards) and that interventions targeting low-income populations usually have higher returns in terms of avoided well-being losses per dollar invested.Publication Tradeoffs over Rate Cycles(Washington, DC: World Bank, 2025-05-23)Central banks often face tradeoffs in how their monetary policy decisions impact economic activity (including employment), inflation and the price level. This paper assesses how these tradeoffs have evolved over time and varied across countries, with a focus on understanding the post-pandemic adjustment. To make these comparisons, we compile a cross-country, historical database of “rate cycles” (i.e., easing and tightening phases for monetary policy) for 24 advanced economies from 1970 through 2024. This allows us to quantify the characteristics of interest rate adjustments and corresponding macroeconomic outcomes and tradeoffs. We also calculate Sacrifice Ratios (output losses per inflation reduction) and document a historically low “sacrifice” during the post-pandemic tightening. This popular measure, however, ignores adjustments in the price level—which increased by more after the pandemic than over the past four decades. A series of regressions and simulations suggest monetary policy (and particularly the timing and aggressiveness of rate hikes) play a meaningful role in explaining these tradeoffs and how adjustments occur during tightening phases. Central bank credibility is the one measure we assess that corresponds to only positive outcomes and no difficult tradeoffs.Publication From Patriarchy to Policy(Washington, DC: World Bank, 2025-05-29)Legal institutions play an important role in shaping gender equality in economic domains, from inheritance to labor markets. But where do gender equal laws come from? Using cross-country data on social norms and legal equality, this paper investigates the socio-cultural roots of gender inequity in the legal system and its implications for female labor force participation. To identify the impact of social norms, the analysis uses an empirical strategy that exploits pre-modern differences in ancestral patriarchal culture as an instrument for present-day gender norms. The findings show that ancestral patriarchal culture is a strong predictor of contemporary norms, and conservative social norms are associated with more gender inequality in the de jure legal framework, the de facto implementation of laws, and the labor market. The paper presents evidence for a political selection mechanism linking norms to laws: countries with more conservative norms elect political leaders who are more hostile to gender equality, who then pass less progressive legislation. The results highlight the cultural roots and political drivers of legalized gender inequality.