Publication: From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria
Loading...
Date
2025-05-20
ISSN
Published
2025-05-20
Editor(s)
Abstract
This study evaluates the impact of a program leveraging large language models for virtual tutoring in secondary education in Nigeria. Using a randomized controlled trial, the program deployed Microsoft Copilot (powered by GPT-4) to support first-year senior secondary students in English language learning over six weeks. The intervention demonstrated a significant improvement of 0.31 standard deviation on an assessment that included English topics aligned with the Nigerian curriculum, knowledge of artificial intelligence and digital skills. The effect on English, the main outcome of interest, was of 0.23 standard deviations. Cost-effectiveness analysis revealed substantial learning gains, equating to 1.5 to 2 years of ’business-as-usual’ schooling, situating the intervention among some of the most cost-effective programs to improve learning outcomes. An analysis of heterogeneous effects shows that while the program benefits students across the baseline ability distribution, the largest effects are for female students, and those with higher initial academic performance. The findings highlight that artificial intelligence-powered tutoring, when designed and used properly, can have transformative impacts in the education sector in low-resource settings.
Link to Data Set
Citation
“De Simone, Martin; Tiberti, Federico; Barron Rodriguez, Maria; Manolio, Federico; Mosuro, Wuraola; Dikoru, Eliot Jolomi. 2025. From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria. Policy Research Working Paper; 11125. © World Bank. http://hdl.handle.net/10986/43212 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Geopolitics and the World Trading System(Washington, DC: World Bank, 2024-12-23)Until the beginning of this century, the GATT/WTO system worked. Economic research provided a compelling explanation. It showed that if governments maximize the well-being of their own countries broadly defined, GATT/WTO principles would facilitate mutually beneficial cooperation over their trade policy choices. Now heightened geopolitical rivalry seems to have undermined the WTO. A simple transposition of the previous rationalization suggests that geopolitics and trade cooperation are not compatible. The paper shows that this is only true if rivalry eclipses any consideration of own-country well-being. In all other circumstances, there are gains from trade cooperation even with geopolitics. Furthermore, the WTO’s relevance is in question only if it adheres too rigidly to its existing rules and norms. Through measured adaptation to the geopolitical imperative, the WTO can continue to thrive as a forum for multilateral trade cooperation in the age of geopolitics.Publication From Patriarchy to Policy(Washington, DC: World Bank, 2025-05-29)Legal institutions play an important role in shaping gender equality in economic domains, from inheritance to labor markets. But where do gender equal laws come from? Using cross-country data on social norms and legal equality, this paper investigates the socio-cultural roots of gender inequity in the legal system and its implications for female labor force participation. To identify the impact of social norms, the analysis uses an empirical strategy that exploits pre-modern differences in ancestral patriarchal culture as an instrument for present-day gender norms. The findings show that ancestral patriarchal culture is a strong predictor of contemporary norms, and conservative social norms are associated with more gender inequality in the de jure legal framework, the de facto implementation of laws, and the labor market. The paper presents evidence for a political selection mechanism linking norms to laws: countries with more conservative norms elect political leaders who are more hostile to gender equality, who then pass less progressive legislation. The results highlight the cultural roots and political drivers of legalized gender inequality.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication Rethinking Fiscal Policies(Washington, DC: World Bank, 2025-05-27)This paper examines the redistributive impact of fiscal policy—specifically taxes and transfers—on poverty and inequality in eight countries in the Middle East and North Africa: the Arab Republic of Egypt, Djibouti, the Islamic Republic of Iran, Iraq, Jordan, Morocco, Tunisia, and the West Bank and Gaza. Utilizing the Commitment to Equity framework, the analysis evaluates how fiscal interventions alter income distribution across these diverse national contexts. The results indicate that direct cash transfers and social assistance programs are generally effective in reducing poverty and shielding vulnerable populations, while in-kind benefits—particularly in education and healthcare—significantly contribute to mitigating income inequality. In contrast, generalized subsidies, especially in the energy sector, are fiscally burdensome and largely regressive, offering limited equity gains. Indirect taxes, although important for revenue generation, often exacerbate income disparities. The study underscores the need for comprehensive fiscal reforms, including the expansion of well-targeted transfers, adoption of progressive taxation, and reallocation of inefficient subsidies toward investments in human capital. Successful initiatives, such as Egypt’s Takaful and Karama and Jordan’s Takaful and bread subsidy compensation programs, illustrate scalable models of effective redistribution. Moreover, the Islamic Republic of Iran’s progressive tax policies highlight viable pathways to equitable revenue mobilization. Strengthening investment in education and health is essential for promoting long-term equity, enhancing upward mobility, and supporting inclusive and sustainable development across the region.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Efficient Learning for the Poor : Insights from the Frontier of Cognitive Neuroscience(Washington, DC : World Bank, 2006)This book integrates research into applications that extend from preschool brain development to the memory of adult educators. In layman's terms, it provides explanations and answers to questions such as: Why do children have to read fast before they can understand what they read? How do health, nutrition, and stimulation influence brain development? Why should students learn basic skills in their maternal language? Is there such a thing as an untrained teacher? What signs in a classroom show whether students are getting a quality education? How must information be presented in class so that students can retain it and use it? What training techniques are most likely to help staff put their learning into use? This book is intended for use by policymakers, donor agency staff, teacher trainers, supervisors, and inspectors, as well as university professors and students.Publication Developing Cross-Language Metrics for Reading Fluency Measurement(World Bank, Washington, DC, 2012-07-10)Since 2005, over 70 oral reading fluency tests have been given in many languages and scripts, either as part of the Early Grade Reading Assessment (EGRA) or as individual one-minute tests. Particularly in multilingual countries, reading speed and comprehension measures have been taken in multiple languages and also in multiple scripts. The development of language has a significant genetic component, which tends to create common grammatical structures. Then languages must conform to information processing limitations, notably to working memory capacity. On the basis of such features, it may be possible to develop common standards for performance improvement compare findings cross linguistically. Languages are most comparable when large chunks are used rather than single words. To arrive at some comparisons, several methods may be tried. These include: a) counting actual words in connected texts or in lists, using some conventions if needed; b) using computational solutions to arrive at coefficients of certain languages vis a vis others, such as 1 Swahili word being equivalent roughly to 1.3 English words; c) using in multiple languages lists of words of a defined length, e.g. 4 letters; d) measuring phonemes or syllables per minute, possibly dividing by average word length; and e) rapid serial visual presentation, potentially also measuring perception at the letter feature level. Overall, reading rate as words per minute seems to be a valid and reliable indicator of achievement, with 45-60 words being a range that is usable as a benchmark.Publication Vocational Education in the New EU Member States : Enhancing Labor Market Outcomes and Fiscal Efficiency(Washington, DC: World Bank, 2007)This report explores the fiscal aspects of vocational education reform in the context of secondary education as a whole and considers the implications of any changes in the vocational education (VE) system for post-secondary and other modes of skill development. The report begins by describing the inherited system of vocational education in the former socialist countries of Central and Eastern Europe which was based on the assumption that everyone had to be trained for a specific occupation before starting work and that it was the function of vocational schools to provide such training. The report explores the scope for improvements in fiscal efficiency via a number of propositions about VE in the EU8 countries today: a) It would not be possible or advisable to fund adequately a traditional VE system which would provide ready-to-work recruits with narrowly specialized skills for the economy's enterprises; b) One way to reduce costs to government would be to locate practical training entirely in-plant but this is increasingly difficult; c) EU8 employers' traditional expectations of a fully-subsidized VE system delivering ready-to-work, specifically-skilled recruits are unreasonable; d) Traditional VE was the traditional answer to the question "What to do with those who have performed less well in basic education?" but this answer no longer convinces; and e) Parents and students are showing an increasing preference for general education (GE) over VE. Each of these propositions was discussed in this report not with a view to prescribing a detailed "one-size-fits-all" strategy for all the EU8 countries, but rather to deriving some principles that continued reform of VE could take into account, to the benefit of fiscal efficiency.Publication Developing Social-Emotional Skills for the Labor Market : The PRACTICE Model(World Bank Group, Washington, DC, 2014-11)Although there is a general agreement in the literature of the importance of social-emotional skills for labor market success, there is little consensus on the specific skills that should be acquired or how and when to teach them. The psychology, economics, policy research, and program implementation literatures all touch on these issues, but they are not sufficiently integrated to provide policy direction. The objective of this paper is to provide a coherent framework and related policies and programs that bridge the psychology, economics, and education literature, specifically that related to skills employers value, non-cognitive skills that predict positive labor market outcomes, and skills targeted by psycho-educational prevention and intervention programs. The paper uses as its base a list of social-emotional skills that employers value, classifies these into eight subgroups (summarized by PRACTICE), then uses the psychology literature -- drawing from the concepts of psycho-social and neuro-biological readiness and age-appropriate contexts -- to map the age and context in which each skill subset is developed. The paper uses examples of successful interventions to illustrate the pedagogical process. The paper concludes that the social-emotional skills employers value can be effectively taught when aligned with the optimal stage for each skill development, middle childhood is the optimal stage for development of PRACTICE skills, and a broad international evidence base on effective program interventions at the right stage can guide policy makers to incorporate social-emotional learning into their school curriculum.Publication Exports, University-Industry Linkages, and Innovation Challenges in Bangalore, India(World Bank, Washington, DC, 2006-04)The success of the Indian software industry is now internationally recognized. Consequently, scholars, policymakers, and industry officials everywhere generally anticipate the increasing competitiveness of India in high technology activities. Using a structural framework, the author argues that Bangalore's (and India's) information technology (IT) industry is predicated on an Indian business model which does not encourage thick institutional linkages such as those encapsulated by the triple helix model. Under this institutional arrangement there is cross-fertilization of new ideas and new modes of institutional interaction between industry, academia, and government. Though there are several hundred IT businesses in a milieu of numerous engineering and science colleges and high-end public sector research institutes, the supposed thick institutional architecture is in reality quite thin. This is due to a particular type of an export-oriented model which is based on off-shore development of software services, targeted mainly to the United States. Neither domestic market nor non-U.S. markets such as East Asia are pursued aggressively by Indian firms, which offer alternative forms of learning. Consequently, Bangalore's dynamism in the IT industry stems from linear and extensive growth rather than nonlinear and intensive growth. The author argues that Bangalore has serious innovation challenges with weak university-industry linkages, lack of inter-firm collaboration, and the absence of cross-fertilization between the knowledge-intensive defense/public sector and the commercial IT industry. To strengthen Bangalore's and India's innovation system, the Indian business model must be reformed by diversifying geographical and product markets, stemming international and internal brain drain, and contributing to urban infrastructure.
Users also downloaded
Showing related downloaded files
Publication Global Socio-economic Resilience to Natural Disasters(Washington, DC: World Bank, 2025-05-22)Most disaster risk assessments use damages to physical assets as their central metric, often neglecting distributional impacts and the coping and recovery capacity of affected people. To address this shortcoming, the concepts of well-being losses and socio-economic resilience—the ability to experience asset losses without a decline in well-being—have been proposed. This paper uses microsimulations to produce a global estimate of well-being losses from, and socio-economic resilience to, natural disasters, covering 132 countries. On average, each $1 in disaster-related asset losses results in well-being losses equivalent to a $2 uniform national drop in consumption, with significant variation within and across countries. The poorest income quintile within each country incurs only 9% of national asset losses but accounts for 33% of well-being losses. Compared to high-income countries, low-income countries experience 67% greater well-being losses per dollar of asset losses and require 56% more time to recover. Socio-economic resilience is uncorrelated with exposure or vulnerability to natural hazards. However, a 10 percent increase in GDP per capita is associated with a 0.9 percentage point gain in resilience, but this benefit arises indirectly—such as through higher rate of formal employment, better financial inclusion, and broader social protection coverage—rather than from higher income itself. This paper assess ten policy options and finds that socio-economic and financial interventions (such as insurance and social protection) can effectively complement asset-focused measures (e.g., construction standards) and that interventions targeting low-income populations usually have higher returns in terms of avoided well-being losses per dollar invested.Publication Tradeoffs over Rate Cycles(Washington, DC: World Bank, 2025-05-23)Central banks often face tradeoffs in how their monetary policy decisions impact economic activity (including employment), inflation and the price level. This paper assesses how these tradeoffs have evolved over time and varied across countries, with a focus on understanding the post-pandemic adjustment. To make these comparisons, we compile a cross-country, historical database of “rate cycles” (i.e., easing and tightening phases for monetary policy) for 24 advanced economies from 1970 through 2024. This allows us to quantify the characteristics of interest rate adjustments and corresponding macroeconomic outcomes and tradeoffs. We also calculate Sacrifice Ratios (output losses per inflation reduction) and document a historically low “sacrifice” during the post-pandemic tightening. This popular measure, however, ignores adjustments in the price level—which increased by more after the pandemic than over the past four decades. A series of regressions and simulations suggest monetary policy (and particularly the timing and aggressiveness of rate hikes) play a meaningful role in explaining these tradeoffs and how adjustments occur during tightening phases. Central bank credibility is the one measure we assess that corresponds to only positive outcomes and no difficult tradeoffs.Publication Beyond Aggregates(Washington, DC: World Bank, 2025-05-21)This paper develops a bottom-up, sector-specific approach to modeling potential output that overcomes limitations of traditional top-down estimates for long-term projections and policy analysis. The model disaggregates total-factor productivity (TFP) growth into within-sector productivity effects and between-sector reallocations. Such endogenous between effects capture structural transformation, notably the shift from low-productivity sectors like agriculture to higher-productivity industrial and service sectors—a key driver of growth in developing countries. At the heart of the framework, wedges in sectoral factor prices, substitution elasticities, and productivity differentials describe the contribution of between-effects to aggregate productivity. Although the approach here can be applied to any macro-structural model, its benefits are illustrated by introducing it into the World Bank’s semi-structural models for Ghana and the Kyrgyz Republic to showcase its potential to enhance the analysis of long-run growth dynamics through structural change.Publication From Patriarchy to Policy(Washington, DC: World Bank, 2025-05-29)Legal institutions play an important role in shaping gender equality in economic domains, from inheritance to labor markets. But where do gender equal laws come from? Using cross-country data on social norms and legal equality, this paper investigates the socio-cultural roots of gender inequity in the legal system and its implications for female labor force participation. To identify the impact of social norms, the analysis uses an empirical strategy that exploits pre-modern differences in ancestral patriarchal culture as an instrument for present-day gender norms. The findings show that ancestral patriarchal culture is a strong predictor of contemporary norms, and conservative social norms are associated with more gender inequality in the de jure legal framework, the de facto implementation of laws, and the labor market. The paper presents evidence for a political selection mechanism linking norms to laws: countries with more conservative norms elect political leaders who are more hostile to gender equality, who then pass less progressive legislation. The results highlight the cultural roots and political drivers of legalized gender inequality.Publication Rethinking Fiscal Policies(Washington, DC: World Bank, 2025-05-27)This paper examines the redistributive impact of fiscal policy—specifically taxes and transfers—on poverty and inequality in eight countries in the Middle East and North Africa: the Arab Republic of Egypt, Djibouti, the Islamic Republic of Iran, Iraq, Jordan, Morocco, Tunisia, and the West Bank and Gaza. Utilizing the Commitment to Equity framework, the analysis evaluates how fiscal interventions alter income distribution across these diverse national contexts. The results indicate that direct cash transfers and social assistance programs are generally effective in reducing poverty and shielding vulnerable populations, while in-kind benefits—particularly in education and healthcare—significantly contribute to mitigating income inequality. In contrast, generalized subsidies, especially in the energy sector, are fiscally burdensome and largely regressive, offering limited equity gains. Indirect taxes, although important for revenue generation, often exacerbate income disparities. The study underscores the need for comprehensive fiscal reforms, including the expansion of well-targeted transfers, adoption of progressive taxation, and reallocation of inefficient subsidies toward investments in human capital. Successful initiatives, such as Egypt’s Takaful and Karama and Jordan’s Takaful and bread subsidy compensation programs, illustrate scalable models of effective redistribution. Moreover, the Islamic Republic of Iran’s progressive tax policies highlight viable pathways to equitable revenue mobilization. Strengthening investment in education and health is essential for promoting long-term equity, enhancing upward mobility, and supporting inclusive and sustainable development across the region.