Publication: From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria
Loading...
Date
2025-05-20
ISSN
Published
2025-05-20
Editor(s)
Abstract
This study evaluates the impact of a program leveraging large language models for virtual tutoring in secondary education in Nigeria. Using a randomized controlled trial, the program deployed Microsoft Copilot (powered by GPT-4) to support first-year senior secondary students in English language learning over six weeks. The intervention demonstrated a significant improvement of 0.31 standard deviation on an assessment that included English topics aligned with the Nigerian curriculum, knowledge of artificial intelligence and digital skills. The effect on English, the main outcome of interest, was of 0.23 standard deviations. Cost-effectiveness analysis revealed substantial learning gains, equating to 1.5 to 2 years of ’business-as-usual’ schooling, situating the intervention among some of the most cost-effective programs to improve learning outcomes. An analysis of heterogeneous effects shows that while the program benefits students across the baseline ability distribution, the largest effects are for female students, and those with higher initial academic performance. The findings highlight that artificial intelligence-powered tutoring, when designed and used properly, can have transformative impacts in the education sector in low-resource settings.
Link to Data Set
Citation
“De Simone, Martin; Tiberti, Federico; Barron Rodriguez, Maria; Manolio, Federico; Mosuro, Wuraola; Dikoru, Eliot Jolomi. 2025. From Chalkboards to Chatbots: Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria. Policy Research Working Paper; 11125. © World Bank. http://hdl.handle.net/10986/43212 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Geopolitics and the World Trading System(Washington, DC: World Bank, 2024-12-23)Until the beginning of this century, the GATT/WTO system worked. Economic research provided a compelling explanation. It showed that if governments maximize the well-being of their own countries broadly defined, GATT/WTO principles would facilitate mutually beneficial cooperation over their trade policy choices. Now heightened geopolitical rivalry seems to have undermined the WTO. A simple transposition of the previous rationalization suggests that geopolitics and trade cooperation are not compatible. The paper shows that this is only true if rivalry eclipses any consideration of own-country well-being. In all other circumstances, there are gains from trade cooperation even with geopolitics. Furthermore, the WTO’s relevance is in question only if it adheres too rigidly to its existing rules and norms. Through measured adaptation to the geopolitical imperative, the WTO can continue to thrive as a forum for multilateral trade cooperation in the age of geopolitics.Publication Chinese Imports and Industrialization in Africa(Washington, DC: World Bank, 2025-05-12)The rise of China in the global economy has been linked with negative impacts on employment across many high- and middle-income countries. However, evidence for African countries is limited. This paper investigates the causal relationship between Chinese imports and manufacturing employment in Ethiopia. Imports may harm domestic firms through a revenue effect (lower market shares) or benefit them, indirectly if competition spurs innovation or directly through access to better quality or cheaper inputs. The analysis shows that a one unit increase in import penetration leads to a 15.2 percent increase in industry employment. The inputs effect is disentangled from the other two effects by decomposing total Chinese imports by their end-use category using input-output tables. The evidence shows that imported intermediate inputs are driving the employment gains. The findings are consistent with the idea that employment gains are a result of productivity gains and increases in capacity utilization. These employment gains appear to benefit large firms and labor-intensive industries disproportionately.Publication VAT Exemptions, Embedded Tax, and Unintended Consequences(Washington, DC: World Bank, 2025-05-15)The value-added tax (VAT) has proved to be a highly effective tool at raising revenue in developed and developing countries alike. However, the effective operation of the VAT breaks down in the presence of exemptions. Unlike zero rates, exemptions deny input tax credits, thereby increasing production costs and resulting in VAT being embedded within the prices of goods and services. This paper develops a VAT model based on input-output table and household budget survey data for 29 European countries to examine the effects of VAT exemptions on final prices and to assess the merits of their use. Simulation results show that exemptions suffer from the same targeting problems as reduced VAT rates, but, in addition, they are non-transparent and have unpredictable and counterproductive indirect effects. These effects are in addition to the well-known distortionary impact of exemptions on production decisions, and their creation of incentives to self-supply. The paper concludes that the use of exemptions should be limited to addressing pragmatic concerns, such as the disproportionate compliance costs of small businesses and the practical difficulty in taxing margin-based financial services.Publication Disentangling the Key Economic Channels through Which Infrastructure Affects Jobs(Washington, DC: World Bank, 2025-04-03)This paper takes stock of the literature on infrastructure and jobs published since the early 2000s, using a conceptual framework to identify the key channels through which different types of infrastructure impact jobs. Where relevant, it highlights the different approaches and findings in the cases of energy, digital, and transport infrastructure. Overall, the literature review provides strong evidence of infrastructure’s positive impact on employment, particularly for women. In the case of electricity, this impact arises from freeing time that would otherwise be spent on household tasks. Similarly, digital infrastructure, particularly mobile phone coverage, has demonstrated positive labor market effects, often driven by private sector investments rather than large public expenditures, which are typically required for other large-scale infrastructure projects. The evidence on structural transformation is also positive, with some notable exceptions, such as studies that find no significant impact on structural transformation in rural India in the cases of electricity and roads. Even with better market connections, remote areas may continue to lack economic opportunities, due to the absence of agglomeration economies and complementary inputs such as human capital. Accordingly, reducing transport costs alone may not be sufficient to drive economic transformation in rural areas. The spatial dimension of transformation is particularly relevant for transport, both internationally—by enhancing trade integration—and within countries, where economic development tends to drive firms and jobs toward urban centers, benefitting from economies scale and network effects. Turning to organizational transformation, evidence on skill bias in developing countries is more mixed than in developed countries and may vary considerably by context. Further research, especially on the possible reasons explaining the differences between developed and developing economies, is needed.Publication Economic Consequences of Trade and Global Value Chain Integration(World Bank, Washington, DC, 2025-04-04)This paper introduces a new approach to measuring Global Value Chains (GVC), crucial for informed policy-making. It features a tripartite classification (backward, forward, and two-sided) covering trade and production data. The findings indicate that traditional trade-based GVC metrics significantly underestimate global GVC activity, especially in sectors like services and upstream manufacturing, and overstate risks in early trade liberalization stages. Additionally, conventional backward-forward classifications over-estimate backward linkages. The paper further applies these measures empirically to assess how GVC participation mediates the impact of demand shocks on domestic output, highlighting both the exposure and stabilizing potential of GVC integration. These new measures are comprehensively available on the World Bank’s WITS Platform, providing a key resource for GVC analysis.
Journal
Journal Volume
Journal Issue
Citations
Collections
Related items
Showing items related by metadata.
Publication Efficient Learning for the Poor : Insights from the Frontier of Cognitive Neuroscience(Washington, DC : World Bank, 2006)This book integrates research into applications that extend from preschool brain development to the memory of adult educators. In layman's terms, it provides explanations and answers to questions such as: Why do children have to read fast before they can understand what they read? How do health, nutrition, and stimulation influence brain development? Why should students learn basic skills in their maternal language? Is there such a thing as an untrained teacher? What signs in a classroom show whether students are getting a quality education? How must information be presented in class so that students can retain it and use it? What training techniques are most likely to help staff put their learning into use? This book is intended for use by policymakers, donor agency staff, teacher trainers, supervisors, and inspectors, as well as university professors and students.Publication Developing Cross-Language Metrics for Reading Fluency Measurement(World Bank, Washington, DC, 2012-07-10)Since 2005, over 70 oral reading fluency tests have been given in many languages and scripts, either as part of the Early Grade Reading Assessment (EGRA) or as individual one-minute tests. Particularly in multilingual countries, reading speed and comprehension measures have been taken in multiple languages and also in multiple scripts. The development of language has a significant genetic component, which tends to create common grammatical structures. Then languages must conform to information processing limitations, notably to working memory capacity. On the basis of such features, it may be possible to develop common standards for performance improvement compare findings cross linguistically. Languages are most comparable when large chunks are used rather than single words. To arrive at some comparisons, several methods may be tried. These include: a) counting actual words in connected texts or in lists, using some conventions if needed; b) using computational solutions to arrive at coefficients of certain languages vis a vis others, such as 1 Swahili word being equivalent roughly to 1.3 English words; c) using in multiple languages lists of words of a defined length, e.g. 4 letters; d) measuring phonemes or syllables per minute, possibly dividing by average word length; and e) rapid serial visual presentation, potentially also measuring perception at the letter feature level. Overall, reading rate as words per minute seems to be a valid and reliable indicator of achievement, with 45-60 words being a range that is usable as a benchmark.Publication Vocational Education in the New EU Member States : Enhancing Labor Market Outcomes and Fiscal Efficiency(Washington, DC: World Bank, 2007)This report explores the fiscal aspects of vocational education reform in the context of secondary education as a whole and considers the implications of any changes in the vocational education (VE) system for post-secondary and other modes of skill development. The report begins by describing the inherited system of vocational education in the former socialist countries of Central and Eastern Europe which was based on the assumption that everyone had to be trained for a specific occupation before starting work and that it was the function of vocational schools to provide such training. The report explores the scope for improvements in fiscal efficiency via a number of propositions about VE in the EU8 countries today: a) It would not be possible or advisable to fund adequately a traditional VE system which would provide ready-to-work recruits with narrowly specialized skills for the economy's enterprises; b) One way to reduce costs to government would be to locate practical training entirely in-plant but this is increasingly difficult; c) EU8 employers' traditional expectations of a fully-subsidized VE system delivering ready-to-work, specifically-skilled recruits are unreasonable; d) Traditional VE was the traditional answer to the question "What to do with those who have performed less well in basic education?" but this answer no longer convinces; and e) Parents and students are showing an increasing preference for general education (GE) over VE. Each of these propositions was discussed in this report not with a view to prescribing a detailed "one-size-fits-all" strategy for all the EU8 countries, but rather to deriving some principles that continued reform of VE could take into account, to the benefit of fiscal efficiency.Publication Developing Social-Emotional Skills for the Labor Market : The PRACTICE Model(World Bank Group, Washington, DC, 2014-11)Although there is a general agreement in the literature of the importance of social-emotional skills for labor market success, there is little consensus on the specific skills that should be acquired or how and when to teach them. The psychology, economics, policy research, and program implementation literatures all touch on these issues, but they are not sufficiently integrated to provide policy direction. The objective of this paper is to provide a coherent framework and related policies and programs that bridge the psychology, economics, and education literature, specifically that related to skills employers value, non-cognitive skills that predict positive labor market outcomes, and skills targeted by psycho-educational prevention and intervention programs. The paper uses as its base a list of social-emotional skills that employers value, classifies these into eight subgroups (summarized by PRACTICE), then uses the psychology literature -- drawing from the concepts of psycho-social and neuro-biological readiness and age-appropriate contexts -- to map the age and context in which each skill subset is developed. The paper uses examples of successful interventions to illustrate the pedagogical process. The paper concludes that the social-emotional skills employers value can be effectively taught when aligned with the optimal stage for each skill development, middle childhood is the optimal stage for development of PRACTICE skills, and a broad international evidence base on effective program interventions at the right stage can guide policy makers to incorporate social-emotional learning into their school curriculum.Publication Exports, University-Industry Linkages, and Innovation Challenges in Bangalore, India(World Bank, Washington, DC, 2006-04)The success of the Indian software industry is now internationally recognized. Consequently, scholars, policymakers, and industry officials everywhere generally anticipate the increasing competitiveness of India in high technology activities. Using a structural framework, the author argues that Bangalore's (and India's) information technology (IT) industry is predicated on an Indian business model which does not encourage thick institutional linkages such as those encapsulated by the triple helix model. Under this institutional arrangement there is cross-fertilization of new ideas and new modes of institutional interaction between industry, academia, and government. Though there are several hundred IT businesses in a milieu of numerous engineering and science colleges and high-end public sector research institutes, the supposed thick institutional architecture is in reality quite thin. This is due to a particular type of an export-oriented model which is based on off-shore development of software services, targeted mainly to the United States. Neither domestic market nor non-U.S. markets such as East Asia are pursued aggressively by Indian firms, which offer alternative forms of learning. Consequently, Bangalore's dynamism in the IT industry stems from linear and extensive growth rather than nonlinear and intensive growth. The author argues that Bangalore has serious innovation challenges with weak university-industry linkages, lack of inter-firm collaboration, and the absence of cross-fertilization between the knowledge-intensive defense/public sector and the commercial IT industry. To strengthen Bangalore's and India's innovation system, the Indian business model must be reformed by diversifying geographical and product markets, stemming international and internal brain drain, and contributing to urban infrastructure.
Users also downloaded
Showing related downloaded files
No results found.