Publication: Adaptive Experiments for Policy Choice: Phone Calls for Home Reading in Kenya
Loading...
Published
2022-06
ISSN
Date
2022-06-28
Author(s)
Esposito, Bruno
Editor(s)
Abstract
Adaptive sampling in experiments with multiple waves can improve learning for “policy choice problems” where the goal is to select the optimal intervention or treatment among several options. This paper uses a real-world policy choice problem to demonstrate the advantages of adaptive sampling and propose solutions to common issues in applying the method. The application is a test of six formats for automated calls to parents in Kenya that encourage reading with children at home. The adaptive ‘exploration sampling’ algorithm is used to efficiently identify the call with the highest rate of engagement. Simulations show that adaptive sampling increased the posterior probability of the chosen arm being optimal from 86 to 93 percent and more than halved the posterior expected regret. The paper discusses a range of implementation aspects, including how to decide about research design parameters such as the number of experimental waves.
Link to Data Set
Citation
“Esposito, Bruno; Sautmann, Anja. 2022. Adaptive Experiments for Policy Choice: Phone Calls for Home Reading in Kenya. Policy Research Working Paper;10098. © World Bank. http://hdl.handle.net/10986/37602 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication The Evolution of Local Participatory Democracy in Nepal(Washington, DC: World Bank, 2025-11-05)Nepal is, according to its constitution, among the world’s most decentralized countries, with a long and complex tradition of local-level public participation. This paper traces the evolution of Nepal’s modern participatory institutions, examining the extent to which they are “induced” by external interventions versus being “organically” rooted in indigenous practices. The paper identifies three broad phases: an initial focus on participation in project implementation; a subsequent phase that expanded citizen engagement; and a third phase of citizen empowerment, culminating in the 2015 federal constitution, which granted unprecedented local autonomy. The analysis yields five key findings. First, over the past 50 years, successive reforms have progressively expanded opportunities for citizens to influence local decision-making. Second, these reforms have integrated traditional participatory mechanisms into formal institutions of local government. Third, although central-level initiatives exist, most participatory platforms continue to operate at the local level. Fourth, the federal constitution has created a new landscape of local democracy, embedding autonomy and accountability. Fifth, although they are still valued in many ethnic and territorial communities, traditional participatory practices are gradually disappearing. The paper concludes by offering policy recommendations to help donor agencies and governments strengthen Nepal’s democratic trajectory. It argues that effective interventions should build on Nepal’s deep participatory traditions while recognizing the constitutional reality of far-reaching local autonomy.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication Closing the Gender Gap in Entrepreneurship: Overcoming Challenges in Law and Practice for Female Entrepreneurs(Washington, DC: World Bank, 2026-01-07)Despite significant strides toward gender equality, women around the world continue to encounter systemic obstacles that hinder their entrepreneurial success. This paper systematically reviews the literature on the barriers female entrepreneurs face and the solutions proposed to overcome these challenges. It discusses institutional factors, financial factors, human capital factors, and social and cultural factors. The literature overview is complemented by a series of stylized facts that illustrate how overcoming some of these existing barriers is correlated with improved women’s entrepreneurship and female labor force participation, drawing on the World Bank’s Women, Business and the Law database as well as the World Bank’s Enterprise Surveys. The findings underscore the need for creating an enabling environment where women can thrive as entrepreneurs.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Developing Social-Emotional Skills for the Labor Market : The PRACTICE Model(World Bank Group, Washington, DC, 2014-11)Although there is a general agreement in the literature of the importance of social-emotional skills for labor market success, there is little consensus on the specific skills that should be acquired or how and when to teach them. The psychology, economics, policy research, and program implementation literatures all touch on these issues, but they are not sufficiently integrated to provide policy direction. The objective of this paper is to provide a coherent framework and related policies and programs that bridge the psychology, economics, and education literature, specifically that related to skills employers value, non-cognitive skills that predict positive labor market outcomes, and skills targeted by psycho-educational prevention and intervention programs. The paper uses as its base a list of social-emotional skills that employers value, classifies these into eight subgroups (summarized by PRACTICE), then uses the psychology literature -- drawing from the concepts of psycho-social and neuro-biological readiness and age-appropriate contexts -- to map the age and context in which each skill subset is developed. The paper uses examples of successful interventions to illustrate the pedagogical process. The paper concludes that the social-emotional skills employers value can be effectively taught when aligned with the optimal stage for each skill development, middle childhood is the optimal stage for development of PRACTICE skills, and a broad international evidence base on effective program interventions at the right stage can guide policy makers to incorporate social-emotional learning into their school curriculum.Publication Expanding Access to Early Childhood Development : Using Interactive Audio Instruction(World Bank Group, Washington, DC, 2015-02)The returns to investments in early childhood development (ECD) are manifold and can include improved school readiness, reduced drop-out rates, higher labor force productivity and greater social cohesion. Despite these high returns, enrollment in early childhood education is just 18 percent across Africa, with disproportionately high enrollment from children in urban areas and from wealthier families. Interactive Audio Instruction (IAI) is a distance learning technology that can deliver low-cost, culturally appropriate education via radio or mobile audio technology. It is a highly effective tool to reach children who can be hard to reach through conventional programs, including the rural poor and children with disabilities. IAI can also be an effective form of service delivery in unstable and conflict-affected regions.Publication Efficient Learning for the Poor : Insights from the Frontier of Cognitive Neuroscience(Washington, DC : World Bank, 2006)This book integrates research into applications that extend from preschool brain development to the memory of adult educators. In layman's terms, it provides explanations and answers to questions such as: Why do children have to read fast before they can understand what they read? How do health, nutrition, and stimulation influence brain development? Why should students learn basic skills in their maternal language? Is there such a thing as an untrained teacher? What signs in a classroom show whether students are getting a quality education? How must information be presented in class so that students can retain it and use it? What training techniques are most likely to help staff put their learning into use? This book is intended for use by policymakers, donor agency staff, teacher trainers, supervisors, and inspectors, as well as university professors and students.Publication Intelligence, Personality, and Creativity : Unleashing the Power of Intelligence and Personality Traits to Build a Creative and Innovative Economy(World Bank, Washington, DC, 2014-11-04)Cognitive ability, as measured by IQ and background factors such as socioeconomic status and demographics have historically been seen as the principal determinants of a student s academic success. However, a growing body of research from psychology, education, behavioral economics and neuroscience is showing that personality traits also predict academic and work performance. This change in paradigm suggests that education systems face a more complex challenge than traditionally recognized: to work not only with the different types of intelligence possessed by students but also with their different personality traits in order to produce academic success measured by cognitive and non-cognitive skills. This paper reviews the research findings from the different literatures (psychology, education, behavioral economics, and neuroscience) that relate to these questions. Several good reviews summarize the findings on aspects of these questions, but rarely address all of the questions above. In particular, those reviews do not shed light on how education can improve both cognitive and non-cognitive skills and how such skills promote creativity and labor market outcomes. The scientific literatures on human intelligence and personality are large, but our focus is on the subset of research findings that relate intelligence and personality to academic performance. Likewise, the literature on creativity, innovation, and productivity is extensive, but our focus will be on the research findings that relate academic performance to creativity and, ultimately, to productivity in the workplace.Publication Charter School Entry and School Choice(World Bank, Washington, DC, 2015-07)This paper develops and estimates an equilibrium model of charter school entry and school choice. In the model, households choose among public, private, and charter schools, and a regulator authorizes charter entry and mandates charter exit. The model is estimated for Washington, D.C. According to the estimates, charters generate net social gains by providing additional school options, and they benefit non-white, low-income, and middle-school students the most. Further, policies that raise the supply of prospective charter entrants in combination with high authorization standards enhance social welfare.
Users also downloaded
Showing related downloaded files
Publication How to Improve Education Outcomes Most Efficiently? A Comparison of 150 Interventions Using the New Learning-Adjusted Years of Schooling Metric(World Bank, Washington, DC, 2020-10)Many low- and middle-income countries lag far behind high-income countries in educational access and student learning. Limited resources mean that policymakers must make tough choices about which investments to make to improve education. Although hundreds of education interventions have been rigorously evaluated, making comparisons between the results is challenging. Some studies report changes in years of schooling; others report changes in learning. Standard deviations, the metric typically used to report learning gains, measure gains relative to a local distribution of test scores. This metric makes it hard to judge if the gain is worth the cost in absolute terms. This paper proposes using learning-adjusted years of schooling (LAYS) -- which combines access and quality and compares gains to an absolute, cross-country standard -- as a new metric for reporting gains from education interventions. The paper applies LAYS to compare the effectiveness (and cost-effectiveness, where cost is available) of interventions from 150 impact evaluations across 46 countries. The results show that some of the most cost-effective programs deliver the equivalent of three additional years of high-quality schooling (that is, schooling at quality comparable to the highest-performing education systems) for just $100 per child -- compared with zero years for other classes of interventions.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Social Exclusion(World Bank, Washington, DC, 2022-06)There are multiple estimates of global monetary and multidimensional poverty, but there are still no estimates of populations at risk of social exclusion worldwide. This paper fills this gap by estimating the share and number of populations at risk of exclusion globally and regionally. It develops a conceptual framework of social exclusion that builds on Sen’s capability approach and emphasizes the relative, multidimensional, and dynamic features of exclusion. The paper also develops a macro counting measure of population groups that are particularly vulnerable to exclusion based on identity, circumstances, and socioeconomic conditions. The empirical strategy surveys the most reliable sources of vulnerable populations across countries and develops a protocol to avoid double-counting of individuals at risk of social exclusion. Overall, between 2.33 billion and 2.43 billion people—roughly 32 percent of the global population—are estimated to be at risk of being socially excluded. The South Asia and East Asia and Pacific regions contain 1.3 billion such people, with India and China alone home to 840 million of them. Meanwhile, 52 percent of Sub-Saharan Africa’s population is vulnerable to exclusion, the greatest share of any region. The paper also discusses several implications of these estimates, emphasizing that policies targeting the poor might not be sufficient to tackle social exclusion.Publication World Bank Annual Report 2024(Washington, DC: World Bank, 2024-10-25)This annual report, which covers the period from July 1, 2023, to June 30, 2024, has been prepared by the Executive Directors of both the International Bank for Reconstruction and Development (IBRD) and the International Development Association (IDA)—collectively known as the World Bank—in accordance with the respective bylaws of the two institutions. Ajay Banga, President of the World Bank Group and Chairman of the Board of Executive Directors, has submitted this report, together with the accompanying administrative budgets and audited financial statements, to the Board of Governors.Publication Choosing Our Future(Washington, DC: World Bank, 2024-09-04)Education can propel faster and better climate action in two crucial ways. First, education can galvanize behavior change at scale - not just for tomorrow, but also for today. Second, education can unlock skills and innovation to shift economies onto greener trajectories for growth. At the same time, education needs to be protected from climate change. Extreme climate events and temperatures are already eroding hard-won progress on schooling and learning. Climate change is causing school closures, learning losses, and dropouts. These will turn into long-run inter-generational earnings losses putting into jeopardy education’s powerful potential for spurring poverty alleviation and economic growth. Governments can act now to adapt schools for climate change in cost-effective ways. This report outlines new data, evidence, and examples on how countries can harness education to propel climate action. It provides an actionable policy agenda to meet development, education, and climate goals together, recognizing that tackling climate change requires changes to individual beliefs, behaviors, and skills – changes that education is uniquely positioned to catalyze.