Publication: Local Knowledge, Formal Evidence, and Policy Decisions
Loading...
Published
2024-12-10
ISSN
Date
2024-12-10
Author(s)
Editor(s)
Abstract
How do policymakers value advice from local experts versus formal evidence from impact evaluations when making policy decisions Using a discrete choice experiment conducted in collaboration with the World Bank and Inter-American Development Bank, we show that policymakers were willing to accept a program that had a 5.0 percentage point smaller estimated effect on enrollment rates if it were recommended by a local expert. They also preferred programs supported by evidence from a different region over programs supported by local evaluations only if the former had a 5.8 percentage point higher estimated impact. These premiums are large, surpassing the effects of many programs aimed at improving enrollment rates. This highlights the substantial weight that policymakers place on local evidence.
Link to Data Set
Citation
“Vivalt, Eva; Coville, Aidan; KC, Sampada. 2024. Local Knowledge, Formal Evidence, and Policy Decisions. © World Bank. http://hdl.handle.net/10986/42523 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Labor Demand in the Age of Generative AI: Early Evidence from the U.S. Job Posting Data(Washington, DC: World Bank, 2025-11-18)This paper examines the causal impact of generative artificial intelligence on U.S. labor demand using online job posting data. Exploiting ChatGPT’s release in November 2022 as an exogenous shock, the paper applies difference-in-differences and event study designs to estimate the job displacement effects of generative artificial intelligence. The identification strategy compares labor demand for occupations with high versus low artificial intelligence substitution vulnerability following ChatGPT’s launch, conditioning on similar generative artificial intelligence exposure levels to isolate substitution effects from complementary uses. The analysis uses 285 million job postings collected by Lightcast from the first quarter of 2018 to the second quarter of 2025Q2. The findings show that the number of postings for occupations with above-median artificial intelligence substitution scores fell by an average of 12 percent relative to those with below-median scores. The effect increased from 6 percent in the first year after the launch to 18 percent by the third year. Losses were particularly acute for entry-level positions that require neither advanced degrees (18 percent) nor extensive experience (20 percent), as well as those in administrative support (40 percent) and professional services (30 percent). Although generative artificial intelligence generates new occupations and enhances productivity, which may increase labor demand, early evidence suggests that some occupations may be less likely to be complemented by generative artificial intelligence than others.Publication The Lasting Effects of Working while in School(Washington, DC: World Bank, 2025-08-18)This paper provides the first experimental evidence on the long-term effects of work-study programs, leveraging a randomized lottery design from a national program in Uruguay. Participation leads to a persistent 11 percent increase in formal labor earnings, observable seven years after the program. Effects are stronger for youth who participate during pivotal educational transitions and are larger for vulnerable youth and men, while remaining positive for women and non-vulnerable youth. The program is highly cost-effective, with average impacts exceeding those of job training programs and comparable to early childhood investments.Publication It’s Not (Just) the Tariffs: Rethinking Non-Tariff Measures in a Fragmented Global Economy(Washington, DC: World Bank, 2025-10-22)As tariffs have declined, non-tariff measures (NTMs) have become central to trade policy, especially in high-income countries and regulated sectors like food and green technologies. Although NTMs may serve legitimate goals, they could also sort countries and firms into or out of markets based on compliance capacity and differences in product mix. Documenting recent advances in the estimation of ad valorem equivalents (AVEs), this paper uncovers new patterns of use and exposure of NTMs. High-income countries rely more heavily on NTMs relative to tariffs, while low- and middle-income countries face steeper AVEs on their exports. Firm-level evidence shows that NTMs disproportionately affect smaller firms, leading to market exit and concentration. Poorly designed NTMs can harm productivity and welfare, while coordinated, capacity-aware use can deliver inclusive outcomes. Policy design, transparency, and diagnostics must evolve to reflect the growing role—and risks—of NTMs in a fragmented global trade landscape.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication External Validity and Policy Adaptation(Published by Oxford University Press on behalf of the World Bank, 2020-07)With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy’s design and implementation? This paper suggests that failures of external validity (both in transporting and scaling up policy) can be understood as arising from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. The paper surveys existing approaches to analyzing external validity, and suggests that there has been more focus on the generalizability of impact evaluation results than on the applicability of evidence to specific contexts. To help fill this gap, the study develops a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength of evidence on the policy from other contexts and the policy maker’s information about the local context.Publication Nigeria Poverty : Environmental Linkages in the Natural Resource Sector - Empirical Evidence from Nigerian Case Studies with Policy Implications and Recommendations(Washington, DC, 2003-06-30)This study explores the international development community's understanding of poverty and illustrates how it is related to environmental degradation. the study relies on three sources: a comprehensive general literature review, a review of past donor interventions in Nigeria, and original empirical evidence. The linkages found between poverty and environmental degradation are based on 240 household surveys analyzed by income quintiles investigated at two sites in Nigeria: the Hadejia-Nguru Wetlands in the North and the Niger Delta in the South. Lessons relating "Causality and Linkages" (Chapter 2) explore various perceptions of how poverty "causes" environmental degradation, how environmental degradation "causes" poverty, or how other ofseting or reinforcing factors may influence the linkages between poverty and environmental quality. The empirical findings from the two study sites provide the primary basis for subsequent policy recommendations. Chapter 3 outlines the site selection process, summarizes the methods used, and provides detailed descriptive statistics for the two sites. the chapter concludes with an interpretation of key findings. Chapter 4 comences with a general discussion of available policies and strategies, including issues relating to self-sufficiency, precautionary principle, and adaptive co-management.Specific recommended strategies for Nigeria are based on the empirical findings from the case studies. An agenda for incorporating these strategies within ongoing Nigerian and donor initiatives concludes this chapter.Publication Participatory Management and Local Culture : Proverbs and Paradigms(World Bank, Washington, DC, 2000-03)Whereas evaluation has often been considered an activity required by donors, but fundamentally foreign to local culture, there is however plenty which has been done recently to develop participatory, and empowering modes of program evaluation, giving local stakeholders active roles, and a say in how evaluation is performed. Furthermore, unanticipated results of participatory evaluation practices in West Africa has brought to light local attitudinal approaches to evaluation, thus creating a basis for the development of an appropriate evaluation methodology. Incredibly, one of the means for such results was the use of proverbs, which encapsulate local attitudes, and provide insight in evaluation-related issues, such as accountability, performance, and social responsibility. Such "proverbial" culture placed evaluation at its best form of collective decision-making, making the participatory approach to evaluation a leitmotif. This attitude creates the basis for helping beneficiaries develop a culturally-appropriate technology of democratic self-governance.Publication Learning from the Evidence on Forced Displacement(Washington, DC: World Bank, 2024-08-01)In recent years, forced displacement has become a phenomenon of tragic proportions. Every year, more people are forced to flee their homes to safer shelter, either within their countries’ borders or in the low- and middle-income countries where 76 percent of the forcibly displaced find refuge. A historian writing in the 22nd century may regard the first quarter of the 21st century as a pivotal period for the history of forced displacement, when the number of forcibly displaced persons more than doubled from approximately 40 million people in the early 2000s to 108.4 million people at the end of 2022. This amounts to a staggering 1 in every 74 people on earth. These figures are provoked by protracted conflicts and new conflicts, violence, persecution, or severe political and economic crises taking place in many parts of the world. Displacement is rarely a short-term predicament. Many who become displaced remain displaced for years. At the end of 2022, 67 percent of the 108.4 million people who have fled their homes endured protracted displacement. For the displaced, healthcare, education, and employment opportunities become uncertain. Most of the displaced take refuge in low- and middle-income countries. Media coverage often focuses on refugees fleeing into affluent nations, such as the influx of Syrians and Ukrainians to countries in Europe. However, nearly seven out of ten people who flee violence are internally displaced within the borders of their home countries or live as refugees in neighboring low- and middle-income countries (LMICs). A new policy resolved started taking shape at the height of the Syrian refugee crisis in 2015. Increasingly, forced displacements were viewed as a humanitarian and development challenge. Development and humanitarian practitioners recognized the protracted nature of forced displacement situations and their impact on many already struggling low- and middle-income countries. They also recognized the need for more cooperation and coordination between humanitarian and development actors in these contexts. The ratification in 2019 of the Global Compact on Refugees (GCR) was an important milestone in support of a shift to an improved forced displacement response, anchored on the principle of responsibility sharing.Publication Promoting Handwashing and Sanitation : Evidence from a Large-Scale Randomized Trial in Rural Tanzania(World Bank Group, Washington, DC, 2015-01)The association between hygiene, sanitation, and health is well documented, yet thousands of children die each year from exposure to contaminated fecal matter. At the same time, evidence on the effectiveness of at-scale behavior change interventions to improve sanitation and hygiene practices is limited. This paper presents the results of two large-scale, government-led handwashing and sanitation promotion campaigns in rural Tanzania. For the campaign, 181 wards were randomly assigned to receive sanitation promotion, handwashing promotion, both interventions together, or neither. One year after the end of the program, sanitation wards increased latrine construction rates from 38.6 to 51 percent and reduced regular open defecation from 23.1 to 11.1 percent. Households in handwashing wards show marginal improvements in handwashing behavior related to food preparation, but not at other critical junctures. Limited interaction is observed between handwashing and sanitation on intermediate outcomes: wards that received both handwashing and sanitation promotion are less likely to have feces visible around their latrine and more likely to have a handwashing station close to their latrine facility relative to individual treatment groups. Final health effects on child health measured through diarrhea, anemia, stunting, and wasting are absent in the single-intervention groups. The combined-treatment group produces statistically detectable, but biologically insignificant and inconsistent, health impacts. The results highlight the importance of focusing on intermediate outcomes of take-up and behavior change as a critical first step in large-scale programs before realizing the changes in health that sanitation and hygiene interventions aim to deliver.
Users also downloaded
Showing related downloaded files
Publication Europe and Central Asia Economic Update, Fall 2024: Better Education for Stronger Growth(Washington, DC: World Bank, 2024-10-17)Economic growth in Europe and Central Asia (ECA) is likely to moderate from 3.5 percent in 2023 to 3.3 percent this year. This is significantly weaker than the 4.1 percent average growth in 2000-19. Growth this year is driven by expansionary fiscal policies and strong private consumption. External demand is less favorable because of weak economic expansion in major trading partners, like the European Union. Growth is likely to slow further in 2025, mostly because of the easing of expansion in the Russian Federation and Turkiye. This Europe and Central Asia Economic Update calls for a major overhaul of education systems across the region, particularly higher education, to unleash the talent needed to reinvigorate growth and boost convergence with high-income countries. Universities in the region suffer from poor management, outdated curricula, and inadequate funding and infrastructure. A mismatch between graduates' skills and the skills employers are seeking leads to wasted potential and contributes to the region's brain drain. Reversing the decline in the quality of education will require prioritizing improvements in teacher training, updated curricula, and investment in educational infrastructure. In higher education, reforms are needed to consolidate university systems, integrate them with research centers, and provide reskilling opportunities for adult workers.Publication Global Economic Prospects, June 2023(Washington, DC: World Bank, 2023-06-06)Global growth is projected to slow significantly in the second half of this year, with weakness continuing in 2024. Inflation pressures persist, and tight monetary policy is expected to weigh substantially on activity. The possibility of more widespread bank turmoil and tighter monetary policy could result in even weaker global growth. Rising borrowing costs in advanced economies could lead to financial dislocations in the more vulnerable emerging market and developing economies (EMDEs). In low-income countries, in particular, fiscal positions are increasingly precarious. Comprehensive policy action is needed at the global and national levels to foster macroeconomic and financial stability. Among many EMDEs, and especially in low-income countries, bolstering fiscal sustainability will require generating higher revenues, making spending more efficient, and improving debt management practices. Continued international cooperation is also necessary to tackle climate change, support populations affected by crises and hunger, and provide debt relief where needed. In the longer term, reversing a projected decline in EMDE potential growth will require reforms to bolster physical and human capital and labor-supply growth.Publication Good Practice Note on Dam Safety(World Bank, Washington, DC, 2020-10)The objective of this good practice note (GPN) on dam safety is to provide additional guidance to World Bank staff on the application of relevant requirements under the environmental and social framework (ESF). This GPN provides guidance on using a risk management approach to the application of the dam safety requirements. The guidance contained in this note is designed to enhance the quality of practice without creating new requirements for the application of the ESF. The GPN provides guidance on compliance requirements, a risk management approach to dam safety, risk analysis tools, quality of information and capacity, application to World Bank operations, and procedural aspects. The GPN pertains to: (a) construction of new dams or dams under construction (DUC) under investment project financing (IPF); (b) rehabilitation of existing dams under IPF; and (c) existing dams or DUC that are not financed under IPF, on which the project relies or may rely.Publication MIGA Annual Report 2021(Washington, DC: Multilateral Investment Guarantee Agency, 2021-10-01)In FY21, MIGA issued 5.2 billion US Dollars in new guarantees across 40 projects. These projects are expected to provide 784,000 people with new or improved electricity service, create over 14,000 jobs, generate over 362 million US Dollars in taxes for the host countries, and enable about 1.3 billion US Dollars in loans to businesses—critical as countries around the world work to keep their economies afloat. Of the 40 projects supported during FY21, 85 percent addressed at least one of the strategic priority areas, namely, IDA-eligible countries (lower-income), fragile and conflict affected situations (FCS), and climate finance. As of June 2021, MIGA has also issued 5.6 billion US Dollars of guarantees through our COVID-19 Response Program and anticipate an expansion to 10–12 billion US Dollars over the coming years, a testament to the countercyclical role that MIGA can play in mobilizing private investment in the face of the pandemic. A member of the World Bank Group, MIGA is committed to strong development impact and promoting projects that are economically, environmentally, and socially sustainable. MIGA helps investors mitigate the risks of restrictions on currency conversion and transfer, breach of contract by governments, expropriation, and war and civil disturbance, as well as offering credit enhancement on sovereign obligations.Publication Implementation Know-how Briefs to Support Countries to Prioritize, Connect and Scale for a Digital-in-Health Future(Washington, DC: World Bank, 2023-08-18)Technology and data are integral to daily life. As health systems face increasing demands to deliver new, more, better, and seamless services affordable to all people, data and technology are essential. With the potential and perils of innovations like artificial intelligence the future of health care is expected to be technology-embedded and data-linked. This shift involves expanding the focus from digitization of health data to integrating digital and health as one: Digital-in-Health. The World Bank’s report, Digital-in-Health: Unlocking the Value for Everyone, calls for a new digital-in-health approach where digital technology and data are infused into every aspect of health systems management and health service delivery for better health outcomes. The report proposes ten recommendations across three priority areas for governments to invest in: prioritize, connect and scale. The Implementation Know-How Briefs serve as practical guides for countries as they implement the ten recommendations. Every Implementation Know-How Brief provides practical information to start planning and implementing how to implement the recommendations. It also contains key terminologies for those not familiar with a particular topic, provides key questions to ask, and a general orientation as to typical issues in these sectors. Topics covered are: 1.) Digital health assessments; 2.) Telemedicine and virtual health care; 3.) Private sector involvement in digital health; 4.) Interoperability in health sector; 5.) Data governance for health data; 6.) Cybersecurity for health sector; 7.) Digital health records; 8.) Determining value of digital technology in health; 9.) Certification and regulatory sandboxes for digital technologies in health; 10.) Workflow mapping for digital technology (re)design in health systems.