Publication: Bayesian Impact Evaluation with Informative Priors: An Application to a Colombian Management and Export Improvement Program
Loading...
Date
2023-01
ISSN
Published
2023-01
Author(s)
Editor(s)
Abstract
Policymakers often test expensive new programs on relatively small samples. Formally incorporating informative Bayesian priors into impact evaluation offers the promise to learn more from these experiments. A Colombian government program which aimed to increase exporting was trialed experimentally on 200 firms with this goal in mind. Priors were elicited from academics, policymakers, and firms. Contrary to these priors, frequentist estimation can not reject 0 effects in 2019, and finds some negative impacts in 2020. For binary outcomes like whether firms export, frequentist estimates are relatively precise, and Bayesian credible posterior intervals update to overlap almost completely with standard confidence intervals. For outcomes like increasing export variety, where the priors align with the data, the value of these priors is seen in posterior intervals that are considerably narrower than frequentist confidence intervals. Finally, for noisy outcomes like export value, posterior intervals show almost no updating from the priors, highlighting how uninformative the data are about such outcomes.
Link to Data Set
Citation
“Iacovone, Leonardo; McKenzie, David; Meager, Rachael. 2023. Bayesian Impact Evaluation with Informative Priors: An Application to a Colombian Management and Export Improvement Program. Policy Research Working Paper; 10274. © World Bank. http://hdl.handle.net/10986/38493 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Future of Poverty(Washington, DC: World Bank, 2025-07-15)Climate change is increasingly acknowledged as a critical issue with far-reaching socioeconomic implications that extend well beyond environmental concerns. Among the most pressing challenges is its impact on global poverty. This paper projects the potential impacts of unmitigated climate change on global poverty rates between 2023 and 2050. Building on a study that provided a detailed analysis of how temperature changes affect economic productivity, this paper integrates those findings with binned data from 217 countries, sourced from the World Bank’s Poverty and Inequality Platform. By simulating poverty rates and the number of poor under two climate change scenarios, the paper uncovers some alarming trends. One of the primary findings is that the number of people living in extreme poverty worldwide could be nearly doubled due to climate change. In all scenarios, Sub-Saharan Africa is projected to bear the brunt, contributing the largest number of poor people, with estimates ranging between 40.5 million and 73.5 million by 2050. Another significant finding is the disproportionate impact of inequality on poverty. Even small increases in inequality can lead to substantial rises in poverty levels. For instance, if every country’s Gini coefficient increases by just 1 percent between 2022 and 2050, an additional 8.8 million people could be pushed below the international poverty line by 2050. In a more extreme scenario, where every country’s Gini coefficient increases by 10 percent between 2022 and 2050, the number of people falling into poverty could rise by an additional 148.8 million relative to the baseline scenario. These findings underscore the urgent need for comprehensive climate policies that not only mitigate environmental impacts but also address socioeconomic vulnerabilities.Publication Exports, Labor Markets, and the Environment(Washington, DC: World Bank, 2025-07-14)What is the environmental impact of exports? Focusing on 2000–20, this paper combines customs, administrative, and census microdata to estimate employment elasticities with respect to exports. The findings show that municipalities that faced increased exports experienced faster growth in formal employment. The elasticities were 0.25 on impact, peaked at 0.4, and remained positive and significant even 10 years after the shock, pointing to a long and protracted labor market adjustment. In the long run, informal employment responds negatively to export shocks. Using a granular taxonomy for economic activities based on their environmental impact, the paper documents that environmentally risky activities have a larger share of employment than environmentally sustainable ones, and that the relationship between these activities and exports is nuanced. Over the short run, environmentally risky employment responds more strongly to exports relative to environmentally sustainable employment. However, over the long run, this pattern reverses, as the impact of exports on environmentally sustainable employment is more persistent.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication The Asymmetric Bank Distress Amplifier of Recessions(Washington, DC: World Bank, 2025-07-11)One defining feature of financial crises, evident in U.S. and international data, is asymmetric bank distress—concentrated losses on a subset of banks. This paper proposes a model in which shocks to borrowers’ productivity dispersion lead to asymmetric bank losses. The framework exhibits a “bank distress amplifier,” exacerbating economic downturns by causing costly bank failures and raising uncertainty about the solvency of banks, thereby pushing banks to deleverage. Quantitative analysis shows that the bank distress amplifier doubles investment decline and increases the spread by 2.5 times during the Great Recession compared to a standard financial accelerator model. The mechanism helps explain how a seemingly small shock can sometimes trigger a large crisis.Publication Impact of Heat Waves on Learning Outcomes and the Role of Conditional Cash Transfers(Washington, DC: World Bank, 2025-07-14)This paper evaluates the impact of higher temperatures on learning outcomes in Peru. The results suggest that 1 degree above 20°C is equivalent to 7 and 6 percent of a standard deviation of what a student learns in a year for math and reading tests, respectively. These results hold true when the main specification is changed, splitting the sample, collapsing the data at school level, and using other climate specifications. The paper aims to improve understanding of how to deal with the impacts of climate change on learning outcomes in developing countries. The evidence suggests that conditional cash transfer programs can mitigate the negative effects of higher temperatures on students’ learning outcomes in math and reading.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Mixtures of g-priors for Bayesian Model Averaging with Economic Application(2011-07-01)This paper examines the issue of variable selection in linear regression modeling, where there is a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset. In this context, Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. The main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. The authors combine a Binomial-Beta prior on model size with a g-prior on the coefficients of each model. In addition, they assign a hyperprior to g, as the choice of g has been found to have a large impact on the results. For the prior on g, they examine the Zellner-Siow prior and a class of Beta shrinkage priors, which covers most choices in the recent literature. The authors propose a benchmark Beta prior, inspired by earlier findings with fixed g, and show it leads to consistent model selection. Inference is conducted through a Markov chain Monte Carlo sampler over model space and g. The authors examine the performance of the various priors in the context of simulated and real data. For the latter, they consider two important applications in economics, namely cross-country growth regression and returns to schooling. Recommendations for applied users are provided.Publication Shortening Supply Chains(World Bank, Washington, DC, 2019-08)Small trading activities are a prevalent form of self-employment in developing countries, but their integration into supply value chains is not efficient, especially when it comes to perishable produce. This study tests a novel approach to improve their efficiency by reducing the time and cost of sourcing produce by aggregating purchases through the use of an app and centralized distribution system. Fruit and vegetable vendors in Bogotá currently travel most days to a central market to purchase produce, incurring substantial time and monetary costs. A social enterprise attempted to shorten the supply chain between farmers and vendors by aggregating orders from many small stores, sourcing directly from farmers, and delivering them to the stores. The introduction of this new service was randomized at the market block level. Initial interest was high and offering the service reduced travel time for users by almost two hours a week, reduced travel costs, and increased work-life balance for store owners. Firms offered the service saved an average of 6 to 8 percent on purchase costs, and although some of this passed through into lower prices for consumers, there was incomplete pass-through, so that markups rose. However, stores reduced their sales of products that were not originally offered by this new service, and their total sales and profits appear to have fallen in the short run, with service usage falling over time. The results highlight the potential for new technologies to solve firm coordination problems, offer a window into the nature of competition among small retailers, and point to the challenges in achieving economies of scale when disrupting centralized markets for multi-product firms.Publication Beyond Baseline and Follow-up : The Case for More T in Experiments(2011-04-01)The vast majority of randomized experiments in economics rely on a single baseline and single follow-up survey. If multiple follow-ups are conducted, the reason is typically to examine the trajectory of impact effects, so that in effect only one follow-up round is being used to estimate each treatment effect of interest. While such a design is suitable for study of highly autocorrelated and relatively precisely measured outcomes in the health and education domains, this paper makes the case that it is unlikely to be optimal for measuring noisy and relatively less autocorrelated outcomes such as business profits, household incomes and expenditures, and episodic health outcomes. Taking multiple measurements of such outcomes at relatively short intervals allows the researcher to average out noise, increasing power. When the outcomes have low autocorrelation, it can make sense to do no baseline at all. Moreover, the author shows how for such outcomes, more power can be achieved with multiple follow-ups than allocating the same total sample size over a single follow-up and baseline. The analysis highlights the large gains in power from ANCOVA rather than difference-in-differences when autocorrelations are low and a baseline is taken. The paper discusses the issues involved in multiple measurements, and makes recommendations for the design of experiments and related non-experimental impact evaluations.Publication Testing the Importance of Search Frictions, Matching, and Reservation Prestige through Randomized Experiments in Jordan(World Bank Group, Washington, DC, 2014-09-01)Unemployment rates for tertiary-educated youth in Jordan are high, as is the duration of unemployment. Two randomized experiments in Jordan were used to test different theories that may explain this phenomenon. The first experiment tested the role of search and matching frictions by providing firms and job candidates with an intensive screening and matching service based on educational backgrounds and psychometric assessments. Although more than 1,000 matches were made, youth rejected the opportunity to even have an interview in 28 percent of cases, and when a job offer was received, they rejected this offer or quickly quit the job 83 percent of the time. A second experiment built on the first by examining the willingness of educated, unemployed youth to apply for jobs of varying levels of prestige. Youth applied to only a small proportion of the job openings they were told about, with application rates higher for higher prestige jobs than lower prestige jobs. Youth failed to show up for the majority of interviews scheduled for low prestige jobs. The results suggest that reservation prestige is an important factor underlying the unemployment of educated Jordanian youth.Publication Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment in Jordan(World Bank, Washington, DC, 2012-07)Throughout the Middle East, unemployment rates of educated youth have been persistently high and female labor force participation, low. This paper studies the impact of a randomized experiment in Jordan designed to assist female community college graduates find employment. One randomly chosen group of graduates was given a voucher that would pay an employer a subsidy equivalent to the minimum wage for up to 6 months if they hired the graduate; a second group was invited to attend 45 hours of employability skills training designed to provide them with the soft skills employers say graduates often lack; a third group was offered both interventions; and the fourth group forms the control group. The analysis finds that the job voucher led to a 40 percentage point increase in employment in the short-run, but that most of this employment is not formal, and that the average effect is much smaller and no longer statistically significant 4 months after the voucher period has ended. The voucher does appear to have persistent impacts outside the capital, where it almost doubles the employment rate of graduates, but this appears likely to largely reflect displacement effects. Soft-skills training has no average impact on employment, although again there is a weakly significant impact outside the capital. The authors elicit the expectations of academics and development professionals to demonstrate that these findings are novel and unexpected. The results suggest that wage subsidies can help increase employment in the short term, but are not a panacea for the problems of high urban female youth unemployment.
Users also downloaded
Showing related downloaded files
Publication The Journey Ahead(Washington, DC: World Bank, 2024-10-31)The Journey Ahead: Supporting Successful Migration in Europe and Central Asia provides an in-depth analysis of international migration in Europe and Central Asia (ECA) and the implications for policy making. By identifying challenges and opportunities associated with migration in the region, it aims to inform a more nuanced, evidencebased debate on the costs and benefits of cross-border mobility. Using data-driven insights and new analysis, the report shows that migration has been an engine of prosperity and has helped address some of ECA’s demographic and socioeconomic disparities. Yet, migration’s full economic potential remains untapped. The report identifies multiple barriers keeping migration from achieving its full potential. Crucially, it argues that policies in both origin and destination countries can help maximize the development impacts of migration and effectively manage the economic, social, and political costs. Drawing from a wide range of literature, country experiences, and novel analysis, The Journey Ahead presents actionable policy options to enhance the benefits of migration for destination and origin countries and migrants themselves. Some measures can be taken unilaterally by countries, whereas others require close bilateral or regional coordination. The recommendations are tailored to different types of migration— forced displacement as well as high-skilled and low-skilled economic migration—and from the perspectives of both sending and receiving countries. This report serves as a comprehensive resource for governments, development partners, and other stakeholders throughout Europe and Central Asia, where the richness and diversity of migration experiences provide valuable insights for policy makers in other regions of the world.Publication Fixing the Foundation(Washington, DC: World Bank, 2023-09-20)Countries in middle-income East Asia and the Pacific were already experiencing serious learning deficits prior to the COVID-19 pandemic. COVID-related school disruptions have only made things worse. Learning poverty -- defined as the percentage of 10-year-olds who cannot read and understand an age-appropriate text -- is as high as 90 percent in several countries. Several large Southeast Asian countries consistently perform well below expectations on adolescent learning assessments. This report examines key factors affecting student learning in the region, with emphasis on the central role of teachers and teaching quality. It also analyzes the role education technologies, which came into widespread use during the pandemic, and examines the political economy of education reform. The report presents recommendations on how countries can strengthen teaching to improve learning and, in doing so, can enhance productivity, growth, and future development in the region.Publication Services Unbound(Washington, DC: World Bank, 2024-12-09)Services are a new force for innovation, trade, and growth in East Asia and Pacific. The dramatic diffusion of digital technologies and partial policy reforms in services--from finance, communication, and transport to retail, health, and education--is transforming these economies. The result is higher productivity and changing jobs in the services sector, as well as in the manufacturing sectors that use these services. A region that has thrived through openness to trade and investment in manufacturing still maintains innovation-inhibiting barriers to entry and competition in key services sectors. 'Services Unbound: Digital Technologies and Policy Reform in East Asia and Pacific' makes the case for deeper domestic reforms and greater international cooperation to unleash a virtuous cycle of increased economic opportunity and enhanced human capacity that would power development in the region.Publication Business Ready 2024(Washington, DC: World Bank, 2024-10-03)Business Ready (B-READY) is a new World Bank Group corporate flagship report that evaluates the business and investment climate worldwide. It replaces and improves upon the Doing Business project. B-READY provides a comprehensive data set and description of the factors that strengthen the private sector, not only by advancing the interests of individual firms but also by elevating the interests of workers, consumers, potential new enterprises, and the natural environment. This 2024 report introduces a new analytical framework that benchmarks economies based on three pillars: Regulatory Framework, Public Services, and Operational Efficiency. The analysis centers on 10 topics essential for private sector development that correspond to various stages of the life cycle of a firm. The report also offers insights into three cross-cutting themes that are relevant for modern economies: digital adoption, environmental sustainability, and gender. B-READY draws on a robust data collection process that includes specially tailored expert questionnaires and firm-level surveys. The 2024 report, which covers 50 economies, serves as the first in a series that will expand in geographical coverage and refine its methodology over time, supporting reform advocacy, policy guidance, and further analysis and research.Publication Global Economic Prospects, June 2024(Washington, DC: World Bank, 2024-06-11)After several years of negative shocks, global growth is expected to hold steady in 2024 and then edge up in the next couple of years, in part aided by cautious monetary policy easing as inflation gradually declines. However, economic prospects are envisaged to remain tepid, especially in the most vulnerable countries. Risks to the outlook, while more balanced, are still tilted to the downside, including the possibility of escalating geopolitical tensions, further trade fragmentation, and higher-for-longer interest rates. Natural disasters related to climate change could also hinder activity. Subdued growth prospects across many emerging market and developing economies and continued risks underscore the need for decisive policy action at the global and national levels. Global Economic Prospects is a World Bank Group Flagship Report that examines global economic developments and prospects, with a special focus on emerging market and developing economies, on a semiannual basis (in January and June). Each edition includes analytical pieces on topical policy challenges faced by these economies.