Publication: Bayesian Impact Evaluation with Informative Priors: An Application to a Colombian Management and Export Improvement Program
Loading...
Published
2023-01
ISSN
Date
2023-01-18
Author(s)
Editor(s)
Abstract
Policymakers often test expensive new programs on relatively small samples. Formally incorporating informative Bayesian priors into impact evaluation offers the promise to learn more from these experiments. A Colombian government program which aimed to increase exporting was trialed experimentally on 200 firms with this goal in mind. Priors were elicited from academics, policymakers, and firms. Contrary to these priors, frequentist estimation can not reject 0 effects in 2019, and finds some negative impacts in 2020. For binary outcomes like whether firms export, frequentist estimates are relatively precise, and Bayesian credible posterior intervals update to overlap almost completely with standard confidence intervals. For outcomes like increasing export variety, where the priors align with the data, the value of these priors is seen in posterior intervals that are considerably narrower than frequentist confidence intervals. Finally, for noisy outcomes like export value, posterior intervals show almost no updating from the priors, highlighting how uninformative the data are about such outcomes.
Link to Data Set
Citation
“Iacovone, Leonardo; McKenzie, David; Meager, Rachael. 2023. Bayesian Impact Evaluation with Informative Priors: An Application to a Colombian Management and Export Improvement Program. Policy Research Working Paper; 10274. © World Bank. http://hdl.handle.net/10986/38493 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication The Evolution of Local Participatory Democracy in Nepal(Washington, DC: World Bank, 2025-11-05)Nepal is, according to its constitution, among the world’s most decentralized countries, with a long and complex tradition of local-level public participation. This paper traces the evolution of Nepal’s modern participatory institutions, examining the extent to which they are “induced” by external interventions versus being “organically” rooted in indigenous practices. The paper identifies three broad phases: an initial focus on participation in project implementation; a subsequent phase that expanded citizen engagement; and a third phase of citizen empowerment, culminating in the 2015 federal constitution, which granted unprecedented local autonomy. The analysis yields five key findings. First, over the past 50 years, successive reforms have progressively expanded opportunities for citizens to influence local decision-making. Second, these reforms have integrated traditional participatory mechanisms into formal institutions of local government. Third, although central-level initiatives exist, most participatory platforms continue to operate at the local level. Fourth, the federal constitution has created a new landscape of local democracy, embedding autonomy and accountability. Fifth, although they are still valued in many ethnic and territorial communities, traditional participatory practices are gradually disappearing. The paper concludes by offering policy recommendations to help donor agencies and governments strengthen Nepal’s democratic trajectory. It argues that effective interventions should build on Nepal’s deep participatory traditions while recognizing the constitutional reality of far-reaching local autonomy.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication Closing the Gender Gap in Entrepreneurship: Overcoming Challenges in Law and Practice for Female Entrepreneurs(Washington, DC: World Bank, 2026-01-07)Despite significant strides toward gender equality, women around the world continue to encounter systemic obstacles that hinder their entrepreneurial success. This paper systematically reviews the literature on the barriers female entrepreneurs face and the solutions proposed to overcome these challenges. It discusses institutional factors, financial factors, human capital factors, and social and cultural factors. The literature overview is complemented by a series of stylized facts that illustrate how overcoming some of these existing barriers is correlated with improved women’s entrepreneurship and female labor force participation, drawing on the World Bank’s Women, Business and the Law database as well as the World Bank’s Enterprise Surveys. The findings underscore the need for creating an enabling environment where women can thrive as entrepreneurs.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Mixtures of g-priors for Bayesian Model Averaging with Economic Application(2011-07-01)This paper examines the issue of variable selection in linear regression modeling, where there is a potentially large amount of possible covariates and economic theory offers insufficient guidance on how to select the appropriate subset. In this context, Bayesian Model Averaging presents a formal Bayesian solution to dealing with model uncertainty. The main interest here is the effect of the prior on the results, such as posterior inclusion probabilities of regressors and predictive performance. The authors combine a Binomial-Beta prior on model size with a g-prior on the coefficients of each model. In addition, they assign a hyperprior to g, as the choice of g has been found to have a large impact on the results. For the prior on g, they examine the Zellner-Siow prior and a class of Beta shrinkage priors, which covers most choices in the recent literature. The authors propose a benchmark Beta prior, inspired by earlier findings with fixed g, and show it leads to consistent model selection. Inference is conducted through a Markov chain Monte Carlo sampler over model space and g. The authors examine the performance of the various priors in the context of simulated and real data. For the latter, they consider two important applications in economics, namely cross-country growth regression and returns to schooling. Recommendations for applied users are provided.Publication Shortening Supply Chains(World Bank, Washington, DC, 2019-08)Small trading activities are a prevalent form of self-employment in developing countries, but their integration into supply value chains is not efficient, especially when it comes to perishable produce. This study tests a novel approach to improve their efficiency by reducing the time and cost of sourcing produce by aggregating purchases through the use of an app and centralized distribution system. Fruit and vegetable vendors in Bogotá currently travel most days to a central market to purchase produce, incurring substantial time and monetary costs. A social enterprise attempted to shorten the supply chain between farmers and vendors by aggregating orders from many small stores, sourcing directly from farmers, and delivering them to the stores. The introduction of this new service was randomized at the market block level. Initial interest was high and offering the service reduced travel time for users by almost two hours a week, reduced travel costs, and increased work-life balance for store owners. Firms offered the service saved an average of 6 to 8 percent on purchase costs, and although some of this passed through into lower prices for consumers, there was incomplete pass-through, so that markups rose. However, stores reduced their sales of products that were not originally offered by this new service, and their total sales and profits appear to have fallen in the short run, with service usage falling over time. The results highlight the potential for new technologies to solve firm coordination problems, offer a window into the nature of competition among small retailers, and point to the challenges in achieving economies of scale when disrupting centralized markets for multi-product firms.Publication Beyond Baseline and Follow-up : The Case for More T in Experiments(2011-04-01)The vast majority of randomized experiments in economics rely on a single baseline and single follow-up survey. If multiple follow-ups are conducted, the reason is typically to examine the trajectory of impact effects, so that in effect only one follow-up round is being used to estimate each treatment effect of interest. While such a design is suitable for study of highly autocorrelated and relatively precisely measured outcomes in the health and education domains, this paper makes the case that it is unlikely to be optimal for measuring noisy and relatively less autocorrelated outcomes such as business profits, household incomes and expenditures, and episodic health outcomes. Taking multiple measurements of such outcomes at relatively short intervals allows the researcher to average out noise, increasing power. When the outcomes have low autocorrelation, it can make sense to do no baseline at all. Moreover, the author shows how for such outcomes, more power can be achieved with multiple follow-ups than allocating the same total sample size over a single follow-up and baseline. The analysis highlights the large gains in power from ANCOVA rather than difference-in-differences when autocorrelations are low and a baseline is taken. The paper discusses the issues involved in multiple measurements, and makes recommendations for the design of experiments and related non-experimental impact evaluations.Publication Testing the Importance of Search Frictions, Matching, and Reservation Prestige through Randomized Experiments in Jordan(World Bank Group, Washington, DC, 2014-09-01)Unemployment rates for tertiary-educated youth in Jordan are high, as is the duration of unemployment. Two randomized experiments in Jordan were used to test different theories that may explain this phenomenon. The first experiment tested the role of search and matching frictions by providing firms and job candidates with an intensive screening and matching service based on educational backgrounds and psychometric assessments. Although more than 1,000 matches were made, youth rejected the opportunity to even have an interview in 28 percent of cases, and when a job offer was received, they rejected this offer or quickly quit the job 83 percent of the time. A second experiment built on the first by examining the willingness of educated, unemployed youth to apply for jobs of varying levels of prestige. Youth applied to only a small proportion of the job openings they were told about, with application rates higher for higher prestige jobs than lower prestige jobs. Youth failed to show up for the majority of interviews scheduled for low prestige jobs. The results suggest that reservation prestige is an important factor underlying the unemployment of educated Jordanian youth.Publication Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment in Jordan(World Bank, Washington, DC, 2012-07)Throughout the Middle East, unemployment rates of educated youth have been persistently high and female labor force participation, low. This paper studies the impact of a randomized experiment in Jordan designed to assist female community college graduates find employment. One randomly chosen group of graduates was given a voucher that would pay an employer a subsidy equivalent to the minimum wage for up to 6 months if they hired the graduate; a second group was invited to attend 45 hours of employability skills training designed to provide them with the soft skills employers say graduates often lack; a third group was offered both interventions; and the fourth group forms the control group. The analysis finds that the job voucher led to a 40 percentage point increase in employment in the short-run, but that most of this employment is not formal, and that the average effect is much smaller and no longer statistically significant 4 months after the voucher period has ended. The voucher does appear to have persistent impacts outside the capital, where it almost doubles the employment rate of graduates, but this appears likely to largely reflect displacement effects. Soft-skills training has no average impact on employment, although again there is a weakly significant impact outside the capital. The authors elicit the expectations of academics and development professionals to demonstrate that these findings are novel and unexpected. The results suggest that wage subsidies can help increase employment in the short term, but are not a panacea for the problems of high urban female youth unemployment.
Users also downloaded
Showing related downloaded files
Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication The Global Findex Database 2021(Washington, DC: World Bank, 2022-06-29)The fourth edition of the Global Findex offers a lens into how people accessed and used financial services during the COVID-19 pandemic, when mobility restrictions and health policies drove increased demand for digital services of all kinds. The Global Findex is the world’s most comprehensive database on financial inclusion. It is also the only global demand-side data source allowing for global and regional cross-country analysis to provide a rigorous and multidimensional picture of how adults save, borrow, make payments, and manage financial risks. Global Findex 2021 data were collected from national representative surveys of about 128,000 adults in more than 120 economies. The latest edition follows the 2011, 2014, and 2017 editions, and it includes a number of new series measuring financial health and resilience and contains more granular data on digital payment adoption, including merchant and government payments. The Global Findex is an indispensable resource for financial service practitioners, policy makers, researchers, and development professionals.Publication World Bank East Asia and the Pacific Economic Update, October 2024: Jobs and Technology(Washington, DC: World Bank, 2024-10-07)East Asia and the Pacific, seen in the context of the world economy, stands out as a paragon of development. Despite the recent ravages of the pandemic and the persistent tensions of geopolitics, the region is growing at stably high rates and the benefits are widely shared. But compared to its own past and potential, the region’s economic performance is less impressive. Growth is still below pre-pandemic rates, except in Indonesia, and output has not yet recovered to pre-pandemic levels in several countries, especially in the Pacific. This Economic Update highlights three key developments: shifting regional growth dynamics as China’s growth slows, changing trade patterns due to global tensions, and the impact of technologies such as robots, artificial intelligence, and digital platforms on jobs. The report calls for productivity-enhancing structural reforms to strengthen domestic growth drivers through; deeper international trade agreements to foster more open and stable trade regimes; deeper technical, digital, and soft skills while addressing impediments to labor mobility, factor price distortions and expanding social protection for workers in the digital informal economy to boost productivity and employment.Publication South Asia Development Update, October 2024: Women, Jobs, and Growth(Washington, DC: World Bank, 2024-10-10)South Asia’s growth is on track to exceed earlier expectations, in a broad-based upturn. The region is expected to remain the fastest-growing among emerging market and developing economies (EMDEs). Several risks could upend this generally promising outlook, including extreme weather events, social unrest, and policy missteps, such as reform delays. But South Asian countries also have considerable untapped potential that could help them further boost productivity growth and employment and adapt to climate change. In particular, with about two-thirds of the region’s working-age women out of the labor force, raising female employment rates to those of men could increase per capita income by as much as one-half. Measures to accelerate job creation, remove obstacles to women working, and equalize gender rights would be more effective if combined with a shift toward social norms that looked more favorably on working women. Also, most South Asian countries rank among the EMDEs least open to global trade and investment. Greater openness could boost women’s employment, spur the growth of firms, and allow the region to take better advantage of the reshaping of global supply chains and trade. Reducing the cost of conducting business could help the region better harness large-scale remittance inflows.Publication Remarks to the G20 Finance Ministers and Central Bank Governors Meeting(World Bank, Washington, DC, 2020-11-20)World Bank Group President David Malpass said that while some countries are recovering, the pandemic is still taking a terrible toll, with poverty levels rising sharply. He highlighted on the health emergency response programs in one hundred twelve countries using a fast-track mechanism that is now able to access a further window of twelve billion in funding for vaccine purchases and delivery. He also mentioned that the World Bank is already at work in cooperation with WHO, UNICEF, the Global Fund and GAVI on rapid vaccine deployment readiness assessments for one hundred countries. He spoke about IFC working in coordination with CEPI to invest a further four billion in manufacturing and distribution of vaccines and products that support vaccination programs. He recognized that fragile conflict and violence (FCV) states are most in need, and World Bank's engagement with them. Under his Presidency, the World Bank Group has invested more in climate finance than at any time in its history. He mentioned that IDA is frontloading its financing to make more resources available for the poorest countries. He highlighted on an important step that the G20 call on DSSI beneficiary countries to commit to disclose all public sector financial commitments. The Development Committee that asked the Bank and the IMF to propose more actions to address the unsustainable debt burdens of low- and middle-income countries. He concluded that the fuller transparency is the only way to balance the interests of the people with the interests of those signing the debt and investment contracts.