Publication: Distributional Impact Analysis: Toolkit and Illustrations of Impacts Beyond the Average Treatment Effect
Loading...
Published
2017-07
ISSN
Date
2017-07-19
Editor(s)
Abstract
Program evaluations often focus on average treatment effects. However, average treatment effects miss important aspects of policy evaluation, such as the impact on inequality and whether treatment harms some individuals. A growing literature develops methods to evaluate such issues by examining the distributional impacts of programs and policies. This toolkit reviews methods to do so, focusing on their application to randomized control trials. The paper emphasizes two strands of the literature: estimation of impacts on outcome distributions and estimation of the distribution of treatment impacts. The article then discusses extensions to conditional treatment effect heterogeneity, that is, to analyses of how treatment impacts vary with observed characteristics. The paper offers advice on inference, testing, and power calculations, which are important when implementing distributional analyses in practice. Finally, the paper illustrates select methods using data from two randomized evaluations.
Link to Data Set
Citation
“Bedoya, Guadalupe; Bittarello, Luca; Davis, Jonathan; Mittag, Nikolas. 2017. Distributional Impact Analysis: Toolkit and Illustrations of Impacts Beyond the Average Treatment Effect. Policy Research Working Paper;No. 8139. © World Bank. http://hdl.handle.net/10986/27642 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication The Evolution of Local Participatory Democracy in Nepal(Washington, DC: World Bank, 2025-11-05)Nepal is, according to its constitution, among the world’s most decentralized countries, with a long and complex tradition of local-level public participation. This paper traces the evolution of Nepal’s modern participatory institutions, examining the extent to which they are “induced” by external interventions versus being “organically” rooted in indigenous practices. The paper identifies three broad phases: an initial focus on participation in project implementation; a subsequent phase that expanded citizen engagement; and a third phase of citizen empowerment, culminating in the 2015 federal constitution, which granted unprecedented local autonomy. The analysis yields five key findings. First, over the past 50 years, successive reforms have progressively expanded opportunities for citizens to influence local decision-making. Second, these reforms have integrated traditional participatory mechanisms into formal institutions of local government. Third, although central-level initiatives exist, most participatory platforms continue to operate at the local level. Fourth, the federal constitution has created a new landscape of local democracy, embedding autonomy and accountability. Fifth, although they are still valued in many ethnic and territorial communities, traditional participatory practices are gradually disappearing. The paper concludes by offering policy recommendations to help donor agencies and governments strengthen Nepal’s democratic trajectory. It argues that effective interventions should build on Nepal’s deep participatory traditions while recognizing the constitutional reality of far-reaching local autonomy.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication Closing the Gender Gap in Entrepreneurship: Overcoming Challenges in Law and Practice for Female Entrepreneurs(Washington, DC: World Bank, 2026-01-07)Despite significant strides toward gender equality, women around the world continue to encounter systemic obstacles that hinder their entrepreneurial success. This paper systematically reviews the literature on the barriers female entrepreneurs face and the solutions proposed to overcome these challenges. It discusses institutional factors, financial factors, human capital factors, and social and cultural factors. The literature overview is complemented by a series of stylized facts that illustrate how overcoming some of these existing barriers is correlated with improved women’s entrepreneurship and female labor force participation, drawing on the World Bank’s Women, Business and the Law database as well as the World Bank’s Enterprise Surveys. The findings underscore the need for creating an enabling environment where women can thrive as entrepreneurs.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Investing in Communities Achieves Results : Findings from an Evaluation of Community Responses to HIV and AIDS(Washington, DC: World Bank, 2013-01)The overview summarizes the evaluation of community responses (15 studies, including 11 evaluations carried out in 8 countries). It presents the evaluation questions, the methodology, the key results achieved by community responses along the continuum of prevention, treatment, care and support, and the resulting policy and programmatic implications. Before the scale-up of the international response to the AIDS pandemic, community responses in developing countries played a crucial role in providing services and care for those affected. This study is the first comprehensive, mixed-method evaluation of the impact of that response. The evaluation finds that community response can be effective at increasing knowledge of HIV, promoting social empowerment, increasing access to and use of HIV services, and even decreasing HIV incidence, all through the effective mobilization of limited resources. By effectively engaging with this powerful community structure, future HIV and AIDS programs can ensure that communities continue to contribute to the global response to HIV and AIDS.Publication Final Report with Recommendations for Reforming Bulgaria’s Pharmaceutical Sector(Washington, DC, 2015-05)This report presents a review of current issues in the pharmaceutical sector in Bulgaria, examining drug policy, regulation, pricing, formulary selection, distribution, expenditure, and to the extent possible, patterns of use in Bulgaria. Its recommendations are intended to serve as options for reform, by articulating short and long term strategies for managing pharmaceutical expenditure, improving system sustainability, and driving value for money in Bulgaria, thereby improving efficiency, equity, affordability and ultimately, access to prescription medicines. Although small, the Bulgarian pharmaceutical market is showing strong growth. Medicines comprise not only a disproportionate share of health care expenditure (38 percent of total health expenditure, compared with an EU average of around 25 percent), the burden of out of pocket (OOP) costs is also excessive, possibly as high as 81 percent of total pharmaceutical expenditure. Of perhaps greatest concern is that rapid expenditure growth is taking place without obvious improvements in health outcomes, and at the expense of population equity. Bulgaria does not yet have an integrated national medicines policy, and the pharmaceutical sector is characterized by various highly prescriptive and at times, arguably inconsistent policy levers. While the regulatory framework has been largely brought into line with current EU standards, existing mechanisms for listing, pricing and subsidizing medicines are not ensuring adequate value for money for the National Health Insurance Fund (NHIF), and are contributing to inefficiencies in the health sector. Current pharmaceutical policy settings appear focused on limiting NHIF outlays rather than prioritizing access and affordability, and afford little financial protection to patients.Publication Impact Evaluation of a Large-scale Rural Sanitation Project in Indonesia(World Bank, Washington, DC, 2013-02)Lack of sanitation and poor hygiene behavior cause a tremendous disease burden among the poor. This paper evaluates the impact of the Total Sanitation and Sanitation Marketing project in Indonesia, where about 11 percent of children have diarrhea in any two-week period and more than 33,000 children die each year from diarrhea. The evaluation utilizes a randomized controlled trial but is unusual in that the program was evaluated when implemented at scale across the province of rural East Java in a way that was designed to strengthen the enabling environment and so be sustainable. One hundred and sixty communities across eight rural districts participated, and approximately 2,100 households were interviewed before and after the intervention. The authors found that the project increased toilet construction by approximately 3 percentage points (a 31 percent increase in the rate of toilet construction). The changes were primarily among non-poor households that did not have access to sanitation at baseline. Open defecation among these households decreased by 6 percentage points (or 17 percent). Diarrhea prevalence was 30 percent lower in treatment communities than in control communities at endline (3.3 versus 4.6 percent). The analysis cannot rule out that the differences in drinking water and handwashing behavior drove the decline in diarrhea. Reductions in parasitic infestations and improvements in height and weight were found for the non-poor sample with no sanitation at baseline.Publication Promoting Handwashing and Sanitation : Evidence from a Large-Scale Randomized Trial in Rural Tanzania(World Bank Group, Washington, DC, 2015-01)The association between hygiene, sanitation, and health is well documented, yet thousands of children die each year from exposure to contaminated fecal matter. At the same time, evidence on the effectiveness of at-scale behavior change interventions to improve sanitation and hygiene practices is limited. This paper presents the results of two large-scale, government-led handwashing and sanitation promotion campaigns in rural Tanzania. For the campaign, 181 wards were randomly assigned to receive sanitation promotion, handwashing promotion, both interventions together, or neither. One year after the end of the program, sanitation wards increased latrine construction rates from 38.6 to 51 percent and reduced regular open defecation from 23.1 to 11.1 percent. Households in handwashing wards show marginal improvements in handwashing behavior related to food preparation, but not at other critical junctures. Limited interaction is observed between handwashing and sanitation on intermediate outcomes: wards that received both handwashing and sanitation promotion are less likely to have feces visible around their latrine and more likely to have a handwashing station close to their latrine facility relative to individual treatment groups. Final health effects on child health measured through diarrhea, anemia, stunting, and wasting are absent in the single-intervention groups. The combined-treatment group produces statistically detectable, but biologically insignificant and inconsistent, health impacts. The results highlight the importance of focusing on intermediate outcomes of take-up and behavior change as a critical first step in large-scale programs before realizing the changes in health that sanitation and hygiene interventions aim to deliver.Publication Stimulating Demand for AIDS Prevention : Lessons from the RESPECT Trial(2012-02-01)HIV-prevention strategies have yielded only limited success so far in slowing down the AIDS epidemic. This paper examines novel intervention strategies that use incentives to discourage risky sexual behaviors. Widely-adopted conditional cash transfer programs that offer payments conditioning on easily monitored behaviors, such as well-child health care visits, have shown positive impact on health outcomes. Similarly, contingency management approaches have successfully used outcome-based rewards to encourage behaviors that are not easily monitored, such as stopping drug abuse. These strategies have not been used in the sexual domain, so this paper assesses how incentives can be used to reduce risky sexual behavior. After discussing theoretical pathways, it discusses the use of sexual-behavior incentives in the Tanzanian RESPECT trial. There, participants who tested negative for sexually transmitted infections are eligible for outcome-based cash rewards. The trial was well-received in the communities, with high enrollment rates and more than 90 percent of participants viewing the incentives favorably. After one year, 57 percent of enrollees in the "low-value" reward arm stated that the cash rewards "very much" motivated sexual behavioral change, rising to 79 percent in the "high-value" reward arm. Despite its controversial nature, the authors argue for further testing of such incentive-based approaches to encouraging reductions in risky sexual behavior.
Users also downloaded
Showing related downloaded files
Publication China’s Export Growth and the China Safeguard : Threats to the World Trading System?(2010-05-01)Is there evidence from China's pre-WTO accession period that newly imposed U.S. or EU import restrictions deflect Chinese exports to third markets? The authors examine this question by drawing on a newly constructed data set of U.S. and EU product-level import restrictions on Chinese trade imposed between 1992 and 2001 and estimate their impact on Chinese exports to 38 alternative markets. There is no systematic evidence that the import restrictions imposed during this period resulted in Chinese exports surging to such alternate destinations. To the contrary, there is weak evidence of a chilling effect on China's exports to third markets.Publication Road Capacity, Domestic Trade and Regional Outcomes(World Bank, Washington, DC, 2021-09)What is the impact on intranational trade and regional economic outcomes when the quality and lane capacity of an existing paved road network is expanded significantly? This paper investigates this question for the case of Turkey, which undertook a large-scale public investment in roads during the 2000s. Using spatially disaggregated data on road upgrades and domestic transactions, the paper estimates a large positive impact of reduced travel times on trade as well as local manufacturing employment and wages. A quantitative exercise using a workhorse model of spatial equilibrium implies heterogeneous effects across locations, with aggregate real income gains reaching 2–3 percent in the long run. Reductions in travel times increased the local employment-to-population ratio but had no effect on local population. The model is extended by endogenizing the labor supply decision to capture this finding. The model-implied elasticity of employment rates to travel time reductions captures about one-third of the empirical elasticity.Publication Increasing Female Labor Force Participation(World Bank, Washington, DC, 2023-01)Gender gaps in labor force participation persist worldwide. Closing this gap can lead to sizeable gains for economies—a 20 percent increase in GDP per capita, on average. Female labor force participation (FLFP) remains low due to lack of skills, assets and networks, time-based constraints, limited mobility, gender discrimination in hiring and promotion, and restrictive gender norms. Effective evidence-backed policy options can increase FLFP. They include providing childcare services, disseminating information on work opportunities and returns to employment, training in socio-emotional skills, addressing norms by engaging partners and family members, and targeting women via social protection, safety net, and public-works programs. The World Bank Group actively supports countries in boosting FLFP through development policy lending, advisory and analytical work, and supporting reforms to address constraining contextual factors, including legal barriers, social norms, and gender-based violence. This note sheds light on an array of policy options that are effective or show promise in improving FLFP.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Monetary Policy Strategies for Latin America(World Bank, Washington, DC, 2001-10)The authors examine possible monetary policy strategies for Latin America that may help lock in the gains the region attained in the fight against inflation in the 1990s. Instead of focusing the debate about the conduct of monetary policy on whether the nominal exchange rate should be fixed or flexible, the focus should be on whether the monetary policy regime appropriately constrains discretion in monetary policymaking. Three basic frameworks deserve serious discussion as possible long-run strategies for monetary policy in Latin America. The authors examine the advantages and disadvantages of a hard exchange-rate peg, monetary targeting, and inflation targeting, in light of monetary policy's recent track record in several Latin American countries, looking for clues about which of the strategies might be best suited to economies in the region. The answer: It depends on the country's institutional environment. Some countries appear not to have the institutions to constrain monetary policy if discretion is allowed. In those countries, there is a strong argument for hard pegs, including full dollarization, that allow little or no discretion to monetary authorities. In countries such as Chile, which can constrain discretion, inflation targeting is likely to produce a monetary policy that keeps inflation low yet appropriately copes with domestic and foreign shocks. Monetary targeting as a strategy for Latin America is not viable because of the likely instability of the relationship between inflation and monetary aggregates, of which there is ample international evidence. No monetary strategy can solve the basic problems that have existed in Latin American economies for a long time. The authors welcome the recent move in Latin American countries toward inflation targeting, but say no policy will succeed unless government policies also create the right institutional environment.