Publication: Designing Experiments to Measure Spillover Effects
Loading...
Date
2014-03
ISSN
Published
2014-03
Author(s)
Editor(s)
Abstract
This paper formalizes the design of experiments intended specifically to study spillover effects. By first randomizing the intensity of treatment within clusters and then randomly assigning individual treatment conditional on this cluster-level intensity, a novel set of treatment effects can be identified. The paper develops a formal framework for consistent estimation of these effects, provides explicit expressions for power calculations, and shows that the power to detect average treatment effects declines precisely with the quantity that identifies the novel treatment effects. A demonstration of the technique is provided using a cash transfer program in Malawi.
Link to Data Set
Citation
“Baird, Sarah; Bohren, Aislinn; McIntosh, Craig; Özler, Berk. 2014. Designing Experiments to Measure Spillover Effects. Policy Research Working Paper;No. 6824. © http://hdl.handle.net/10986/17738 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication Geopolitical Fragmentation and Friendshoring(Washington, DC: World Bank, 2025-06-26)This paper examines the relationship between geopolitical fragmentation and friendshoring of foreign investments over time, countries, and sectors. The analysis uses comprehensive data on foreign direct investments covering greenfield projects, mergers and acquisitions, and stocks of affiliates, as well as data on four alternative measures of geopolitical distance between countries. The gravity estimations suggest that, first, geopolitical differences have a negative effect on foreign investments and the magnitude has heightened in the post-pandemic period compared to a decade ago. Second, it is primarily the companies from advanced Western economies whose foreign investment decisions are increasingly shaped by friendshoring forces. Finally, the paper shows that friendshoring is not only confined to strategic industries, implying that allocations of foreign direct investments may not solely reflect national security or resilience considerations.Publication Soaring Food Prices Threaten Recent Economic Gains in the EU(Washington, DC: World Bank, 2025-07-02)The surge in food prices following the 2021 economic rebound has become a significant concern for households, particularly low-income ones, in Bulgaria, Croatia, Poland, and Romania. Food price inflation, which surpasses general inflation rates, risks worsening poverty and food insecurity in these countries. This paper explores the distributional impacts of rising food prices and the effectiveness of government response measures. Low-income households, who allocate a larger share of their income to food, are disproportionately affected and are struggling to cope with unexpected expenses, leading to increased difficulties in accessing proper nutrition. Simulations indicate that rising food prices contribute to higher poverty rates and greater income inequality, especially among vulnerable populations. They also suggest that the main poverty-targeted social assistance schemes offer critical support for the extreme poor, but expanding both coverage and benefits is vital to shield all at-risk individuals. Targeted policies that balance immediate relief with long-term resilience-building are essential to addressing the challenges posed by escalating food prices.Publication Disentangling the Key Economic Channels through Which Infrastructure Affects Jobs(Washington, DC: World Bank, 2025-04-03)This paper takes stock of the literature on infrastructure and jobs published since the early 2000s, using a conceptual framework to identify the key channels through which different types of infrastructure impact jobs. Where relevant, it highlights the different approaches and findings in the cases of energy, digital, and transport infrastructure. Overall, the literature review provides strong evidence of infrastructure’s positive impact on employment, particularly for women. In the case of electricity, this impact arises from freeing time that would otherwise be spent on household tasks. Similarly, digital infrastructure, particularly mobile phone coverage, has demonstrated positive labor market effects, often driven by private sector investments rather than large public expenditures, which are typically required for other large-scale infrastructure projects. The evidence on structural transformation is also positive, with some notable exceptions, such as studies that find no significant impact on structural transformation in rural India in the cases of electricity and roads. Even with better market connections, remote areas may continue to lack economic opportunities, due to the absence of agglomeration economies and complementary inputs such as human capital. Accordingly, reducing transport costs alone may not be sufficient to drive economic transformation in rural areas. The spatial dimension of transformation is particularly relevant for transport, both internationally—by enhancing trade integration—and within countries, where economic development tends to drive firms and jobs toward urban centers, benefitting from economies scale and network effects. Turning to organizational transformation, evidence on skill bias in developing countries is more mixed than in developed countries and may vary considerably by context. Further research, especially on the possible reasons explaining the differences between developed and developing economies, is needed.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Optimal Design of Experiments in the Presence of Interference(The MIT Press, 2018-01-12)We formalize the optimal design of experiments when there is interference between units, i.e. an individual's outcome depends on the outcomes of others in her group. We focus on randomized saturation designs, two-stage experiments that first randomize treatment saturation of a group, then individual treatment assignment. We map the potential outcomes framework with partial interference to a regression model with clustered errors, calculate standard errors of randomized saturation designs, and derive analytical insights about the optimal design. We show that the power to detect average treatment effects declines precisely with the ability to identify novel treatment and spillover effects.Publication Designing Cost-Effective Cash Transfer Programs to Boost Schooling among Young Women in Sub-Saharan Africa(2009-10-01)As of 2007, 29 developing countries had some type of conditional cash transfer program in place, with many others planning or piloting one. However, the evidence base needed by a government to decide how to design a new conditional cash transfer program is severely limited in a number of critical dimensions. This paper presents one-year schooling impacts from a conditional cash transfer experiment among teenage girls and young women in Malawi, which was designed to address these shortcomings: conditionality status, size of separate transfers to the schoolgirl and the parent, and village-level saturation of treatment were all independently randomized. The authors find that the program had large impacts on school attendance: the re-enrollment rate among those who had already dropped out of school before the start of the program increased by two and a half times and the dropout rate among those in school at baseline decreased from 11 to 6 percent. These impacts were, on average, similar in the conditional and the unconditional treatment arms. Although most schooling outcomes examined here were unresponsive to variation in the size of the transfer to the parents, higher transfers given directly to the schoolgirls were associated with significantly improved school attendance and progress - but only if the transfers were conditional on school attendance. There were no spillover effects within treatment communities after the first year of program implementation. Policymakers looking to design cost-effective cash transfer programs targeted toward young women should note the relative insensitivity of these short-term program impacts with respect to conditionality and total transfer size.Publication Cash or Condition? Evidence from a Cash Transfer Experiment(2010-03-01)Conditional Cash Transfer programs are "...the world's favorite new anti-poverty device," (The Economist, July 29 2010) yet little is known about the specific role of the conditions in driving their success. In this paper, we evaluate a unique cash transfer experiment targeted at adolescent girls in Malawi that featured both a conditional (CCT) and an unconditional (UCT) treatment arm. We find that while there was a modest improvement in school enrollment in the UCT arm in comparison to the control group, this increase is only 43 percent as large as the CCT arm. The CCT arm also outperformed the UCT arm in tests of English reading comprehension. The schooling condition, however, proved costly for important non-schooling outcomes: teenage pregnancy and marriage rates were substantially higher in the CCT than the UCT arm. Our findings suggest that a CCT program for early adolescents that transitions into a UCT for older teenagers would minimize this trade-off by improving schooling outcomes while avoiding the adverse impacts of conditionality on teenage pregnancy and marriage.Publication The Short-Term Impacts of a Schooling Conditional Cash Transfer Program on the Sexual Behavior of Young Women(2009-10-01)Recent evidence suggests that conditional cash transfer programs for schooling are effective in raising school enrollment and attendance. However, there is also reason to believe that such programs can affect other outcomes, such as the sexual behavior of their young beneficiaries. Zomba Cash Transfer Program is a randomized, ongoing conditional cash transfer intervention targeting young women in Malawi that provides incentives (in the form of school fees and cash transfers) to current schoolgirls and recent dropouts to stay in or return to school. An average offer of US$10/month conditional on satisfactory school attendance plus direct payment of secondary school fees led to significant declines in early marriage, teenage pregnancy, and self-reported sexual activity among program beneficiaries after just one year of program implementation. For program beneficiaries who were out of school at baseline, the probability of getting married and becoming pregnant declined by more than 40 percent and 30 percent, respectively. In addition, the incidence of the onset of sexual activity was 38 percent lower among all program beneficiaries than the control group. Overall, these results suggest that conditional cash transfer programs not only serve as useful tools for improving school attendance, but may also reduce sexual activity, teen pregnancy, and early marriage.Publication The Heterogeneous Effects of HIV Testing(World Bank, Washington, DC, 2014-03)An extensive multi-disciplinary literature examines the effects of learning one's HIV status on subsequent risky sexual behaviors. However, many of these studies rely on non-experimental designs; use self-reported outcome measures, or both. This study investigates the effects of a randomly assigned home based HIV testing and counseling (HTC) intervention on risky sexual behaviors and schooling investments among school-age females in Malawi. The study finds no overall effects on HIV, Herpes Simplex Virus (HSV-2), or achievement test scores at follow-up. However, among the small group of individuals who tested positive for HIV, a large increase in the probability of contracting HSV-2 is found, with this effect stronger among those surprised by their test results. Similarly, those surprised by HIV-negative test results see a significant improvement in achievement test scores, consistent with increased returns to investments in human capital. The finding of increased HSV-2 prevalence among HIV-positive individuals suggests that the conventional wisdom that those who learn they are HIV-positive will adopt safer sexual practices should be treated with caution.
Users also downloaded
Showing related downloaded files
Publication Strengthening Regional Water Security for Greater Resilience in the G5 Sahel(World Bank, Washington, DC, 2021-07-12)The World Bank’s historical engagement in transboundary water in West Africa is at a turning point, at a time when the G5 Sahel region faces unprecedented challenges. Therefore, it is time for the World Bank to broaden its water sector approach in the G5 Sahel and shift its focus to establishing a regional water security framework. The dual objectives of this report on the G5 Sahel region are to: (i) do a high-level analysis of water security challenges and their impacts on regional socio-economic development and stability, and (ii) suggest directions for future World Bank engagement on regional water security. The focus of this note is more exclusively on regional water challenges and local challenges with cross-border or even regional spillover effects. The report takes a development-driven approach to: (i) identify some of the ways in which water security affects socio-economic development in the G5 Sahel, (ii) explore the linkages between water security, resilience and conflict prevention, and (iii) present a set of guiding principles for the next regional engagements on water security in the region, both in terms of types of investment and implementation modalities. This report will also serve as a basis for deepening the dialogue with counterparts in the next fiscal year.Publication Benin Country Climate and Development Report(Washington, DC: World Bank, 2023-12-05)This Country Climate and Development Report (CCDR) proposes that Benin focuses on building a resilient economy, with investment and policy options primarily targeted at adapting to climate change risks. The dependence of Benin’s economic structure on agriculture and informal employment makes its development path highly vulnerable to climate change in the absence of proper adaptation. The government and the private sector need to be better prepared to deal with climate change – building adequate institutions and governance structures will be crucial. While all sectors will have to become more resilient, this is especially urgent for agriculture and land use, urban and network infrastructure, and human development (education, health). Mitigation efforts should focus on avoiding carbon lock-ins and reducing deforestation. Investing in renewable energy whilst expanding the population’s access to electricity should be a priority for Benin. A higher share of renewable energy can bring about co-benefits for other sectors (agriculture, water, transport, and forestry). To maintain its growth trajectory, Benin needs to pay special attention to its most vulnerable people, including women. To protect the poor and vulnerable the just transition should focus on reconciling development and climate goals while addressing inequality (income and gender related), and spatial exclusion.Publication Brazil Country Climate and Development Report(World Bank Group, Washington DC, 2023-05-04)Brazil is highly exposed to climate change risks. The impacts of global climate change risks and local practices on the Amazon and Cerrado biomes are of particular concern, as they provide vital ecosystem services to Brazil, the South American region, and the world. The Brazil Country Climate and Development Report (CCDR) examines the implications of climate change and climate action for Brazil's development objectives and priorities. It identifies opportunities for Brazil to achieve both its development goals and its climate commitments. It lays out a combination of sectoral and economy-wide policy reforms, as well as targeted investments in near- and medium-term mitigation and adaptation measures to achieve more rapid and inclusive development with lower greenhouse gas (GHG) emissions. The idea is to maximize synergies between climate and development objectives, while addressing trade-offs among policy objectives and key transition challenges.Publication Availability and Pricing of Essential Medicines in Myanmar(Washington, DC: World Bank, 2024-04-08)This report on the Availability and Pricing of Essential Medicines in Myanmar relies on multiple assessments (2021 - 2023) conducted jointly by the World Bank and World Health Organization as well as other publicly available information. This report provides an overview of findings from a mixed-method study to assess the impact of the COVID-19 pandemic and political instability on Myanmar’s healthcare system, focusing on the availability of essential medicines. This study combines quantitative data collection from retail pharmacies and qualitative interviews with key stakeholders in the pharmaceutical industry of the private sector. The study further explored the coping mechanisms households and healthcare providers employ to address the rising prices of essential medicines. Many individuals with chronic conditions reported reducing or skipping medication intake due to financial constraints. Healthcare providers adjusted their prescription choices, often opting for more affordable brands, to mitigate the impact on patients. However, these coping mechanisms were insufficient to alleviate the negative consequences of rising medicine prices. Based on the findings, this study emphasizes the urgent need to address the root causes of rising medicine prices in order to ensure the availability and affordability of essential medicines for all individuals in Myanmar. By gaining a comprehensive understanding of the prevailing issues, targeted interventions and programs can be designed to safeguard the health and wellbeing of Myanmar’s population.Publication Cash in the City(World Bank, Washington, DC, 2021-01)Poverty and crises are rapidly “urbanizing†. Yet experience with operationalizing cash transfers in urban areas is limited. This paper captures early lessons from a new generation of urban cash transfer responses to Covid-19 in eleven African countries. The analysis contextualizes such initiatives within a longer-term trajectory of urban social protection programs from the early 2000s. A range of lessons emerge around design and implementation, partnerships, institutions and political economy, strategic issues, and evidence and learning.