Publication: In Pursuit of Balance : Randomization in Practice in Development Field Experiments
Randomized experiments are increasingly used in development economics, with researchers now facing the question of not just whether to randomize, but how to do so. Pure random assignment guarantees that the treatment and control groups will have identical characteristics on average, but in any particular random allocation, the two groups will differ along some dimensions. Methods used to pursue greater balance include stratification, pair-wise matching, and re-randomization. This paper presents new evidence on the randomization methods used in existing randomized experiments, and carries out simulations in order to provide guidance for researchers. Three main results emerge. First, many researchers are not controlling for the method of randomization in their analysis. The authors show this leads to tests with incorrect size, and can result in lower power than if a pure random draw was used. Second, they find that in samples of 300 or more, the different randomization methods perform similarly in terms of achieving balance on many future outcomes of interest. However, for very persistent outcome variables and in smaller sample sizes, pair-wise matching and stratification perform best. Third, the analysis suggests that on balance the re-randomization methods common in practice are less desirable than other methods, such as matching.
Link to Data Set
“Bruhn, Miriam; McKenzie, David. 2008. In Pursuit of Balance : Randomization in Practice in Development Field Experiments. Policy Research Working Paper; No. 4752. © World Bank, Washington, DC. http://hdl.handle.net/10986/6910 License: CC BY 3.0 IGO.”
Other publications in this report series
PublicationHow to Deal with Exchange Rate Risk in Infrastructure and Other Long-Lived Projects(World Bank, Washington, DC, 2023-09-19)Most developing economies rely on foreign capital to finance their infrastructure needs. These projects are usually structured as long-term (25–35 years) franchises that pay in local currency. If investors evaluate their returns in terms of foreign currency, exchange rate volatility introduces risk that may reduce the level of investment below what would be socially optimal. This paper proposes a mechanism with very general features that hedges exchange rate fluctuation by adjusting the concession period. Such mechanism does not imply additional costs to the government and could be offered as a zero-cost option to lenders and investors exposed to currency fluctuations. This general mechanism is illustrated with three alternative specifications and data from a 25-year highway franchise is used to simulate how they would play out in eight different countries that exhibit diverse exchange rate trajectories.
PublicationRebel with a Cause: Effects of a Gender Norms Intervention for Adolescents in Somalia(World Bank, Washington, DC, 2023-09-15)Gender inequality and restrictive norms are often reinforced and internalized during adolescence, influencing pivotal life choices. This paper presents results from a randomly-assigned gender norms intervention for young adolescents in Somalia that led to greater support for gender equality in reported attitudes among both girls and boys. In a novel lab-in-the-field experiment designed to observe social group dynamics, treated adolescents were also found to be less likely to succumb to peer pressure to conform when stating their gender attitudes in public. Perceptions of gender norms appears to shift for boys, leading to a greater public expression of gender egalitarian ideals. Furthermore, the findings show improved adolescent mental health, increased caring behavior towards siblings of the opposite sex, and a higher likelihood of involvement in household chores by boys. A complementary gender norms intervention for parents had limited marginal impact on the attitudes and behaviors of adolescents. The results suggest that gender norms interventions can be effective in influencing the attitudes and public discourse around gender equality, even in early adolescence.
PublicationCorruption as a Push and Pull Factor of Migration Flows: Evidence from European Countries(World Bank, Washington, DC, 2023-09-14)Conclusive evidence on the relationship between corruption and migration has remained scant in the literature to date. Using data from 2008 to 2018 on bilateral migration flows across European Union and European Free Trade Association countries and four measures of corruption, this paper shows that corruption acts as both a push factor and a pull factor for migration patterns. Based on a gravity model, a one-unit increase in the corruption level in the origin country is associated with a 11 percent increase in out-migration. The same one-unit increase in the destination country is associated with a 10 percent decline in in-migration.
PublicationGlobal Trends in Child Monetary Poverty According to International Poverty Lines(World Bank, Washington, DC, 2023-09-19)This paper analyzes extreme child poverty ($2.15/day poverty line) trends, as well as child poverty based on the higher international poverty lines of $3.65 and $6.85. The paper provides a trajectory of extreme child poverty (children living in extremely poor households) from 2013 to 2019 (based on the most recent surveys included in the Global Monitoring Database), complemented by nowcasting for 2020 to 2022. Children continue to be disproportionately affected by extreme poverty. Children who are younger than 18 years comprise more than 50 percent of those living in extreme poverty, although their share of the population is 31 percent. The paper estimates that in 2019, 15.8 percent of children in the world (319 million) younger than 18 years lived on less than $2.15 (2017 purchasing power parity) per day, as opposed to 6.6 percent of adults ages 18 and older. More recent “nowcasted” estimates suggest that at least 333 million children were expected to be living in extremely poor households in 2022, implying that 14 million more children were extremely poor in 2022 than in 2019. Following an increase in extreme child poverty at the height of the pandemic in 2020, nowcasted estimates show that the rate of extreme child poverty fell again in 2021 and 2022, but only at the slow rate of progress seen prior to the COVID-19 crisis. If the COVID-19 pandemic had not occurred, an estimated 79.7 million fewer children would have been living in extreme poverty between 2013 and 2022; however, the estimates suggest that the number of children living in extreme poverty decreased by 49.2 million, due to pandemic disruptions.
PublicationMachine Learning Imputation of High Frequency Price Surveys in Papua New Guinea(World Bank, Washington, DC, 2023-09-28)Capabilities to track fast-moving economic developments re-main limited in many regions of the developing world. This complicates prioritizing policies aimed at supporting vulnerable populations. To gain insight into the evolution of fluid events in a data scarce context, this paper explores the ability of recent machine-learning advances to produce continuous data in near-real-time by imputing multiple entries in ongoing surveys. The paper attempts to track inflation in fresh produce prices at the local market level in Papua New Guinea, relying only on incomplete and intermittent survey data. This application is made challenging by high intra-month price volatility, low cross-market price correlations, and weak price trends. The modeling approach uses chained equations to produce an ensemble prediction for multiple price quotes simultaneously. The paper runs cross-validation of the prediction strategy under different designs in terms of markets, foods, and time periods covered. The results show that when the survey is well-designed, imputations can achieve accuracy that is attractive when compared to costly–and logistically often infeasible–direct measurement. The methods have wider applicability and could help to fill crucial data gaps in data scarce regions such as the Pacific Islands, especially in conjunction with specifically designed continuous surveys.