Publication: Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
Date
2012-12
ISSN
Published
2012-12
Author(s)
Abstract
Matching grants are one of the most
common policy instruments used by developing country
governments to try to foster technological upgrading,
innovation, exports, use of business development services
and other activities leading to firm growth. However, since
they involve subsidizing firms, the risk is that they could
crowd out private investment, subsidizing activities that
firms were planning to undertake anyway, or lead to pure
private gains, rather than generating the public gains that
justify government intervention. As a result, rigorous
evaluation of the effects of such programs is important. The
authors attempted to implement randomized experiments to
evaluate the impact of seven matching grant programs offered
in six African countries, but in each case were unable to
complete an experimental evaluation. One critique of
randomized experiments is publication bias, whereby only
those experiments with "interesting" results get
published. The hope is to mitigate this bias by learning
from the experiments that never happened. This paper
describes the three main proximate reasons for lack of
implementation: continued project delays, politicians not
willing to allow random assignment, and low program take-up;
and then delves into the underlying causes of these
occurring. Political economy, overly stringent eligibility
criteria that do not take account of where value-added may
be highest, a lack of attention to detail in "last
mile" issues, incentives facing project implementation
staff, and the way impact evaluations are funded, and all
help explain the failure of randomization. Lessons are drawn
from these experiences for both the implementation and the
possible evaluation of future projects.
Link to Data Set
Citation
“Campos, Francisco; Coville, Aidan; Fernandes, Ana M.; Goldstein, Markus; McKenzie, David. 2012. Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa. Policy Research Working Paper; No. 6296. © World Bank, Washington, DC. http://hdl.handle.net/10986/12200 License: CC BY 3.0 IGO.”
Report Series
Report Series
Other publications in this report series
-
PublicationHow to Deal with Exchange Rate Risk in Infrastructure and Other Long-Lived Projects(World Bank, Washington, DC, 2023-09-19)Most developing economies rely on foreign capital to finance their infrastructure needs. These projects are usually structured as long-term (25–35 years) franchises that pay in local currency. If investors evaluate their returns in terms of foreign currency, exchange rate volatility introduces risk that may reduce the level of investment below what would be socially optimal. This paper proposes a mechanism with very general features that hedges exchange rate fluctuation by adjusting the concession period. Such mechanism does not imply additional costs to the government and could be offered as a zero-cost option to lenders and investors exposed to currency fluctuations. This general mechanism is illustrated with three alternative specifications and data from a 25-year highway franchise is used to simulate how they would play out in eight different countries that exhibit diverse exchange rate trajectories.
-
PublicationRebel with a Cause: Effects of a Gender Norms Intervention for Adolescents in Somalia(World Bank, Washington, DC, 2023-09-15)Gender inequality and restrictive norms are often reinforced and internalized during adolescence, influencing pivotal life choices. This paper presents results from a randomly-assigned gender norms intervention for young adolescents in Somalia that led to greater support for gender equality in reported attitudes among both girls and boys. In a novel lab-in-the-field experiment designed to observe social group dynamics, treated adolescents were also found to be less likely to succumb to peer pressure to conform when stating their gender attitudes in public. Perceptions of gender norms appears to shift for boys, leading to a greater public expression of gender egalitarian ideals. Furthermore, the findings show improved adolescent mental health, increased caring behavior towards siblings of the opposite sex, and a higher likelihood of involvement in household chores by boys. A complementary gender norms intervention for parents had limited marginal impact on the attitudes and behaviors of adolescents. The results suggest that gender norms interventions can be effective in influencing the attitudes and public discourse around gender equality, even in early adolescence.
-
PublicationCorruption as a Push and Pull Factor of Migration Flows: Evidence from European Countries(World Bank, Washington, DC, 2023-09-14)Conclusive evidence on the relationship between corruption and migration has remained scant in the literature to date. Using data from 2008 to 2018 on bilateral migration flows across European Union and European Free Trade Association countries and four measures of corruption, this paper shows that corruption acts as both a push factor and a pull factor for migration patterns. Based on a gravity model, a one-unit increase in the corruption level in the origin country is associated with a 11 percent increase in out-migration. The same one-unit increase in the destination country is associated with a 10 percent decline in in-migration.
-
PublicationGlobal Trends in Child Monetary Poverty According to International Poverty Lines(World Bank, Washington, DC, 2023-09-19)This paper analyzes extreme child poverty ($2.15/day poverty line) trends, as well as child poverty based on the higher international poverty lines of $3.65 and $6.85. The paper provides a trajectory of extreme child poverty (children living in extremely poor households) from 2013 to 2019 (based on the most recent surveys included in the Global Monitoring Database), complemented by nowcasting for 2020 to 2022. Children continue to be disproportionately affected by extreme poverty. Children who are younger than 18 years comprise more than 50 percent of those living in extreme poverty, although their share of the population is 31 percent. The paper estimates that in 2019, 15.8 percent of children in the world (319 million) younger than 18 years lived on less than $2.15 (2017 purchasing power parity) per day, as opposed to 6.6 percent of adults ages 18 and older. More recent “nowcasted” estimates suggest that at least 333 million children were expected to be living in extremely poor households in 2022, implying that 14 million more children were extremely poor in 2022 than in 2019. Following an increase in extreme child poverty at the height of the pandemic in 2020, nowcasted estimates show that the rate of extreme child poverty fell again in 2021 and 2022, but only at the slow rate of progress seen prior to the COVID-19 crisis. If the COVID-19 pandemic had not occurred, an estimated 79.7 million fewer children would have been living in extreme poverty between 2013 and 2022; however, the estimates suggest that the number of children living in extreme poverty decreased by 49.2 million, due to pandemic disruptions.
-
PublicationMachine Learning Imputation of High Frequency Price Surveys in Papua New Guinea(World Bank, Washington, DC, 2023-09-28)Capabilities to track fast-moving economic developments re-main limited in many regions of the developing world. This complicates prioritizing policies aimed at supporting vulnerable populations. To gain insight into the evolution of fluid events in a data scarce context, this paper explores the ability of recent machine-learning advances to produce continuous data in near-real-time by imputing multiple entries in ongoing surveys. The paper attempts to track inflation in fresh produce prices at the local market level in Papua New Guinea, relying only on incomplete and intermittent survey data. This application is made challenging by high intra-month price volatility, low cross-market price correlations, and weak price trends. The modeling approach uses chained equations to produce an ensemble prediction for multiple price quotes simultaneously. The paper runs cross-validation of the prediction strategy under different designs in terms of markets, foods, and time periods covered. The results show that when the survey is well-designed, imputations can achieve accuracy that is attractive when compared to costly–and logistically often infeasible–direct measurement. The methods have wider applicability and could help to fill crucial data gaps in data scarce regions such as the Pacific Islands, especially in conjunction with specifically designed continuous surveys.