The World Bank Economic Review, 36(1), 2022, 244–267 https://doi.org10.1093/wber/lhab007 Article What We Learn about Girls’ Education from Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Interventions That Do Not Focus on Girls David K. Evans and Fei Yuan Abstract What is the best way to improve access and learning outcomes for girls? This review brings together evidence from 267 educational interventions in 54 low- and middle-income countries – regardless of whether the inter- ventions specifically target girls – and identifies their impacts on girls. To improve access and learning, general interventions deliver average gains for girls that are comparable to girl-targeted interventions. General inter- ventions have similar impacts for girls as for boys. Taken together, these findings suggest that many educational gains for girls may be achieved through nontargeted programs. Many of the most effective interventions to im- prove access for girls relax household-level constraints (such as cash transfer programs), and many of the most effective interventions to improve learning for girls involve improving the pedagogy of teachers. Girl-targeted interventions may make the most sense when addressing constraints that are unique to, or most pronounced for, girls. JEL classification: I21, I24, J16, O1 Keywords: education, girls, effect size, impact evaluation, economic development 1. Introduction Investing in girls’ education has been called “the world’s best investment” (Sperling and Winthrop 2015). But how can policymakers do so most effectively? Evidence on what works to improve the quality of education is accumulating at an unprecedented rate (World Bank 2018b).1 In recent years, hundreds of impact evaluations in low- and middle-income countries have demonstrated the effective- ness – or lack thereof – of a range of interventions at improving education outcomes, for girls and boys David K. Evans (corresponding author) is a Senior Fellow at the Center for Global Development, Washington, DC, USA; his email address is devans@cgdev.org. Fei Yuan (corresponding author) is a doctoral student at the Harvard Graduate School of Education, Cambridge, MA, USA; her email address is fyuan@g.harvard.edu. The order of authors was determined alphabetically. The research for this article was financed by Echidna Giving, the Umbrella Facility for Gender Equality at the World Bank, and the Bill & Melinda Gates Foundation. The authors thank Eric Edmonds, Deon Filmer, Erin Ganju, Markus Goldstein, Pamela Jakiela, Oni Lusk-Stover, Mary Obelnicki, Owen Ozier, Pauline Rose, Dana Schmidt, Craig Silverstein, Lexie Wagner, Kim Wright-Violich, Louise Yorke, and various seminar audiences and anonymous referees for feedback and suggestions. The authors also thank Amina Mendez Acosta, Tara Siegel, Danielle Sobol, and Shikhty Sunny for excellent research assistance. A supplementary online appendix is available with this article at The World Bank Economic Review website. 1 Specifically, fig. S4.1 in World Bank (2018b) demonstrates the rapid growth of studies that examine learning outcomes. Studies that examine access outcomes have also grown. © The Author(s) 2021. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com The World Bank Economic Review 245 (Evans and Popova 2016; J-PAL 2017). Reviews that examine the most effective ways to boost girls’ ed- ucation tend to focus on interventions that target girls – for example, building girls’ latrines at schools or providing scholarships for girls – potentially missing large educational benefits for girls from interventions that are not gender-specific (Unterhalter et al. 2014; Sperling and Winthrop 2015; Haberland, McCarthy, and Brady 2018).2 This paper reports the results of a systematic review identifying the most effective interventions to Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 improve girls’ access to education and learning outcomes within an evidence base that includes both girl-targeted and general education interventions. The study poses three research questions: (1) Are girl- targeted interventions more effective for girls’ outcomes than general interventions? (2) For general, non- targeted interventions, do impacts on girls tend to be larger? and (3) In absolute terms, what are the most effective interventions for girls? To answer these questions, the study collected and examined a large database of education studies with access or learning outcomes for students. It categorized the studies as either evaluating girl-targeted or non-targeted (i.e., general) interventions and identified all studies that reported gender-differentiated impacts. Only one in three studies of interventions not targeted to girls report disaggregated impacts by gender, so a first implication of this work is that in order to understand how best to improve girls’ education, studies should consistently report impacts for girls. For those studies that did not report gender- differentiated impacts, the study contacted their authors asking them either to run the additional gender- differentiated analysis or to share the data. The effects of different programs were then standardized to increase comparability of effect sizes across studies. Ultimately, the effects for girls from 175 studies were synthesized. (The full list of studies is available in S1 of the supplementary online appendix.) The study finds that general, nontargeted interventions perform similarly to girl-targeted interventions on average to increase both girls’ access to school and their learning in school. General interventions tend to have similar effects for girls and for boys. (The evidence suggests that if anything, girls benefit slightly more from general interventions, although the differences are not statistically significant.) In examining the most and least effective interventions for girls’ education, the study finds that girls’ access to school is more responsive to changes in costs, distance, and sanitation conditions; while girls’ learning is more likely to be improved by structured pedagogy and interventions that help teachers to teach at the right level. Later sections of the paper discuss the implications for inequality between boys and girls, cost-effectiveness of programs, and what circumstances lend themselves to general versus targeted interventions. General, nontargeted interventions may be more politically palatable for scaling up by national gov- ernments – since constituents have both daughters and sons.3 General interventions offer a wider array of evaluated interventions, giving policy makers a richer menu of options among nontargeted interventions to improve girls’ education. In countries where boys also struggle to achieve quality education, general interventions can simultaneously improve girls’ learning while benefitting boys as well. None of this sug- gests that programs will not benefit from considering gender issues in their design. For programs with a per-pupil expenditure (like cash transfers or scholarships), targeted versions will be significantly less costly for the simple reason that they will only need to budget for the cost of transfers or scholarships to girls (and not to boys). Some investments in quality, like pedagogical programs delivered through teachers, may be less sensitive to the number of beneficiary students. 2 J-PAL (2017) looks at nontargeted programs, but focuses only on access outcomes and, within that, only on randomized controlled trials. 3 Those preferences may be different among international donors. Among the general, nontargeted interventions in this analysis, 43 percent were implemented by government agencies, whereas only 28 percent of girl-targeted interventions were conducted by governments. Furthermore, government-implemented interventions benefited more girls (at least, as proxied by the size of the evaluation treatment groups), with an average treatment group sample size (5,158) about four times that of interventions implemented by nongovernment organizations (NGOs) (1,276). 246 Evans and Yuan This analysis is subject to certain limitations. First, there are far more general interventions than girl- targeted interventions. Second, not all general studies report gender-disaggregated results, although those results were obtained from the authors whenever possible, including for many studies that did not initially report them. Third, this study focuses on access to school and on learning outcomes, whereas some girl- targeted interventions may focus on other outcomes. Fourth, many of the interventions included in this review focus on primary education, and as girls reach adolescence, they may face more gender-specific Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 constraints. Bearing those caveats in mind, these results suggest that to achieve access and quality, especially in primary education, specifically targeting girls may not always be necessary to help those girls succeed. If policy makers want to help girls learn, one strategy will be to make schools better for all children. 2. Method The project gathered a large collection of studies that report education outcomes, either access or learning. For each of the studies, the project identified whether or not they separately report impacts for boys and girls. For studies that separately report impacts for boys and girls, this project extracted those data, stan- dardized the estimates, and used them to compare the impacts for boys versus girls and across programs for girls. For studies that do not separately report, the researchers contacted the authors and asked them either to share the data or to provide the separate estimates themselves. This section reports on each step in detail. Literature Search The study began with a large database of education impact evaluations compiled for Evans and Popova (2016) and subsequently updated it. The database consists of 495 studies that were cited in 10 recent systematic reviews of evidence on what works to improve learning and access in low- and middle-income countries.4 All the reviews were published or made publicly available between 2013 and 2015, and the studies included were conducted between 1980 and 2015. Another systematic review of interventions with a special focus on access outcomes came out in 2017 (J-PAL 2017); its references added four studies to the database. To increase the coverage of studies that were published (either as working papers or peer-reviewed articles) after 2015, the researchers conducted an additional literature search between October 2017 and January 2018. They searched Google Scholar and the websites of major institutions that conduct research related to low- and middle-income countries for working papers or research reports that were published between 2015 and 2017 containing the keywords “evidence,” “education,” “access,” “learning,” “enroll- ment,” “dropout,” “attendance,” or “score.” The same search terms were applied to several economics and education journals, which are listed in S2 of the supplementary online appendix. These two additional searches yielded 19 new studies. In total, 518 papers were reviewed. Inclusion Criteria The project included studies of education interventions (such as teacher professional development and providing textbooks), health interventions (such as providing deworming drugs and micronutrients), and safety net interventions (such as cash transfers). The present study only included studies that took place in 4 The 10 reviews are Kremer, Brannen, and Glennerster (2013), Krishnaratne and White (2013); Glewwe, Maïga, and Zheng (2014), Ganimian and Murnane (2016); McEwan (2015), Masino and Niño-Zarazúa (2016), Glewwe and Muralidharan (2016); Asim et al. (2017), Snilstveit et al. (2017), and Conn (2017). Conn, Glewwe et al., McEwan, and Masino and Niño-Zarazúa only include studies with learning outcomes. The other reviews include studies with learning outcomes and studies with access outcomes. The database is available at “Database of Education Studies,” https://sites.google.com/site/davidkevans/database-of-education-studies. The World Bank Economic Review 247 preprimary, primary, and secondary schools in low- or middle-income countries, according to the World Bank definition (World Bank 2020). To be included, studies had to be published – either as a working paper or a journal article – between 1980 and 2017 and had to report at least one of the following education outcomes: access outcomes (enrollment, dropout, or attendance) or learning outcomes (composite test score or any subject score). Nonacademic skill development programs for adolescents were not included. The project only included studies that used an experimental or quasi-experimental design. To be in- Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 cluded, studies needed to have a valid counterfactual – in other words, a credible way of determining what would have happened in the absence of the program. The ways that studies could construct such a counterfactual included random assignment of treatment, difference-in-differences analysis, regression discontinuity, instrumental variables, and propensity score matching. At the same time, the analysis was restricted to studies where girls are included in the intervention group. Data Collection Upon reviewing the 518 identified studies, 328 studies met the inclusion criteria. These studies were further divided into two groups: girl-targeted interventions and general interventions. Girl-targeted interventions include any intervention that is explicitly designed to boost education outcomes for girls specifically. For example, this includes programs that provide girls with cash or in-kind transfers, reduce tuition or other school costs for girls, offer merit scholarships to girls, build latrines for girls in schools, reduce travel dis- tance to schools for girls by building village schools or providing transportation, provide female teachers, or implement girls’ empowerment curricula in schools. In general, if the program either specifically targets girls for benefits or explicitly states its objective as improving girls’ educational outcomes, it is counted as “girl-targeted.” The researchers identified 18 studies designed to increase access or learning specifically for girls. The other 310 studies were general interventions.5 “General interventions” refer to programs that are gender neutral in their design. Examples include programs that offer computer-assisted learning for all students, provide school meals for all students, and distribute free school uniforms or textbooks to all students. A general intervention may disproportionately benefit girl students, but it is not explicitly designed to do so, nor is it targeted specifically to girls. To collect the impacts of interventions on girls for the 18 girl-targeted studies, the project used the results on girls reported in the studies. For general interventions, the average effect reported in the study covered an average across boys and girls, so the researchers verified which studies also reported effects separately: 105 studies reported heterogeneous intervention impacts by gender in their original papers, and those results were incorporated in the review. However, that left 205 studies that did not report gender-differentiated impacts in their original papers. The authors of these studies were contacted up to three times between January 2018 and July 2018, with the request that they either provide additional estimates of intervention effects by gender or share the data of their studies with the project to perform the analysis on their behalf. Authors were given at least three months to reply with either new estimates or their data if they were interested. Of the 205 studies, the project received replies from the authors of 104 studies. Among them, the authors of 32 studies indicated that the data were no longer available or that gender data were not collected. Another 72 sets of authors expressed their willingness to run the additional analysis (50 papers) or share their data (22 papers). By the end of July 2018, it was possible to obtain new estimates of effects by gender of 52 studies. Figure 1 displays the review process. Combining girl-targeted interventions, general interventions that report impacts on girls, and the new estimates the project collected from authors, the final sample of this review consists of 175 studies evaluating 267 total interventions. Among those studies, 86 measured access 5 There are three general intervention studies that contain a girl-targeted intervention arm, but for the purpose of counting, because the bulk of the benefits do not target girls, the project includes them in the general intervention group. 248 Evans and Yuan Figure 1. Review Method [328 studies] Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Source: Authors’ calculations. Table 1. Descriptive Statistics of the Overall Sample Number of studies Number of interventions General studies 157 238 General studies – Access 69 95 General studies – Learning 106 178 Girl-targeted studies 18 29 Girl-targeted studies – Access 17 25 Girl-targeted studies – Learning 12 15 Total 175 267 Source: Authors’ calculations. Note: Access and learning do not sum to the total because multiple studies report both learning and access outcomes. outcomes such as enrollment, attendance, or dropout; and 118 measured learning outcomes including a composite test score, math score, or language score (table 1). Coding of Effect Sizes In this paper, the unit of analysis is the estimated impact of an intervention, where a group that received an intervention is compared to another group that did not receive the intervention. For studies with multiple treatment arms, the project coded the impact of each treatment arm separately (as its own intervention) and recorded the education outcomes corresponding to that intervention. For example, Berlinski et al. (2016) tested the effects of four interventions or treatment arms: (1) an active learning approach to the teaching of math, (2) an active learning approach plus an interactive white board, (3) an active learn- ing approach plus a computer lab, and (4) an active learning approach plus one computer per student. The project coded these four experiments as four separate interventions. Furthermore, if studies reported The World Bank Economic Review 249 multiple estimates for a given intervention, the project coded all of those estimates separately rather than creating a composite variable. Since studies in the sample reported different outcomes using different measures, in order to compare the effectiveness of the interventions on the same scale, individual point estimates needed to be standard- ized. This paper used Cohen’s d to standardize effect sizes, following McEwan (2015) and Conn (2017). Cohen’s d can be estimated using the raw mean difference (D) between a treatment group (Y ¯ T ) and a Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 control group (Y ¯ C ) measured at the follow-up, as well as the pooled standard deviation for the treatment and control groups combined (S pooled ) (see eq. 1).6 D ¯T − Y Y ¯C d= = (1) S pooled S pooled In cases where the pooled standard deviation was not directly reported in the study, it was calculated using eq. (2) from Borenstein et al. (2009): nT nC S pooled = SED (2) nT + nC where nT and nc are the sample sizes in the treatment and control groups at the follow-up, and SED is the reported standard error of D. Qualitative Variables A set of additional variables was collected to better characterize the most effective interventions for girls. The variables included country, region, implementation agency, location (rural or urban), intervention level (village, school, household, or individual), duration of intervention (single contact or repeated con- tact), number of intervention components (single or multiple), the level of education at which the inter- vention was implemented (preprimary, primary, or secondary), student age, major program components (such as reducing school costs, a health intervention, additional teaching and learning materials, or school grants), the presence of components identified by program implementers as “girl friendly,” cost data (if any), quality of the outcome data (e.g., administrative data, self-reported data, national tests, interna- tional tests, program designed tests). For each study, its publication type and evaluation design were also coded. 3. Results Where and What Are the Girl-Targeted Interventions? Girl-targeted interventions are distributed differently from general interventions, both in terms of their characteristics and where they are placed (table 2). Girl-targeted interventions are more likely to cover both primary and secondary than general interventions. Girl-targeted and general interventions have a similar urban/rural distribution, but the smaller sample of girl-targeted interventions means that almost all are in either rural areas or both urban and rural areas. Girl-targeted and general interventions are, similarly, likely to have a single component or multiple components. Studies of girl-targeted interventions are slightly more likely to be published as journal articles (72 percent) rather than working papers, as opposed to general interventions (64 percent). The starkest difference between the samples is that almost all of the girl-targeted interventions are located in either South Asia (52 percent) or Sub-Saharan Africa (45 percent). While there are many general interventions from those regions, there are also many general interventions in Latin America and the Caribbean. This difference is unsurprising, given that the gender gap has flipped in Latin America and the 6 This paper, wherever applicable, collected the mean difference with controls for observable variables. 250 Evans and Yuan Table 2. Descriptive Statistics of Included Interventions All interventions General interventions Girl-targeted interventions Education level Primary 150 (56%) 142 (60%) 8 (28%) Secondary (lower and/or upper) 40 (15%) 36 (15%) 4 (14%) Both primary and secondary 47 (18%) 31 (13%) 16 (55%) Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Other 30 (11%) 29 (12%) 1 (3%) Location Urban 29 (11%) 27 (11%) 2 (7%) Rural 101 (38%) 88 (37%) 13 (45%) Both 137 (51%) 123 (52%) 14 (48%) Components Single 127 (48%) 113 (47%) 14 (48%) Multiple 140 (52%) 125 (53%) 15 (52%) Region East Asia and Pacific 45 (17%) 44 (18%) 1 (3%) Europe and Central Asia 1 1(0.5%) – Latin America and the Caribbean 55 (21%) 55 (23%) – Middle East and North Africa 1 1 (0.5%) – South Asia 77 (29%) 62 (26%) 15 (52%) Sub-Saharan Africa 88 (33%) 75 (32%) 13 (45%) Publication type Published paper 173 (65%) 152 (64%) 21 (72%) Working paper 94 (35%) 86 (36%) 8 (28%) Source: Authors’ calculations. Caribbean, with girls tending to outperform boys, whereas South Asia and Sub-Saharan Africa remain the regions with the widest gender gaps favoring boys (Evans, Akmal, and Jakiela 2021). Girl-targeted interventions are also concentrated in a subset of classes of interventions (table 3). Most girl-targeted interventions fall into three categories: individual transfers – that is, cash transfers, in-kind transfers, or scholarships (35 percent), school infrastructure (28 percent), or girls’ empowerment pro- grams (21 percent). The latter group includes gender awareness education, marriage delay incentives and skill development, and so forth. General interventions fall in some of the same categories (e.g., 29 percent are individual transfers) but also in others: 29 percent involve some sort of teacher-focused interven- tion (such as professional development, improved pedagogy, or teacher incentives). The study’s results include analysis of subgroups where there is a significant concentration of both girl-targeted and general interventions. Finally, the study examines the association between whether an intervention is girl-targeted and cer- tain additional characteristics (table S3.1 in the supplementary online appendix). Interventions imple- mented by nongovernment organizations are 11.7 percentage points more likely to be targeted to girls. Secondary school programs are also more likely to be targeted to girls. Girl-targeted interventions are slightly more common in countries with lower girls’ primary school completion and enrollment rates, lower performance of girls on the human capital index, and in countries with higher adolescent mar- riage and adolescent childbirth rates. These associations may not be causal, but they are suggestive, both that girl-targeted interventions are more common among NGOs and in places where girls face more obstacles. Are Girl-Targeted Interventions the Most Effective for girls? In terms of increasing girls’ participation in school, girl-targeted interventions and general interventions perform similarly on average, although there are some girl-targeted interventions that outstrip general The World Bank Economic Review 251 Table 3. Categories of Included Interventions All interventions General interventions Girl-targeted interventions Student-level Individual transfers 78 (29%) 68 (29%) 10 (35%) Girls’ empowerment 6 (2%) – 6 (21%) Health, sanitation, nutrition (including school meals) 25 (9%) 24 (10%) 1 (3%) Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Information 8 (3%) 8 (3%) – Remedial education 6 (2%) 6 (3%) – Teacher-level Teacher professional development and support 20 (7%) 20 (8%) – Curriculum, instruction, and pedagogy 36 (13%) 36 (15%) – Teaching and learning materials 4 (1%) 3(1%) 1 (3%) Teacher incentive 11 (4%) 11 (5%) – School-level Infrastructure 17 (6%) 9 (4%) 8 (28%) School grants 24 (9%) 22 (9%) 2 (6%) Technology 11 (4%) 11 (5%) Community-level Community engagement 12 (5%) 12 (5%) Other 9 (3%) 8 (3%) 1 (3%) Source: Authors’ calculations. interventions. Figure 2 demonstrates the result of a random effects meta-analysis comparing the effect sizes for schooling access for both general interventions and girl-targeted interventions. The average effect size is larger for girl-targeted interventions (0.15 SDs) than for general interventions (0.11 SDs), but the differences are not statistically significant. (Significance is indicated with asterisks in fig. 2 and reported explicitly in table S3.2 in the supplementary online appendix.) In South Asia, where there is a significant sample of both general interventions and girl-targeted in- terventions with access outcomes, the point estimates are very similar: 0.08 SDs for girl-targeted in- terventions and 0.09 SDs for general interventions. In Sub-Saharan Africa, girl-targeted interventions have a larger average effect (0.23 SDs versus 0.09 SDs for general interventions), but the confidence in- terval for girl-targeted interventions is enormous, from 0.01 to 0.45 SDs, suggesting wide variation in performance. When examining specific classes of programs for which several girl-targeted interventions are available, similar point estimates are seen for school infrastructure (0.10 SDs for girl-targeted interventions and 0.12 SDs for general interventions). For individual transfers, larger point estimates are seen for girl-targeted interventions (0.23 SDs) than for general interventions (0.15 SDs), but the confidence interval for girl- targeted interventions is – like the estimates for Sub-Saharan Africa – very wide, suggesting wide variation in performance of these programs. For studies that took place in rural areas, general studies have larger point estimates (0.15 SDs) than girl-targeted studies (0.06 SDs), and the difference is statistically significant (fig. 2 and table S3.2 in the supplementary online appendix). The study likewise does not see any clear differences for programs in primary school or secondary school only. In an ideal scenario, one might control for multiple characteristics, but the relatively small sample of girl-targeted interventions examined in this study makes that econometrically infeasible. An alternative way to compare these interventions is to examine the distribution of effect sizes for girl-targeted interventions and for general interventions – that is, list all the estimates for targeted and for general interventions separately and compare the two distributions (table S3.3 in the supplementary online appendix). As with the meta-analytic results, similar distributions of effect sizes are found for the two sets of interventions. The median effect size for these two categories is very similar, increasing girls’ enrollment 252 Evans and Yuan Figure 2. Effect Sizes of Access Outcomes for Girls Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Source: Authors’ calculations. Note: A random-effects model was used to estimate the meta results for each group of studies. Error bars report the 95 percent confidence interval for the estimates. Country groups followed the World Bank country group definitions (World Bank 2020). * Difference in effect sizes between general studies and girl-targeted studies is significant at the 0.05 level, and p-values are reported in table S3.2 in the supplementary online appendix. or attendance by 0.07–0.09 standard deviations. The effect sizes of less effective interventions – at the 10th and 25th percentiles – are also similar. However, the girl-targeted interventions at the 90th percentile have effect sizes that are 0.07 standard deviations larger than those of general interventions. That said, there are also general interventions with large effect sizes. The effect size of the most effective general intervention (a conditional cash transfer in South Africa, Eyal, Woolard, and Burns (2014)) –1.66 standard deviations – is comparable in size to that of the most effective girl-targeted intervention (a conditional cash transfer to girls in Malawi, Baird et al. (2016)), at 1.54 standard deviations. None of these differences are statistically significant at standard levels (table S3.4 in the supplementary online appendix). There are also far more evaluated general interventions than girl-targeted interventions. As seen in table 1, the number of general interventions is more than three times that of girl-targeted interventions. This means that in each of the effect size bins, general interventions provide a larger menu for tested op- tions (fig. 3). Even among the most effective interventions, there are almost as many general interventions with large effect sizes (greater than 0.4 standard deviations) because so many more general interventions have been tested. Therefore, general interventions –together with girl-targeted interventions – constitute an important source of ways to improve girls’ access to education. For learning, smaller point estimates are observed for girl-targeted interventions (0.11 SDs versus 0.18 SDs for general interventions), and the difference is statistically significant at the 5 percent level (fig. 4). In this case, girl-targeted interventions have – on average – similar or smaller effects as com- pared to general interventions in every category: primary only, secondary only, South Asia, Sub-Saharan Africa, rural, transfer programs, and infrastructure programs. Some of those subgroup differences are sta- The World Bank Economic Review 253 Figure 3. Number of Access Outcomes by Effect Size Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Source: Authors’ calculations. Note: Effect sizes are in the unit of standard deviations. tistically significantly different (table S3.2 in the supplementary online appendix): in both South Asia and Sub-Saharan Africa, effect sizes for general studies are around 0.10 SDs bigger than those for girl-targeted studies.7 Comparing the distribution of effect sizes, girl-targeted and general interventions have comparable impacts on girls’ learning (table S3.3 in the supplementary online appendix). The median interventions increase learning by 0.12 and 0.11 standard deviations. The top programs (90th percentile) among general interventions have slightly bigger measured effect sizes (0.52 standard deviations) than the top programs among girl-targeted interventions (0.33 standard deviations), although the differences are not statistically significant (table S3.4 in the supplementary online appendix).8 However, as with the access studies, the difference in the number of general interventions and girl- targeted interventions is significant (fig. 5). This is even more the case in learning outcomes: there are 178 general learning interventions (from 106 studies) compared to only 14 girl-targeted learning inter- ventions (table 1). With just 15 girl-targeted interventions (from 12 studies), the distribution of effect sizes might be affected by outliers: in fact, the large effect size of the top girl-targeted intervention (at the 90th percentile) is purely driven by a school construction intervention (Kazianga et al. 2013), which was designed specifically to increase girls’ access to schools.9 When schools were built in villages in rural Burk- ina Faso, learning outcomes for girls dramatically improved. But taking out this intervention, the effect size of girl-targeted interventions at the 90th percentile drops to 0.10 standard deviations. Alternatively, if one drops the largest two interventions from the general interventions, the effect size changes hardly at all. These findings have two potential implications. The first is that while general and girl-targeted interven- tions perform similarly on average, there are more proven general interventions that deliver high impacts for girls’ learning than there are girl-targeted interventions. As a result, policy makers have more options 7 In additional analysis, the study compares both access and learning based on whether the studies are published or not. The study finds that for access outcomes, there is no statistically significant difference in the estimates between published papers and nonpublished papers. For learning outcomes, the point estimates for published general studies (0.13 SDs) are actually smaller than the estimates (0.24 SDs) for nonpublished papers, whereas published girl-targeted studies have slightly larger point estimates (0.11 SDs) than nonpublished studies (0.09 SDs). 8 The largest effect size for learning outcomes is a very large 2.56 SDs (Piper 2009), whereas the largest for girl-targeted interventions is 0.41 SDs (Kazianga et al. 2013). 9 The Burkina Faso program, evaluated in (Kazianga et al. 2013), includes girls explicitly in the name of the program: the Burkinabe Response to Improve Girls’ Chances to Succeed. 254 Evans and Yuan Figure 4. Effect Sizes of Learning Outcomes for Girls Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Source: Authors’ calculations. Note: A random-effects model was used to estimate the meta results for each group of studies. The error bars show the 95 percent confidence interval for the estimates. The error bars for girl-targeted outcomes in the primary only group are not included due to values larger than the display, but they are reported to the right of the figure. Country groups followed the World Bank country group definitions (World Bank 2020). * Difference in effect sizes between general studies and girl-targeted studies is significant at the 0.05 level, and exact p-values are reported in table S3.2 in the supplementary online appendix. to draw from among the general interventions. The second is that insofar as governments and other actors are experimenting with innovative girl-targeted interventions, there may be value in evaluating these to build the evidence base. For General Interventions, Do Impacts on Girls Tend to Be Larger? Previous research shows that the demand for girls’ schooling tends to be more responsive than that for boys’ to gender-neutral education policies (Glick 2008; J-PAL 2017). Similar point estimates are observed for girls and boys for both access and learning outcomes (fig. 6).10 Point estimates are slightly higher for girls so that – if anything – general interventions seem to be slightly more effective for girls than for boys, confirming previous work. Similar patterns are found when comparing the distribution of effect sizes (table S3.5 in the supplementary online appendix), with no significant differences between the distribu- tions (table S3.4 in the supplementary online appendix).11 10 Statistical significance is indicated with asterisks in fig. 6 and is reported fully in table S3.2 in the supplementary online appendix. 11 This study also examines whether impacts on girls are larger in places with low levels of initial performance, using the harmonized learning indicators from the World Bank’s Human Capital Index (World Bank 2018a) as well as various access indicators from the World Development Indicators (World Bank 2017). No relationship is found. The World Bank Economic Review 255 Figure 5. Number of Learning Outcomes by Effect Size Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Source: Authors’ calculations. Note: Effect sizes are in the unit of standard deviations. Figure 6. Effect Sizes for Girls and Boys (General Interventions Only) Source: Authors’ calculations. Note: A random-effects model was used to estimate the meta results for each group of studies. The error bars suggest the 95 percent confidence interval for the estimates. What are the Most Effective Interventions for Girls? To summarize the most effective interventions for girls, this section presents the 10 access and learning interventions with the largest effect sizes and seeks to understand their attributes. These are contrasted with the 10 least effective interventions in terms of access and learning outcomes. The effect sizes from all studies underlying the analysis are included in S4 in the supplementary online appendix. An alternative approach would be to carry out a formal meta-analysis: As the results demonstrate, there is a great deal of variation within categories of interventions (such as cash transfers), such that taking the average effect of a category is unlikely to yield meaningful insights.12 Because the study codes each estimate of the impact of an intervention separately, the fact that 1 estimate appears in the 10 most effective or least effective does not mean that all estimates of the impact of that intervention are among the most or least effective. 12 Analysis of previous meta-analyses of education interventions suggests that high heterogeneity within categories limits the predictive power of meta-analysis in education (Masset 2019). 256 Evans and Yuan Table 4. The 10 Most Effective Interventions to Improve Access to Education for Girls Evaluation Effect size Program description Country Region design Level of school Outcome (SD) 1 Conditional cash transfer South Africa SSA DID Primary and Enrollment 1.657 (Eyal, Woolard, and secondary Burns 2014) (ages 6–18) Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 2* Conditional cash transfer Malawi SSA RCT Primary Enrollment, Yr 2 1.536 for dropped out girls (Baird et al. 2016) 3 Conditional cash transfer Nicaragua LAC DID Primary Enrollment, Yr 1 0.883 (Maluccio, Murphy, and Regalia 2010) Enrollment, Yr 2 0.617 4* Hygiene promotion + Kenya SSA RCT Primary Enrollment 0.634 water treatment + sanitation + water supply (Garn et al. 2013) 5 Free secondary education Ghana SSA RCT Secondary Enrollment 0.542 (Duflo et al. 2017) 6 Malaria prevention Gambia SSA RCT Early Enrollment – 0.555 (Jukes et al. 2006) childhood cohort w/o (3–59 months) contamination Enrollment – 0.457 cohort w/minimal contamination 7 Conditional cash transfer: Nepal SA RCT Primary and Attendance rate 0.517 school stipend (Edmonds secondary and Shrestha 2014) (ages 10–16) 8 Labelled cash transfer for Morocco MENA RCT Primary Dropped out by 0.486 (abs. education (Benhassine the end of year 2 value) et al. 2015) 9 Village-based school Afghanistan SA RCT Primary Enrollment 0.478 (Burde and Linden 2013) 10* Providing subsidies to Pakistan SA RCT Primary Enrollment 0.441 private schools (Kim, Alderman, and Orazem 1999) Source: Authors’ calculations based on cited works. * Girl-targeted interventions. Access The 10 studies that report the largest impacts in improving access to education for girls report greatly im- proved girls’ participation in school, with an average effect size of 0.73 standard deviation (table 4). Three of the 10 are girl-targeted interventions, including cash transfers to girls who had previously dropped out of school – conditional on school attendance in Malawi (Baird et al. 2016), improving school water and sanitation systems in Kenya (Garn et al. 2013), and providing private school subsidies for girls in Pak- istan (Kim, Alderman, and Orazem 1999). Six of the general interventions are similarly related to offering cash for education in different countries (Maluccio, Murphy, and Regalia 2010; Edmonds and Shrestha 2014; Eyal, Woolard, and Burns 2014; Benhassine et al. 2015; Duflo, Dupas, and Kremer 2017), building village schools in Afghanistan (Burde and Linden 2013) and another intervention is focused on malaria prevention in The Gambia (Jukes et al. 2006). Altogether, 6 of the 10 involve cash transfers, and one more – subsidies in Pakistan – similarly involves reducing the cost of schooling. The World Bank Economic Review 257 Table 5. The 10 Least Effective Interventions to Improve Access to Education for Girls Evaluation Effect size Program description Country Region design Level of school Outcome (SD) 1 School canteen Burkina SSA RCT Primary and Absenteeism –0.200 Faso secondary (ages 6–15) Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 * Conditional take-home Absenteeism –0.182 rations for girls (Kazianga et al. 2012) 2* Unconditional cash Malawi SSA RCT Primary Attendance, Yr2 –0.152 transfer to girls (Baird, McIntosh, and Özler 2011) 3 Early financial China EAP RCT Secondary Dropout –0.1401 commitment (Yi et al. 2015) 4* Hygiene promotion + Kenya SSA RCT Primary Enrollment –0.138 water treatment (Garn et al. 2013) 5 School meal (Afridi 2011) India SA DID Primary Enrollment –0.120 6 Education cash saving Uganda SSA RCT Primary Enrollment Y1 –0.110 account with parent outreach (Karlan and Attendance overall –0.107 Leiden 2014) Enrollment Y2 –0.107 Education cash saving Enrollment Y1 –0.049 account without parent outreach (Karlan and Leiden 2014) 7 Conditional cash transfer Philippines EAP RCT Early child- Enrolled in school –0.098 (Chaudhury, Friedman, hood + pri- 15–17 yrs and Onishi 2013) mary (ages 0–14) 8* Sanitary products (Oster Nepal SA RCT Secondary Attendance –0.083 and Thornton 2011) 9 Conditional cash transfer Uruguay LAC DID Secondary School attendance, –0.056 (Amarante, Ferrando, and 18-month follow up Vigorito 2013) School attendance, –0.047 30-month follow up 10 Unconditional Cash South Africa SSA DID Early Attendance –0.043 Transfer (Santana 2008) childhood + primary (ages 0–13) Source: Authors’ calculations based on cited works. * Girl-targeted interventions. 1 Adjusted negative value for comparison. These top interventions demonstrate that reducing the cost of schooling is likely the single most effec- tive way to bring girls into school. Most of these are conditional cash transfers, although fewer uncondi- tional transfers have been tested. In addition, reducing indirect costs – such as the commuting distance to school for girls by building village schools – has been effective in increasing access. Note, however, that one unconditional cash transfer – without a schooling condition – is among the less effective interventions (table 5) (Baird, McIntosh, and Özler 2011). Improving health conditions through either better sanitation facilities or controlling malaria tends to attract more girls to school as well. 258 Evans and Yuan There are concerns about the effectiveness of conditional cash transfer programs if only considering the most effective interventions. One of them is that the popularity of conditional cash transfers has led to an emergence of impact evaluations in this field, which might lead this class of interventions to be overrepresented in the evidence base. Cash transfer interventions could be among both the most effective and the least effective interventions. To test this, this study summarizes the bottom 10 interventions to increase access for girls in table 5. There are three transfer programs – conditional, cash, or in-kind – that Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 were particularly ineffective in bringing girls into school, such as those in Burkina Faso (Kazianga, De Walque, and Alderman 2012), the Philippines (Chaudhury, Friedman, and Onishi 2013), and Uruguay (Amarante, Ferrando, and Vigorito 2013), but transfer programs represent far more of the most effective than the least effective programs. There is more variation in the least effective programs, ranging from providing school meals to targeted savings accounts for education. Interestingly, it is seen that within the same study (Garn et al. 2013), while promoting hygiene, improving water treatment, improved sanitation, and safe water storage in Kenyan primary schools is one of the best ways to increase girls’ enrollment, promoting hygiene and improving water storage alone actually reduced enrollment for girls. Although it is likely that girls are more responsive to sanitation conditions, different environments face different challenges: in Nepal providing sanitary products did not increase girls’ school attendance, likely in part because very few girls reported missing school due to a lack of sanitary products (Oster and Thornton 2011). Learning For learning, the average effect size of the top interventions for girls is 0.98 standard deviation (table 6). Compared to access interventions, there is more variation in the design of learning interventions. First, only 2 in 10 studies are girl-targeted interventions. One of the two girl-targeted interventions is a public-private partnership initiative in schools in Pakistan providing a gender-differentiated subsidy that increased girls’ test score by 0.77 standard deviation. The other intervention arm in the same initiative provided a gender- neutral subsidy and also yielded sizeable effects, albeit smaller than the gender-differentiated one (Barrera- Osorio et al. 2017). The other top-10 girl-targeted intervention is the Afghan village school program for girls that delivered significant impacts on girls’ access and learning outcomes (Burde and Linden 2013). A general (nontargeted) community school program in Honduras greatly improved girls’ math score (Di Gropello and Marshall 2011). Several of the most effective general interventions for girls among the top 10 involve structured peda- gogy in early grades, or providing teachers with clear guidance on teaching or even scripted lesson plans. These interventions have been shown to be highly effective in several Sub-Saharan African countries in- cluding South Africa, Liberia, and Kenya (Piper 2009; Piper and Korda 2010; Piper and Mugenda 2014; Piper, Zuilkowski, and Ong’ele 2016). Another category of interventions that work well for girls (and boys) are those that help teachers to teach children at their current level of learning (e.g., teaching at the right level), either through diagnostic feedback or software as reported in Banerjee et al. (2016) and Imbrogno (2014). On the other hand, the least effective programs for girls’ learning are all general interventions (table 7). Various interventions actually had negative impacts on learning for girls compared to “schooling-as-usual,” but often, those same programs did not work for boys either. For example, technology interventions – whether substituting teachers with computers or providing students with laptops – did not help improve learning (Linden 2008; Sharma 2014). Although there are teacher professional development programs that work to improve student learning (Popova et al. forthcoming), the present study’s findings demonstrate that introducing new pedagogical methods through a short teacher training program is less likely to be effective to improve girls’ learning; and this is true no matter which education level the intervention targets (Yoshikawa et al. 2015; Berlinski and Busso 2017). In The World Bank Economic Review 259 Table 6. The 10 Most Effective Interventions to Improve Learning for Girls Evaluation Level of Effect size Program description Country Region design school Outcome (SD) 1 Literacy intervention (Piper South Africa SSA DID Primary Letter sounding 2.563 2009) fluency Word naming fluency 1.840 Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Reading 1.757 comprehension Oral reading fluency 1.658 2 Mother tongue instruction Kenya SSA RCT Primary Reading 1.360 (Piper, Zuilkowski, and comprehension, Ong’ele 2016) Lubukusu, class 1 Reading 1.250 comprehension, Kikamba, class 2 3 TaRL 10-day Camp1 India SA RCT Primary Language 1.050 (Banerjee et al. 2016) TaRL 10-day Camp Math 0.870 (Banerjee et al. 2016) TaRL 20-day Camp Language 0.830 (Banerjee et al. 2016) TaRL 20-day Camp Math 0.730 (Banerjee et al. 2016) 4 Structured pedagogy (Piper Liberia SSA DID Primary Listening 1.030 and Korda 2010) comprehension Reading 0.830 comprehension Unfamiliar word 0.780 fluency Letter-naming fluency 0.680 Oral reading fluency 0.680 5 Primary literacy Kenya SSA RCT Primary English letter sound 0.845 intervention (Piper and Mugenda 2014) English segmenting 0.690 Kiswahili letter sound 0.646 * 6 PPP gender subsidy Pakistan SA RCT Primary Test score 0.770 (Barrera-Osorio et al. 2017) PPP subsidy pooled Test score 0.661 (Barrera-Osorio et al. 2017) PPP uniform subsidy Test score 0.655 (Barrera-Osorio et al. 2017) 7* Village-based schools Afghanistan SA RCT Primary Test score–2nd 0.661 (Burde and Linden 2013) semester Test score–1st 0.654 semester 8 Math tutor software Mexico LAC RCT Secondary Test scores 0.660 (Imbrogno 2014) 9 Community school Honduras LAC IV Primary Math 0.630 program (Di Gropello and Marshall 2011) 10 Math tutor software Chile LAC RCT Secondary Test scores 0.611 (Imbrogno 2014) Source: Authors’ calculations based on cited works. * Girl-targeted interventions. 1 TaRL: Teaching at the Right Level. 260 Evans and Yuan Table 7. The 10 Least Effective Interventions to Improve Learning for Girls Evaluation Level of Effect size Program description Country Region design school Outcome (SD) 1 Computer-assisted India SA RCT Primary Math and English –0.613 learning in school (Linden 2008) Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 2 School management Madagascar SSA RCT Primary Test score –0.403 (Glewwe and Maïga (district-level 2011) intervention) 3 New curriculum + OLPC Costa Rica LAC RCT Secondary Math-geometry –0.378 (Berlinski and Busso 2017) New curriculum + Math-geometry –0.216 computer lab (Berlinski and Busso 2017) New curriculum Math-geometry –0.142 (Berlinski and Busso 2017) New curriculum + white Math-geometry –0.136 board (Berlinski and Busso 2017) 4 Teacher training Chile LAC RCT Pre-school Vocabulary –0.305 (Yoshikawa et al. 2015) 5 One laptop per Child Nepal SA DID Primary English –0.244 (Sharma 2014) 6 Mobile school librarian India SA RCT Primary Language –0.232 (Borkum, He, and Linden 2012) 7 Preschool voucher (Wong China EAP RCT Pre-school Test score – school –0.223 et al. 2013) readiness 8 School report card Sri Lanka SA DID Secondary Science 8th grade –0.221 (Aturupane et al. 2014) Math 8th grade –0.215 9 Attendance reward India SA RCT Primary Test score –0.207 (Visaria et al. 2016) 10 Teacher training (Özler Malawi SSA RCT Primary Early Grade Math, –0.124 et al. 2016) 36-month follow-up Source: Authors’ calculations based on cited works. addition, school accountability interventions such as distributing school report cards to students and parents were not effective for girls in Sri Lanka (Piper and Korda 2010; Aturupane et al. 2014). 4. Discussion Inequality Up until this point, this paper has focused on identifying the interventions that deliver the highest absolute learning gains for girls. An alternative approach would be to identify those programs that benefit girls most relative to boys. In other words, this approach would focus on closing inequalities (or increasing them, in contexts where girls are ahead in school) rather than merely improving girls’ access and learning without regard to boys’ performance. Figure 7 shows the gains in access and learning for boys versus those for girls. The programs with the most unequal impacts – both favoring girls and favoring boys – are general interventions. Almost all of the girl-targeted interventions for which data are available on both girls and The World Bank Economic Review 261 Figure 7. Relative Effect Sizes for Boys versus Girls Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Source: Authors’ calculations. Note: Grey points represent effect sizes of general interventions, black dots represent effects sizes of girl-targeted interventions, and the dashed line represents same effect sizes for girls and boys. For access outcomes, 171 effect sizes of general interventions and 12 (out of 34) effect sizes of girl-targeted interventions are plotted; for learning outcomes, 423 effect sizes of general interventions and 5 (out of 20) effect sizes of girl-targeted interventions are plotted. The missing girl-targeted interventions are those that did not report outcomes for boys. boys have similar results for both genders, with slightly better results for girls: 12 in 18 girl-targeted studies do not report outcomes for boys. If one’s objective were purely inequality reduction, then cash transfers in South Africa had dramatically larger access impacts on girls than on boys, despite not being gender- targeted (Eyal et al. 2014). A mother tongue learning instruction in Kenya in the Lubukusu language had no discernible impact on boys’ learning but a sizeable impact for girls (Piper, Zuilkowski, and Ong’ele 2016). However, there are no clear patterns as to which classes of interventions are inequality-enhancing versus inequality-reducing. For this inequality analysis, the project drops girl-targeted studies that do not report impacts for boys. If one were to assume that those girl-targeted studies that did not report outcomes for boys had zero impact on boys, then girl-targeted interventions would decrease inequality more than general interventions. Costs While the study standardized effect sizes across interventions in this review, incorporating cost data would enhance the analysis, as the most effective programs may not be the most cost-effective and there- fore not easy to scale up. However, despite a strong demand for cost data, few studies report them. McEwan (2015) in his review stated that 56 percent of studies reported no cost details, and most of the rest reported minimal information. The present study encountered similar problems when it tried to collect cost data. In addition, even when cost data are reported, they are often not comparable due to different accounting methods. Taking an early childhood development program in rural Mexico as an example, the cost per child estimated by World Bank researchers was $76 (Cárdenas, Evans, and Holland 2015), but when evaluated by another group of researchers at Brookings, the cost per child 262 Evans and Yuan almost doubled to $174–$202 (Gustafsson-Wright, Boggild-Jones, and Gardiner 2017). Ideally, a sep- arate initiative would collect cost data following a standard set of guidelines such as those laid out in Dhaliwal et al. (2013). This review finds that general interventions are often comparable in impact to girl-targeted interven- tions in improving access to school and learning once at school. But if a policy maker’s primary concern is improving girls’ education, then perhaps investing in girl-targeted interventions would allow similar Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 gains at much lower cost – that is, just paying for the girls rather than girls and boys. This argument plays out differently for access versus learning interventions. For access interventions such as cash transfers, the cost could indeed be potentially reduced by targeting only girls. Indeed, several of the most effective general interventions were cash transfer programs that happened to not target girls specifically. One could imagine replacing those programs with girl-only cash transfer programs and potentially achieving similar gains. For learning interventions, such as structured pedagogy interventions, many are introduced at the level of the school, so that in mixed-gender schools, there is no clear cost gain to trying to limit the impact to girls only. Program Attributes This study gathered data on a number of program attributes with the aim to provide more information on the most effective programs. For example, the average program size of the most effective access inter- ventions is 262 students, and for learning interventions it is 556 students. With the exception of the cash transfer programs, all others among the 10 most and least effective programs are pilot programs. This is a result of the fact that most interventions that are carefully evaluated tend to be pilots. Therefore, it is not possible from this sample to infer whether or not pilot programs are more effective than those that have been implemented at scale. Another attribute that was examined was the level of education that the top programs targeted. In terms of access interventions, 7 of the top 10 interventions targeted school-aged chil- dren in general, often between age 6 up to age 16 – working through the household rather than the school, trying to get out-of-school children into school. For learning interventions, 9 in 10 focused on the primary level, and half of them were designed to improve learning in grades 1–3. There is great interest in programs for adolescent girls, but many of those programs focus on building life skills and increasing earning capac- ity directly (see, for example, Adoho et al. 2014; Bandiera et al. 2018; Bandiera et al. 2019) rather than keeping girls in school and increasing their learning ability. In many low- and middle-income countries, children and youth can still significantly improve their literacy and numeracy all through primary and sec- ondary school (Evans and Yuan 2019), and so there will be great value in continuing to evaluate programs and increase learning and access for adolescents. The study also examined if authors included any gender component in their interventions. It found that besides girl-targeted programs, only 1 general intervention in the top 20 had a girl-friendly component, which was to provide gender-differentiated school subsidies (Barrera-Osorio et al. 2017). What Has Been Studied A key limitation of this work is that it only surveys those interventions that have been evaluated. One can imagine a wide array of girl-targeted interventions that could still be tried or that have been tried but not yet rigorously evaluated. In the context of strict budget constraints, having clear data on the best in- vestments among those interventions that have been evaluated can be useful, and it can help governments and other education stakeholders to avoid investing in programs that have proven ineffective. However, it should not stop policy makers from continuing to innovate and test new programs that relax constraints on girls’ access and learning. Another, related limitation is that there are far fewer girl-targeted interven- tions in this study’s sample. As researchers continue to test girl-targeted interventions, this evidence will continue to evolve. The World Bank Economic Review 263 When Should Programs Target Girls’ Education? Children in low- and middle-income countries face a wide range of constraints to their education. Psaki et al. (forthcoming) group these constraints into three categories. The first category of constraints almost exclusively affects girls, such as adolescent pregnancy, child marriage, or social norms that devalue girls’ education. The second category affects both boys and girls but may disproportionately affect girls’ edu- cation because of inequitable gender norms, such as the cost of schooling or a lack of access to schools. Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 The third category affects both boys and girls (although of course there may still be differences), such as low-quality pedagogy in schools. This study’s finding that nontargeted interventions can be as effec- tive as targeted interventions points to the fact that in many contexts, the third category of constraints may be binding. However, contexts vary, and whether the right intervention is targeted or non-targeted – or a combination of the two – depends on the collection of constraints that children face in a given setting. 5. Conclusion Previous reviews of what works to improve girls’ education tend to focus on girl-targeted interventions. That approach omits key evidence of the impact of general education interventions on girls. This review brings together a large evidence base of general interventions that report effects for girls and collects additional estimates for a sample of studies that did not report effects for girls. Based on 175 studies from 54 countries, this review finds that girls’ access to school is more responsive to changes in costs, distance, and health conditions; while girls’ learning is more likely to be improved by structured pedagogy and interventions that help teachers to teach at the right level. While this review focuses on girls’ education, the global learning crisis impoverishes both girls and boys (World Bank 2018b). This study’s findings demonstrate that gender-neutral interventions hold great promise for girls’ learning as well as for boys. Considering the limited resources that education systems in most low- and middle-income countries possess, the most practical approach to help girls learn may be to make schools better for all children. Such an approach may also be more politically palatable to voters – who have sons as well as daughters – than programs that restrict their benefits to girls. At the same time, this approach comes at a cost: interventions that involve per-pupil transfers will obviously be cheaper if targeted. Finally, attending school and acquiring learning are not the finish line for girls’ education. The ul- timate objective is that girls can empower themselves through education and achieve their life aspira- tions. To this point, very few evaluations have included either long-term follow-ups or these broader measures of well-being. But gaining literacy and numeracy are the foundation for positive longer-term outcomes. References Adoho, Franck, Shubha Chakravarty, Jr, Dala T. Korkoyah, Mattias K. A. Lundberg, and Afia Tasneem. 2014. “The Impact of an Adolescent Girls Employment Program. The EPAG Project in Liberia.” Policy Research Working Paper, The World Bank. Washington, DC, USA. Afridi, Farzana. 2011. “The Impact of School Meals on School Participation: Evidence from Rural India.” Journal of Development Studies 47 (11): 1636–56. Amarante, Verónica, Mery Ferrando, and Andrea Vigorito. 2013. “Teenage School Attendance and Cash Transfers: An Impact Evaluation of PANES.” Economía 14: 61–96. Asim, Salman, Robert S., Chase, Amit Dar, and Achim Schmillen. 2017. “Improving Learning Outcomes in South Asia: Findings from a Decade of Impact Evaluations.” World Bank Research Observer 32 (1): 75–106. Aturupane, Harsha, Paul Glewwe, Renato Ravina, Upul Sonnadara, and Suzanne Wisniewski. 2014. “An Assessment of the Impacts of Sri Lanka’s Programme for School Improvement and School Report Card Programme on Students’ Academic Progress.” Journal of Development Studies 50: 1647–69. 264 Evans and Yuan Baird, Sarah J., Ephraim Chirwa, Jacobus De Hoop, and Berk Özler. 2016. “Girl Power: Cash Transfers and Adolescent Welfare. Evidence from a Cluster-Randomized Experiment in Malawi.” In African Successes, Volume 2: Human Capital, edited by Sebastian Edwards, Simon Johnson and David N. Weil, 139–64. Chicago: University of Chicago Press. Baird, Sarah, Craig McIntosh, and Berk Özler. 2011. “Cash or Condition? Evidence from a Cash Transfer Experi- ment.” Quarterly Journal of Economics 126(4): 1709–53. Bandiera, Oriana, Niklas Buehren, Robin Burgess, Markus Goldstein, Selim Gulesci, Imran Rasul, and Munshi Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Sulaiman. 2019. “Women’s Empowerment in Action: Evidence from a Randomized Control Trial in Africa.” Amer- ican Economic Journal: Applied Economics Forthcoming. Bandiera, Oriana, Niklas Buehren, Markus Goldstein, Imran Rasul, and Andrea Smurra. 2018. “The Economic Lives of Young Women in the Time of Ebola: Lessons from an Empowerment Program.” Unpublished Manuscript. Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukerji, Marc Shotland, and Michael Walton. 2016. “Mainstreaming an Effective Intervention: Evidence form Randomized Evaluations of “Teaching at the Right Level” in India.” Working Paper No. 22746. National Bureau of Economic Research. Cambridge, MA, USA. Barrera-Osorio, Felipe, David S. Blakeslee, Matthew Hoover, Leigh L. Linden, Dhushyanth Raju, and Stephen P. Ryan. 2017. “Delivering Education to the Underserved through a Public-Private Partnership Program in Pakistan.” Policy Research Working Paper No. 8177. World Bank. Washington, DC, USA. Benhassine, Najy, Florencia Devoto, Esther Duflo, Pascaline Dupas, and Victor Pouliquen. 2015. “Turning a Shove into a Nudge? A“Labeled Cash Transfer” for Education.” American Economic Journal: Economic Policy 7 (3): 86–125. Berlinski, Samuel, Matias Busso, Taryn Dinkelman, and Claudia Martinez. 2016. “Reducing Parent-School Informa- tion Gaps and Improving Education Outcomes: Evidence from High Frequency Text Messaging in Chile.” Unpub- lished Manuscript. Berlinski, Samuel, and Matias Busso. 2017. “Challenges in Educational Reform: An Experiment on Active Learning in Mathematics.” Economics Letters 156: 172–75. Borenstein, Michael et al. 2009. “Effect Sizes for Continuous Data.” The Handbook of Research Synthesis and Meta- Analysis. 2nd ed. Edited by Harris Cooper, V. Hedges and Jeffrey C. Valentine, 221–35. New York: Russell Sage Foundation. Borkum, Evan, Fang He, and Leigh L. Linden. 2012. “ The Effects of School Libraries on Language Skills: Evidence from a Randomized Controlled Trial in India.” No. w18183. National Bureau of Economic Research. Burde, Dana, and Leigh L. Linden. 2013. “Bringing Education to Afghan Girls: A Randomized Controlled Trial of Village-Based Schools.” American Economic Journal: Applied Economics 5 (3): 27–40. Cárdenas, Sergio, David K. Evans, and Peter Holland. 2015. “Early Childhood Benefits at Low Cost: Evidence from a Randomized Trial in Mexico.” Unpublished Manuscript. Chaudhury, Nazmul, Jed Friedman, and Junko Onishi. 2013. “ Philippines Conditional Cash Transfer Program Impact Evaluation 2012.” World Bank Report. Manila, The Philippines. Conn, Katharine M. 2017. “Identifying Effective Education Interventions in Sub-Saharan Africa: A Meta-Analysis of Impact Evaluations.” Review of Educational Research 87 (5): 863–98. Dhaliwal, Iqbal, Esther Duflo, Rachel Glennerster, and Caitlin Tulloch. 2013. “Comparative Cost-Effectiveness Analy- sis to Inform Policy in Developing Countries: A General Framework with Applications for Education.” In Education Policy in Developing Countries. Edited by Paul Glewwe, 285–338. Chicago: University of Chicago Press. Di Gropello, Emanuela, and Jeffery H. Marshall. 2011. “Decentralization and Educational Performance: Ev- idence from the PROHECO Community School Program in Rural Honduras.” Education Economics 19: 161–80. Duflo, Esther, Pascaline Dupas, and Michael Kremer. 2017. “The Impact of Free Secondary Education: Experimental Evidence from Ghana.” Working Paper. Massachusetts Institute of Technology. Cambridge, MA, USA. Edmonds, Eric V., and Maheshwor Shrestha. 2014. “You Get What You Pay For: Schooling Incentives and Child Labor.” Journal of Development Economics 111: 196–211. Evans, D. K., M. Akmal, and P. Jakiela. 2021. “Gender Gaps in Education: The Long View.” IZA Journal of Devel- opment and Migration 12 (1). The World Bank Economic Review 265 Evans, David K., and Anna Popova. 2016. “What Really Works to Improve Learning in Developing Countries? An Analysis of Divergent Findings in Systematic Reviews.” World Bank Research Observer 31: 242–70. Evans, David K., and Fei Yuan. 2019. “Economic Returns to Interventions that Increase Learning.” Policy Research Working Paper. The World Bank. Washington, DC, USA. Eyal, Katherine, Ingrid Woolard, and Justine Burns. 2014. “Cash Transfers and Teen Education: Evidence from South Africa.” School of Economics, University of Capetown. Unpublished manuscript. Ganimian, Alejandro J., and Richard J. Murnane. 2016. “Improving Education in Developing Countries: Lessons from Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Rigorous Impact Evaluations.” Review of Educational Research 86 (3): 719–55. Garn, Joshua V., Leslie E. Greene, Robert Dreibelbis, Shadi Saboori, Richard D. Rheingans, and Matthew C. Freeman. 2013. “A Cluster-Randomized Trial Assessing the Impact of School Water, Sanitation and Hygiene Improvements on Pupil Enrolment and Gender Parity in Enrolment.” Journal of Water Sanitation and Hygiene for Development 3: 592–601. Glewwe, Paul, and Eugenie W. H. Maïga. 2011. “The Impacts of School Management Reforms in Madagascar: Do the Impacts Vary by Teacher Type?” Journal of Development Effectiveness 3 (4): 435–69. Glewwe, P., Maïga E., and Zheng H. 2014. “The Contribution of Education to Economic Growth: A Review of the Evidence, with Special Attention and an Application to Sub-Saharan Africa.” World Development 59: 379–93. Glewwe, Paul, and Karthik Muralidharan. 2016. “Improving Education Outcomes in Developing Countries: Evidence, Knowledge Gaps, and Policy Implications.” Handbook of the Economics of Education. Vol. 5. Eric A. Hanushek, Stephen J. Machin and Ludger Woessmann, 653–743. North Holland: Elsevier. Glick, Peter. 2008. “What Policies Will Reduce Gender Schooling Gaps in Developing Countries: Evidence and Inter- pretation.” World Development 36: 1623–46. Gustafsson-Wright, Emily, Izzy Boggild-Jones, and Sophie Gardiner. 2017. “SECT: The Standardized Early Childhood Development Costing Tool.” Center for Universal Education at Brookings. Washington, DC, USA. Haberland, Nicole A., Katharine J. McCarthy, and Martha Brady. 2018. “A Systematic Review of Adolescent Girl Pro- gram Implementation in Low-and Middle-Income Countries: Evidence Gaps and Insights.” Journal of Adolescent Health 63 (1): 18–31. Imbrogno, Jason. 2014. “Essays on the Economics of Education.” Doctoral dissertation. Carnegie Mellon University. Pittsburgh, Pennsylvania, USA. J-PAL. 2017. “Roll Call: Getting Children into School.” J-PAL Policy Bulletin. Jukes, Matthew C. H., Margaret Pinder, Elena L. Grigorenko, Helen Baños Smith, Gijs Walraven, Elisa Meier Bariau, Robert J. Sternberg, Lesley J. Drake, Paul Milligan, and Yin Bun Cheung. 2006. “Long-Term Impact of Malaria Chemoprophylaxis on Cognitive Abilities and Educational Attainment: Follow-Up of a Controlled Trial.” PLoS Clinical Trials 1: e19. Karlan, Dean, and Leigh L. Linden. 2014. “Loose Knots: Strong versus Weak Commitments to Save for Education in Uganda.” Columbia University. Working Paper 19863. National Bureau of Economic Research. Cambridge, MA, USA. Kazianga, Harounan, Damien De Walque, and Harold Alderman. 2012. “Educational and Child Labour Impacts of Two Food-for-Education Schemes: Evidence from a Randomised Trial in Rural Burkina Faso.” Journal of African Economies 21 (5): 723–60. Kazianga, Harounan, Dan Levy, Leigh L. Linden, and Matt Sloan. 2013. “The Effects of ‘Girl-Friendly’ Schools: Evidence from the BRIGHT School Construction Program in Burkina Faso.” American Economic Journal: Applied Economics 5: 41–62. Kim, Jooseop, Harold Alderman, and Peter F Orazem. 1999. “Can Private School Subsidies Increase Enrollment for the Poor? The Quetta Urban Fellowship Program.” World Bank Economic Review 13: 443–65. Kremer, Michael, Conner Brannen, and Rachel Glennerster. 2013. “The Challenge of Education and Learning in the Developing World.” Science 340 (6130): 297–300. Krishnaratne, Shari, and Howard White. 2013. “Quality Education for All Children? What Works in Education in De- veloping Countries.” 3ie Publications No. 0000-0. International Initiative for Impact Evaluation (3ie). Washington, DC, USA; New Delhi, India. Linden, Leigh L. 2008. “Complement or Substitute? The Effect of Technology on Student Achievement in India.” Working Paper, Columbia University. 266 Evans and Yuan Maluccio, John A., Alexis Murphy, and Ferdinando Regalia. 2010. “Does Supply Matter? Initial Schooling Conditions and the Effectiveness of Conditional Cash Transfers for Grade Progression in Nicaragua.” Journal of Development Effectiveness 2: 87–116. Masino, Serena, and Miguel Niño-Zarazúa. 2016. “What Works to Improve the Quality of Student Learning in Developing Countries?” International Journal of Educational Development 48: 53–65. Masset, Edoardo. 2019. “Impossible Generalisations: Meta-Analyses of Education Interventions in International Development.” RISE Annual Conference 2019. Washington, DC, USA. June 19–20, 2019. Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 McEwan, Patrick J. 2015. “Improving Learning in Primary Schools of Developing Countries: A Meta-Analysis of Randomized Experiments.” Review of Educational Research 85: 353–94. Oster, Emily, and Rebecca Thornton. 2011. “Menstruation, Sanitary Products, and School Attendance: Evidence from a Randomized Evaluation.” American Economic Journal: Applied Economics 3: 91–100. Özler, Berk, Lia C.H. Fernald, Patricia Kariger, Christin McConnell, Michelle Neuman, and Eduardo Fraga. 2016. “ Combining Preschool Teacher Training with Parenting Education: A Cluster-Randomized Controlled Trial.” Work- ing Paper. The World Bank. Washington, DC, USA. Piper, Benjamin. 2009. “ Integrated Education Program: Impact Study of SMRS Using Early Grade Reading Assess- ment in Three Provinces in South Africa.” RTI International. Research Triangle Park, NC. Piper, Benjamin, and Medina Korda. 2010. “ Early Grade Reading Assessment (EGRA) Plus: Liberia. Program Eval- uation Report.” RTI International. Research Triangle Park, NC. Piper, Benjamin, and A. Mugenda. 2014. The Primary Math and Reading (PRIMR) Initiative: Endline Impact Eval- uation. RTI International, Research Triangle Park, NC. Piper, Benjamin, Stephanie S. Zuilkowski, and Salome Ong’ele. 2016. “Implementing Mother Tongue Instruction in the Real World: Results from a Medium-Scale Randomized Controlled Trial in Kenya.” Comparative Education Review 60: 776–807. Popova, Anna, David K. Evans, Mary E. Breeding, and Violeta Arancibia. Forthcoming. “Teacher Professional Devel- opment around the World: The Gap between Evidence and Practice.” World Bank Research Observer. Psaki, Stephanie, Nicole Haberland, Barbara Mensch, Erica Chuang, and Lauren Woyczynski. Forthcoming. “Policies and Interventions to Remove Gender-Related Barriers to Girls’ School Participation and Learning in Low- and Middle-Income Countries: A Systematic Review of the Evidence.” Campell Systematic Reviews. Santana, Maria Isabel. 2008. “An Evaluation of the Impact of South Africa’s Child Support Grant on School Atten- dance.” Centro de Estudios Distributivos, Laborales y Sociales, Universidad Nacional de La Plata, Argentina. Sharma, Uttam. 2014. “Can Computers Increase Human Capital in Developing Countries? An Evaluation of Nepal’s One Laptop per Child Program.” Annual Meeting of the Agricultural and Applied Economics Association, Min- neapolis MN, July 27–29. Snilstveit, Birte, Jennifer Stevenson, Daniel Phillips, Martina Vojtkova, Emma Gallagher, Tanja Schmidt, Han- nah Jobse, Maisie Geelen, Maria Grazia Pastorello, and John Eyers. 2017. “Interventions for Improv- ing Learning Outcomes and Access to Education in Low- and Middle-Income Countries: A Systematic Review.” Campbell Systematic Reviews 13 (1): 1–82. Sperling, Gene B, and Rebecca Winthrop. 2015. What Works in Girls’ Education: Evidence for the World’s Best Investment. Washington, DC: Brookings Institution Press. Unterhalter, Elaine, Amy North, Madeleine Arnot, Cynthia Lloyd, Lebo Moletsane, Erin Murphy-Graham, Jenny Parkes, and Mioko Saito. 2014. “Interventions to Enhance Girls’ Education and Gender Equality.” Education Rigorous Literature Review. Department for International Development. Visaria, Sujata, Rajeev Dehejia, Melody M. Chao, and Anirban Mukhopadhyay. 2016.“Unintended Consequences of Rewards for Student Attendance: Results from a Field Experiment in Indian Classrooms.” Economics of Education Review 54: 173–84. Wong, Ho Lun, Renfu Luo, Linxiu Zhang, and Scott Rozelle. 2013. “The Impact of Vouchers on Preschool Atten- dance and Elementary School Readiness: A Randomized Controlled Trial in Rural China.” Economics of Education Review 35: 53–65. World Bank. 2017. “World Development Indicators.” ———. 2018a. “Human Capital Project.” World Bank, Washington, DC, USA. ———. 2018b. “ World Development Report 2018: Learning to Realize Education’s Promise.” World Bank, Wash- ington, DC, USA. The World Bank Economic Review 267 ———. 2020. “World Bank Country and Lending Groups.” World Bank, Washington, DC, USA. Yi, Hongmei, Yingquan Song, Chengfang Liu, Xiaoting Huang, Linxiu Zhang, Yunli Bai, Baoping Ren, Yaojiang Shi, Prashant Loyalka, James Chu, and Scott Rozelle. “Giving Kids a Head Start: The Impact and Mechanisms of Early Commitment of Financial Aid on Poor Students in Rural China.” Journal of Development Economics 113 (2015): 1–15. Yoshikawa, Hirokazu, Diana Leyva, Catherine E. Snow, Ernesto Treviño, M. Barata, Christina Weiland, Celia J. Gomez, Lorenzo Moreno, Andrea Rolla, and Nikhit D’Sa. 2015. “Experimental Impacts of a Teacher Professional Downloaded from https://academic.oup.com/wber/article/36/1/244/6278419 by LEGVP Law Library user on 08 December 2023 Development Program in Chile on Preschool Classroom Quality and Child Outcomes.” Developmental Psychology 51: 309–22.