Policy Research Working Paper 10420 The Longer Students Were Out of School, the Less They Learned Harry Anthony Patrinos Europe and Central Asia Region Office of the Chief Economist April 2023 Policy Research Working Paper 10420 Abstract COVID-19 led to school closures and emergency remote vaccination—and causally estimate the impact of duration learning systems. It is feared that students learned less when on learning. It is estimated that for every week that schools they were remote. This paper analyzes school closures during were closed, learning levels declined by almost 1 percent of the pandemic using a unique data base. The determinants a standard deviation. This means that a 20 week closure, for of the duration of school closures estimates were used to example, would reduce learning outcomes by 0.20 standard instrument school closures—stringency of lockdown and deviation, almost one year of schooling. This paper is a product of the Office of the Chief Economist, Europe and Central Asia Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The author may be contacted at hpatrinos@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team The Longer Students Were Out of School, the Less They Learned Harry Anthony Patrinos1 World Bank, 1818 H Street N.W., Washington DC, 20433, USA; hpatrinos@worldbank.org JEL classification: I10; I31; J13 Keywords: School closures; COVID-19 pandemic; learning loss 1 This is a working paper, and hence it represents research in progress. This paper represents the opinions of the author and should not be attributed to the World Bank Group. Any errors are the fault of the author. I benefited from discussions at seminars at Wesleyan University, University of Mississippi, University of Maryland, Carleton University, University of Arkansas, GLO conference, ICFES, Global Schools Forum and FHi360. I received excellent comments from George Psacharopoulos, Xiaoxue Zhao, Chris Sakellariou, Vicente Garcia Moreno, Kevin Macdonald, Gustavo Arcia, David Marshall, Karthika Radhakrishnan-Nair, and Noah Yarrow. Introduction COVID-19 school closures led to emergency remote-learning systems almost immediately. At its peak, nearly 1.6 billion learners in more than 190 countries, or 94 percent of the world’s student population, were impacted by school closures (UNESCO 2020). Students learned less when they were remote and attending high-poverty schools, which were hit hardest (Engzell et al. 2021). Learning losses could cost this generation of students close to $15 trillion in lifetime earnings (Psacharopoulos et al. 2021). Worldwide, learning losses on average amount to 0.17 of a standard deviation (Patrinos et al. 2023), equivalent to roughly a one-half year’s worth of learning. I estimate the impact of the duration of school closures on learning. The longer the duration of the closures, then the greater the losses. For countries with robust learning loss data, average school closures were 21 weeks, leading to average losses of 0.23 standard deviation, almost a whole year’s worth of learning. Each week of closures increases learning loss by 1 percent of a standard deviation. This will increase the education gaps between high and low socioeconomic status students (Agostinelli et al. 2022), especially in lower-income countries (Kaffenberger 2021). The main lesson learned is that if COVID continues as a low-intensity pandemic, or if a similar situation arises, keeping schools open should be a priority, as the evidence shows that the health benefits of school closures seemed to have been lower than the cost of learning losses (Allen 2022; Davies et al. 2021, 2020; Gandini et al. 2021; Lee and Raszka 2020; Ludvigsson 2020; Raffetti and Baldassarre 2022). The priority now should be to minimize the long-term impacts of school closures. This paper contributes to several strands of the literature. First, it adds to the growing research on the impact of school closures on learning outcomes (Agostinelli et al. 2022; Belot and Webbink 2010; Haelermans et al 2022), especially on the impact of duration on learning loss (Goldhaber et al 2022; Jack et al 2022). It also adds to the literature on the determinants of school closure duration (Kurmann and Lalé 2021; Nitsche and Hudde 2022). This paper uses unique data to estimate the precise impact per week of closures on learning loss across a variety of countries. From a human capital perspective, it makes sense that school closures would lead to learning loss since schooling is a productive endeavor (Becker 1994); therefore, extending the time away from in-person instruction should lead to a reduction in learning outcomes. Programs to support learning loss mitigation have been explored. 1. Data We supplement the robust data on learning losses compiled by Patrinos et al. (2023) with a few more recent studies (see Annex 1 and https://microdata.worldbank.org/index.php/catalog/5367). Most refer to primary schooling. The number of weeks of school closure is compiled from the same studies. Not all surveys are nationally representative. In the United States, the learning loss surveys are representative, but school closure duration varied state to state, and our figure represents an average. In Nepal, the data come from a study that only includes adolescent girls from one disadvantaged district. Besides school closure and duration, other possible correlates of learning loss are included. One of those controls is the pandemic itself. We include the COVID- 19 death rate per 100,000 population; the lockdown stringency index; and the vaccination rate. 2 We also control for national income. Since there are some reports that students in private schools lost less learning than others (Arenas and Gortázar 2022; Jack et al 2022; Wolf et al 2021), we include the proportion of private schools as a control. To gauge to what extent connectivity helped, we include individuals using the internet (percent of the population). To measure pre-existing school quality, we include the score from the World Bank’s Harmonized Learning Outcomes (HLO). To measure the strength of trade unions, we include the trade union density rate. We use the Polity5 dataset as the measure of democracy. See Table 1 for means, standard deviations, and sources. Table 1: Means and Standard Deviations Variable Mean Std. Dev. Source Learning loss (SD) 0.23 0.16 see Annex 1 (Learning Loss COVID-19 2020-2022) Weeks closed 20.78 18.21 see Annex 1 Death rate/100k 152.36 125.56 Johns Hopkins Coronavirus Resource Center GDP p/c $US 21709 22717 World Bank Private school (%) 14.84 14.10 World Bank Internet (%) 69.44 25.12 International Telecommunication Union School quality 462.08 89.97 Angrist et al. 2021 Vaccination rate 59.77 32.17 Oxford Coronavirus Government Response Tracker Stringency index 56.29 8.90 Hale et al 2021 Union density 22.38 16.05 International Labour Organization Democracy 7.02 4.76 Polity 5 2. School Closures and Learning Losses The literature suggests that school closures are an efficient strategy to reduce the overall duration of a pandemic (Bin Nafisah et al 2018). The decision to close down schools was primarily aimed at mitigating the spread of COVID-19 during the beginning of the pandemic (Raffetti and Baldassarre 2022). Models from previous epidemics such as influenza suggested the role of schools as places that facilitate the spreading and the possible benefit from closure (Cauchemez et al. 2008; Jackson et al. 2014). It was estimated that since school holidays could lead to a 20-29 percent reduction in the rate at which influenza is transmitted to children, then this might work for COVID-19. School closures delay the epidemic peak if implemented earlier but do not eliminate it. Imperfect knowledge led to no guidance on how long the duration of closures should be. Democratic countries tended to implement closures more quickly from the start of the pandemic than those with a more authoritarian regime (Cronert 2020). Researchers using national data – for example, the United States national achievement test, NAEP – have shown that there is a significant correlation between the length of school closures and learning loss, confirming earlier research (Barnum 2022; Goldhaber et al 2022; Jack et al 2022; Lehrer-Small 2022; West 2022). In the United States, public schools averaged less in-person teaching than private schools. These results are explained in large part by political preferences, vaccination rates, teacher unionization rates, and local labor conditions (Kurmann and Lalé 2021). (Teacher) unionization did not necessarily always favor school closures. For example, in Greece, teachers went on strike opposing the government’s distance education plan (Lambropoulos et al 3 2022). Nitsche and Hudde (2022) show that societal gender ideology has likely influenced school reopening policies – that is, countries with more supportive attitudes toward maternal employment reopened schools significantly sooner than those with less supportive attitudes toward maternal employment. There is a clear link between weeks of school closures and learning loss (Figure 1). School closures were an exogenous event, brought about by the pandemic and the need to control the infection rates. Closures as part of national lockdowns were imposed. It turns out that an average learning loss of 0.23 standard deviation occurred on average with a school closure duration of 21 weeks in a sample of 41 countries (which represent 2/3 of the world’s population). Figure 1: Learning Loss and Duration 0.90 0.80 Nepal 0.70 0.60 Learning loss (SD) India 0.50 Malawi Cambodia Argentina 0.40 Turkey Mexico Kenya Brazil 0.30 Botswana Poland Bangladesh Ghana Chile Norway Colombia Belgium Finland China USA GreeceSouth Africa 0.20 Switzerland Germany Italy England Russia Netherlands Canada Hungary Uganda Ethiopia Indonesia 0.10 Czech Spain Korea Denmark France Japan Australia 0.00 Sweden 0 10 20 30 40 50 60 70 80 90 100 Weeks closed We estimate learning loss, LL, in terms of standard deviations, as a result of weeks closed, W, and several control variables, Xn: LL = β1W+ βnXn+ ε (1) In terms of learning loss, it makes little difference if the controls are included; all are insignificant, and they do not change the coefficient on school closures much (see Table 2). With or without controls, every week of school closures increases learning loss by almost 0.01 of a standard deviation; or 1% of a standard deviation. This means that a 20-week closure will reduce learning outcomes by the equivalent of almost ¾ of a year’s worth of schooling. No other observed variable has an impact, so it is merely duration that counts. Given that there is little evidence that school 4 closures reduced infection rates, then it is a high cost to pay, especially after it was deemed secure to open schools safely. Table 2: Determinants of Learning Loss Variable 1 2 Weeks 0.006*** 0.007*** (0.001) (0.002) Death rate 0.000 (0.000) Private -0.0025 (0.002) Internet 0.0012 (0.003) School quality -0.001 (0.001) Vaccination 0.001 (0.001) Stringency 0.000 (0.003) Union 0.002 (0.002) Democracy 0.005 (0.006) Log GDP p/c -0.0499 (0.048) Constant 0.110* 0.568* (0.029) (0.295) Observations 41 41 R-squared 0.415 0.542 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 3. The Determinants of Duration of School Closures While closures were exogenous, the decision to remain closed longer was a conscious choice, especially after the American Academy of Pediatrics (2020) released, on 24 June 2020, the first school guidance for safe in-person learning and the European Centre for Disease Prevention and Control (2020) released its recommendation on 6 August 2020. By the time of the 2020-21 school year, it had become clear that it was possible to safely open schools (de Hoyos and Saavedra 2021). By August 2020 evidence supported a marginal role of children as drivers of the first wave of infections (Munday 2021). School reopening did not play a causal role in the increased number of new cases and hospitalizations in Italy during the period of September–October 2020 (Gandini et al. 2021), and it was not the main driver of the increased spreading of the COVID-19 UK variant in England at the beginning of 2021 (Davies et al. 2021). Moreover, school closures have no detectable effect on the contact patterns of adults (Cauchemez et al. 2008). Other studies suggested that school closure during the 2009 H1N1 influenza epidemic in Pennsylvania would not have 5 been worth the cost (Brown et al. 2011). Costs such as productivity losses of education professionals and parental wages outweigh the benefits in terms of decreased infection. Simulations showed that closing schools resulted in substantially higher net costs than not closing schools (Brown et al. 2011). Therefore, during the 2009 H1N1 epidemic, school closures could have resulted in substantial costs to society as the potential costs of lost productivity and childcare could have far outweighed the cost savings in preventing influenza cases. In other words, the likely beneficial impact of school closures is limited. At most, studies suggested that school closure can be a useful control measure during an influenza pandemic, particularly for reducing peak demand on health services, but they were not precise on the quantifiable benefits (Jackson et al. 2014). It turned out that the actual benefits of school closure during the COVID-19 pandemic were less than expected (Raffetti and Baldassarre 2022). Children have been less contagious than adults (Ludvigsson 2020; Lee and Raszka 2020; Davies et al. 2020). Allen (2022) calculates a number of cost/benefit ratios of lockdowns in terms of life-years saved in Canada. Using a mid-point estimate for costs and benefits, the reasonable estimate for Canada is a cost/benefit ratio of just 141. We estimate the determinants of weeks closed using the same control variables. In our analysis, the only significant variables are GDP, private schooling, school quality, the stringency of lockdowns, and the vaccination rate (Table 3). Higher-income countries have a shorter duration of school closure. A low vaccination rate is associated with longer school closures. Similarly, more stringent lockdowns also increase the length of closures. Higher test scores before COVID-19 are correlated with longer school closures. This might suggest that when authorities believe their education system and therefore its distance education version is of high quality, then they tend to prolong school closures. A higher proportion of private schools in a country leads to a longer duration of school closures. Interestingly, the death rate is not a factor, suggesting that school closures were not based on incidence of COVID-related fatalities. 6 Table 3: Determinants of Duration of School Closures Variable Coefficient Death rate -0.009 (0.021) Private 0.409** (0.166) Internet -0.0243 (0.267) School quality 0.139*** (0.039) Vaccination -0.255** (0.098) Stringency 0.486* (0.280) Union 0.112 (0.150) Democracy -0.355 (0.552) Log GDP p/c -8.742* (4.566) Constant 21.800 (29.15) Observations 41 R-squared 0.639 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 While the unplanned closures were imposed on schools due to an external event, the duration of the closures was a choice, to some extent at least. To properly estimate the determinants of the duration and its impact on learning outcomes, then one needs to address potential endogeneity issues. Table 3 suggests that national income, private schooling, school quality, the stringency of lockdowns, and the vaccination rate are associated with the duration of school closures. Since stringency is related to closures, and quality and private schooling are related to learning outcomes, then we experiment with the vaccination rate as a possible instrument. It could be argued that a high vaccination rate may lead to shorter school closures. A study shows that school closures can be minimized by regular testing and vaccination against COVID-19 (Colosi et al. 2022). The stringency index, a composite measure based on the strength of lockdown measures, can be used to measure how restrictive the measures were (Gros et al. 2021). We instrument the duration of school closures using the vaccination rate and the stringency index using 2SLS. The first stage and IV results are presented in Table 4. 7 Table 4: Determinants of Learning Loss Variable First stage IV learning weeks closed loss Vaccination -0.301** (0.073) Stringency 1.021* (0.262) Weeks 0.006*** (0.002) Constant -18.723 0.111 (14.567) (0.034) N 41 41 R-square 0.409 0.415 F-test 13.13 10.98 [p-value] 0.000 0.002 Sargan (overidentification) statistic 0.409 [p-value] [0.5223] The vaccination rate and the stringency index turn out to be useful instruments. Since we do not have many degrees of freedom, we estimate a very parsimonious equation, explaining weeks closed with the vaccination rate. It turns out that a lower (higher) vaccination rate leads to a longer (shorter) duration of school closures, while a higher stringency index prolongs school closures. Therefore, we can tentatively conclude that the duration of school closures leads to a higher level of learning loss. In the IV estimates, a week of school closures leads to a 0.006 standard deviation increase in learning loss, just slightly higher than the OLS estimate. 4. Implications In terms of learning loss, it makes little difference if the controls are included; all are insignificant. This suggests that school closures themselves were responsible for learning loss and the severity of that loss was primarily due to how long schools were kept closed. What matters is time spent in school. While pre-COVID learning levels varied considerably across countries, depending on many factors, learning happens if children go to school. Keeping schools open reduces learning losses, even in the poorest countries and in countries with low pre-COVID learning levels. This is especially important because research has found little evidence showing that school closures reduced infection rates. Students learn less when they are not being taught in school buildings. Duration played a key role. The long-term impact may be increased education gaps, especially between students from high and low socioeconomic status. The main lesson learned here is that if COVID continues as a low-intensity pandemic, or if a new, similar situation arises, keeping schools open should be a priority, as the evidence shows that the benefits of school closures seemed to have been lower than the cost of learning losses. However, the task at hand is to figure out how to accelerate learning and make up for the lost time. Fortunately, there are several things that work in terms of mitigating learning losses. These include online tutoring programs, which were shown to reduce learning loss significantly in several 8 randomized controlled trials (see, for example, Carlana and La Ferrara 2021; Gortazar et al. 2022; Kraft et al. 2022). A large-scale randomized trial testing low-technology interventions—SMS messages and phone calls—with parents to support their child in Botswana improved learning by 0.12 standard deviation (Angrist et al. 2022). Compensatory education works as well. In Tamil Nadu, India, two-thirds of the school closure learning deficit was made up within 6 months after school reopening, with a government-run after-school remediation program contributing one- quarter of the recovery (Singh et al. 2022). In Nigeria, a program designed to make up missed lessons during school closures led to a rebound within 2 months, and a recovery of all learning losses among students in low-cost schools (Adeniran et al. 2022). 5. Conclusion COVID-19 led to school closures and emergency remote-learning systems. The analysis investigated the impact of school closures using a unique data base. The determinants of the duration of school closures estimates were used to instrument school closures – stringency of lockdown and vaccination – and causally estimate the impact of duration on learning. It is concluded that for every week that schools were closed, learning levels declined by almost 1 percent of a standard deviation. This means that a 20 week closure, for example, would reduce learning outcomes by 0.20 standard deviation, almost one year of schooling. The findings here could influence education policy decision-making. It would be wise for policy makers to be aware of the linkages between public health decisions – such as vaccinations – and the spillover effects on class time and learning. Such linkages have clear implications for the development of learning technologies (such as different applications for distance learning) and the resilience of learning outcomes. 9 References Abufhele, A., Bravo, D., López Bóo, F., Soto-Ramirez, P. 2022. Developmental Losses in Young Children from Pre-Primary Program Closures During the COVID-19 Pandemic. IZA Discussion Paper No. 15179. Adeniran, A., Okoye, D., Yedomiffi, M.P. and Wantchekon, L. 2022. COVID-19 Learning Losses, Parental Investments, and Recovery: Evidence from Low-Cost Private Schools in Nigeria. RISE Working Paper Series 22/120. Agostinelli, F., Doepke, M., Sorrenti, G. and Zilibotti, F. 2022. When the great equalizer shuts down: Schools, peers, and parents in pandemic times. Journal of Public Economics 206. Allen, D.W. 2022. Covid-19 Lockdown Cost/Benefits: A Critical Assessment of the Literature. International Journal of the Economics of Business 29(1): 1-32. Amelia, R., Kadarisma, G., Fitriani, N., Ahmadi, Y. 2020. The effect of online mathematics learning on junior high school mathematic resilience during the covid-19 pandemic. Journal of Physics: Conference Series 1657(1). American Academy of Pediatrics. 2020. COVID-19 Guidance for Safe Schools and Promotion of In-Person Learning. https://www.aap.org Angrist, N., Bergman, P., Matsheng, M. 2022. Experimental evidence on learning using low-tech when school is out. Nature Human Behavior 6: 941–950. Angrist, N., Djankov, S., Goldberg, P.K., Patrinos, H.A. 2021. Measuring human capital using global learning data. Nature 592: 403–408. Ardington, C, Willis, G., Kotze, J. 2021. COVID-19 Learning Losses: Early Grade Reading in South Africa. International Journal of Educational Development 86. Arenas, A., Gortázar, L. 2022. Learning Loss One Year After School Closures: Evidence From The Basque Country. IEB Working Paper 2022/03. Argentina, Ministerio de Educación de la Nación. 2022. Aprender 2021: Educación Primaria: informe nacional de resultados: análisis sobre los logros de aprendizaje y sus condiciones. https://www.argentina.gob.ar/sites/default/files/informe_aprender_2021_1.pdf Asakawa, S., Ohtake, F. 2021. Impact of Temporary School Closure Due to COVID-19 on the Academic Achievement of Elementary School Students. Discussion Papers in Economics and Business 21-14, Osaka University, Graduate School of Economics. Asim, S., Gera, R., Singhal, A. 2022. Learning Loss from Covid in Sub-Saharan Africa: Evidence from Malawi. World Bank. Barnum, M. 2022. Nation’s report card: Massive drop in math scores, slide in reading linked to COVID disruption. Chalkbeat. Becker, G.S. 1994. Human capital revisited. In Human Capital: A Theoretical and Empirical Analysis with Special Reference to Education, Third Edition (pp. 15-28). The University of Chicago Press. Belot, M., Webbink, D. 2010. Do Teacher Strikes Harm Educational Attainment of Students? Labour 24(4): 391–406. Bielinski, J., Brown, R., Wagner, K. 2021. No Longer a Prediction: What New Data Tell Us About the Effects of 2020 Learning Disruptions. Irvine, CA: Illuminate Education. Bin Nafisah, S., Alamery, A.H., Al Nafesa, A., Aleid, B. and Brazanji, N.A. 2018. School closure during novel influenza: a systematic review. Journal of Infection and Public Health 11(5): 657-661. Birkelund, J.F., Karlson, K.B. 2022. No Evidence of a Major Learning Slide 14 Months into the COVID-19 Pandemic in Denmark. European Societies. 10 Brown, S.T., Tai, J.H., Bailey, R.R. 2011. Would school closure for the 2009 H1N1 influenza epidemic have been worth the cost?: a computational simulation of Pennsylvania. BMC Public Health 11: 353. Carlana, M. and La Ferrara, E. 2021. Apart but connected: Online tutoring and student outcomes during the COVID-19 pandemic. IZA Discussion Papers No. 14094. Cauchemez, S., Valleron, A.J., Boëlle, P.Y., Flahault, A., Ferguson, N.M. 2008. Estimating the impact of school closure on influenza transmission from Sentinel data. Nature 452: 750- 754. Chaban, T.Y., Rameeva, R.S., Denisov, I.S., Kersha, Y.D., Zvyagintsev, R.S. 2022. Rossiyskaya shkola v period pandemii COVID-19: effekty pervykh dvukh voln i kachestvo obrazovaniya [Russian Schools during the COVID-19 Pandemic: Impact of the First Two Waves on the Quality of Education]. Voprosy obrazovaniya/Educational Studies Moscow 1: 160–188. Clark, A.E., Nong, H., Zhu, H., Zhu, R. 2021. Compensating for academic loss: Online learning and student performance during the COVID-19 pandemic. China Economic Review 68. Colosi, E., Bassignana, G., Contreras, D.A., Poirier, C., Boëlle, P.Y., Cauchemez, S., Yazdanpanah, Y., Lina, B., Fontanet, A., Barrat, A. and Colizza, V., 2022. Screening and vaccination against COVID-19 to minimise school closure: a modelling study. The Lancet Infectious Diseases 22(7): 977-989. Contini, D., Di Tommaso, M.L., Muratori, C., Piazzalunga, D. and Schiavon, L. 2022. Who Lost the Most? Mathematics Achievement during the COVID-19 Pandemic. The BE Journal of Economic Analysis and Policy 22(2): 399-408. Coskun, K., Kara, C. 2022. The impact of the COVID-19 pandemic on primary school students’ mathematical reasoning skills: a mediation analysis. London Review of Education 20(1): 1-15. Cronert, A. 2020. Democracy, state capacity, and COVID-19 related school closures. APSA Preprints.https://preprints.apsanet.org/engage/apsa/article-details/5ea8501b68bfcc00122e96ac Davies, N.G., Klepac, P., Liu, Y., Prem, K., Jit, M., Eggo, R.M. 2020. Age-dependent effects in the transmission and control of COVID-19 epidemics. Nature Medicine 26: 1205–1211. Davies, N.G., Abbott, S., Barnard, R.C., Jarvis, C.I., Kucharski, A.J., Munday, J.D., Pearson, C.A.B., Russell, T.W., Tully, D.C., Washburne, A.D. 2021. Estimated transmissibility and impact of SARS-CoV-2 lineage B.1.1.7 in England. Science 372: 1-10. de Hoyos, R., J. Saavedra. 2021. It is time to return to learning. World Bank. https://blogs.worldbank.org/education/it-time-return-learning Domingue, B.W., Hough, H.J., Lang, D., Yeatman, J. 2021. Changing Patterns of Growth in Oral Reading Fluency During the COVID-19 Pandemic—Policy Brief. Policy Analysis for California Education (PACE). Education Policy Institute. 2021. Understanding progress in the 2020/21 academic year: Interim findings. Department for Education UK. Renaissance Learning. Engzell, P, Frey, A, Verhagen, M. 2021. Learning loss due to school closures during the COVID- 19 pandemic. Proceedings of the National Academy of Sciences 118(17): 1-7. European Centre for Disease Prevention and Control. 2020. COVID-19 in children and the role of school settings in COVID-19 transmission. Stockholm: ECDC. 11 Gajderowicz, T.J., Jakubowski, M.J., Patrinos, H.A. and Wrona, S.M., 2022. Capturing the Educational and Economic Impacts of School Closures in Poland. World Bank Policy Research Working Paper No. 10253. Gambi, L., de Witte, K. 2021. The resiliency of school outcomes after the COVID-19 pandemic. Standardised test scores and inequality one year after long term school closures. FEB Research Report Department of Economics. Gandini, S., Rainisio, M., Iannuzzo, M.L., Bellerba, F., Cecconi, F., Scorrano, L. 2021. A cross- sectional and prospective cohort study of the role of schools in the SARS-CoV-2 second wave in Italy. Lancet Regional Health-Europe 5. Georgiou, G. 2021. Covid-19’s Impact on Children’s Reading Scores: Data Trends and Complementary Interview. The Reading League Journal 2(2): 3-33. Goldhaber, D., Kane, T., McEachin, A., Morton E., Patterson, T., Staiger, D. 2022. The Consequences of Remote and Hybrid Instruction During the Pandemic. NBER Working Paper No. w30010. Gore, J., Fray, L., Miller, A., Harris, J., Taggart, W. 2021. The impact of COVID-19 on student learning in New South Wales primary schools: an empirical study. Australian Educational Researcher 48: 605-637. Gortazar, L., Hupkau, C. and Roldán, A. 2022. Online tutoring works: Experimental evidence from a program with vulnerable children. ESADE Center for Economic Policy and School of Public Policy, Spain, Working Paper (2). Gros, D., Ounnas, A. and Yeung, T.Y.C. 2021. A new COVID policy stringency index for Europe. Covid Economics 115. Haelermans, C., Jacobs, M., van der Velden, R., van Vugt, L., van Wetten, S. 2022. Inequality in the Effects of Primary School Closures Due to the COVID-19 Pandemic: Evidence from the Netherlands. AEA Papers and Proceedings 112: 303-07. Hale , T., Angrist , N., Goldszmidt, R., Kira, B., Petherick , A., Phillips, T., Webster, S., Cameron- Blake , E., Hallas, L., Majumdar, S., Tatlow, H. 2021. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nature Human Behaviour 5(4): 529-538. Hallin, A.E., Danielsson, H., Nordström, T., Fälth, L. 2022. No learning loss in Sweden during the pandemic: Evidence from primary school reading assessments. International Journal of Educational Research 114. Hassan, H., Islam, A., Siddique, A., Choon Wang, L. 2021. Telementoring and homeschooling during school closures: A randomized experiment in rural Bangladesh. No. 13. TUM School of Governance at the Technical University of Munich. Hevia, F.J., Vergara-Lope Tristan, S., Velásquez-Durán, A., Calderón Martín del Campo, D. 2021. Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico. International Journal of Educational Development 88. Jack, R., Halloran, C, Okun, J, Oster, E. 2022. Pandemic Schooling Mode and Student Test Scores: Evidence from U.S. School Districts. American Economic Review: Insights. Jackson, C., Mangtani, P., Hawker, J., Olowokure, B., Vynnycky, E. 2014. The effects of school closures on influenza outbreaks and pandemics: Systematic review of simulation studies. PLoS ONE 9(5): e97297. Kaffenberger, M. 2021. Modelling the long-run learning impact of the Covid-19 learning shock: Actions to (more than) mitigate loss. International Journal of Educational Development 81. 12 Kim, J., Rose, P., Tirunch, D.T, Sabates, R., Woldehanna, T. 2021. Learning Inequalities Widen Following COVID-19 School Closures in Ethiopia. Research on Improving Systems of Education (Rise), 4. Kogan, V., Lavertu, S. 2021. The COVID-19 pandemic and student achievement on Ohio’s third- grade English language arts assessment. Ohio State University. Korbel, V., Prokop, D. 2021. Czech students lost 3 months of learning after a year of the COVID- 19 pandemic. Prague: PAQ Research. Kraft, M.A., List, J.A., Livingston, J.A. and Sadoff, S. 2022, May. Online tutoring by college volunteers: Experimental evidence from a pilot program. American Economic Review Papers and Proceedings 112: 614-618. Kuhfeld, L.K., Ruzek, E., McEachin, A. 2021. Learning during COVID-19: Reading and math achievement in the 2020-21 school year. NWEA Kurmann, A., Lalé, E. 2021. School Closures and Effective In-Person Learning during COVID- 19: When, Where, and for Whom. School of Economics Working Paper Series 2021-18 LeBow College of Business Drexel University. Lambropoulos, H., Vathi-Sarava, P., Karatzia-Stavlioti, E. 2022. The Impact of the COVID-19 Pandemic on Learning in Greece: Investigation of the University Entrance Exams. European Journal of Education Studies 9(4): 67-89. Lee, B., Raszka, W.V. 2020. COVID-19 Transmission and Children: The Child Is Not to Blame. Pediatrics 146. Lehrer-Small, A. 2022. Strong Link in Big City Districts’ 4th-Grade Math Scores to School Closures. The 74. https://www.the74million.org/article/strong-link-in-big-city-districts- 4th-grade-math-scores-to-school-closures/ Lerkkanen, MK., Pakarinen, E., Salminen, J., Torppa, M. 2022. Reading and math skills development among Finnish primary school children before and after COVID-19 school closure. Reading and Writing 36: 263-288. Lewis, K., Kuhfeld, M., Ruzek, E., McEachin, A. 2021. Learning during COVID-19: Reading and math achievement in the 2020-21 school year. Center for School and Student Progress. Lichand, G., Doria, C.A., Leal-Neto, O. 2022. The impacts of remote learning in secondary education during the pandemic in Brazil. Nature Human Behaviour 6: 1079-1086. Locke, V.N., Patarapichayatham, C., Lewis, S. 2021. Learning Loss in Reading and Math in US Schools Due to the COVID-19 Pandemic. Dallas, TX: Istation. Ludewig, U., Kleinkorres, R., Schaufelberger, R., Schlitter, T., Lorenz, R., Koenig, C., Frey, A., McElvany, N. 2022. COVID-19 Pandemic and Student Reading Achievement – Findings from a School Panel Study Ludvigsson, J.F.2020. Children are unlikely to be the main drivers of the COVID-19 pandemic— A systematic review. Acta Paediatrica 109(8): 1525-1530. Molnár, G., Hermann, Z. 2023. Short- and long-term effects of COVID-related kindergarten and school closures on first- to eighth-grade students’ school readiness skills and mathematics, reading and science learning. Learning and Instruction 83. Munday, J.D., Sherratt, K., Meakin, S., Endo, A., Pearson, C.A.B., Hellewell, J., Abbott, S., Bosse, N.I., Atkins, K.E., Wallinga, J. 2021. Implications of the school-household network structure on SARS-CoV-2 transmission under school reopening strategies in England. Nature Communications 12:1942. 13 Nitsche, N., Hudde, A. 2022. Countries embracing maternal employment opened schools sooner after Covid-19 lockdowns. No. WP-2022-008. Max Planck Institute for Demographic Research, Rostock, Germany. Patrinos, H., Vegas, E., Carter-Rau, R. 2023. An Analysis of COVID-19 Student Learning Loss. In the Oxford Research Encyclopedia of Economics and Finance. Oxford University Press. Pier, L., Christian, M., Tymeson, H., Meyer, R.H. 2021. COVID-19 Impacts on Student Learning: Evidence from Interim Assessments in California. Policy Analysis for California Education, PACE. Psacharopoulos, G., Collis, V., Patrinos, H.A., Vegas, E. 2021. The COVID-19 Cost of School Closures in Earnings and Income across the World. Comparative Education Review 65(2): 271-287. Raffetti, E., Di Baldassarre, G. 2022. Do the Benefits of School Closure Outweigh Its Costs?. International Journal of Environmental Research and Public Health 19(5). Singh, A., Romero, M., Muralidharan, K. 2022. COVID-19 Learning Loss and Recovery: Panel Data Evidence from India. RISE Working Paper Series. 22/112. Skar, G.B.U., Graham, S., Huebner, A. 2021. Learning Loss During the COVID-19 Pandemic and the Impact of Emergency Remote Instruction on First Grade Students’ Writing: A Natural Experiment. Journal of Educational Psychology 114(7): 1553-1566. Tetra Tech. 2022. Independent Evaluation of the Girls’ Education Challenge Phase II Evaluation Study 1: Effects of Covid-19 on Access and Learning in the GEC II. Thorn, W., Vincent-Lancrin, S. 2021. Schooling During a Pandemic: The Experience and Outcomes of Schoolchildren During the First Round of COVID-19 Lockdowns. OECD. Tomasik, M., Helbling, L., Moser, U. 2020. Educational gains of in-person vs. distance learning in primary and secondary schools: a natural experiment during the COVID-19 pandemic 29 school closures in Switzerland. International Journal of Psychology 56(4): 566-576. UNESCO. 2020. UN Secretary-General warns of education catastrophe, pointing to UNESCO estimate of 24 million learners at risk of dropping out. UNICEF. 2022. Learning Loss in the Covid-19 Pandemic Era: Evidence from the 2016-2021 Grade Six National Learning Assessment in Cambodia: Supplementary Technical Report for the 2021 Grade Six National Learning Assessment. Uwezo Uganda. 2021. Are Our Children Learning? Illuminating the Covid-19 Learning Losses and Gains in Uganda. Uwezo National Learning Assessment Report, 2021. Kampala: Uwezo Uganda. Vegas, E. 2022. COVID-19’s impact on learning losses and learning inequality in Colombia. Center for Universal Education at Brookings 599. West, M.R. 2022. Nation’s Report Card Shows Steep Declines in Student Learning. Education Next. Whizz Education. 2021. Measuring Impact of COVID-19 on Learning in Rural Kenya. Wolf, S., Aurino, E., Suntheimer, N., Avornyo, E., Tsinigo, E., Jordan, J., Samanhyia, S., Aber, J.L., Behrman, J.R. 2021. Learning in the Time of a Pandemic and Implications for Returning to School: Effects of COVID-19 in Ghana. CPRE Working Papers. Yarrow, N.B., Kim, H., Yoo, J., Pfutze, T. 2022. EdTech in COVID Korea: Learning with Inequality. Innovation and Technology Note Series No. 7. World Bank. 14 Annex 1: Learning loss database Average learning losses Country (SD) Weeks closed Source Nepal 0.80 40 Tetra Tech 2022 India 0.52 93 Singh et al 2022 Malawi 0.47 30 Asim et al 2022 Cambodia 0.45 40 UNICEF 2022 Argentina 0.44 39 Argentina 2022 Mexico 0.37 48 Hevia et al 2021 Türkiye 0.37 40 Coskun and Kara 2022 Kenya 0.36 28 Whizz Education 2021 Brazil 0.32 38 Lichand et al 2022 Poland 0.30 20 Gajderowicz et al. 2022 Botswana 0.29 11 Angrist et al 2022 Bangladesh 0.28 63 Hassan et al 2021 Chile 0.25 14 Abufhele et al 2022 Ghana 0.25 10 Wolf et al 2021 Colombia 0.24 23 Vegas 2022 Norway 0.24 7 Skar et al 2021 Belgium 0.23 9 Gambi and de Witte 2021 China 0.22 7 Clark et al 2021 Finland 0.22 8 Lerkkanen et al 2022 Greece 0.22 12 Lambropoulos et al 2022 South Africa 0.22 15 Ardington et al 2021 United States 0.22 8 Bielinski 2021; Domingue 2021; Kogan/Lavertu 2021; Kuhfeld 2021; Lewis 2021; Locke 2021; Pier 2021 Switzerland 0.20 8 Tomasik et al 2020 Italy 0.19 15 Contini et al 2022 Germany 0.18 10 Ludewig et al 2022 Canada 0.17 26 Georgiou 2021 England 0.17 10 Education Policy Institute 2021 Netherlands 0.17 8 Haelermans et al 2022 Russia Federation 0.17 14 Chaban et al 2022 Hungary 0.16 13 Molnár and Hermann 2023 Uganda 0.14 39 Uwezo Uganda 2021 Ethiopia 0.11 21 Kim et al 2021 Indonesia 0.11 20 Amelia et al 2020 Czechia 0.08 9 Korbel and Prokop 2021 Spain 0.05 10 Arenas and Gortazar 2022 Korea, Rep. 0.04 11 Yarrow et al 2022 Denmark 0.03 8 Birkelund and Karlson 2022 France 0.02 8 Thorn and Vincent-Lancrin 2021 Japan 0.02 11 Asakawa and Ohtake 2021 Australia 0.01 8 Gore et al 2021 Sweden 0.00 0 Hallin et al 2022 15