The effect of the H1N1 pandemic on learning. What to expect with Covid-19? Vivian Amorim, Caio Piza e Ildo José Lautharte Júnior World Bank affect the labor productivity of the workforce in H1N1, use data from Prova Brasil test scores Key messages: the long-term. in Portuguese language and mathematics. The focus is on fifth graders as in 2005 only 4 out of Despite the fact that some schools have adopted 1 The analysis indicates that the extension the 13 municipalities that postponed the return remote learning, many children lack the required of the winter break, adopted in 13 of schooling had proficiency score data for the equipment, access to the broadband Internet, municipalities of the state of São Paulo ninth grade. a suitable environment for studying, and/or during the H1N1 pandemic, significantly the presence of an adult who can help with Our analysis then considers as the treatment decreased learning outcomes among the new teaching routine. Compounding these group the schools in the municipalities that students in the 5th grade of elementary difficulties, few teachers have the skills required extended the winter break, while the comparison school (ES). to teach remotely (World Bank, 2020). group is formed by schools in the other 632 2 The decrease in math performance resulting municipalities of São Paulo state, which followed from the two to three weeks winter break As far as the emotional consequences of the the school calendar normally. extension is equivalent to two months of crisis are concerned, it is likely that both teachers learning. and students will be negatively affected during As the school closures occurred just after the 3 The impacts were more pronounced among the period of social isolation. The duration of winter break of 2009, the proposed exercise schools in the bottom quintile of test quarantine, the loss of a family member, the fear compares two groups of students: those who score distribution, which suggests that the of contagion, the reduction of household income, had the usual school break period and those consequences were more severe among the and the disrupted family environment due to whose school break was extended by two to three most vulnerable groups. increased domestic violence are likely to scale weeks. If one assumes that the longer the time 4 These results suggest that the effect of up the damage of school closures on learning away from school, the greater the potential for Covid-19 will be relevant, and remedial outcomes.4 forgetting the school content, our estimates will policies will be required to mitigate learning overstate the impacts of interrupting classes for This note aims to shed light on the potential losses. two to three weeks. effects of the Covid-19 pandemic on the learning of fifth grade students. We leverage the 2009 Intuitively, we estimate the effect of the winter School closures and learning H1N1 pandemic to benchmark the impacts of break extension by comparing the difference the current crisis. There are limitations to this in the evolution of learning in the treatment School closures have been adopted as a exercise that are worth pointing out. At the and comparison groups in 2007 (before the preventive measure against the spread of the time of H1N1, no quarantines were adopted, the pandemic) and 2009 (after the pandemic).6 For Covid-19 pandemic by almost all countries in risk of contagion and mortality were lower, and the estimates to be interpreted as a causal the world. Such measures are unprecedented the emotional and economic impacts were also effect of the school closures on the learning in scale: at the end of April 2020, in 190 incomparably lower. Thus, by relying on a less outcomes of fifth grade students, it is necessary countries, more than 1.5 billion students were severe pandemic crisis, our main findings should that the evolution of learning of the comparison out of school. Most of Brazil’s 48 million 1 be seen as lower bound effects of the Covid-19 group emulates what would have happened with students in Primary and Secondary Education on learning. student learning in the treated schools in the have being affected.2 Unique in modern history, absence of the treatment (school closures). In 7 there is a great debate about the consequences H1N1 in São Paulo fact, data from Prova Brasil, for the years 2005 for students and education professionals. and 2007, suggest that the learning outcomes In July 2009, in the midst of the H1N1 of these two groups followed the same trend as Although it is too early to determine the impact pandemic, all state schools and municipal before the H1N1 outbreak. of school closures on children’s learning, few schools of 13 municipalities in the State of would disagree that such effects will be negative. São Paulo extended the winter break by two The research exploring the interruption of to three weeks. More than 5.5 million Primary 5 Impacts on learning classes has found an increase in the risk of and Secondary Education students were affected school dropout (Meyers and Thomasson, 2017; by this measure, which in that year represented Our results indicate that extension of the winter Bandiera et al., 2018); a decrease in literacy 69.3% of students from state and local schools. break due to the H1N1 pandemic had a negative (Marcotte and Helmut, 2008) and proficiency The other municipalities did not alter their average effect on proficiency in Portuguese (Baker, 2013); an increase in the knowledge school calendars. language and mathematics, equivalent to a gap between students of different socioeconomic 0.15 standard deviation.8 This magnitude is backgrounds (Alexander et al., 2007); and lower high, especially when compared to a preliminary How to estimate school closure probability of holding a college degree (Belot exercise by Burgess and Sievertsen; and with effects on learning? and Webbink, 2010). Together, these results 3 the average effect found for educational policies indicate that, in addition to the high individual To estimate the impact of school closures on aimed to increase student learning in developing costs, the current crisis may also significantly the learning of fifth grade students during the countries (McKewan, 2015). 9 World Bank, June 2020 The effect of the H1N1 pandemic on learning. What to expect with Covid-19? Vivian Amorim, Caio Piza e Ildo José Lautharte Júnior World Bank The effect in mathematics corresponds to a 4.5 already behind in mathematics, which explains extrapolate these results to Brazil, this analysis point drop in performance on the SAEB scale, the relatively small increase of 3.5 percentage suggests that there will be a significant drop in or a 0.2 standard deviation. For Portuguese 10 points (6.1%) of students below the adequate student performance. language, the estimated effect is negative, but it level of mathematics proficiency. However, everyone agrees that the reopening is not statistically different from zero (Figure 1). These results also suggest that the effect was of schools should only happen when the health The higher effect on math is in agreement with higher among the most vulnerable students, of students and teachers can be guaranteed, Baker’s work (2013) and with the literature that since local schools with the worst grades, in line with the recommendations of health explores the effect of summer break on learning compared to top 20% schools, had a higher professionals. In the process of returning in the United States (Cooper et al., 1996). proportion of students who had already dropped to schools, some of the main policies to be Figura 1: Estimated impact in Portuguese out (7.7 percentage points higher) or repeated adopted to mitigate the negative effects include: language and mathematics for ES fifth grade some grade of ES (9.6 percentage points higher) administering proficiency tests to identify the students, SAEB scale and whose mother had not completed high content that should be prioritized and the school (20.7 percentage points higher). most vulnerable students who will need special Placebo Extension of winter break 95% CI attention; extending the daily course load; 4.0 It is important to note that the magnitude of Portuguese Math shortening the planned holiday for December the effect reflects not only the postponement 2.0 and January; holding extra tuition; promoting of return to school, but also the influence of campaigns to raise awareness about returning to 0.0 other factors associated with the context of school (via TV or digital media); and continuing the pandemic. The risk of contagion and the γ, SAEB scale and improving distance learning platforms to -2.0 average number of hospitalizations was higher complement classroom learning. among the municipalities that extended the -4.0 winter break. Parents and guardians were Notes -6.0 likely more reluctant to send children back to 1 UNESCO and World Bank. school, causing them to miss more school days.11 Synopsis of Primary and Secondary Education, 2019. INEP. 2 3 In the mentioned articles, the schools were closed because of the polio pandemic in 1916 (Meyers and Thomasson, 2017); more severe winters in the USA (Marcotte and Helmut, 2008); the school 2007 2009 2007 2009 -8.0 The fear of contagion could also have affected holiday period (Alexander et al., 2007); and a teachers’ strike (Belot and Webbink, 2010). versus 2005 versus 2007 versus 2005 versus 2007 4 The return to school in the context of the Covid-19 pandemic. Todos Pela Educação. May 2020. the leisure activities of students, potentially 5 São Paulo, Campinas, Diadema, Embu das Artes, Indaiatuba, Mairiporã, Osasco, São Bernardo do Campo, Santo André, São Caetano do Sul, Sumaré, Ribeirão Preto and Taboão da Serra. triggering psychological factors such as anxiety 6 The proposed estimator is the difference-in-differences: https://github.com/worldbank/h1n1- school-closures-sp-2009. and depression. 7 As Prova Brasil was applied in 2005 for the first time, it was not possible to test the hypothesis In an ideal scenario, one estimates that of parallel trends in previous years. In 2005, 03 of 13 municipalities which chose to postpone the return to classes did not present proficiency data of ES 5th grade students (Campinas, Indaiatuba In addition, the school is an environment in the average annual gain in proficiency in and São Caetano do Sul). Thus, as it is not possible to check the hypothesis of parallel trends for their local schools, they were excluded from the analysis. which the student is protected from cases of mathematics between the end of the fifth and In 2007, in the local system of São Paulo, the standardized score of fifth grade students presented 8 an average of 5.2, with a standard deviation of 0.66. Thus, a drop of -0.1 point of the grade is domestic violence, at least temporarily. Being ninth grade, should be 20 points on the SAEB equivalent to a 0.15 standard deviation. 9 Simon Burgess and Hans Henrik Sievertsen argue that closing schools for 12 weeks could lead to more exposed to risky situations can also impact scale per academic year (Alves, Soares, and a 0.06 sd drop in learning. (https://voxeu.org/article/impact-covid-19- education). 10 In 2007, in the local system of São Paulo, the mathematics score of fifth grade students presented their emotional state. All of these factors may an average of 203.41 with a standard deviation of 19.43. Xavier, 2016). Therefore, the estimated drop 11 Until August 2009, the number of hospitalizations for H1N1 were 35 and 20 per 100,000 inhabitants have contributed to the decreases in learning. in the treated and comparison municipalities, respectively. of 4.5 points would correspond to 9 weeks or about 2 months (200 school days × 4.5/20) of References learning. This effect is large, especially when one Alexander, K. L., D. R. Entwisle, and L. S. Olson(2007): “Lasting consequences ofthe summer What to expect with Covid-19? learning gap”. American sociological review, 72, 167–180. considers that the learning base was already low: Alves, M. T. G., Soares, J. F., e Xavier, F. P. (2016). Desigualdades educacionais no ensino fundamental de 2005 a 2013: hiato entre grupos sociais. Revista Brasileira de Sociologia, 4(7), even before the pandemic, 57.2% of students in 49-82. Estimates show that the closure of schools Baker, M. (2013). Industrial Actions in Schools: Strikes and Student Achievement. The Canadian the schools affected by the measure were below Journal of Economics, 46(3), 1014–1036. in São Paulo, in the midst of the H1N1 Bandiera, O., Buehren, N., Goldstein, M. P., Rasul, I., and Smurra, A. (2019). The Economic Lives of Young Women in the Time of Ebola: Lessons from an Empowerment Program. The World Bank. the adequate level of proficiency in mathematics. pandemic, resulted in a significant drop in Belot, M. and D. Webbink(2010): “Do teacher strikes harm educational attainment ofstudents?”. Labour, 24, 391–406. To check whether the policy had distributional student performance in mathematics for ES fifth Cooper, H., B. Nye, K. Charlton, J. Lindsay, and S. Greathouse(1996): “The effects of summer vacation on achievement test scores: A narrative and meta-analytic review”. Review of educational effects, we estimate treatment effects on grade students. The two to three week extension research, 66, 227–268. Eyles, A., Gibbons, S., and Montebruno, P. (2020). Covid-19 school shutdowns: What will they do the quintiles of the mathematics proficiency of July holidays is estimated to be equivalent to our children’s education? (No. cepcovid-19-001). Centre for Economic Performance, LSE. Marcotte, D. E. and S. W. Hemelt(2008): “Unscheduled school closings and student performance”. distribution. The effect is negative throughout to the loss of two months of learning. If this Education Finance and Policy, 3, 316–338. McEwan, P. J. (2015). Improving learning in primary schools of developing countries: A meta- analysis of randomized experiments. Review of Educational Research, 85(3), 353-394. the distribution, but higher in the first quintile estimate is a good proxy of what will happen Meyers, K. and M. A. Thomasson(2017): “Paralyzed by panic: Measuring the effectof school closures during the 1916 polio pandemic on educational attainment”. Tech. rep., National Bureau (-5.2 points on the SAEB scale). The strongest with the actual school closures, one can expect of Economic Research. Todos pela Educação. O retorno às aulas presenciais no contexto da pandemia da Covid-19. Maio effect on the lower tail suggests that the a learning loss equivalent to half-semester of de 2020. The Covid-19 Pandemic: Schocks to Education and Policy Responses. World Bank Grouo. Maio negative effects were higher among students a school year. Although it is not possible to de 2020. World Bank, June 2020