The eﬀect of the H1N1 pandemic on learning.
What to expect with Covid-19?
Vivian Amorim, Caio Piza e Ildo José Lautharte Júnior
World Bank
aﬀect the labor productivity of the workforce in H1N1, use data from Prova Brasil test scores
Key messages: the long-term. in Portuguese language and mathematics. The
focus is on ﬁfth graders as in 2005 only 4 out of
Despite the fact that some schools have adopted
1 The analysis indicates that the extension the 13 municipalities that postponed the return
remote learning, many children lack the required
of the winter break, adopted in 13 of schooling had proﬁciency score data for the
equipment, access to the broadband Internet,
municipalities of the state of São Paulo ninth grade.
a suitable environment for studying, and/or
during the H1N1 pandemic, signiﬁcantly
the presence of an adult who can help with Our analysis then considers as the treatment
decreased learning outcomes among
the new teaching routine. Compounding these group the schools in the municipalities that
students in the 5th grade of elementary
diﬃculties, few teachers have the skills required extended the winter break, while the comparison
school (ES).
to teach remotely (World Bank, 2020). group is formed by schools in the other 632
2 The decrease in math performance resulting
municipalities of São Paulo state, which followed
from the two to three weeks winter break As far as the emotional consequences of the
the school calendar normally.
extension is equivalent to two months of crisis are concerned, it is likely that both teachers
learning. and students will be negatively aﬀected during As the school closures occurred just after the
3 The impacts were more pronounced among the period of social isolation. The duration of winter break of 2009, the proposed exercise
schools in the bottom quintile of test quarantine, the loss of a family member, the fear compares two groups of students: those who
score distribution, which suggests that the of contagion, the reduction of household income, had the usual school break period and those
consequences were more severe among the and the disrupted family environment due to whose school break was extended by two to three
most vulnerable groups. increased domestic violence are likely to scale weeks. If one assumes that the longer the time
4 These results suggest that the eﬀect of up the damage of school closures on learning away from school, the greater the potential for
Covid-19 will be relevant, and remedial outcomes.4 forgetting the school content, our estimates will
policies will be required to mitigate learning overstate the impacts of interrupting classes for
This note aims to shed light on the potential
losses. two to three weeks.
eﬀects of the Covid-19 pandemic on the learning
of ﬁfth grade students. We leverage the 2009 Intuitively, we estimate the eﬀect of the winter
School closures and learning H1N1 pandemic to benchmark the impacts of break extension by comparing the diﬀerence
the current crisis. There are limitations to this in the evolution of learning in the treatment
School closures have been adopted as a exercise that are worth pointing out. At the and comparison groups in 2007 (before the
preventive measure against the spread of the time of H1N1, no quarantines were adopted, the pandemic) and 2009 (after the pandemic).6 For
Covid-19 pandemic by almost all countries in risk of contagion and mortality were lower, and the estimates to be interpreted as a causal
the world. Such measures are unprecedented the emotional and economic impacts were also eﬀect of the school closures on the learning
in scale: at the end of April 2020, in 190 incomparably lower. Thus, by relying on a less outcomes of ﬁfth grade students, it is necessary
countries, more than 1.5 billion students were severe pandemic crisis, our main ﬁndings should that the evolution of learning of the comparison
out of school. Most of Brazil’s 48 million
1 be seen as lower bound eﬀects of the Covid-19 group emulates what would have happened with
students in Primary and Secondary Education on learning. student learning in the treated schools in the
have being aﬀected.2 Unique in modern history, absence of the treatment (school closures). In
7
there is a great debate about the consequences H1N1 in São Paulo fact, data from Prova Brasil, for the years 2005
for students and education professionals. and 2007, suggest that the learning outcomes
In July 2009, in the midst of the H1N1 of these two groups followed the same trend as
Although it is too early to determine the impact pandemic, all state schools and municipal before the H1N1 outbreak.
of school closures on children’s learning, few schools of 13 municipalities in the State of
would disagree that such eﬀects will be negative. São Paulo extended the winter break by two
The research exploring the interruption of to three weeks. More than 5.5 million Primary
5 Impacts on learning
classes has found an increase in the risk of and Secondary Education students were aﬀected
school dropout (Meyers and Thomasson, 2017; by this measure, which in that year represented Our results indicate that extension of the winter
Bandiera et al., 2018); a decrease in literacy 69.3% of students from state and local schools. break due to the H1N1 pandemic had a negative
(Marcotte and Helmut, 2008) and proﬁciency The other municipalities did not alter their average eﬀect on proﬁciency in Portuguese
(Baker, 2013); an increase in the knowledge school calendars. language and mathematics, equivalent to a
gap between students of diﬀerent socioeconomic 0.15 standard deviation.8 This magnitude is
backgrounds (Alexander et al., 2007); and lower high, especially when compared to a preliminary
How to estimate school closure
probability of holding a college degree (Belot exercise by Burgess and Sievertsen; and with
eﬀects on learning?
and Webbink, 2010). Together, these results
3 the average eﬀect found for educational policies
indicate that, in addition to the high individual To estimate the impact of school closures on aimed to increase student learning in developing
costs, the current crisis may also signiﬁcantly the learning of ﬁfth grade students during the countries (McKewan, 2015). 9
World Bank, June 2020
The eﬀect of the H1N1 pandemic on learning.
What to expect with Covid-19?
Vivian Amorim, Caio Piza e Ildo José Lautharte Júnior
World Bank
The eﬀect in mathematics corresponds to a 4.5 already behind in mathematics, which explains extrapolate these results to Brazil, this analysis
point drop in performance on the SAEB scale, the relatively small increase of 3.5 percentage suggests that there will be a signiﬁcant drop in
or a 0.2 standard deviation. For Portuguese
10
points (6.1%) of students below the adequate student performance.
language, the estimated eﬀect is negative, but it level of mathematics proﬁciency.
However, everyone agrees that the reopening
is not statistically diﬀerent from zero (Figure 1).
These results also suggest that the eﬀect was of schools should only happen when the health
The higher eﬀect on math is in agreement with
higher among the most vulnerable students, of students and teachers can be guaranteed,
Baker’s work (2013) and with the literature that
since local schools with the worst grades, in line with the recommendations of health
explores the eﬀect of summer break on learning
compared to top 20% schools, had a higher professionals. In the process of returning
in the United States (Cooper et al., 1996).
proportion of students who had already dropped to schools, some of the main policies to be
Figura 1: Estimated impact in Portuguese out (7.7 percentage points higher) or repeated adopted to mitigate the negative eﬀects include:
language and mathematics for ES ﬁfth grade some grade of ES (9.6 percentage points higher) administering proﬁciency tests to identify the
students, SAEB scale and whose mother had not completed high content that should be prioritized and the
school (20.7 percentage points higher). most vulnerable students who will need special
Placebo Extension of winter break 95% CI
attention; extending the daily course load;
4.0 It is important to note that the magnitude of
Portuguese Math
shortening the planned holiday for December
the eﬀect reﬂects not only the postponement
2.0 and January; holding extra tuition; promoting
of return to school, but also the inﬂuence of
campaigns to raise awareness about returning to
0.0
other factors associated with the context of
school (via TV or digital media); and continuing
the pandemic. The risk of contagion and the
γ, SAEB scale
and improving distance learning platforms to
-2.0 average number of hospitalizations was higher
complement classroom learning.
among the municipalities that extended the
-4.0
winter break. Parents and guardians were
Notes
-6.0 likely more reluctant to send children back to 1
UNESCO and World Bank.
school, causing them to miss more school days.11 Synopsis of Primary and Secondary Education, 2019. INEP.
2
3
In the mentioned articles, the schools were closed because of the polio pandemic in 1916 (Meyers
and Thomasson, 2017); more severe winters in the USA (Marcotte and Helmut, 2008); the school
2007 2009 2007 2009
-8.0
The fear of contagion could also have aﬀected holiday period (Alexander et al., 2007); and a teachers’ strike (Belot and Webbink, 2010).
versus 2005 versus 2007 versus 2005 versus 2007 4
The return to school in the context of the Covid-19 pandemic. Todos Pela Educação. May 2020.
the leisure activities of students, potentially 5
São Paulo, Campinas, Diadema, Embu das Artes, Indaiatuba, Mairiporã, Osasco, São Bernardo do
Campo, Santo André, São Caetano do Sul, Sumaré, Ribeirão Preto and Taboão da Serra.
triggering psychological factors such as anxiety 6
The proposed estimator is the diﬀerence-in-diﬀerences: https://github.com/worldbank/h1n1-
school-closures-sp-2009.
and depression. 7
As Prova Brasil was applied in 2005 for the ﬁrst time, it was not possible to test the hypothesis
In an ideal scenario, one estimates that of parallel trends in previous years. In 2005, 03 of 13 municipalities which chose to postpone the
return to classes did not present proﬁciency data of ES 5th grade students (Campinas, Indaiatuba
In addition, the school is an environment in
the average annual gain in proﬁciency in and São Caetano do Sul). Thus, as it is not possible to check the hypothesis of parallel trends for
their local schools, they were excluded from the analysis.
which the student is protected from cases of
mathematics between the end of the ﬁfth and In 2007, in the local system of São Paulo, the standardized score of ﬁfth grade students presented
8
an average of 5.2, with a standard deviation of 0.66. Thus, a drop of -0.1 point of the grade is
domestic violence, at least temporarily. Being
ninth grade, should be 20 points on the SAEB equivalent to a 0.15 standard deviation.
9
Simon Burgess and Hans Henrik Sievertsen argue that closing schools for 12 weeks could lead to
more exposed to risky situations can also impact
scale per academic year (Alves, Soares, and a 0.06 sd drop in learning. (https://voxeu.org/article/impact-covid-19- education).
10
In 2007, in the local system of São Paulo, the mathematics score of ﬁfth grade students presented
their emotional state. All of these factors may an average of 203.41 with a standard deviation of 19.43.
Xavier, 2016). Therefore, the estimated drop 11
Until August 2009, the number of hospitalizations for H1N1 were 35 and 20 per 100,000 inhabitants
have contributed to the decreases in learning. in the treated and comparison municipalities, respectively.
of 4.5 points would correspond to 9 weeks or
about 2 months (200 school days × 4.5/20) of References
learning. This eﬀect is large, especially when one Alexander, K. L., D. R. Entwisle, and L. S. Olson(2007): “Lasting consequences ofthe summer
What to expect with Covid-19? learning gap”. American sociological review, 72, 167–180.
considers that the learning base was already low: Alves, M. T. G., Soares, J. F., e Xavier, F. P. (2016). Desigualdades educacionais no ensino
fundamental de 2005 a 2013: hiato entre grupos sociais. Revista Brasileira de Sociologia, 4(7),
even before the pandemic, 57.2% of students in 49-82.
Estimates show that the closure of schools Baker, M. (2013). Industrial Actions in Schools: Strikes and Student Achievement. The Canadian
the schools aﬀected by the measure were below Journal of Economics, 46(3), 1014–1036.
in São Paulo, in the midst of the H1N1 Bandiera, O., Buehren, N., Goldstein, M. P., Rasul, I., and Smurra, A. (2019). The Economic Lives
of Young Women in the Time of Ebola: Lessons from an Empowerment Program. The World Bank.
the adequate level of proﬁciency in mathematics.
pandemic, resulted in a signiﬁcant drop in Belot, M. and D. Webbink(2010): “Do teacher strikes harm educational attainment ofstudents?”.
Labour, 24, 391–406.
To check whether the policy had distributional student performance in mathematics for ES ﬁfth Cooper, H., B. Nye, K. Charlton, J. Lindsay, and S. Greathouse(1996): “The eﬀects of summer
vacation on achievement test scores: A narrative and meta-analytic review”. Review of educational
eﬀects, we estimate treatment eﬀects on grade students. The two to three week extension research, 66, 227–268.
Eyles, A., Gibbons, S., and Montebruno, P. (2020). Covid-19 school shutdowns: What will they do
the quintiles of the mathematics proﬁciency of July holidays is estimated to be equivalent to our children’s education? (No. cepcovid-19-001). Centre for Economic Performance, LSE.
Marcotte, D. E. and S. W. Hemelt(2008): “Unscheduled school closings and student performance”.
distribution. The eﬀect is negative throughout to the loss of two months of learning. If this Education Finance and Policy, 3, 316–338.
McEwan, P. J. (2015). Improving learning in primary schools of developing countries: A meta-
analysis of randomized experiments. Review of Educational Research, 85(3), 353-394.
the distribution, but higher in the ﬁrst quintile estimate is a good proxy of what will happen Meyers, K. and M. A. Thomasson(2017): “Paralyzed by panic: Measuring the eﬀectof school
closures during the 1916 polio pandemic on educational attainment”. Tech. rep., National Bureau
(-5.2 points on the SAEB scale). The strongest with the actual school closures, one can expect of Economic Research.
Todos pela Educação. O retorno às aulas presenciais no contexto da pandemia da Covid-19. Maio
eﬀect on the lower tail suggests that the a learning loss equivalent to half-semester of de 2020.
The Covid-19 Pandemic: Schocks to Education and Policy Responses. World Bank Grouo. Maio
negative eﬀects were higher among students a school year. Although it is not possible to de 2020.
World Bank, June 2020