Policy Research Working Paper 10879 Learning When Schools Shutdown Impacts of H1N1 Outbreak on Learning Loss and Learning Gaps Vivian Amorim Caio Piza Ildo José Lautharte Jr. Development Economics A verified reproducibility package for this paper is Development Impact Group available at http://reproducibility.worldbank.org, August 2024 click here for direct access. Policy Research Working Paper 10879 Abstract This paper contributes to the growing body of evidence on performing below the expected proficiency level in math, the effects of school closures on learning outcomes, focus- suggesting disproportionate effects on schools with a higher ing on a recent event in a developing country. During the percentage of academically challenged students. Moreover, 2009 H1N1 pandemic outbreak, a significant number the research underscores the role of school decentralization, of public schools in São Paulo state, Brazil, extended the revealing that municipal schools exhibited greater resilience winter break by two to three weeks. By employing double- in mitigating the negative shock compared to state-run and triple-difference estimates, the study reveals that even schools. This observation aligns with the broader evidence such a relatively short period of school closure can result highlighting the advantages of decentralized governance in a learning deficit equivalent to six to nine weeks of reg- structures in responding to crises within the education ular schooling. Furthermore, the findings indicate that the sector. adverse impacts were more pronounced among students This paper is a product of the Development Impact Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at viviamamorin5@gmail.com, caiopiza@worldbank.org, and ilautharthe@worldbank.org. A verified reproducibility package for this paper is available at http://reproducibility.worldbank.org, click here for direct access. RESEA CY LI R CH PO TRANSPARENT ANALYSIS S W R R E O KI P NG PA The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Learning When Schools Shutdown: Impacts of H1N1 Outbreak on Learning Loss and Learning Gaps Vivian Amorim* Caio Piza† Ildo José Lautharte Júnior ‡§ JEL Codes: C21, H12, I18, I21 * Education Division at the Inter-American Development Bank. Contact: vamorim@iadb.org † Development Impact Department (DIME) at the World Bank. Contact: caiopiza@worldbank.org ‡ LAC Education Unit at the World Bank. Contact: ilautharte@worldbank.org § The authors wish to express their sincere gratitude to Christian Lehmann and Rafael Terra for their significant contributions, to Arianna Legovini and an anonymous referee. We thank the World Bank Brazil Country Office and the DEC Support Budget for funding support. 1 Introduction School closures are one of the most drastic measures to contain the spread of infectious diseases. Existing evidence shows that school shutdowns, even for a short period, can have large negative effects on students’ numeracy and literacy skills (Andrabi et al. (2020), Marcotte and Hemelt (2008)). Although most of the existing evidence applies to developed countries, and more recently the development community is eager to understand the consequences of the long period of school closure during the COVID-19 pandemic crisis on learning loss (Lichand et al., 2021; Hevia et al., 2022; Azevedo et al. (2020); Donnelly and Patrinos (2021)), our study focuses on the impacts of a relatively recent episode of school shutdown in a large developing country. In July 2009, amid the H1N1 outbreak, the Health Department of São Paulo state, Brazil, recommended postponing the end of winter break due to the increasing number of H1N1 cases.1 All state schools extended the winter break by two to three weeks. A few municipalities in the state followed the state Health Department guidelines, whereas the majority did not change the school schedule. The shutdowns affected 12,957 state and municipal schools and more than 5.5 million students (70% of the students). We leverage the variations within and across municipalities over time to estimate the impacts of school closures on students’ learning loss and learning gaps. We first explore the policy variation across municipalities using a difference-in-differences (DiD) design to estimate the impacts of school closure on students’ learning in municipal schools. We then leverage a within-municipality variation across municipal and state schools using a triple difference-in-differences design to estimate the impact of the shutdowns on learning outcomes in state-managed schools. To assess whether this episode of school closure had distributional effects, we estimate quantile treatment effects using the changes-in-changes design. The results show that extending the winter break by two to three weeks reduced math scores by at least 0.18 of a standard deviation in municipal schools and 0.26 in state schools. These estimates are equivalent to at least six weeks of learning loss. We find evidence that the impact was slightly stronger in schools in the bottom deciles of the math test score distribution, suggesting that the effects of the school shutdown were more detrimental in schools with a higher share of students 1 We focus on the state of São Paulo since the state has the largest public network in the country and we could find, by checking local newspapers, the names of the municipalities whose local authorities opted to extend the winter break due to the pandemic crisis. 1 already behind in math skills. The effects on reading skills reached at least 0.19 of a standard deviation, but they are restricted to state schools. This result suggests that the impacts of school closure vary across schools’ levels of administration - i.e., whether the education policy is centralized at the state level or decentralized to the municipality (local authority). These findings are consistent with the existing evidence on the impacts of school closures on learning, even though the point estimates are slightly larger than what has been documented in developed countries. We investigate three potential mechanisms underlying these results: principals’ managerial skills, teacher absenteeism, and the shorter time frame to cover school curricula in closed schools. First, for the state-managed schools, we find suggestive evidence that the higher the teachers’ perception of principals’ skills, the more the negative impacts of the shutdowns are offset. Second, in state-managed schools where teacher absenteeism is seen as a big issue, the learning loss is at least 60% higher compared to state-managed schools where absenteeism is not a concern. The data suggest that state-managed schools did not extend the length of the school year to compensate for the period of shutdowns. Finally, the shorter length of an average school day seems to have made learning recovery more challenging for locally managed schools. This paper has three contributions. First, it provides evidence of the effects of school closure in a developing country, where mitigating mechanisms are more limited than in developed contexts. Second, it shows that locally managed (municipal) schools coped better with the negative school shock. This result is consistent with the school decentralization literature that points out that local authorities, which are closer to where the policy is delivered, can better identify and act on population needs. Third, it suggests that some schools’ characteristics contributed to offsetting the adverse effects of school closures, giving rise to direct implications for policy recommendations. Apart from this introduction, the paper is organized as follows: Section 2 provides some background information on the H1N1 outbreak in the Brazilian context. Section 3 describes the data used and presents the descriptive statistics. Section 4 discusses the empirical strategy, whereas Section 5 presents the main findings. We then conclude with a discussion and policy implications in Section 6. 2 2 Background: The H1N1 pandemic and school closures in São Paulo In June 2009, a new influenza outbreak, H1N1 (also known as swine flu), was declared a pandemic by the World Health Organization (WHO).2 From April to December 2009, Brazil confirmed 54,171 cases and 2,196 deaths from H1N1.3 However, the number is likely to be much higher as many people with flu symptoms do not seek help, not all the ones that look for health assistance are tested, and the under-reporting of hospitalizations and deaths are publicly known. At the end of June 2009, amid the increasing number of cases, the Health Department of the State of São Paulo, where 8% of all national cases were registered, recommended that public schools extend the winter break of July 2009 for a few weeks. The recommendation was based on the fact that children can act as vectors of transmission as they are less likely to adopt behavioral changes, such as washing hands regularly and avoiding physical contact for prolonged times (Klaiman et al. (2011)).4 The state authority postponed the return to school for two weeks in all the schools under their management across the 645 municipalities of São Paulo state. However, each local authority of the 645 municipalities had the autonomy to decide whether or not to follow the state guidelines. By checking local newspapers from 2009, we found that 13 local authorities followed the guidance of the state Health Department (Figure A.1)5 . These 13 municipalities postponed the winter break of their locally-managed schools for two to three weeks. These municipalities are among the one-fifth most populous in the state. On average, they concentrated most confirmed cases of H1N1 per 100,000 inhabitants between April and July in the state (10 versus 2 cases, Figure A.2). Overall, more than half of the public schools were closed, affecting about 70% of the students (Table 1). 2 By May 2010, 214 countries had reported cases and an estimated death toll of more than 200,000 people (The World Health Organization (2010), Dawood et al. (2012)). 3 DATASUS, 2009. Available through: http://tabnet.datasus.gov.br/cgi/tabcgi.exe?sinannet/cnv/influbr.def. 4 Check the official statement of the state government of São Paulo in this link: https://www.saopaulo.sp.gov.br/ultimas-noticias/nota-oficial-da-secretaria-da-saude-sobre-retorno-as-aulas/. 5 São Paulo, the state capital, Campinas, Diadema, Embu das Artes, Indaiatuba, Mairiporã, Osasco, São Bernardo do Campo, Santo André, São Caetano do Sul, Sumaré, Ribeirão Preto and Taboão da Serra 3 Table 1: State and locally-managed schools in São Paulo (2009) Students Schools Total Pre-K Primary Lower Upper Total Pre-K Primary Lower Upper Education Secondary Secondary Education Secondary Secondary Schools shutdown, extension of the winter-break State-managed schools 4,338,887 1,193 852,116 1,870,373 1,615,205 9,787 39 2,144 3,833 3,771 located in all the 645 municipalities in the state of São Paulo Locally-managed schools 1,202,386 454,499 454,703 281,392 11,792 3,170 1,579 972 596 23 located in 13 out of the 645 municipalities sin the state of São Paulo Total 5,541,273 455,692 1,306,819 2,151,765 1,626,997 12,957 1,618 3,116 4,429 3,794 No school shutdown, no extension of the winter break Locally-managed 2,458,858 727,745 1,380,053 326,645 24,415 12,192 6,204 4,956 947 85 4 located in 632 out of the 645 municipalities in the state of São Paulo % of students affected by the shutdowns % of schools affected by the shutdowns 69.2% 51.5% Notes: Pre-K: Kindergarten and pre-school. Primary Education: first to fifth grades. Lower secondary: sixth to ninth grades. Upper secondary education: tenth to twelfth grades (high school). The locally-managed schools: schools managed by the local authorities of the 645 municipalities in São Paulo. The state-managed schools: schools managed by the state government of São Paulo. Source: Census of Education, 2009. We use a linear probability model to uncover whether some observed characteristics at the municipal level help explain the 13 municipalities’ self-selection. We regress a dummy variable that takes the value of 1 if the municipality extended the winter break of municipal schools in 2009 and 0 otherwise on a vector of pre-determined covariates: the municipality GDP per capita in 2007, the municipality population size in 2007, the number of confirmed cases of H1N1 per 100,000 inhabitants (until July 25, 2009), and municipal averages of students’, teachers’ and principals’ characteristics. (Table A.2).6 Although several coefficients are statistically significant, most are very small. The adjusted R2 suggests that the municipalities’ observed characteristics do not explain much of the local authority’s decision to follow the state guidelines. 3 Data The Brazilian public education system is decentralized. The 26 states, the Federal district, and the 5,570 municipalities share the responsibilities of provisioning education. According to the Brazilian Constitution of 1988, the municipal governments should prioritize early childhood, primary (grades 1 to 5), and lower secondary education (grades 6 to 9), and the state authorities should prioritize primary, lower secondary, and upper secondary education (grades 10 to 12). Hence, the schools located in all the Brazilian municipalities can be managed either by the state government or by the municipal government. We will refer to the first group of schools as state- managed and the second group of schools as locally-managed. In each of the 645 municipalities in São Paulo state, there is at least one locally-managed school and one state-managed school offering either preschool, primary, lower, or upper secondary education. 6 All the descriptive statistics presented are relative to 2007 and based on the Brazilian Ministry of Education questionnaires that students, teachers, and principals fill out. Tables are available in the online appendix. The principal managerial skills is an index ranging from 0 to 1. It is calculated based on teachers’ answers of how frequently they believe that the principal pays attention to students’ learning, administrative norms, and school maintenance; motivates the teachers and encourages new ideas, and takes into consideration teachers’ inputs; and whether teachers trust the principal and can participate in decisions related to their work. All these variables have four possible answers: never (value 0), sometimes (0.33), often (0.66), and always (1). The principal managerial skills index at the teacher level is an average of these answers, and the index at the school level is an average of teachers’ answers. The student motivation index is similarly defined. It is calculated based on the teachers’ answers on whether students’ learning deficit is caused by low student motivation or bad behavior in the classroom. These variables have two possible answers: yes (0) and no (1). The index includes the variable ’students’ absenteeism’ which can take three possible values: a moderate/big issue (0), a small issue (0.5), and not a problem (1). The student motivation index is an average of these answers. Finally, the teacher motivation index also ranges from 0 to 1, and it is calculated based on students’ answers of how frequently the teacher corrects their reading and math homework. The variable has three possible answers: always (1), sometimes (0.5), and never (0). The teacher motivation at the school level averages students’ answers. 5 The Brazilian legislation for primary, lower, and upper secondary education determines a school year with a minimum of 200 days and 800 hours of instruction time. The state-managed schools are monitored by the State Department of Education, under the state government administration, and locally-managed schools are monitored by the Municipal Department of Education, under the municipal government administration. The respective Departments of Education are in charge of educational policies implemented at the school level, hiring teachers, providing textbooks, appointing principals, financing school infrastructure, and determining the length of the school day, and school breaks, as well as the beginning and the end of the school year in the schools under their responsibility. Usually, the academic year starts in February, and ends in November, summer break occurs in December and January, and winter breaks cover three to four weeks in July. Data on students’ proficiency in reading and math come from the School Census, the Brazilian Institute of Geography and Statistics (IBGE), and from Prova Brasil, which is the standardized national proficiency exam used to assess students’ learning in primary and lower secondary education.7 Since 1995, all private and public schools offering primary and secondary education have participated in the annual School Census. The Census is implemented by the National Institute of Educational Studies and Research (INEP), a research agency under the Brazilian Ministry of Education.8 The School Census collects information on (i) school facilities, such as libraries, sports courts, and science and computer labs; (ii) school infrastructure such as filtered water, electricity, and internet access; (iii) social services, for example, school transportation and provision of meals; (iv) students, such as gender, color of the skin, age, physical disabilities or mental illness, grade level, instruction time per day, class-size, subjects they are enrolled in, grade promotion, repetition and dropout rates; and (v) teachers, such as educational attainment, age, physical disabilities, subjects taught, and classes they are in charge of. Every two years, the INEP applies a standardized national exam, Prova Brasil, to assess students’ proficiency in reading and math. Since 2005, the test has been applied to 5th and 9th graders of all public schools.9 Prova Brasil is one of the proficiency tests within the scope of 7 IBGE stands for Instituto Brasileiro de Geografia e Estatística. IBGE has information on GDP per capita and the population at the municipality level. 8 INEP stands for Instituto Nacional de Estudos e Pesquisas Educacionais. 9 Schools with at least 20 students enrolled in fifth or ninth grade. Proficiency tests are also applied to students in the last grade of high school. 6 the Education Assessment System (SAEB).10 Students take the test at the end of the school year (between October and November). In 2007, children took the test between November 5 and 20; and in 2009 between October 19 and 31 (approximately, two months after the school shutdowns). For fifth-graders, proficiency in reading has a scale ranging from 0 to 325, and math has a scale ranging from 0 to 350 (SAEB scale). The student’s proficiency can be classified as insufficient, basic, or advanced. In addition to answering reading and math questions, students filled out a socioeconomic ques- tionnaire with information on their parents’ educational attainment; incentive from the family to pursue an education; time watching TV, time spent on the internet, time dedicated to read- ing books, and to homework; if they already dropped out or repeated a grade; and if they at- tended kindergarten. Data from the School Census and Prova Brasil are used to calculate the National Education Development Index (IDEB), the most important educational indicator in Brazil, that monitors students’ grade promotion and learning levels.11 Brazilian State and municipal governments use this indicator to monitor the quality of public education and compare the schools’ performance within and between municipalities. 4 Empirical strategy To estimate the treatment effects of the 2009 school closures due to the H1N1 outbreak on fifth- graders proficiency in reading and math, we explore the policy variation between and within the municipalities of the state of São Paulo. We focus the analysis on grade five because only five of the 13 municipalities that opted for extending the winter break had proficiency scores available for grade nine in 2007, our ’baseline’ year. To estimate the impact of the shutdowns on learning, we first look at the impacts on locally-managed schools. To do so, we explore the policy variation between municipalities 10 SAEB stands for Sistema de Avaliação da Educação Básica. 11 IDEB stands for Índice de Desenvolvimento da Educação Básica. To compute the index, the student’s reading and math performance is transformed into a standardized proficiency ranging from 0 to 10. The index is then multiplied by the student’s grade promotion rate (on a scale from 0 to 1) to obtain the IDEB at school, municipal, state, and country levels. For example, for primary education, the index is the product of the standardized performance of fifth-graders and the average grade promotion from first to fifth grade. 7 using a difference-in-differences design. We compare locally-managed schools in the municipalities whose local authorities extended children’s winter break (treatment group) with locally-managed schools in the municipalities that did not change the school calendar (comparison group). For locally-managed schools, student-level proficiency in Prova Brasil has been available since 2005.12 However, this proficiency assessment does not include all the primary education schools, mostly due to their size.13 A key assumption of the DiD design is that in the absence of the policy, the trajectories in proficiency in treated and comparison groups would be the same. To test the plausibility of the parallel trends assumption, we compare the proficiency of both groups before the school closures (2005 and 2007). Due to data limitations, the test is restricted to a subset of treated (10 out of 13) and comparison municipalities (469 out of 629) (Table 2). Because the estimation takes the municipalities as clusters, we use wild bootstrap to estimate standard errors to account for the small number of treated clusters. To estimate the impact of school closures on state-managed schools, we explore a within-municipality variation. For the same municipality, we can compare test scores in both state and locally-managed schools in which students experienced the same shocks at the municipality level. We restrict the sample to municipalities with at least one state and one locally-managed school offering primary education. This is the case of 112 municipalities where the local authority did not extend the winter break (a group that we classify as = 1) and for 10 out of the 13 municipalities that did ( = 0). For schools in = 1, we compare students’ outcomes of 929 state-managed schools that extended winter break with 1,334 locally-managed schools that remained open before and after the H1N1 shock. For schools in = 0, we compare students’ outcomes of 868 state-managed schools against 759 locally-managed schools, all closed before and after the shutdown. The triple difference leverages these two double differences in G0 and G1 respectively. In doing so, it accounts for any differences in the proficiency trajectories of state and locally-managed schools within and across municipalities (Table 2). 12 Proficiency data at the school level for state-managed schools are available starting in 2007. 13 As described in Section 3, to participate in the assessment, the class size has to have at least 20 students in grade 5. 8 Table 2: Sample of the study, São Paulo (2005-2009) (A) Triple difference-in-differences. Locally and state managed schools Municipalities where Other municipalities where the local authorities extended the winter break (G = 0) the local authorities did not extend the winter break (G = 1) State Locally State Locally managed managed managed managed network network network network Number of: Municipalities 10 10 112 112 Schools 869 759 929 1,334 Students 144,655 110,606 99,172 139,017 9 (B) Difference-in-differences. Municipalities in São Paulo where there is at least one school managed by the local authority Locally-managed schools only Municipalities where Other municipalities where the local authorities extended the winter break (G = 0) the local authorities did not extend the winter break (G = 1) Locally Locally managed managed Number of: network network Municipalities 10 469 Schools 795 2,569 Students 118,804 239,778 Notes : To estimate the DiD, we consider all locally-managed schools with proficiency available since 2005 to be able to use more than one data point before the school closures. For the triple difference design, our sample includes the municipalities with at least one state and one locally-managed school available. The rows students show the number of students enrolled in fifth grade. Source: Census of Education, 2009. 4.0.1 DiD on the sample of locally-managed schools To estimate the impacts of the school shutdowns on the locally managed schools we estimate the following regression equation: ′ = 0 + 1 + 2 1 1 + × 1 1 + 3 + + (1) in which is the school-average test score of fifth-graders in school , in municipality , in year ; is a dummy that takes the value of 1 if municipality extended the winter break of all municipal schools in 2009, and 0 otherwise; 1 1 is a dummy that is equal to 1 in 2009, ′ and 0 in 2007; is a vector of school characteristics, and school-average students’, teachers’, principals’ characteristics; and are municipalities fixed effects. The parameter of interest, , is the average treatment effect on the treated (ATT), i.e., the causal impact of school closures on learning. Standard errors are clustered at the unit of intervention level, i.e., the municipality. Because the number of clusters in the treatment group is small (10), standard errors are estimated using wild-bootstrap with 1,000 replications (Roodman et al. (2019)). All the regressions are weighted by the fifth-grade enrollment at the school level. Intuitively, the effect of the winter break extension is estimated by comparing the differences in the average test scores between the treatment and comparison groups in 2009 (after the pandemic) and 2007 (before the pandemic). To claim the causal effects of the school closures on students’ proficiency, the comparison group is assumed to emulate what would have happened with the average test scores in the treatment group in the absence of the school shutdowns. As shown in Figure 1, we find suggestive evidence that the learning outcomes in both T and C groups were on the same time trend before the H1N1 outbreak. 10 Figure 1: Students’ proficiency in locally-managed schools, fifth-grade (2005-2009) Extended winter break Extended winter break Other municipalities Other municipalities Contrafactual Contrafactual 220 200 210 Test score, SAEB scale Test score, SAEB scale 190 200 190 180 180 170 2005 2007 2009 2005 2007 2009 (a) Math (b) reading Note: For fifth-graders, the proficiency scale (SAEB scale) ranges from 0 to 350 in reading, and from 0 to 350 in Math. The proficiency for treatment and comparison groups is the average of the proficiency scores of a sample of locally-managed schools in Sao Paulo state, the ones that have the performance of fifth-graders available since 2005. These schools are located in 10 municipalities of the treatment group and 469 municipalities of the comparison group (Table 2). The contrafactual is calculated assuming that the average score of the treatment group would increase at the same pace as that of the comparison group in the absence of the school closures. Because the first year of Prova Brasil assessment was 2005, we cannot test for parallel trends using a longer time frame. Figure A.3 shows the time-trend for grade promotion, repetition, and dropout rates. Source: Data from Prova Brasil /INEP. Aiming to increase precision and account for potential time-variant confounders, we include a vector of covariates at the school level, as these variables might be correlated with both the decision of the municipality to close its schools and students’ performance in the standardized tests. We use a Lasso (Least Absolute Shrinkage and Selection Operator ) regression to select ′ , that best predict the variation in the school average proficiency the vector of covariates, score (Ahrens et al. (2019)). The vector of covariates includes dummy variables indicating whether the school has a science and computer lab, sports court, library, and access to the internet; instruction hours per day; the number of students per class; GDP per capita of the municipality; and socioeconomic characteristics of fifth-graders. The vector of socioeconomic variables includes the percentage of mothers with a high school diploma; the percentage of students that have already been retained in one specific grade or dropped out of school; the percentage of white students; the percentage of female students; the percentage of students that work for pay; the percentage of students that previously studied in a private school; the percentage of students who have a computer at 11 home; the percentage of students whose parents incentivize them to study, to do the homework, to read, to not miss classes, and that talk about what happens in the school; the percentage of teachers with tenure; whether there is a lack of textbooks in the school; the quality of the textbooks; the number of enrollments in the school; and whether the students are allocated into their classes at the beginning of the school year based on their previous academic performance.14 The sample used in this analysis consists of 795 schools with 118,804 fifth-grade students in the treatment group and 2,568 schools with 239,778 students in the comparison group (Table 2). 4.0.2 Triple DiD on the sample of state and locally-managed schools Because the DiD design considers only 13 treated municipalities and these are considerably larger than most municipalities in the comparison group, one may wonder whether some unobserved time-varying factors could bias the estimates through the selection mechanism. For instance, it could be that the municipalities decided to postpone school return because they had more resources and better infrastructure to put in place mitigating policies later on; or because these municipalities registered the higher incidence of H1N1 cases, it is more likely that parents, teachers and students of these municipalities were more concerned with the risks of contamination and this concern affect their both teachers and students’ attendance rates. As long as these factors could affect learning, the DiD estimates would be biased. To overcome this identification threat, we leverage a within-municipality variation. We consider the municipalities with at least one state and one locally-managed school and split them into two groups. Let = 1 denote the 112 municipalities where state-managed schools were closed and locally-managed schools remained open, and = 0 the ten municipalities where state and municipals schools were closed (Table 2). We assume that idiosyncratic shocks at the municipal level affect students and schools (either state or locally managed) similarly. Due to the differences in the learning trajectories between state and locally-managed schools pre- school shutdown, a simple comparison of state and locally-managed schools in = 1 would likely 14 Table A.3, Table A.4, Table A.5, Table A.6, Table A.7, Table A.8, Table A.9 present the descriptive statistics for these covariates. The model does not include variables that might have been affected by the shutdowns, such as absenteeism of students and teachers, principal managerial skills, student motivation, and teacher motivation. By including these variables, it would be more likely to underestimate the effects of the shutdowns. 12 lead to biased estimates of the impacts of the school shutdowns (Figure A.4).15 Between 2007 and 2009, there was a higher increase in the students’ test scores in the state-managed schools that extended the winter break compared to test scores in municipal schools that remained open which could suggest that the state government was more successful than locally managed schools in counterbalancing the negative effects of school shutdowns.16 To account for different learning trajectories across state and locally-managed schools, we compare state and locally-managed schools in = 1 and = 0 using a triple difference-in-differences design (TD). This design accounts for three sources of variations: time variation, across-municipality variation, and within-municipality school variation. The TD is computed as the difference between two double differences. The first double difference (Δ1 ) is given by the differences in average proficiency across the state (closed), and locally-managed schools (opened) in = 1, between 2007 (before the shutdowns) and 2009 (after the shutdowns). The second double difference (Δ2 ) consists of differences in average proficiency across state and locally-managed schools between 2007 and 2009 in = 0. The TD is given by (Δ1 − Δ2 ) and it therefore measures the effect of the shutdowns on average proficiency in state-managed schools (Muralidharan and Prakash (2017)). Given the full compliance with the policy mandate, we run the following regression equation to estimate the ATT in the TD setup: =0 + 1 + 2 + 3 1 1 + 4 × + 5 × 1 1 ′ + 6 × 1 1 + × × 1 1 + 7 + + in which is equal to 1 if school in municipality is state-managed and 0, otherwise. ′ is a vector consisting of school average students’, teachers’, principals’ characteristics, and schools’ characteristics.17 The parameter of interest, , is the average treatment effect on the treated (ATT), that is, the average effect of school closures on student learning in state- managed schools. The standard error is clustered at the municipality level. All the regressions 15 The first column of Table A.11 and Table A.12 shows that state and locally-managed schools did not follow the same time trend. 16 In fact, in 2008 the state government implemented a program aimed at increasing managerial practices of schools in the bottom 5th percentile of the test score distribution. Despite the relatively small number of state- managed schools targeted by this program (621 of 5,977), we excluded them from the evaluation sample to mitigate the risk of bias. In 2009, the state government had 5,977 state-managed schools offering first to ninth grades. These schools were the target population of the school management upgrading program. 17 The controls were selected using a Lasso regression. 13 were weighted by the fifth-grade enrollment at the school level. Table A.11 shows that with this strategy, we can control for the differences in learning trajectories between state and locally- managed schools across municipalities, as the coefficient of × × is insignificant before the shutdowns (there is no significant difference in the learning gap of state and locally-managed schools in = 1 and = 0). To summarize: ∙ = 1 are the 112 municipalities where the municipal governments did not extend the winter break, and where at least one state and one locally-managed school offering first to fifth grade. These municipalities have 929 state-managed schools with 99,172 students and 1,334 locally-managed schools with 139,017 students. – locally-managed schools opened. – state-managed schools closed. – 1 difference-in-differences: 1 0 1 0 Δ1 = (, =1 − ,=1 ) − (,=1 − ,=1 ) = + 5 (2) 1 – in which , 0 =1 and ,=1 are the proficiency of state-managed schools in = 1 1 in = 1 (2009) and = 0 (2007), respectively; and , 0 =1 and ,=1 are the proficiency of locally-managed schools in = 1 in = 1 and = 0, respectively. ∙ = 0 are the 10 municipalities in which the municipal authority extended the winter break, and where there is at least one state and one locally-managed school. These municipalities have 868 state-managed schools with 144,655 students and 759 locally- managed schools with 110,606 students. – locally-managed schools closed. – state-managed schools closed. – 2 difference-in-differences: 1 0 1 0 Δ2 = (, =0 − ,=0 ) − (,=0 − ,=0 ) = 5 (3) 1 – in which , 0 =0 and ,=0 are the proficiency of state-managed schools in = 0 14 1 in = 1 (2009) and = 0 (2007), respectively; and , 0 =0 and ,=0 are the proficiency of locally-managed schools in = 0 in = 1 and = 0, respectively. ∙ The TD is given by: Δ1 - Δ2 = 5 Results The estimates show that the school shutdowns during the H1N1 outbreak had a negative impact on students’ learning, especially in math. The baseline estimates point to a decrease in math scores equivalent to -0.21 and -0.28 of a standard deviation in locally and state-managed schools, respectively (Columns 1 and 4 of Table 3). The decrease in reading scores is equivalent to -0.24 of a standard deviation and is restricted to the state-managed schools. 15 Table 3: Impact of the school shutdowns on students’ learning, fifth-grade (2007-2009) Estimated decrease in Math and Portuguese Proficiency, SAEB scale Math Portuguese DiD DiD DiD TD TD TD TD TD DiD DiD DiD TD TD TD TD TD (1) (2) (3) (4) (5) (6) (7) (8) (1) (2) (3) (4) (5) (6) (7) (8) H1N1 -3.26** -3.27** -2.75** -4.35*** -4.25*** -4.38*** -4.29*** -4.25*** -0.76 -0.78 -0.54 -3.49*** -3.40*** -3.51*** -4.29*** -2.73*** sd (1.18) (1.22) (1.23) (0.96) (0.97) (0.96) (0.97) (1.06) (0.86) (0.91) (0.95) (0.81) (0.81) (0.81) (0.97) (0.90) Wild-bootstrap p-value 0.0400 0.0260 0.0410 0.0000 0.0000 0.0000 0.0000 0.0001 0.3920 0.4150 0.6060 0.0000 0.0000 0.0000 0.0000 0.0025 95% CI [-5.6,-0.9] [-5.7,-0.9] [-5.2,-0.3] [-6.2,-2.5] [-6.1,-2.4] [-6.3,-2.5] [-6.2,-2.4] [-6.3,-2.2] [-2.5,0.9] [-2.6,1.0] [-2.4,1.3] [-5.1,-1.9] [-5.0,-1.8] [-5.1,-1.9] [-6.2,-2.4] [-4.5,-1.0] N. schools 3912 3912 3912 5329 5329 5329 5329 5329 3912 3912 3912 5329 5329 5329 5329 5329 Adj. R2 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.6 Proficiency - Treatment Group before the school shutdowns (2007) Mean 193.38 193.38 193.38 195.3 195.3 195.3 195.3 195.3 175.44 175.44 175.44 177.88 177.88 177.88 195.3 177.88 Sd 15.44 15.44 15.44 16.09 16.09 16.09 16.09 16.09 15.4 15.4 15.4 15.46 15.46 15.46 16.09 15.46 ATT est (in sd) -0.21 -0.21 -0.18 -0.27 -0.26 -0.27 -0.27 -0.26 -0.05 -0.05 -0.03 -0.23 -0.22 -0.23 -0.27 -0.18 Estimated increase in the percentage of students below the basic level of learning in Math and Portuguese, in % Math Portuguese DiD DiD DiD TD TD TD TD TD DiD DiD DiD TD TD TD TD TD (1) (2) (3) (4) (5) (6) (7) (8) (1) (2) (3) (4) (5) (6) (7) (8) H1N1 2.59** 2.61** 2.20** 4.63*** 4.59*** 4.66*** 4.62*** 4.89*** 1.19 1.2 0.97 3.12*** 3.08*** 3.13*** 3.09*** 2.62*** sd (0.91) (0.94) (0.95) (0.86) (0.87) (0.86) (0.87) (0.97) (0.86) (0.88) (0.89) (0.74) (0.74) (0.74) (0.74) (0.89) Wild-bootstrap p-value 0.0310 0.0210 0.0340 0.0000 0.0000 0.0000 0.0000 0.0000 0.2170 0.2160 0.3140 0.0000 0.0000 0.0000 0.0000 0.0034 N. schools 3914 3914 3914 5330 5330 5330 5330 5330 3914 3914 3914 5330 5330 5330 5330 5330 Adj. R2 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.7 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.5 Percentage of students below the basic level of proficiency - Treatment Group before the school shutdowns (2007) Mean 57.49 57.49 57.49 56.36 56.36 56.36 56.36 56.36 70.74 70.74 70.74 69.62 69.62 69.62 69.62 69.62 Increase, in % 5.12 5.17 4.36 8.78 8.69 8.83 8.76 9.26 1.82 1.82 1.47 4.67 4.6 4.69 4.62 3.92 (A) Municipal FE Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes (B) Controls Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes (C) % teachers No Yes Yes No Yes No Yes Yes No Yes Yes No Yes No Yes Yes in both networks (D) % teachers No No Yes No No Yes Yes Yes No No Yes No No Yes Yes Yes management (E) School FE No No No No No No No Yes No No No No No No No Yes Notes: The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. Wild bootstrap p-values. 95% CI in brackets. ***, **, and * indicate significance at the 1, 5, and 10% critical levels. Standard error in parentheses and clustered at the municipality level. All regressions are weighted by fifth-grade enrollment at the school level. Math performance on a scale from 0 to 350 (SAEB scale). reading performance on a scale from 0 to 325 (SAEB scale). The columns DiD refer to the differences-in-differences estimates and the columns TD show the triple difference-in-differences estimates. The sample of municipalities and schools included in the analysis are detailed in Table 2. (A) municipal fixed effects. (B) students’, teachers’, schools’, and principals’ characteristics. (C) the percentage of teachers that work in a state and a locally-managed school simultaneously. (D) the percentage of teachers working in a state-managed school that implemented the intervention of managerial practices. (E) schools’ fixed effects. All TD estimates exclude state-managed schools in which the state government implemented managerial practices intervention. The stronger negative effects on math are in agreement with other estimates of school shutdowns on learning outcomes (Kuhfeld et al. (2020), Thum and Hauser (2015), Baker (2013), and Cooper et al. (1996)). The available evidence suggests that out-of-school enrichment activities during school breaks tend not to focus on math, which is associated with both math anxiety and new instructional methods that differ from what parents themselves learned. In this context, instead of math, families tend to focus more on the promotion of literacy skills at home (McCombs (2011), Murnane (1975), Bryk and Raudenbush (1989), Allinder et al. (1992), Harris and Sass (2009)). These factors help to explain why math skills depreciate faster than reading skills, why school shutdowns have larger negative effects on math than on reading, and inform policies aimed at mitigating learning losses caused by unexpected shocks that can result in school closures. Because the interpretation of learning gains and losses in terms of standard deviation is not straightforward, we convert the estimates into expected years of schooling. To do so, we use the ideal learning gain of an average student between the fifth and ninth grades in the Brazilian school system as a benchmark. An average student who does not repeat a grade should experience an annual increase in proficiency of 20 points in the national standardized exam, on a learning scale that ranges from 0 to 325 for reading and 0 to 350 for math (Alves et al. (2016)). Therefore, the estimated drop in math performance of -3.3 and -4.6 points corresponds to at least six and nine weeks of learning loss in locally and state-managed schools, respectively.18 Given the relatively large effects of the winter break extension on learning loss, we test whether the estimated drop in test scores increased the percentage of students below the basic learning level according to the SAEB scale. The baseline estimates shown in Table 3 point to an increase in the percentage of students below the basic level in math proficiency of 2.6 and 4.9 percentage points in locally and state-managed schools, respectively (a rise of 4.5% and 8.7%, compared to baseline levels). For reading, the point estimate is 3.4 percentage points in state-managed schools (an increase of 4.9% over the baseline levels). The relatively small increase in the proportion of students below the basic learning level suggests that the adverse impacts of the school closures were disproportionately higher among those who were already behind even before the H1N1 pandemic. In fact, in 2007, almost 60% and 70% of 18 200 school days × 3.3/20 = 32 days, or 6.6 weeks (considering five school days a week). 200 school days × 4.6/20 = 46 days, or 9.2 weeks (considering five school days a week). 17 fifth-graders in the affected schools did not achieve basic levels of learning in math and reading, respectively.19 School shutdowns can increase learning gaps between low and top performers if their impacts are more detrimental for low performers.20 To investigate whether the impacts of the school shutdowns are more pronounced at the lower tail of the learning distribution, we estimate quantile treatment effects using the Athey and Imbens (2006) changes-in-changes (CiC) estimator.21 Figure 2 shows that the negative effects of the school’s shutdown were stronger in math, and potentially stronger in the lower tail of the test score distribution, as suggested by the inter-quantile difference between the lower and top deciles estimates in math. Figure 2: Distributional impacts of the school shutdowns, fifth-grade (2007-2009) Extended winter break 95% CI Extended winter break 95% CI 2.0 2.0 0.0 0.0 -2.0 γ, SAEB scale γ, SAEB scale -2.0 -4.0 -4.0 -6.0 -6.0 -8.0 -10.0 -8.0 10 20 30 40 50 60 70 80 90 10 20 30 40 50 60 70 80 90 Percentiles of test score distribution Percentiles of test score distribution (a) Math, TD (b) Reading, TD Note: The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. 95% Confidence Interval. Math performance on a scale from 0 to 350 (SAEB scale). reading performance on a scale from 0 to 325 (SAEB scale). Figures (a) and (c) show the estimates for the TD. Figures (b) and (d) show the estimates for DiD. All regressions are weighted by fifth-grade enrollment at the school level. Standard errors clustered at the municipality level. The controls are municipal fixed effects, students’, teachers’, schools’, and principals’ characteristics, the percentage of teachers that work in a state and a locally-managed school at the same time, and the percentage of teachers that work in a state-managed school that implemented the managerial practices intervention. The triple difference specifications also include schools’ fixed effects. All triple DiD estimates exclude state-managed schools in which the state government implemented managerial practices intervention. We perform the estimates using the cic (changes in changes) Stata command. The more detrimental effects in the state-managed network might indicate that locally-managed 19 The share of students with insufficient performance in 2007 considers all state and locally managed schools affected by the shutdowns. 20 We also assess the effects of the school shutdowns on grade promotion, retention, and dropout. DiD estimates indicate a small increase in promotion rates and a decrease in school retention and dropout rates in locally- managed schools. Triple difference estimates point to nil effects on those outcomes in state-managed schools (Table A.14). 21 We use the command cic in Stata to perform the estimation. This command allows the estimation of the confidence intervals with bootstrapped standard errors. However, it does not perform the Wild Bootstrap to adjust the standard errors when the number of clusters is relatively small. 18 schools could better respond to the students’ needs when schools reopened. Whether the impact of school closures on learning varies with the school’s administration level is still an open question in the literature. Several papers identify benefits of decentralized management of education (Galiani et al. (2008), Faguet (1999), Jimenez and Sawada (1999), Filmer (2002), King and Ozler (2000)). Having a policymaker close to the population helps identify households’ needs. In the context of school shutdowns, parents of students of municipal schools have lower transaction costs to demand actions from the local authority to help students catch up with the school curriculum. 5.1 Robustness Checks Effects on municipal schools In the municipalities where the local authorities opted not to extend the winter break, 11% of the contracted teachers of their locally-managed schools also worked on at least one state- managed school that adopted the school shutdown.22 It could be that some of these teachers opted not to lecture, making students from the locally-managed network miss school content even though the municipal government opted not to extend their winter break. To deal with this potential spillover effect, we rerun the regressions controlling for the percentage of teachers working in a state and a locally-managed school. The estimates are shown in Column 2 of Table 3 and are very similar to the baseline specification. As pointed out in Section 4, the DiD analysis is run on a sample of locally-managed schools. Therefore, none of these schools were included in the state government’s managerial practices program implemented in 2008 and 2009. However, some teachers of the locally-managed network who also worked in a state-managed school where this intervention was implemented. This is the case for 5.4% of teachers in locally-managed schools that extended the winter break, and for 1.3% of teachers in the comparison municipalities. Even though the percentages are small, DiD could be slightly biased if these teachers took better managerial practices to the locally-managed schools. We then add as control the percentage of teachers in each one of the locally-managed schools that also work in a state-managed school that implemented the 22 School Census, 2009. 19 state-government intervention. As shown in Columns 3 of Table 3, the point estimate is lower in absolute terms, but not statistically different from the baseline specification. Effects on state schools The triple difference design considers the sample of state and locally-managed schools in the municipalities that opted to extend the winter break ( = 0) and in the municipalities that followed the school calendar as previously planned ( = 1). To assess whether the managerial practices program implemented by the São Paulo state government in 2008 and 2009 could be confounding the results, we exclude from the analysis all the 621 schools in which this program was implemented. Because 18% of state school teachers also worked in a school that received the managerial practices program, we add the percentage of state school teachers in ‘contaminated’ schools as a control variable. The estimates are similar to the baseline estimate (columns 4 and 5 of Table 3). It is interesting to notice that the first DiD of the triple differences design is run on the sample of state and locally-managed schools of the 112 municipalities ( = 1). The comparison here is among state schools that extended the winter break with the municipal schools that remained open (Equation 2 and Table 2). As pointed out earlier, a small percentage of teachers in these municipalities work in both networks and this could also affect the estimates. We then run a regression adding the percentage of teachers in this situation as a control variable. The point estimate is smaller, but not statistically different than the baseline one (Columns 4 and 6 of Table 3). The second DiD of the triple difference model considers the schools in the 10 municipalities ( = 0) where both state and locally-managed schools were closed (Equation 3 and Table 2). This estimate will inform both the time trend gap between these two school networks (5 ), and indicate how each school network reacted to the shutdown. Although the educational indicators of locally-managed schools are, on average, better than the state-managed schools in = 1 (see Table A.4 and Table A.8), the descriptive statistics presented in Table A.5 and Table A.9 suggest that the state schools seem better in some dimensions, on average, than the municipal ones in = 0. For instance, the percentage of principals who perceive teacher absenteeism as a big issue is 11 percentage points lower in state-managed schools than in locally-managed schools. 20 On average, these state schools had longer school days than locally-managed ones (5.2 hours vs. 4.7 hours) and lower teacher absenteeism. In = 0, state-managed schools also seemed to be better prepared regarding the availability and quality of textbooks.23 To deal with this heterogeneity between state and municipal schools in G0, we estimate the regressions controlling for schools, principals, teachers, and students’ characteristics, and school- fixed effects. Adding school fixed effects allow us to control for time-invariant unobserved school characteristics associated with heterogeneity in schools’ ability to respond to the shock. With all the controls the point estimates become smaller, but not statistically significant different from the baseline ones (Columns 4 and 8 of Table 3.) 5.2 Potential mechanisms We find evidence that a relatively small period of school shutdowns (2 to 3 weeks) caused by the 2009 H1N1 outbreak led to a significant decrease in students’ proficiency in math, equivalent to at least 0.18 of a standard deviation in locally-managed schools and 0.26 in state-managed ones. For reading, the estimated decrease is restricted to the state-managed network and is equivalent to at least 0.19 of a standard deviation. The magnitude of the effect, besides reflecting the short time frame to cover the school curriculum, might also reflect other factors associated with the pandemic context. It could be that teachers, parents, and students were not entirely comfortable with in-person classes soon after the schools reopened. More stressed teachers and students could negatively affect the quality of the classes, student-teacher, and peer-to-peer interactions at school. Teachers might also have been more condescending to the students after schools reopened, leading to children who would put less effort into learning. Also, there were between nine to ten weeks between the reopening of the schools and the application of Prova Brasil.24 Therefore, students and teachers did not have much time to cover the curriculum before the test. Although we do not have any data to test these hypotheses directly, we highlight a few differences on the school management side and in the availability of resources that help elucidate how state and 23 The percentage of teachers that classify the textbooks as great is six percentage points higher, and the percentage of principals stating there is a lack of textbooks is 26 percentage points lower. 24 There are between 44 and 53 business days between the reopening of the schools (August 17, 2009) and the proficiency test (which took place between October 19 and October 31, 2009). 21 municipal schools acted to counterbalance the effects of the school shutdown. In the following analysis, we dig into the heterogeneous effects of the school closure by interacting the treatment status with the ratio of students per teacher, teachers’ perception of principals’ managerial skills, teacher absenteeism, percentage of teachers with adequate university degrees to teach reading and math, and whether the teachers always correct students’ homework. Whenever possible we will also present descriptive statistics to highlight the differences between municipalities, as well as differences between state and municipal schools within municipalities. By doing this, we aim to get some understanding of how school staff and students dealt with the negative shock. School principals We first assess whether schools in which principals have better managerial skills were better able to cope with the shutdowns. The index of principal managerial skills ranges from 0 to 1. It is calculated based on teachers’ answers of how frequently they believe that the principal pays attention to students’ learning, administrative norms, and school maintenance; motivates the teachers and encourages new ideas, and takes into consideration teachers’ inputs; and whether teachers trust the principal and can participate in the decisions related to their work. For the state-managed network, we find suggestive evidence that better-prepared principals can attenuate the negative impacts of the shutdowns (Table 4). A 10% increase in principals’ managerial skills is associated with a negative impact on math proficiency 0.45 points smaller on a SAEB scale, equivalent to five school days, or an effect 10% smaller than the ATT estimate of our baseline specification of -4.6 points in SAEB scale (see Table 3). 25 For reading, the same increase in principals’ managerial skills is associated with an effect 8% smaller.26 25 As the index of principal managerial skills ranges from 0 to 1, we compare a school with the principal managerial skills at 0.91 and with a school with the index of 1 (10% higher). All the same, the effect of the school shutdown in a school with the principals’ managerial skills at 0.91 is -3.3 points on a SAEB scale (−8+5.15 × 0.91), whereas the effect in a school with a principal scoring 1 in managerial skills is -2.85 (−8 + 5.15 × 1). Since in 200 school days students are supposed to increase their proficiency by 20 points, the 0.45 (−2.85 − (−3.3) = 0.45) estimate is equivalent to 4.5 school days (200 school days × 0.45/20 = 4.5 days). 26 We perform the same exercise for reading, i.e., −5.2 + 3.3 × 1 − 5.2 + 3.3 × 0.91 = 0.3. These coefficients are shown in Table 4. Since the baseline estimate for the decrease in reading proficiency is 3.7 on a SAEB scale, the 0.3 difference is equivalent to 8%. 22 Table 4: Principals heterogeneity on the impact of the school shutdowns on students’ learning, fifth-grade (2007-2009) Effects of school shutdowns on the locally-managed network Math Portuguese DiD DiD DiD DiD DiD DiD (1) (2) (3) (1) (2) (3) H1N1 -2.75** -3.62 -2.69 -0.54 -0.34 -0.13 (1.23) (2.04) (1.62) (0.95) (1.77) (1.38) H1N1 versus 1.56 0.01 principal managerial skills (2.15) (2.30) H1N1 versus 0.1 -0.33 program to reduce dropout (0.66) (0.58) N. schools 3912 3838 3857 3912 3838 3857 Adj. R2 0.8 0.8 0.8 0.8 0.8 0.8 Effects of school shutdowns on the state-managed network Math Portuguese TD TD TD TD TD TD (1) (2) (3) (1) (2) (3) H1N1 -4.25*** -8.25*** -4.12*** -2.73*** -5.71*** -1.83* (1.06) (2.10) (1.28) (0.90) (1.69) (1.11) H1N1 versus 5.34** 4.12** principal managerial skills (2.47) (2.01) H1N1 versus -0.04 -0.75 program to reduce dropout (0.61) (0.53) N. schools 5329 5227 5239 5329 5227 5239 Adj. R2 0.7 0.7 0.7 0.6 0.6 0.6 Notes : The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. Wild bootstrap p-values. 95% CI in brackets. ***, **, and * indicate significance at the 1, 5, and 10% critical levels. Standard error in parenthesis and clustered at the municipality level. All regressions are weighted by fifth-grade enrollment at the school level. Math performance on a scale from 0 to 350 (SAEB scale). reading performance on a scale from 0 to 325 (SAEB scale). Columns DiD show the estimates of differences-in-differences as described in subsection 4.0.1 and Columns Triple D show the estimates for triple difference-in-differences as described in subsection 4.0.2. The sample of municipalities and schools included in the analysis are detailed in Table 2. 23 Teachers Teacher absenteeism is an important issue among schools impacted by the shutdowns.27 School principals of municipal schools in the municipalities that extended the winter break are almost 20 percentage points more concerned with teacher absenteeism than school principals of municipal schools in the comparison group (Table A.7). In the 112 municipalities where the state schools closed and the municipal schools remained open, the percentage of principals that saw teacher absenteeism as a big concern was 11 percentage points higher in state schools (Table A.8). Therefore, one may wonder whether the higher absenteeism augmented the effects of the shutdowns. We find evidence that teacher absenteeism exacerbated the impact of the shutdowns on math and reading proficiency by nearly 2 points on the SAEB scale in state schools (Table 5). The estimated learning loss in math and reading in a school where the principal sees teacher absen- teeism as a big issue is 45% and 60% higher compared to schools in which absenteeism is not a concern, respectively.28 Even with the shutdowns, the percentage of teachers in the state-managed network that covered at least 80% of the school curriculum is not statistically different from the municipal schools that remained open (Table A.8). Even with the shutdowns and higher teacher absenteeism, these schools found a way to cover most of the school curriculum on time. 27 Teacher absenteeism is a dummy equal to 1 if the principal sees teacher absenteeism as a big issue and 0 if the principal sees it as a moderate or small issue. 28 The estimated decrease in math and reading performance in a school where teacher absenteeism is not a concern is -3.8 and -2.6, respectively. If teacher absenteeism is a concern, the estimates are equal to -5.5 (-3.8 - 1.7) and -4.2 (-2.4 - 1.8), therefore, 45% and 60% higher, respectively. 24 Table 5: Teachers’ heterogeneity on the impact of the school shutdowns on students’ learning, fifth-grade (2007-2009) Effects of school shutdowns on the locally-managed network Math Portuguese DiD DiD DiD DiD DiD DiD DiD DiD (1) (2) (3) (4) (1) (2) (3) (4) H1N1 -2.75** -3.04 -2.28 -6.24 -0.54 0.12 -0.24 2.74 (1.23) (2.27) (1.36) (2.59) (0.95) (2.11) (1.14) (2.65) H1N1 versus 0.01 -0.03 students per teacher (0.11) (0.11) H1N1 versus -1.94 -1.04 teacher absenteeism (1.17) (1.27) H1N1 versus teachers that 0.04 correct Math homework (0.03) H1N1 versus teachers that -0.04 correct Portuguese homework (0.03) N. schools 3912 3912 3886 3912 3912 3912 3886 3912 Adj. R2 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 Effects of school shutdowns on the state-managed network Math Portuguese Triple Triple Triple Triple Triple Triple Triple Triple DiD DiD DiD DiD DiD DiD DiD DiD (1) (2) (3) (4) (1) (2) (3) (4) H1N1 -4.25*** -3.88*** -3.72*** -15.71*** -2.73*** -2.32* -2.26** -7.05** (1.06) (1.49) (1.11) (3.88) (0.90) (1.32) (0.94) (2.92) H1N1 versus -0.03 -0.03 students per teacher (0.06) (0.05) H1N1 versus -2.08** -1.74** teacher absenteeism (0.94) (0.80) H1N1 versus teachers that 0.14*** correct Math homework (0.04) H1N1 versus teachers that 0.05 correct Portuguese homework (0.03) N. schools 5329 5329 5295 5329 5329 5329 5295 5329 Adj. R2 0.7 0.7 0.7 0.7 0.6 0.6 0.6 0.6 Notes : The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. Wild bootstrap p-values. 95% CI in brackets. ***, **, and * indicate significance at the 1, 5, and 10% critical levels. Standard error in parenthesis and clustered at the municipality level. All regressions are weighted by fifth-grade enrollment at the school level. Math performance on a scale from 0 to 350 (SAEB scale). reading performance on a scale from 0 to 325 (SAEB scale). Columns DiD show the estimates of differences-in-differences as described in subsection 4.0.1 and Columns Triple D show the estimates for triple difference-in-differences as described in subsection 4.0.2. The sample of municipalities and schools included in the analysis are detailed in Table 2. 25 Length of the school day and the quality of the textbooks The fact that the state-managed network could cover as much school content as the locally- managed network that remained open is associated with the longer school hours per day (5.3 hours versus 4.9 hours). This difference is equivalent to an addition of three to four school days in the whole school year compared to locally-managed schools (Table A.4).29 However, the additional time in the classroom due to the longer duration of the classes in the state-managed schools is still inferior to the lost time in the classroom due to the period of the shutdowns. We did not find any source of information that could suggest that the state government extended the length of the school year to compensate for the days the schools were closed. The length of the school year was five days longer in the locally-managed network (Table A.4). These statistics also suggest that teachers of the state-managed network had to rush to cover the school curriculum, probably making it challenging for the students to keep up. A glance at the descriptive statistics presented in Table A.3 also highlights the challenges faced by the municipal schools affected by the shutdowns. One of the issues that stands out is that these schools could not cover the school curriculum in a shorter time frame. While 55% of municipal schools in municipalities that did not extend the winter break covered more than 80% of the school curriculum, only 45% did so in municipalities that extended the winter break. Data from Prova Brasil questionnaire show that teachers faced many barriers to successfully covering the entire school curriculum. For instance, students in treated schools had fewer hours of classes per day (4.8 hours × 5 hours). The 12-minute daily difference resulted in children from the affected schools having two days less of school content between the reopening of the schools and the proficiency test.30 Also, we do not find any source of information of an extension of school days later in the year to account for the missing days during the winter break extension. 29 There are between 44 and 53 business days between the reopening of the schools (August 17, 2009) and the proficiency test (which took place between October 19 and October 31, 2009). Therefore, the 24-minute difference would result in 1,056 to 1,272 additional minutes of class, equivalent to 17.6 to 21.2 hours. If the average number of class hours in the state-managed network in = 1 is 5.3, the estimate in terms of school days is between three and four. 30 There are between 44 and 53 business days between the reopening of the schools (August 17, 2009) and the proficiency test (which took place between October 19 and October 31, 2009). Therefore, the 12-minute difference would result in 528 to 636 fewer minutes of class, equivalent to 8.8 to 10.6 hours. If the average number of class hours in the affected network is 4.8, the estimate in terms of school days is roughly two. 26 The length of the school year was a few days shorter in the affected network (Table A.3). The availability and the quality of textbooks also seem to be a concerning point when the locally-managed network affected by the shutdowns is compared to the unaffected one. The percentage of teachers satisfied with the textbooks was 3.5 percentage points lower in treated schools, whereas the percentage of principals complaining about the lack of textbooks was 22.5 percentage points higher. The elevated teacher and student absenteeism (18 percentage points higher for teachers and 4 pp. higher for students compared to the unaffected schools) and the lack of proper high-quality textbook materials suggest that the treated schools faced more challenges in mitigating the learning loss caused by the school closure (see Table A.7). 6 Discussion and Conclusion The decision to close schools during a pandemic imposes a clear trade-off for policymakers: on the one hand, this policy helps reduce infection rates among students (Adda (2016)). On the other hand, its long-term consequences can be daunting to a whole generation of youth learners, particularly the most vulnerable. In São Paulo, Brazil, amid the H1N1 outbreak in 2009, more than half of the public primary, lower, and upper secondary schools were closed for two to three weeks, affecting more than 5.5 million students. We used double-difference and triple-difference designs to estimate the impacts of school closures on fifth-graders’ reading and math skills. We find evidence that the school shutdowns led to a reduction in math scores of at least 0.18 of a standard deviation in locally-managed schools and 0.26 in state-managed ones, equivalent to more than six weeks of learning loss. For reading, the effects reached at least 0.19 of a standard deviation, but are restricted to state schools. Quantile treatment effects estimates indicate that the effects of the school closure were higher at the lower tail of the math test score distribution. This result helps explain the relatively small increase in the percentage of students below the minimum level of math proficiency (4.5% and 8.7% in locally and state-managed schools, respectively). These results indicate that the shutdowns might have hit harder on students already behind before the pandemic struck. 27 Because a pandemic outbreak can also affect children’s socioemotional skills, the total impact of school closures on human capital accumulation is potentially larger than the effects captured by cognitive tests.31 . To give context to the school system’s challenges, we use a meta-analysis conducted by McEwan (2015) that summarizes the impacts of more than 70 randomized controlled trials of educational interventions in developing countries. According to the author, school interventions in primary education have, on average, positive effects that range from 0.05 to 0.15 of a standard deviation. The upper bound of the overall average effect of the education policies included in the meta- analysis (0.15 of an SD) is smaller than our lowest estimate. The expressive learning losses detected during a relatively small period of school shutdowns amid the H1N1 outbreak illustrate the challenges policymakers will face in designing sustainable learning acceleration policies to counterbalance the unprecedented skill losses caused by the long period of school shutdowns during the COVID-19 pandemic crisis, particularly in developing countries. 31 For instance, the Global Education Monitoring Report (2019) points to the disruptive impact on learning as physical; emotional, with anxiety, fear, sadness, and lack of emotional control; and cognitive, expressed via difficulty paying attention, inability to process information and memory problems. As emotionally nurturing environments produce more capable learners, these factors will have more profound consequences for children’s development (Cunha and Heckman (2007)) 28 References Adda, J. (2016): “Economic activity and the spread of viral diseases: Evidence from high frequency data,” The Quarterly Journal of Economics, 131, 891–941. Ahrens, A., C. B. Hansen, and M. Schaffer (2019): “LASSOPACK: Stata module for lasso, square-root lasso, elastic net, ridge, adaptive lasso estimation and cross-validation,” . Allinder, R. M., L. S. Fuchs, D. Fuchs, and C. L. Hamlett (1992): “Effects of summer break on math and spelling performance as a function of grade level,” The Elementary School Journal, 92, 451–460. Alves, M. T. G., J. F. Soares, and F. P. Xavier (2016): “Desigualdades educacionais no ensino fundamental de 2005 a 2013: hiato entre grupos sociais,” Revista Brasileira de Sociologia, 4, 49–82. Andrabi, T., B. Daniels, and J. Das (2020): “Human Capital Accumulation and Disasters: Evidence from the Pakistan Earthquake of 2005,” OSF. http://doi. org/10.17605/OSF. IO/3QG98. Athey, S. and G. Imbens (2006): “Identification and Inference in Nonlinear Difference-in- Differences Models,” Econometrica, 74, 431–497. Azevedo, J. P., A. Hasan, D. Goldemberg, S. A. Iqbal, and K. Geven (2020): “Simulating the potential impacts of covid-19 school closures on schooling and learning outcomes: A set of global estimates,” . Baker, M. (2013): “Industrial actions in schools: strikes and student achievement,” Canadian Journal of Economics/Revue canadienne d’économique, 46, 1014–1036. Bryk, A. S. and S. W. Raudenbush (1989): “Toward a more appropriate conceptualization of research on school effects: A three-level hierarchical linear model,” in Multilevel analysis of educational data, Elsevier, 159–204. Cooper, H., B. Nye, K. Charlton, J. Lindsay, and S. Greathouse (1996): “The effects of summer vacation on achievement test scores: A narrative and meta-analytic review,” Review of educational research, 66, 227–268. 29 Cunha, F. and J. Heckman (2007): “The technology of skill formation,” American Economic Review, 97, 31–47. Dawood, F. S., A. D. Iuliano, C. Reed, M. I. Meltzer, D. K. Shay, P.-Y. Cheng, D. Bandaranayake, R. F. Breiman, W. A. Brooks, P. Buchy, et al. (2012): “Estimated global mortality associated with the first 12 months of 2009 pandemic influenza A H1N1 virus circulation: a modelling study,” The Lancet infectious diseases, 12, 687–695. Donnelly, R. and H. A. Patrinos (2021): “Learning loss during COVID-19: An early systematic review,” Prospects, 1–9. Faguet, J.-P. (1999): Does decentralization increase responsiveness to local needs? Evidence from Bolivia, The World Bank. Filmer, D. (2002): Autonomy, participation, and learning in Argentine schools: Findings and their implications for decentralization, The World Bank. Galiani, S., P. Gertler, and E. Schargrodsky (2008): “School decentralization: Helping the good get better, but leaving the poor behind,” Journal of public economics, 92, 2106–2120. Global Education Monitoring Report (2019): “Education as healing: Addressing the trauma of displacement through social and emotional learning,” . Harris, D. N. and T. R. Sass (2009): “What Makes for a Good Teacher and Who Can Tell? Working Paper 30.” National Center for Analysis of Longitudinal Data in Education Research. Jimenez, E. and Y. Sawada (1999): “Do community-managed schools work? An evaluation of El Salvador’s EDUCO program,” The world bank economic review, 13, 415–441. King, E. and B. Ozler (2000): “What’s decentralization got to do with learning? Endogenous school quality and student performance in Nicaragua,” Development Research Group, World Bank, Washington, DC. Klaiman, T., J. D. Kraemer, and M. A. Stoto (2011): “Variability in school closure decisions in response to 2009 H1N1: a qualitative systems improvement analysis,” BMC Public Health, 11, 73. 30 Kuhfeld, M., J. Soland, B. Tarasawa, A. Johnson, E. Ruzek, and J. Liu (2020): “Projecting the potential impact of COVID-19 school closures on academic achievement,” Educational Researcher, 49, 549–565. Marcotte, D. E. and S. W. Hemelt (2008): “Unscheduled school closings and student performance,” Education Finance and Policy, 3, 316–338. McCombs, J. S. (2011): Making summer count: How summer programs can boost children’s learning, Rand Corporation. McEwan, P. J. (2015): “Improving learning in primary schools of developing countries: A meta-analysis of randomized experiments,” Review of Educational Research, 85, 353–394. Muralidharan, K. and N. Prakash (2017): “Cycling to school: Increasing secondary school enrollment for girls in India,” American Economic Journal: Applied Economics, 9, 321–50. Murnane, R. J. (1975): “The impact of school resources on the learning of inner city children.” . Roodman, D., M. Ø. Nielsen, J. G. MacKinnon, and M. D. Webb (2019): “Fast and wild: Bootstrap inference in Stata using boottest,” The Stata Journal, 19, 4–60. The World Health Organization (2010): “Pandemic (H1N1) 2009 - update 100,” . Thum, Y. M. and C. H. Hauser (2015): “NWEA 2015 MAP norms for student and school achievement status and growth,” Portland, OR: NWEA. 31 A Appendix Table A.1: Balance test: municipalities that extended the winter break versus municipalities that did not (2009) (1) (2) (1)-(2) 0 1 Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean difference Average enrollment per school 629 316.33 12 531.22 641 -214.89*** (7.52) (62.36) Total enrollment 632 2,700.47 12 19,633.50 644 -16,933.03*** (199.00) (3,515.11) Teachers with tenure 565 0.39 12 0.60 577 -0.21** (0.01) (0.10) H1N1 cases per 100,000 inhabitants 632 1.17 12 7.27 644 -6.10*** (0.14) (0.76) Population, in thousands 632 39.53 12 446.73 644 -407.19*** (3.48) (90.31) GDP per capita 610 26,116.72 12 50,073.24 622 -23,956.52*** (524.71) (6,472.35) Notes : The value displayed for t-tests is the difference in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. The municipality of São Paulo is not included in the Table since it had more than 11 million inhabitants in 2009. Therefore, its inclusion would distort the comparison between treatment and comparison municipalities. Source: Prova Brasil, Census of Education and IBGE. 32 Table A.2: Probability of school closure in 2009 on 2007’s municipality, schools, teachers and students’ characteristics Extended the winter break Proficiency in Math -0.001 0.001 Proficiency in Portuguese 0.001 0.001 Repetition - % 0.000 0.001 Dropout - % -0.002 0.013 Class hours per day -0.022* 0.011 Students per teacher 0.001 0.001 Lack of textbooks according to principals 0.000 0.000 Teachers classify the textbooks as great - % 0.000 0.000 Teachers covered more than 80 percent of the curricula - % 0.000 0.000 Principal managerial skills from teacher perspective -0.038 0.058 Teacher with tenure - % 0.000 0.000 Student absenteeism as a big issue 0.001 0.001 Teacher absenteeism as a big issue 0.001 0.000 Deficit in learning is due to: students’ low effort - % -0.001* 0.000 Deficit in learning is due to: students’ bad behavior - % 0.000 0.000 Students allocated into classrooms according to similar age 0.000 0.000 GDP per capita 0.000** 0.000 Population 0.000*** 0.000 H1N1 cases per 100.000 inhabitants (Late July 2009) 0.005** 0.002 Constant 0.145 0.112 Obs 516.000 Adj. R-squared 0.209 Notes : The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. The Table shows the results of a linear probability model run at the school level, in which the dependent variable assumes the value of 1 if the school is located in a municipality that opted to extend children’s winter break and 0 otherwise. The sample includes all municipalities of São Paulo state for which data on population, number of H1N1 confirmed cases, GDP per capita and education data are available (516 municipalities). 33 Table A.3: Fifth-graders’ characteristics, locally-managed schools (2009) (1) (2) (1)-(2) Comparison Group Treatment Group Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Proficiency in Math [SAEB scale 0 to 350] 2569 219.85 795 202.19 3364 17.66*** (0.40) (0.56) Proficiency in Portuguese [SAEB scale 0 to 325] 2569 194.61 795 182.39 3364 12.22*** (0.33) (0.52) Repetition - % 2567 5.95 794 10.75 3361 -4.80*** (0.12) (0.30) Dropout - % 2567 0.18 794 0.39 3361 -0.22*** (0.01) (0.03) Approval - % 2567 93.87 794 88.85 3361 5.02*** (0.12) (0.30) Parents encourage to study - % 2565 97.94 795 96.93 3360 1.01*** (0.05) (0.08) Parents encourage to do the homework - % 2565 96.74 795 95.50 3360 1.24*** (0.06) (0.10) Parents encourage to read - % 2565 95.01 795 94.61 3360 0.41*** (0.07) (0.10) Parents encourage to go to school - % 2565 97.14 795 95.70 3360 1.44*** (0.06) (0.10) Parents talk about what happens in the school - % 2565 84.07 795 84.89 3360 -0.82*** (0.14) (0.18) White students - % 2566 45.38 795 38.34 3361 7.04*** (0.27) (0.29) Student lives with mother (or legal responsible) - % 2565 94.82 795 94.51 3360 0.31** (0.07) (0.10) Computer in the household - % 2566 49.81 795 58.43 3361 -8.63*** (0.33) (0.45) Students’ mother finished high school - % 2565 33.97 795 37.20 3360 -3.23*** (0.30) (0.46) Student did preschool - % 2565 83.40 795 75.94 3360 7.46*** (0.20) (0.29) Student has ever repeated and 0, otherwise - % 2565 21.55 795 20.53 3360 1.02** (0.21) (0.29) Student has ever dropped and 0, otherwise - % 2565 4.86 795 7.11 3360 -2.25*** (0.08) (0.15) Student works - % 2565 11.86 795 13.27 3360 -1.41*** (0.14) (0.21) Students per class 2568 27.44 795 31.41 3363 -3.97*** (0.10) (0.14) Class hours per day 2568 5.02 795 4.79 3363 0.22*** (0.01) (0.01) Insufficient performance in Math - % 2567 35.52 795 49.47 3362 -13.95*** (0.32) (0.49) Insufficient performance in Portuguese - % 2567 56.86 795 66.07 3362 -9.21*** (0.31) (0.45) Teacher always corrects Portuguese homework - % 2565 84.00 795 78.27 3360 5.73*** (0.22) (0.33) Teacher always corrects Math homework - % 2565 85.57 795 80.91 3360 4.67*** (0.21) (0.30) Schools with computer lab - % 2568 58.68 795 93.58 3363 -34.90*** (0.97) (0.87) Schools with science lab - % 2568 5.45 795 20.00 3363 -14.55*** (0.45) (1.42) Schools sport court - % 2568 66.51 795 90.44 3363 -23.93*** (0.93) (1.04) Schools with library - % 2568 24.65 795 23.40 3363 1.25 (0.85) (1.50) Schools internet access - % 2568 90.50 795 93.71 3363 -3.21*** (0.58) (0.86) Length of the school year (days) 2568 318.76 795 313.95 3363 4.80*** (0.21) (0.53) Total enrollment, all grades 2568 486.22 795 872.01 3363 -385.79*** (5.46) (12.41) GDP per capita of the municipality, in 2019 BRL 2441 30,940.67 795 57,707.66 3236 -26,766.99*** (314.79) (430.57) 34 Notes: Locally-managed schools in São Paulo. The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. Table A.4: Fifth-graders’ characteristics, state and locally-managed schools in = 1 (2009) (1) (2) (1)-(2) Locally-managed State-managed Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Proficiency in Math [SAEB scale 0 to 350] 1334 216.95 929 215.01 2263 1.94** (0.54) (0.62) Proficiency in Portuguese [SAEB scale 0 to 325] 1334 193.39 929 189.67 2263 3.72*** (0.46) (0.52) Repetition - % 1334 5.69 929 6.10 2263 -0.41* (0.16) (0.19) Dropout - % 1334 0.21 929 0.17 2263 0.04 (0.02) (0.02) Approval - % 1334 94.10 929 93.73 2263 0.37 (0.16) (0.19) Parents encourage to study - % 1333 97.92 929 97.80 2262 0.12 (0.06) (0.08) Parents encourage to do the homework - % 1333 96.72 929 96.36 2262 0.36*** (0.08) (0.11) Parents encourage to read - % 1333 94.96 929 95.62 2262 -0.66*** (0.09) (0.10) Parents encourage to go to school - % 1333 97.15 929 96.81 2262 0.34*** (0.07) (0.09) Parents talk about what happens in the school - % 1333 83.36 929 85.08 2262 -1.72*** (0.19) (0.20) White students - % 1333 44.18 929 44.76 2262 -0.58 (0.36) (0.43) Student lives with mother (or legal responsible) - % 1333 95.11 929 94.95 2262 0.16 (0.09) (0.10) Computer in the household - % 1333 53.27 929 52.39 2262 0.88 (0.45) (0.53) Students’ mother finished high school - % 1333 36.09 929 35.75 2262 0.34 (0.43) (0.49) Student did preschool - % 1333 82.26 929 81.52 2262 0.74* (0.28) (0.33) Student has ever repeated and 0, otherwise - % 1333 19.56 929 16.40 2262 3.16*** (0.28) (0.30) Student has ever dropped and 0, otherwise - % 1333 4.90 929 5.79 2262 -0.89*** (0.11) (0.14) Student works - % 1333 11.15 929 12.66 2262 -1.51*** (0.17) (0.23) Students per class 1334 28.71 929 28.82 2263 -0.10 (0.14) (0.16) Class hours per day 1334 4.91 929 5.31 2263 -0.40*** (0.02) (0.03) Insufficient performance in Math - % 1334 37.59 929 40.02 2263 -2.43*** (0.44) (0.50) Insufficient performance in Portuguese - % 1334 57.73 929 60.46 2263 -2.73*** (0.43) (0.48) Teacher always corrects Portuguese homework - % 1333 81.95 929 82.10 2262 -0.16 (0.31) (0.36) Teacher always corrects Math homework - % 1333 83.62 929 83.89 2262 -0.27 (0.30) (0.34) Schools with computer lab - % 1334 57.80 929 92.25 2263 -34.45*** (1.35) (0.88) Schools with science lab - % 1334 6.90 929 6.89 2263 0.01 (0.69) (0.83) Schools sport court - % 1334 61.69 929 84.39 2263 -22.70*** (1.33) (1.19) Schools with library - % 1334 25.86 929 2.91 2263 22.96*** (1.20) (0.55) Schools internet access - % 1334 88.46 929 98.28 2263 -9.82*** (0.88) (0.43) Length of the school year (days) 1334 320.22 929 315.01 2263 5.21*** (0.34) (0.25) Total enrollment, all grades 1334 541.53 929 638.05 2263 -96.52*** (8.28) (12.99) GDP per capita of the municipality, in 2019 BRL 1288 33,284.83 914 33,830.75 2202 -545.92 (370.67) (434.43) 35 Notes: State and locally-managed schools in São Paulo. Group of municipalities in = 1 (Table 2). The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. Table A.5: Fifth-graders’ characteristics, state and locally-managed schools in = 0 (2009) (1) (2) (1)-(2) Locally-managed State-managed Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Proficiency in Math [SAEB scale 0 to 350] 759 201.11 869 210.27 1628 -9.15*** (0.58) (0.54) Proficiency in Portuguese [SAEB scale 0 to 325] 759 181.48 869 188.00 1628 -6.52*** (0.53) (0.47) Repetition - % 758 11.40 868 5.78 1626 5.62*** (0.30) (0.18) Dropout - % 758 0.40 868 0.37 1626 0.03 (0.03) (0.03) Approval - % 758 88.20 868 93.85 1626 -5.65*** (0.31) (0.19) Parents encourage to study - % 759 96.81 868 97.58 1627 -0.77*** (0.09) (0.08) Parents encourage to do the homework - % 759 95.43 868 96.22 1627 -0.79*** (0.10) (0.12) Parents encourage to read - % 759 94.64 868 95.38 1627 -0.74*** (0.10) (0.10) Parents encourage to go to school - % 759 95.64 868 96.39 1627 -0.76*** (0.11) (0.09) Parents talk about what happens in the school - % 759 85.10 868 85.08 1627 0.02 (0.18) (0.16) White students - % 759 38.02 868 40.37 1627 -2.35*** (0.29) (0.32) Student lives with mother (or legal responsible) - % 759 94.42 868 94.78 1627 -0.36*** (0.11) (0.09) Computer in the household - % 759 58.81 868 61.20 1627 -2.39*** (0.43) (0.44) Students’ mother finished high school - % 759 36.59 868 39.64 1627 -3.04*** (0.43) (0.45) Student did preschool - % 759 75.21 868 79.89 1627 -4.68*** (0.29) (0.30) Student has ever repeated and 0, otherwise - % 759 21.16 868 14.98 1627 6.17*** (0.32) (0.26) Student has ever dropped and 0, otherwise - % 759 7.40 868 6.44 1627 0.97*** (0.16) (0.14) Student works - % 759 13.42 868 12.08 1627 1.33*** (0.22) (0.17) Students per class 759 31.76 868 31.37 1627 0.39* (0.14) (0.14) Class hours per day 759 4.74 868 5.17 1627 -0.43*** (0.02) (0.02) Insufficient performance in Math - % 759 50.41 869 42.97 1628 7.44*** (0.51) (0.46) Insufficient performance in Portuguese - % 759 66.70 869 61.65 1628 5.05*** (0.47) (0.43) Teacher always corrects Portuguese homework - % 759 78.19 868 80.30 1627 -2.11*** (0.35) (0.31) Teacher always corrects Math homework - % 759 80.86 868 82.42 1627 -1.56*** (0.32) (0.30) Schools with computer lab - % 759 94.07 868 89.06 1627 5.02*** (0.86) (1.06) Schools with science lab - % 759 20.95 868 11.64 1627 9.31*** (1.48) (1.09) Schools sport court - % 759 89.99 868 85.83 1627 4.16** (1.09) (1.18) Schools with library - % 759 20.95 868 3.80 1627 17.15*** (1.48) (0.65) Schools internet access - % 759 93.68 868 97.47 1627 -3.79*** (0.88) (0.53) Length of the school year (days) 759 313.74 868 313.50 1627 0.24 (0.56) (0.28) Total enrollment, all grades 759 888.28 868 854.12 1627 34.16* (12.22) (15.19) GDP per capita of the municipality, in 2019 BRL 759 58,012.21 869 56,741.82 1628 1,270.38*** (384.71) (306.01) 36 Notes: State and locally-managed schools in São Paulo. Group of municipalities in = 0 (Table 2). The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. Table A.6: Fifth-graders’ characteristics, state-managed schools (2009) (1) (2) (1)-(2) State-managed in G = 0 State-managed in G = 1 Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Proficiency in Math [SAEB scale 0 to 350] 869 210.27 929 215.01 1798 -4.74*** (0.54) (0.62) Proficiency in Portuguese [SAEB scale 0 to 325] 869 188.00 929 189.67 1798 -1.67** (0.47) (0.52) Repetition - % 868 5.78 929 6.10 1797 -0.32 (0.18) (0.19) Dropout - % 868 0.37 929 0.17 1797 0.20*** (0.03) (0.02) Approval - % 868 93.85 929 93.73 1797 0.12 (0.19) (0.19) Parents encourage to study - % 868 97.58 929 97.80 1797 -0.22** (0.08) (0.08) Parents encourage to do the homework - % 868 96.22 929 96.36 1797 -0.14 (0.12) (0.11) Parents encourage to read - % 868 95.38 929 95.62 1797 -0.24* (0.10) (0.10) Parents encourage to go to school - % 868 96.39 929 96.81 1797 -0.42*** (0.09) (0.09) Parents talk about what happens in the school - % 868 85.08 929 85.08 1797 -0.01 (0.16) (0.20) White students - % 868 40.37 929 44.76 1797 -4.40*** (0.32) (0.43) Student lives with mother (or legal responsible) - % 868 94.78 929 94.95 1797 -0.17 (0.09) (0.10) Computer in the household - % 868 61.20 929 52.39 1797 8.81*** (0.44) (0.53) Students’ mother finished high school - % 868 39.64 929 35.75 1797 3.89*** (0.45) (0.49) Student did preschool - % 868 79.89 929 81.52 1797 -1.63*** (0.30) (0.33) Student has ever repeated and 0, otherwise - % 868 14.98 929 16.40 1797 -1.41*** (0.26) (0.30) Student has ever dropped and 0, otherwise - % 868 6.44 929 5.79 1797 0.65*** (0.14) (0.14) Student works - % 868 12.08 929 12.66 1797 -0.58** (0.17) (0.23) Students per class 868 31.37 929 28.82 1797 2.55*** (0.14) (0.16) Class hours per day 868 5.17 929 5.31 1797 -0.14*** (0.02) (0.03) Insufficient performance in Math - % 869 42.97 929 40.02 1798 2.94*** (0.46) (0.50) Insufficient performance in Portuguese - % 869 61.65 929 60.46 1798 1.19* (0.43) (0.48) Teacher always corrects Portuguese homework - % 868 80.30 929 82.10 1797 -1.80*** (0.31) (0.36) Teacher always corrects Math homework - % 868 82.42 929 83.89 1797 -1.46*** (0.30) (0.34) Schools with computer lab - % 868 89.06 929 92.25 1797 -3.19** (1.06) (0.88) Schools with science lab - % 868 11.64 929 6.89 1797 4.75*** (1.09) (0.83) Schools sport court - % 868 85.83 929 84.39 1797 1.44 (1.18) (1.19) Schools with library - % 868 3.80 929 2.91 1797 0.90 (0.65) (0.55) Schools internet access - % 868 97.47 929 98.28 1797 -0.81 (0.53) (0.43) Length of the school year (days) 868 313.50 929 315.01 1797 -1.52*** (0.28) (0.25) Total enrollment, all grades 868 854.12 929 638.05 1797 216.07*** (15.19) (12.99) GDP per capita of the municipality, in 2019 BRL 869 56,741.82 914 33,830.75 1783 22,911.07*** (306.01) (434.43) Notes: State-managed schools in São Paulo. The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. 37 Table A.7: Teachers’ and principals’ characteristics, locally-managed schools (2009) (1) (2) (1)-(2) Comparison Group Treatment Group Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Teacher with tenure - % 2349 48.74 719 67.44 3068 -18.71*** (0.87) (1.21) Teacher with less than 40 years old - % 2354 48.53 722 38.43 3076 10.10*** (0.75) (1.17) Principal managerial skills from teacher perspective 2254 0.76 643 0.72 2897 0.04*** (0.00) (0.01) Index for the violence the teacher faces in the school 2344 0.14 716 0.28 3060 -0.14*** (0.01) (0.01) Teacher expects that almost all students will finish 9th grade - % 2327 88.77 721 87.36 3048 1.41 (0.48) (0.78) Teacher expects that almost all students will finish high school - % 2313 60.74 715 53.35 3028 7.39*** (0.76) (1.26) Teachers covered more than 80 percent of the curricula - % 2352 54.65 722 45.16 3074 9.49*** (0.78) (1.23) Teachers always participate of the work decisions - % 2236 56.40 659 54.45 2895 1.95 (0.80) (1.38) Teachers say that all the students have textbooks - % 2154 70.75 644 64.69 2798 6.06*** (0.83) (1.45) Teachers classify the textbooks as great - % 2179 18.68 667 15.17 2846 3.51** (0.68) (1.07) Teachers’ salary is less than 3 minimum wage - % 1980 33.07 665 13.49 2645 19.58*** (0.89) (0.99) Deficit in learning is due to: students’ low effort - % 2349 81.71 721 76.33 3070 5.38*** (0.59) (0.98) Deficit in learning is due to: students’ absenteeism - % 2211 31.75 650 36.09 2861 -4.34** (0.91) (1.61) Deficit in learning is due to: students’ bad behavior - % 2346 56.93 718 61.24 3064 -4.32*** (0.76) (1.16) Teachers with the correct degree to teach Portuguese - % 2558 39.39 714 24.21 3272 15.17*** (0.75) (1.39) Teachers with the correct degree to teach Math - % 2558 33.12 714 21.12 3272 12.00*** (0.73) (1.33) Principal has organized Teachers’ training last two years, % 1717 56.96 557 64.63 2274 -7.67*** (1.20) (2.03) Lack of textbooks according to principals, % 1672 29.19 553 51.72 2225 -22.53*** (1.11) (2.13) Principal was appointed for the position, % 1732 47.69 565 14.51 2297 33.18*** (1.20) (1.48) Teacher absenteeism as a big issue, % 1745 8.08 561 26.38 2306 -18.30*** (0.65) (1.86) Student absenteeism as a big issue, % 1747 4.41 566 8.30 2313 -3.90*** (0.49) (1.16) Students allocated into classrooms according to similar age, % 1661 34.98 549 27.69 2210 7.29*** (1.17) (1.91) Students allocated into classrooms according to hetero. performance, % 1661 41.12 549 60.29 2210 -19.17*** (1.21) (2.09) Notes: Locally-managed schools in São Paulo. The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. 38 Table A.8: Teachers’ and principals’ characteristics, state and locally-managed schools in = 1 (2009) (1) (2) (1)-(2) Locally-managed State-managed Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Teacher with tenure - % 1220 45.65 852 57.22 2072 -11.57*** (1.24) (1.22) Teacher with less than 40 years old - % 1224 52.24 852 24.83 2076 27.41*** (1.02) (1.04) Principal managerial skills from teacher perspective 1170 0.74 809 0.78 1979 -0.04*** (0.00) (0.01) Index for the violence the teacher faces in the school 1220 0.15 851 0.12 2071 0.03** (0.01) (0.01) Teacher expects that almost all students will finish 9th grade - % 1212 87.37 850 89.54 2062 -2.16** (0.69) (0.71) Teacher expects that almost all students will finish high school - % 1203 56.83 843 66.06 2046 -9.23*** (1.06) (1.19) Teachers covered more than 80 percent of the curricula - % 1222 51.13 852 51.57 2074 -0.45 (1.05) (1.24) Teachers always participate of the work decisions - % 1170 54.84 810 59.90 1980 -5.06*** (1.09) (1.24) Teachers say that all the students have textbooks - % 1133 64.71 787 76.00 1920 -11.29*** (1.19) (1.21) Teachers classify the textbooks as great - % 1126 15.10 791 24.19 1917 -9.09*** (0.84) (1.15) Teachers’ salary is less than 3 minimum wage - % 1073 25.75 731 26.34 1804 -0.59 (1.10) (1.30) Deficit in learning is due to: students’ low effort - % 1219 80.67 853 77.90 2072 2.77** (0.82) (1.00) Deficit in learning is due to: students’ absenteeism - % 1153 37.31 808 35.41 1961 1.90 (1.32) (1.59) Deficit in learning is due to: students’ bad behavior - % 1216 58.82 851 54.25 2067 4.57*** (1.02) (1.23) Teachers with the correct degree to teach Portuguese - % 1331 44.27 927 39.86 2258 4.41*** (1.05) (1.14) Teachers with the correct degree to teach Math - % 1331 37.16 927 33.87 2258 3.29** (1.04) (1.14) Principal has organized Teachers’ training last two years, % 911 60.26 865 65.55 1776 -5.29** (1.62) (1.62) Lack of textbooks according to principals, % 891 34.01 869 25.66 1760 8.35*** (1.59) (1.48) Principal was appointed for the position, % 927 41.75 869 8.17 1796 33.58*** (1.62) (0.93) Teacher absenteeism as a big issue, % 928 8.41 871 19.75 1799 -11.34*** (0.91) (1.35) Student absenteeism as a big issue, % 927 5.39 874 6.98 1801 -1.59 (0.74) (0.86) Students allocated into classrooms according to similar age, % 876 39.16 855 36.96 1731 2.20 (1.65) (1.65) Students allocated into classrooms according to hetero. performance, % 876 39.50 855 38.60 1731 0.90 (1.65) (1.67) Notes: Prova Brasil and Census of Education. State and locally-managed schools in São Paulo. Group of municipalities in = 1 (Table 2). The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. 39 Table A.9: Teachers’ and principals’ characteristics, state and locally-managed schools in = 0 (2009) (1) (2) (1)-(2) Locally-managed State-managed Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Teacher with tenure - % 681 69.63 773 50.75 1454 18.89*** (1.25) (1.18) Teacher with less than 40 years old - % 686 36.58 774 27.67 1460 8.91*** (1.18) (0.94) Principal managerial skills from teacher perspective 608 0.72 710 0.75 1318 -0.03*** (0.01) (0.01) Index for the violence the teacher faces in the school 680 0.29 770 0.26 1450 0.03 (0.01) (0.01) Teacher expects that almost all students will finish 9th grade - % 685 86.80 773 88.21 1458 -1.41 (0.83) (0.69) Teacher expects that almost all students will finish high school - % 680 51.58 769 64.85 1449 -13.27*** (1.29) (1.07) Teachers covered more than 80 percent of the curricula - % 686 42.18 774 51.22 1460 -9.03*** (1.26) (1.13) Teachers always participate of the work decisions - % 625 53.71 723 57.75 1348 -4.04** (1.43) (1.24) Teachers say that all the students have textbooks - % 606 64.71 684 67.25 1290 -2.55 (1.49) (1.32) Teachers classify the textbooks as great - % 630 15.19 730 21.43 1360 -6.24*** (1.10) (1.11) Teachers’ salary is less than 3 minimum wage - % 625 12.50 723 24.54 1348 -12.03*** (0.97) (1.14) Deficit in learning is due to: students’ low effort - % 685 76.95 774 73.05 1459 3.90*** (1.01) (1.00) Deficit in learning is due to: students’ absenteeism - % 614 38.16 708 34.95 1322 3.22 (1.70) (1.55) Deficit in learning is due to: students’ bad behavior - % 682 63.12 771 55.00 1453 8.12*** (1.19) (1.09) Teachers with the correct degree to teach Portuguese - % 678 20.84 868 36.69 1546 -15.85*** (1.34) (1.04) Teachers with the correct degree to teach Math - % 678 17.63 868 33.12 1546 -15.49*** (1.25) (1.03) Principal has organized Teachers’ training last two years, % 540 65.19 795 63.02 1335 2.17 (2.05) (1.71) Lack of textbooks according to principals, % 534 53.00 802 26.81 1336 26.19*** (2.16) (1.57) Principal was appointed for the position, % 548 15.69 798 7.39 1346 8.30*** (1.56) (0.93) Teacher absenteeism as a big issue, % 542 29.52 803 18.43 1345 11.09*** (1.96) (1.37) Student absenteeism as a big issue, % 549 9.11 806 6.95 1355 2.16 (1.23) (0.90) Students allocated into classrooms according to similar age, % 530 27.92 774 36.56 1304 -8.64*** (1.95) (1.73) Students allocated into classrooms according to hetero. performance, % 530 60.57 774 42.64 1304 17.93*** (2.12) (1.78) Notes: State and locally-managed schools in São Paulo. Group of municipalities in = 0 (Table 2). The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. 40 Table A.10: Teachers’ and principals’ characteristics, state-managed schools (2009) (1) (2) (1)-(2) State-managed, G=0 State-managed, G=1 Pairwise t-test Variable N Mean/(SE) N Mean/(SE) N Mean dif Teacher with tenure - % 773 50.75 852 57.22 1625 -6.47*** (1.18) (1.22) Teacher with less than 40 years old - % 774 27.67 852 24.83 1626 2.84** (0.94) (1.04) Principal managerial skills from teacher perspective 710 0.75 809 0.78 1519 -0.04*** (0.01) (0.01) Index for the violence the teacher faces in the school 770 0.26 851 0.12 1621 0.14*** (0.01) (0.01) Teacher expects that almost all students will finish 9th grade - % 773 88.21 850 89.54 1623 -1.32 (0.69) (0.71) Teacher expects that almost all students will finish high school - % 769 64.85 843 66.06 1612 -1.21 (1.07) (1.19) Teachers covered more than 80 percent of the curricula - % 774 51.22 852 51.57 1626 -0.36 (1.13) (1.24) Teachers always participate of the work decisions - % 723 57.75 810 59.90 1533 -2.15 (1.24) (1.24) Teachers say that all the students have textbooks - % 684 67.25 787 76.00 1471 -8.75*** (1.32) (1.21) Teachers classify the textbooks as great - % 730 21.43 791 24.19 1521 -2.76* (1.11) (1.15) Teachers’ salary is less than 3 minimum wage - % 723 24.54 731 26.34 1454 -1.80 (1.14) (1.30) Deficit in learning is due to: students’ low effort - % 774 73.05 853 77.90 1627 -4.85*** (1.00) (1.00) Deficit in learning is due to: students’ absenteeism - % 708 34.95 808 35.41 1516 -0.46 (1.55) (1.59) Deficit in learning is due to: students’ bad behavior - % 771 55.00 851 54.25 1622 0.75 (1.09) (1.23) Teachers with the correct degree to teach Portuguese - % 868 36.69 927 39.86 1795 -3.17** (1.04) (1.14) Teachers with the correct degree to teach Math - % 868 33.12 927 33.87 1795 -0.75 (1.03) (1.14) Principal has organized Teachers’ training last two years, % 795 63.02 865 65.55 1660 -2.53 (1.71) (1.62) Lack of textbooks according to principals, % 802 26.81 869 25.66 1671 1.15 (1.57) (1.48) Principal was appointed for the position, % 798 7.39 869 8.17 1667 -0.78 (0.93) (0.93) Teacher absenteeism as a big issue, % 803 18.43 871 19.75 1674 -1.32 (1.37) (1.35) Student absenteeism as a big issue, % 806 6.95 874 6.98 1680 -0.03 (0.90) (0.86) Students allocated into classrooms according to similar age, % 774 36.56 855 36.96 1629 -0.40 (1.73) (1.65) Students allocated into classrooms according to hetero. performance, % 774 42.64 855 38.60 1629 4.04* (1.78) (1.67) Notes: State-managed schools in São Paulo. The value displayed for t-tests is the differences in the means across the groups. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Source: Prova Brasil and Census of Education. 41 Table A.11: Placebo test for students’ performance in treatment and comparison groups (2005-2007) Fifth-graders Math Portuguese I II I II State-managed -2.979 2.1 -1.357 3.825 (2.078) (6.837) (1.758) (5.810) Post-treament year (2007) 12.693*** 15.375* -0.088 2.869 (1.967) (6.487) (1.664) (5.512) State-managed versus post-treament -6.268* -7.644 -5.789* -8.564 (2.794) (9.173) (2.364) (7.795) G=1 9.026 7.424 (5.041) (4.283) State-managed versus G = 1 -5.079 -5.182 (7.133) (6.062) Post-treatment versus with G = 1 -2.682 -2.957 (6.766) (5.750) Triple Difference in differences 1.376 2.775 (9.572) (8.134) Obs 405 441 405 441 Adj. R-squared 0.148 0.157 0.057 0.057 Notes : The authors’ estimate is based on data from Prova Brasil. Standard errors in parenthesis. The regressions are run at the municipality level for the years 2005 and 2007 as data at the school level is only available for the state-managed network since 2007. Sample of municipalities with at least one state and one locally-managed school. Columns I show the results of the following equation: = 0 + 1 + 2 + 3 × + . Columns II show the results of the following equation: = 0 + 1 + 2 + 3 × + 4 + 5 × + 6 × + 7 × × + . = 1 for state-managed schools and 0 otherwise. = 1 for the 112 municipalities whose local authorities did not extend children’s winter break. = 0 for the group of 10 municipalities whose local authorities extended children’s winter break (Table 2). = 1 for 2007 and = 0 for 2005. 3 is the DiD estimate and 7 is the triple DiD estimate. For primary education, 3 is statistically significant for both math and Portuguese. Therefore, even before the school closures, we do not have evidence that locally and state-managed schools have parallel trends. 7 is not statistically significant, meaning that, before the shutdowns, the variation of students’ proficiency in state and locally-managed schools in = 1 minus the variation of students’ proficiency in state and locally-managed schools in = 0 is not statistically significant. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. 42 Table A.12: Placebo test for grade-promotion, retention, and dropout (2007-2008) Fifth-graders Approval Repetition Dropout I II I II I II State-managed -1.445*** 5.130*** 1.357*** -5.164*** 0.088 0.034 (0.370) (0.473) (0.366) (0.465) (0.052) (0.078) Post-treatment year (2008) 1.233*** 2.193*** -1.191*** -2.040*** -0.042 -0.153 (0.309) (0.490) (0.306) (0.481) (0.044) (0.081) State-managed versus post-treament 0.104 0.475 -0.033 -0.51 -0.071 0.035 (0.495) (0.659) (0.490) (0.648) (0.070) (0.108) G=1 1.508 -1.18 -0.328 (2.647) (2.603) (0.435) State-managed versus G = 1 -6.575*** 6.521*** 0.053 (0.592) (0.582) (0.097) Post-treatment versus with G = 1 -0.96 0.848 0.112 (0.572) (0.563) (0.094) Triple Difference in differences -0.371 0.478 -0.107 (0.812) (0.799) (0.134) Obs 6389 9541 6389 9541 6389 9541 43 Adj. R-squared 0.104 0.137 0.105 0.136 0.01 0.01 Notes : The authors’ estimate is based on data from Prova Brasil. Standard errors in parenthesis. as defined in the empirical strategy (Table 2). The regressions are run at the school level for the years 2007 and 2008. Sample of municipalities with at least one state and one locally-managed school. Columns I show the results of the following equation: = 0 + 1 + 2 + 3 × + . Columns II show the results of the following equation: = 0 + 1 + 2 + 3 × + 4 + 5 × + 6 × + 7 × × + . = 1 for state-managed schools and 0 otherwise. = 1 for the 112 municipalities whose local authorities did not extend children’s winter break. = 0 for the group of 10 municipalities whose local authorities extended children’s winter break (Table 2). = 1 for 2007 and = 0 for 2005. 3 is the DiD estimate and 7 is the triple DiD estimate. For primary education, 3 is statistically significant in all DiD specifications. Therefore, even before the school closures, we do not have evidence that locally and state-managed schools have parallel trends. For primary education, 7 is not statistically significant, meaning that, before the shutdowns, the variation of students’ outcomes in state and locally-managed schools in = 1 minus the variation of students’ outcomes (grade promotion, retention, and dropout) in state and locally-managed schools in = 0 is not statistically significant. Coefficients are converted to percentages (multiplied by 100). ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Table A.13: Impact of the school shutdowns on students’ learning (with placebo), fifth-grade (2005-2009) Math Portuguese DiD DiD DiD TD TD DiD DiD DiD TD TD (1) (2) (3) (4) (5) (1) (2) (3) (4) (5) Placebo 0.27 0.8 (0.97) (0.89) H1N1 -3.26** -2.75** -4.35*** -4.25*** -0.76 -0.54 -3.49*** -4.25*** (1.18) (1.23) (0.96) (1.06) (0.86) (0.95) (0.81) (1.06) N. schools 5291 3912 3912 5329 5329 5291 3912 3912 5329 5329 Adj. R-squared 0.6 0.8 0.8 0.8 0.7 0.5 0.8 0.8 0.8 0.7 Specifications (A) Municipal FE Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes (B) Controls Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes (C) % teachers No No Yes No Yes No No Yes No Yes in both networks (D) % teachers No No Yes No Yes No No Yes No Yes management (E) School FE No No No No Yes No No No No Yes Notes : The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. Wild bootstrap p-values. 95% CI in brackets. ***, **, and * indicate significance at the 1, 5, and 10% critical levels. Standard error in parenthesis and clustered at the municipality level. All regressions are weighted by fifth-grade enrollment at the school level. Math performance on a scale from 0 to 350 (SAEB scale). Portuguese performance on a scale from 0 to 325 (SAEB scale). The Columns DiD show the estimates differences-in-differences as described in subsection 4.0.1 and Columns Triple D show the estimates for triple difference-in-differences as described in subsection 4.0.2. The sample of municipalities and schools included in the analysis are detailed in Table 2. (A) municipal fixed effects. (B) students’, teachers’, schools’ and principals’ characteristics. (C) the percentage of teachers that work in a state and a locally-managed school simultaneously. (D) the percentage of teachers working in a state-managed school that implemented the managerial practices intervention. (E) schools’ fixed effects. All triple DiD estimates exclude state-managed schools in which the state government implemented the managerial practices intervention. The row Placebo shows the estimates in which the pre-treatment year is 2005 and the post-treatment year is 2007. 44 Table A.14: Impact of school shutdowns on grade-promotion, retention and dropout, fifth-grade Grade-promotion Retention Dropout TD DiD TD TD TD DiD TD TD TD DiD TD TD (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) Placebo 0.4 -0.156 -0.244 sd (0.62) (0.55) (0.27) H1N1 1.329** 1.068* -0.061 -1.379** -1.175* -0.056 0.051 0.107 0.117 sd (0.67) (0.63) (0.79) (0.64) (0.62) (0.79) (0.05) (0.08) (0.10) N. schools 8069 3914 5330 5330 8069 3914 5330 5330 8069 3914 5330 5330 Adj. R2 0.3 0.4 0.4 0.1 0.3 0.4 0.4 0.1 0 0.1 0.1 0 Treatment Group before the school shutdowns (2007) Mean 90.62 86.37 90.93 90.93 9.07 13.28 8.78 8.78 0.31 0.35 0.29 0.29 Standard Deviation 9.5 11.28 6.88 6.88 9.36 11.03 6.79 6.79 1.71 0.79 0.81 0.81 Estimate of ATT (in sd) 0.04 0.12 0.16 -0.01 -0.02 -0.13 -0.17 -0.01 -0.14 0.06 0.13 0.14 Specifications 45 (A) Municipal FE Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes (B) Controls Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes (C) % teachers Yes Yes No Yes Yes Yes No Yes Yes Yes No Yes in both networks (D) % teachers Yes Yes No Yes Yes Yes No Yes Yes Yes No Yes management (E) School FE No No No Yes No No No Yes No No No Yes Notes : The authors’ estimate is based on data from Prova Brasil, Census of Education, and IBGE. Wild bootstrap p-values. 95% CI in brackets. ***, **, and * indicate significance at the 1, 5, and 10% critical levels. Standard error in parenthesis and clustered at the municipality level. All regressions are weighted by fifth-grade enrollment at the school level. ATT regressions coefficients are converted to percentages (multiplied by 100). The Columns DiD show the estimates differences-in-differences as described in subsection 4.0.1 and Columns Triple D show the estimates for triple difference-in-differences as described in subsection 4.0.2. The sample of municipalities and schools included in the analysis are detailed in Table 2. (A) municipal fixed effects. (B) students’, teachers’, schools’ and principals’ characteristics. (C) the percentage of teachers that work in a state and a locally-managed school simultaneously. (D) the percentage of teachers working in a state-managed school that implemented the managerial practices intervention. (E) schools’ fixed effects. All triple DiD estimates exclude state-managed schools in which the state government implemented the managerial practices intervention. The row Placebo shows the estimates in which the pre-treatment year is 2007 and the post-treatment year is 2008. state and locally-managed schools closed state-managed closed, locally-managed open Figure A.1: School shutdown policy, São Paulo (2009) Source: Authors’ analysis based on data from Local newspapers. 46 Extend the winter-break Other municipalities Extend the winter-break Other municipalities 12.0 Confirmed cases per 100.000 inhabitants 10.0 8.0 6.0 4.0 2.0 0.0 20 30 40 50 20 30 40 50 Source: DATASUS. Source: DATASUS. 0 10 20 30 40 50 0 10 20 30 40 50 Weeks of 2009, from week 23 (June) to 52 (December) 47 Source: DATASUS. Source: DATASUS. June-July August-December Confirmed cases per 100.000 inhabitants between April-July, 2009 (a) Accumulated, April-July (b) Weekly, April-July Source: The number of confirmed H1N1 cases per week are from https://datasus.saude.gov.br/informacoes-de-saude-tabnet/. (a) The average number of confirmed H1N1 cases between April and July 2009. (b) The weekly number of cases between April and December 2009. Figure A.2: H1N1 confirmed cases per 100.000 inhabitants, São Paulo (2009) Extended winter break Extended winter break Extended winter break Other municipalities Other municipalities Other municipalities 14 94 0.7 0.6 12 92 0.5 10 90 % % % 0.4 88 8 0.3 86 0.2 6 2007 2008 2009 2007 2008 2009 2007 2008 2009 (a) Retention (b) Grade-promotion (c) Dropout 48 Figure A.3: Retention, dropout and grade-promotion, fifth-grade (2007-2009) Note: Retention, grade-promotion, and dropout rates range from o to 100. The indicators for treatment and comparison groups are the average for the sample of locally-managed schools in São Paulo. Source: Census of Education/INEP. Figure A.4: Students’ proficiency in state and locally-managed schools, fifth-grade (2005-2009) G=0 G=1 G=0 G=1 220.00 200.00 216.49 216.84 Portuguese performance, fifth-graders Math performance, fifth-graders 210.04 193.38 191.05 190.00 187.95 200.00 202.91 200.21 185.07 184.37 185.23 195.54 184.09 191.95 180.00 191.70 180.50 190.26 188.26 178.24 187.23 175.17 180.00 170.00 172.07 175.41 169.25 49 2005 2007 2009 2005 2007 2009 2005 2007 2009 2005 2007 2009 State-managed Locally-managed State-managed Locally-managed (a) Math (b) Portuguese Note: The proficiency = 1 and = 0 is the average of the proficiency scores of state and locally-managed schools in São Paulo. The Proficiencies in Portuguese and Math are on the SAEB scale. Source: Prova Brasil /INEP. . 50