Estimating the Potential COVID-19 Impacts on Learning Poverty in Brazil* João Pedro Azevedo and Diana Goldemberg April 2020 learning in every other area; and, (iii) reading proficiency ABSTRACT can serve as a proxy for foundational learning in other subjects, in the same way that the absence of child School closures due to COVID-19 have disrupted stunting is a marker of healthy early childhood education in Brazil. Before this crisis, 42% of children in development. For more details in concept of learning Brazil were learning poor. This note simulates the impacts poverty please see World Bank (2019). on learning poverty, considering different lengths of school closure. In our intermediate scenario, where schools remain closed for one quarter of the academic HOW IS THE BRAZILIAN LEARNING POVERTY MEASURED? year, learning poverty rises 2.6 to 5.2 percentage points. This indicator brings together schooling and learning. It starts with the share of children who haven’t achieved GLOBAL PANDEMIC AND THE LEARNING CRISIS minimum reading proficiency and adjusts it by the proportion of children who are out of school. Before the outbreak of the global coronavirus pandemic, the world was already dealing with a learning crisis, as LP = [LD × (1 − SD)] + SD evidenced by high numbers of Learning Poverty. With the where, spread of the coronavirus, the education system is facing a new crisis, as more than 170 countries (as of March 31) LP is Learning Poverty; mandate some form of school closures impacting at least 1.5 billion children and youth. LD is share of children in learning deprivation, defined by those below a minimum proficiency For as long as school closures persist, facilitating the threshold; continuity of education through remote learning is a priority. Planning for the eventual safe reopening of SD is the share of schooling deprived children; schools is also key. Efforts to mitigate the pandemic and in the case of SD we assume LD = 1. impacts, particularly for more disadvantaged children, are This brief uses the national learning poverty for Brazil, essential to curtail the loss of learning in the short term, which is build using the Brazilian National Learning further loss in human capital and diminished economic Assessment (SAEB) and a national definition of minimum opportunities over the long term. This brief estimate some proficiency (200 points in Portuguese). This indicator of these effects in Brazil. complements, with far greater spatial granularity, the Brazilian international learning poverty indicator, which is WHAT IS LEARNING POVERTY? constructed using the regional assessment LLECE/UNESCO. Learning Poverty means being unable to read and understand a short, age-appropriate text by age 10. All foundational skills are important, but we focus on reading SCHOOL CLOSURES IN BRAZIL because: (i) reading proficiency is an easily understood In Brazil, the school year runs February to December, with measure of learning; (ii) reading is a student’s gateway to 200 school days organized in quarters. All 27 states * The authors would like to thank Andre Loureiro, Pablo Ariel Acosta, Marcelo Becerra, Ildo Lautharte, Omar Arias, Jaime Saavedra, Halsey Rogers, Shwetlena Sabarwal, and Harry Anthony Patrinos. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. The authors may be contacted at jazevedo[at] worldbank.org. Education Global Practice, World Bank. 1 mandated school closures starting March 16-23, with SAEB 2017, at the municipality level. Then, we proceed to calculate BMP as usual, with the MPL threshold of 200 points in Portuguese. most opting for anticipating the winter break. As of April 10, no state has reopened schools yet, and many have Enrollment: dropout rates and proficiency scores are announced remote learning strategies that will be rolled negatively correlated. We use this relationship, computed out as the break ends. nationally, to linearly estimate a decrease in enrollment at Our main scenario considers a loss equivalent to one the municipal level. quarter of the school year. Rather than the duration of the school closures, this magnitude should be interpreted as LEARNING POVERTY AT MUNICIPAL LEVEL the combined effect of fewer school days and lower efficiency of remote learning. Alternative scenarios are presented in the following section. BASELINE (2017) LEARNING POVERTY EFFECTS • Learning Poverty. 42.2 percent of children in Brazil at late primary age today are not proficient in reading, adjusted for the Out-of-School children. Learning Poverty may raise 2.6 pp, to 44.8 percent. This is the equivalent of 84,000 additional children in Learning Poverty (in a single age-grade cohort). • School Deprivation. In Brazil, 4.8 percent of primary school-aged children are not enrolled in school. These children are excluded from learning in school. This number may increase 0.1 pp, to 4.9 percent. • Learning Deprivation. Large-scale learning assessments of students in Brazil indicate that 39.3 percent do not achieve the MPL at the end of primary school, proxied by data from grade 5 in Source: Author’s calculation using SAEB, School Census and IBGE data 2017. This share may raise 2.6 pp, to 41.9 percent. LEARNING POVERTY INCREASE (HOT SPOT ANALYSIS) Brazil has steadily decreased Learning Poverty in recent years, from 60.3% in 2011 to 42.2% in 2017, an average annual improvement of 3.0 percentage points. With the spread of the coronavirus, the education system could backtrack a significant part of the recent progress. WHAT ASSUMPTIONS WERE MADE? Proficiency: learning gains are linear on school days and proxied by the average observed gains in early secondary, at the municipal level. ➢ Since SAEB is vertically scaled, we subtract from the 9th graders mean score, the 5th graders mean score four years prior, obtaining the cohort learning gains during early secondary. The average learning gain for each year of schooling is 15 points (0.3SD), ranging from 2-28 points in all +5,500 municipalities. This annual learning gain, accrued over 200 school days, is linearly scaled to an equivalent loss of N days (24, 50, or 75) and subtracted from individual student scores from Source: Author’s calculation using SAEB, School Census and IBGE data 2 SENSITIVITY ANALYSIS LEARNING EQUITY CONCERNS Our main scenario considers a loss equivalent to one Changes in Learning Poverty are sensitive to the shape of quarter (25%) of the regular school year . Table 1 the learning distribution. For that reason, small changes in contrasts these estimates for learning poverty and its the average learning score, can result much larger shifts in component indicators in other two alternative scenarios. the Learning Poverty Indicator. In the case of Brazil, the These should be interpreted as a lower and upper bound projected decrease on average score is only 1.7%, but of estimates, and the success of the mitigation strategies learning poverty increases 2.6pp (6.6%) (see Table 1, 25% is paramount to determine the real impact. SYE scenario). Table 1. Sensitivity Analysis This estimate considers all children being equally affected by the crisis and mitigation strategies (green curve). Baseline 12.5% 25% 37.5% Indicator (2017) SYE SYE SYE Learning Poverty (%) 42.2 43.5 44.8 46.1 School Deprivation 4.8 4.8 4.9 5.0 Learning Deprivation 39.3 40.6 41.9 43.2 Learning Poverty - 1.3 2.6 3.9 increase (pp) Learning Poverty 3.0 6.1 9.2 Increase (%) - Increase in 5th grade 41.5 83.7 126.9 learning poor (thousands) - Mean score (points) 214.5 212.7 210.9 209.0 Mean score change (%) - -0.9 -1.7 -2.6 Notes: all values are rounded. SYE = School Year Equivalent However, the learning losses may be much higher for LEARNING POVERTY INCREASE (SHARE OF MUNICIPALITIES) vulnerable children. In this case, the impact in learning poverty may be twice as large, increasing 5.2pp (red 50% 25 days curve). 50 days 75 days 40% BENCHMARKING ASSUMPTIONS Share of municipalities 30% The most important parameter in this simulation is the schooling productivity – how much a student learns per 20% year of schooling. This system-level parameter reflects the combined effect of the effort of teachers, students and 10% parents, instructional quality, curricula and school management. 0% 0 1 2 3 4 5 6 7 8 9 10+ There is a vast literature documenting the heterogeneity of schooling productivity. In OECD countries, learning Learning Poverty Increase (pp) gains on most national and international tests during one school year are between 0.25-0.33sd (Woessman, 2016). Source: Author’s calculation using SAEB, School Census and IBGE data A similar range is observed in developing countries. Singh (2019) estimates a much higher productivity in Vietnam (0.45sd) than in Peru (0.2sd), and intermediate values for 3 India and Ethiopia. Jones (2017) estimates schooling REFERENCES productivity of 0.2-0.3sd in Tanzania, Uganda and Kenya. Jones, S. (2017). Do schools in low income countries really In Brazil, both states and municipalities have responsibility produce so little learning? for education within their jurisdictions, with the Singh, A. (2019). Learning More with Every Year: School Year municipality being the dominant provider of primary Productivity and International Learning Divergence. education. Thus, we estimate schooling productivity at the Journal of the European Economic Association. municipal level, finding a range of 0.04-0.56sd and an 10.1093/jeea/jvz033. average of 0.3sd, in line with the literature. Woessmann, L. (2016) The Importance of School Systems: Evidence from International Differences in Student Achievement, Journal of Economic Perspectives, Vol. 30/3, pp. 3-32, http://dx.doi.org/10.1257/jep.30.3.3 World Bank Group and COVID-19 www.worldbank.org/en/who- we-are/news/coronavirus-covid19 World Bank. 2019. Ending Learning Poverty : What Will It Take?. World Bank, Washington, DC. http://hdl.handle.net/10986/32553 Remote Learning, EdTech and COVID-19 www.worldbank.org/en/topic/edutech/brief/edtech- covid-19 4