Capturing the Educational and Economic Impacts of School Closures in Poland

The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations


Introduction
COVID-19 led to strict lockdown measures, which included school closures in most countries. As a result, more than 1.5 billion students were out of school for weeks or months (UNESCO, 2022).
The loss of schooling is expected to negatively impact children's cognitive development, even if distance learning modes are enacted. The loss of in-person teaching could also lead to inequality since the only remaining relevant input is parental involvement during school closures (Agostinelli et al., 2022).
Most studies document significant learning loss. In Europe, the average learning loss is almost a quarter of a school year, but the estimates are available mainly for Western European countries (Donnelly and Patrinos, 2021). Worldwide the loss is even greater, especially in lower-income countries (Patrinos et al., 2022). Poland is an interesting case because it represents countries in Eastern Europe where school closures lasted longer, and research on learning loss is scarce.

Measuring learning loss using PISA as a benchmark
Most countries do not have standardized assessments that allow comparison of student results over time. To measure learning loss in Poland, we implemented an assessment of students in mathematics, science, and reading, that reports results on a scale comparable to OECD's PISA international study. The results are based on a representative random sample of students in 2021 from grades 10 to 12 and compared to PISA results that are available for a random sample of students from Warsaw collected between 2003 and 2018. In each comparison, the sample sizes are greater than 1,000 students .
Two major changes could affect students' results in 2021 compared to previous years. First, it was the first assessment after schools reopened after the closures. All schools in Poland were closed for at least 26 weeks. This was longer than most Western European countries, which were also better prepared for digital instruction. Second, structural changes implemented in 2016 shortened the period of general academic education for all students, switching the 6+3+3 grade structure with the 8+4 system, and replacing national examinations after the 6 th and 9 th grades with the 8 thgrade exam.
To properly estimate the effect of school closures and to distinguish it from the effect of the 2016 structural changes, we compare the expected and actual achievement of three cohorts of students in secondary schools. We assume students should gain a minimum of 0.1 standard deviation (SD) during one year of education. That is a safe assumption but in line with previous studies comparing 15, 16, and 17-year-old student results on the PISA scale in Poland .
International evidence indicates the gains should be larger, around 0.2 SD, which would make our results more significant as they increase the expected achievement (Avvisati and Givord, 2021).
We also assume students tested in autumn (10 th grade in TICKS 2021) have a similar achievement to those tested one grade below in the spring (9 th -grade assessment in PISA 2003-2018). Assuming any achievement progress between spring and autumn makes our results even more significant.  We separate the effects of the pandemic and the structural reform by comparing achievement changes in the cohorts affected by both events (10 th and 11 th grade) to changes in the 12 th grade affected by the pandemic only. We regress the PISA 2003-2018 scores on the results from 2021, adjusted for the expected minimum achievement gain over one year of education. The regression equation is given by: The structural reform effect is captured by 2 , which measures the difference between students who followed the new and the old school structure. The estimate of the learning loss is given by 1 . We also control for student gender and socioeconomic status to minimize individual variance in student outcomes for the main results. We obtain similar results with added controls for test mode (paper vs. computer) or using multiple imputations to include non-complete student responses (for more details, see Jakubowski et al., 2022).
The learning losses due to the pandemic are around -0.3 SD in mathematics and science and are greater than those caused by structural changes (around -0.2 SD). In reading, the learning loss is smaller, around -0.2 SD, and statistically insignificant in most specifications.

Economic consequences of the learning loss
The learning loss consequences can be assessed both by looking at macroeconomic outcomes and individual-level productivity reflected in wages. On the macro level, we used the growth coefficient In the equation, L is the total loss in wages. The first factor is the product of the mean annual earnings Y, the adjustment factor to account for the period in which schools were closed α, and the rate of return of one year of schooling r, all estimated in their present value (PV). The second factor is the product of the number of students S and the efficiency of distance learning β indicates how many students can switch to remote learning without disruptions. The rate of return of one year of 6 schooling r is equal to 7.8%, following the meta-analysis for Central and Eastern European countries (Wincenciak et al., 2022). Using this value and accounting for the 26 weeks of school closures, the value of the adjustment factor to account for the period in which schools were closed α is equal to 0.52. According to the CSO data, the mean annual earnings Y in 2021 are PLN 72,000.
Following Psacharopoulos et al. (2021), the present value is calculated over a student's working life period of 45 years, assuming a 3% discount rate. The number of students S equals 6 million (UNESCO, 2022). Finally, the efficiency of distance learning β was set at 0.6 following the Polish school closures duration (UNESCO, 2022) and arguments of Kuhfeld et al. (2020) and Psacharopoulos et al. (2021). The yearly economic loss at the individual level equals PLN 2,927.
The present value for the 45 years of working life amounts to a loss of PLN 74,693 for each student.
It sums up to PLN 179 billion for the affected population, which equals 7.2% of the 2021 GDP.

Conclusions
The Polish success story of rapid social and economic progress relied strongly on human capital improvement. Unfortunately, this factor is now under significant distress. Significant learning losses have been experienced by Polish students due to the COVID-19-induced school closures. In mathematics and science, the learning losses are equal to more than a year's worth of schooling, even though schools were closed for only part of an academic year. In addition, we show that the 2016 reforms also had a negative impact on student learning. These skills losses are likely to affect the future economic success of the students as well as the country as a whole. Future earnings are projected to decline by PLN 74,693 (more than US$15,000) per year for the affected students. The country would then lose the equivalent of 7.2% of GDP over time.
7 Schools have reopened, and students are back in class. The task of learning recovery must commence, both to recover some of the losses and to accelerate learning. Accelerated programs must include tailored responses for those that fell behind the most, such as tutoring (Fryer and Howard-Noveck 2020), which has proven to be successful before the pandemic and during the school closures, when it was implemented cost-effectively online (Carlana and La Ferrara 2021).
Teachers need to be supported to address the challenge with additional resources. Education systems need to be made more resilient to face future shocks.