Marjorie Chinen • Rafael de Hoyos COVID-19 as an Opportunity to Build Resilient Education Systems Drawing Lessons from Five Countries in Eastern Europe and the South Caucasus COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS © 2023 International Bank for This work is a product of the staff of the World Bank with external contri- Reconstruction and Development / butions. The findings, interpretations, and conclusions expressed in this The World Bank work do not necessarily reflect the views of the World Bank, its Board of Executive Directors, or the governments they represent. 1818 H Street NW, Washington DC 20433 This publication was produced with financial support from the European Telephone: 202-473-1000 Union. Its contents are the sole responsibility of World Bank staff and do Internet: www.worldbank.org not necessarily reflect the views of the European Union. The World Bank does not guarantee the accuracy of the data included in Rights and Permissions this work. The boundaries, colors, denominations, and other information The material in this work is subject presented or shown on any map in this work do not imply any judgment to copyright. Because The World on the part of the World Bank concerning the legal status of any territory Bank encourages dissemination of or the endorsement or acceptance of such boundaries. its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. ii COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Table of Contents Abstract v Acknowledgements vi 1. Introduction and motivation 1 2. Conceptual framework: The technology of skills formation 3 3. The education response to COVID-19 5 3.1 The pre-pandemic context 5 3.2 Measures taken to mitigate the impact of the pandemic on learning 6 3.2.1 Reallocation of school resources 6 3.2.2 Measures to support family inputs 9 3.3 Main challenges related to the rolling out of remote education 10 3.4 Lessons learned 13 3.4.1 Lessons to improve remote education 13 3.4.2 Lessons on the extent of school closures 14 4. Policy options to construct more resilient education systems 18 4.1 Prioritizing, measuring, and improving foundational skills 18 4.2 Opportunities to construct a more resilient education system 20 Final Remarks 23 References 24 iii COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS List of Figures: Figure 1: School closures and school learning outcomes 4 Figure 2: Timeline of school closures since March 2020 6 Figure 3: COVID-19 school closure days and learning poverty 15 (all regions) Figure 4: Average learning losses across various countries 16 Table 1: Main socioeconomic indicators 5 List of Tables: Table 2: Distance education delivery systems deployed at 7 different levels of education Table 3: Measures to support teachers to transition to remote 9 learning Table 4: Measures taken to support families 10 Table 5: Steps taken to assess learning losses because of 16 COVID-related school closures iv COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Abstract In response to the COVID-19 pandemic, education systems had to redeploy inputs typically used in schools to remote education. This significantly reduced average student learning, with disadvantaged students experiencing a disproportionately large decline. Not closing these learning losses will have long-lasting effects on productivity and economic growth and dampen social mobility. In the five Eastern European countries analyzed in this paper, not acquiring sufficient learning is not a challenge that began with the pandemic. Perhaps the pandemic and the attention it is bringing to students’ “learning loss” will create the political conditions to implement long-awaited education reforms to reduce the learning gaps and create better conditions for disadvantaged students, the core element of resilient education systems. This paper shows that using data to guide policy decisions, standardized tests as a diagnostic tool, and remediation policies should become permanent features of education systems. The pandemic pushed forward the use of technology in education. Using technology through online tutoring or Computer Assisted Learning can, when designed appropriately, improve students’ academic performance, socio-emotional skills, and psycholog- ical well-being. v COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Acknowledgements This paper was prepared by Rafael De Hoyos Navarro and Marjorie Chinen under the guidance of Jamele Rigolini (Lead Economist, Human Development) and Fadia Saadah (Regional Director, Human Develop- ment, Europe and Central Asia Region). The paper benefited from the inputs included in the Human Development Education Assessment Reports and the comments received from Task Team Leaders and local consultants from the five countries. The team would like to thank Lucia Casap, Renata Freitas Lemos, James Gresham, and Shiro Nakata for great feedback and contributions, as well as Adrien Samuel, Svitlana Batsiu- kova, Isil Oral Savonitto, Anna Berdzenadze and Anush Shahverdyan for their inputs. This report also benefitted from valuable feedback from Harry Patrinos (Adviser ECACE) and Rita Almeida (Practice Manager, Education, Europe and Central Asia Region). Finally, the team would also like to thank repre- sentatives of the European Commission for useful feedback and inputs, in particular Hoa Binh Adjemian, Thibault Charlet and Corinne Deleu. The note was made possible by generous funding from DG NEAR’s Europe 2020 Trust Fund. vi COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS 1. Introduction and motivation The COVID-19 pandemic exposed the vulnerability rates. This meant that the pandemic interrupted of student learning in many countries. This is par- the human capital formation process, with children ticularly true for disadvantaged children and youth, missing out on education for months or even – in whose human capital formation depends almost some places – for years. Governments implemented entirely on the provision of public services. If public remote education strategies to provide learning education services are interrupted, or their quality alternatives while schools were closed. Evidence is compromised during shocks – pandemics, earth- shows that remote education is, in the best-case quakes, wars, or others – then the unavoidable out- scenario, a poor substitute for face-to-face learn- come is that these shocks will have a long-term ing. Most disadvantaged children and children negative impact on overall well-being, with a higher whose parents have low educational attainment cost for the poor. Essential lessons can be drawn lack devices connected to the internet, so home from the COVID-19 pandemic to avoid interruptions learning is not feasible. International evidence to the learning process during shocks. This paper shows that students’ learning suffered because of aims to identify lessons learned from the COVID-19 school closures and that this learning loss – or for- pandemic for designing the policies needed to build gone learning to be more precise – was more acute more resilient education systems. The analysis is the longer the schools remained closed and among based on policy responses in five Eastern European disadvantaged students.2 Learning losses and their countries (Armenia, Azerbaijan, Georgia, Moldova disproportionate impact on poor students will have and Ukraine), contrasted with international evi- substantial long-term welfare implications due to dence. Most of the lessons learned are relevant for forgone productivity, with countries suffering lower other countries in the region and for middle-income growth and increased inequality. countries worldwide. With some marginal differences in design and The concept of resilience used in this paper refers implementation, remote learning strategies mainly to the capacity of the education system to respond relied on online classes with complementary peda- effectively to shocks and crises1 in four domains: gogical materials available from dedicated portals, preparation, response, adaptation, and building in addition to some efforts to reach the most dis- back better. Resilient education systems ensure advantaged students with educational material by that the learning process is disrupted to the mini- television. Teachers struggled, given their low levels mum possible extent. These systems prepare teach- of digital literacy and lack of experience of deliver- ers for teaching under various crisis scenarios and ing online classes. design programs, to ensure that no students drop out of school because of a crisis. Resilient education But other more critical factors explained the learn- systems use standardized assessments to deter- ing loss caused by school closures. The quality mine the magnitude of the problem and act based and relevance of online education were marginal on that information. Resilient education systems compared with the negative effect of the number acknowledge that disadvantaged students are of days that schools remained closed. Reopen- more likely to be disproportionally affected during ing schools as soon as it became clear that they extended school closures, and thus have compen- were not the infection hub that was thought at the satory and remediating policies and interventions beginning of the pandemic was far more important in place. Furthermore, resilient systems recognize than improving the effectiveness of remote educa- that shocks also create opportunities to introduce tion. The COVID-19 pandemic showed that educa- long-awaited reforms to address the structural tion systems display rigidities (or at least policymak- constraints of the system. ers perceive them) in their decision to open or close schools: in most cases there was a binary decision Most countries decided to close schools at the affecting all schools with no space for school auton- outset of the COVID-19 pandemic, with no infor- omy. Only two of the five countries analyzed in this mation about the virus’ lethality and transmission report opened some schools with less than all of the 1 In this paper, a crisis is defined as any event (such as pandemic, strikes, weather, conflict, or war) that affects the normal operation of schools forcing 1 system-wide school closures for long periods, disrupting students’ education and potentially affecting their academic performance and progress. 2 Patrinos et al., 2022 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS students attending any given day. In the other three countries, the decision was either to keep all schools closed at all times, or fully open with all students attending five days a week. A few countries with education systems showed resilience during the COVID-19 pandemic. Despite closing schools, students in Denmark and France did not experience learning loss. This is partly explained by the initial conditions in these countries and many of the evidence-based decisions that policymakers took during and after the pandemic. France prior- itized the reopening of schools over other activi- ties. French students lost 10 weeks of face-to-face instruction, one of the world's shortest periods of school closure. Denmark kept schools closed for longer (around the average among Organisation for Economic Co-operation and Development – OECD – countries). Still, their education system had reliable digital infrastructure with developed online learning platforms, high levels of digital skills among stu- dents and teachers, and universal broadband con- nectivity. Both countries had robust, standardized tests that identified learning gaps which triggered compensatory or remediating policies – tutoring in small groups and prioritizing foundational skills in the curriculum – right after the pandemic. 2 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS 2. Conceptual framework: The technology of skills formation The stock of an individual's human capital or skills, lative process, with learning today determined by knowledge, and experience is the outcome of a the stock of skills or learning acquired in the past.6 complex, dynamic, cumulative process involv- The cumulative nature of learning implies that, if not ing innate ability, parental investment decisions, adequately addressed, the learning loss caused by and personality traits such as grit and motivation. school closures today will have long-lasting impacts Schooling is one of the most important parental on students’ ability to continue learning throughout investment decisions determining student learning their lives. (T) and, therefore, the stock of individuals’ human capital. Parents can invest in schooling by absorb- These properties of the learning production func- ing the opportunity cost of children going to school tion have important implications for the distribu- (forgone labor remuneration), investing directly in tion of student learning outcomes in general, and private education, choosing a neighborhood partly the effects of school closures. The concentration of based on the quality of public education availa- family and school inputs in better-off households, ble, or making other efforts to improve the quality the possibility of better-off parents compensat- of school inputs available to their children. Paren- ing for adverse changes in school inputs, and the tal investments also determine the availability of cumulative nature of the learning process explain household assets and other family inputs condu- the structural learning disparities observed in most cive to learning, such as learning material at home, a countries even prior to the pandemic, and why good learning environment, and access to technol- school closures exacerbated learning gaps. ogy. Rich parents can afford housing in a neighbor- hood with a high-quality public school, or can pay The simple framework described here can help us for private schooling with qualified teachers, learn- to identify the effects of school closures on learning ing materials, and motivated and committed school outcomes and their distribution (see Figure 1). When directors. Children of better-off parents also enjoy schools closed, school inputs had to be reallocated more family inputs at home, such as books, techno- to remote learning activities, significantly reducing logical devices and – most importantly – educated their ability to produce learning (efficiency effect). parents who create a positive cross-household Teachers, for example, had to deliver classes through covariance between family and school inputs.3 online platforms without previous experience within a few days, significantly reducing their effectiveness The learning outcomes of student ‘i' at age ‘a’ are in producing learning (first arrow in the upper left determined by his or her innate and immutable abil- side of Figure 1). With schools closed, family inputs ity (μ_i), and the flow of past parental investments in became much more important to produce learn- the form of family and school inputs, Fi(a) and Si(a), ing, putting marginalized students at a disadvan- respectively: tage compared to their more affluent peers (upper right side of Figure 1). Students in well-off house- Ti,a= Ta (Fi(a),Si(a), μi) (1) holds had the enabling conditions to mitigate, at least partly, the effects of school closures: devices Family and school inputs in equation (1) have a posi- at home with access to the internet; parents with tive effect on learning outcomes (∂Ti⁄∂Fi , ∂Ti⁄∂Si > 0). sufficient schooling and time to assist them during There is some evidence suggesting that, within their learning process; and quiet spaces to study, households, families adjust their inputs in response concentrate and learn. Meanwhile, disadvantaged to the availability of school inputs, making family students did not have these minimum conditions to and school inputs substitutes (∂2Ti⁄∂Fi∂Si > 0).4 For learn, so every day of school closure represented a instance, affluent families can increase family inputs loss of opportunity for poor children.7 See the lower when the availability or efficiency of school inputs is right side of Figure 1. reduced, as occurred during the COVID-19 school closures.5 A second important characteristic of the To summarize, from a conceptual point of view, the technology of skills formation is that it is a cumu- COVID-driven school closures reduced the effec- 3 This positive covariance is observed across families, although within families the covariance between family and school inputs is most likely neg- 3 ative, showing that parents compensate – in the context of their budget constraints and preferences – for reductions in school inputs (Das et al., 2013) 4 Das et al., 2013 5 Das et al., 2013 6 T_(i,a) is determined by T_(i,(a-1)) plus the parental investment at age “a” in the form of family and school inputs, F_i (a) and S_i (a). Substituting the parental investments in the flow of past student learning outcomes results in equation (1). 7 Agostinelli et al., 2020 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Figure 1: School closures and school learning outcomes Ine ciency e ect due to SCHOOL INPUTS misallocation. Student Learning (cumulative process) Inequity e ect due to more elevance of family inputs in the FAMILY INPUTS production of learning. Vulnerable households do not have the minimum conditions to learn: • Enough and appropriate devices • No or inadequate devices • Good connectivity • No or poor connectivity • Suitable space for children to study • No suitable space for children to study • Learning materials / books • No or minimal learning materials / books • Educated parents, with flexible schedules to support • No educated parents, with fixed schedules to support their children their children Source: Authors’ illustration. tiveness of school inputs, harming overall learning gies targeting disadvantaged students. Otherwise, outcomes. They also made the relatively unequal given the cumulative nature of learning, the learning distribution of family inputs more important in the loss brought about by school closures would have production of learning, therefore increasing learning long-lasting costs, particularly among the poor. disparities between poor and non-poor households. The result is a much more pronounced learning loss among disadvantaged students (See Figure 1). The technology of skills formation is also helpful for identifying the potential contribution of public policy to shaping learning disparities in the con- text of school closures. By offering remote learning during school closures, governments worldwide were mitigating the efficiency effect but also exac- erbating the inequality impact – as the most dis- advantaged students did not benefit at all from remote education. During the pandemic, many governments focused their efforts on improving the effectiveness of online learning through, for exam- ple, better online educational materials or training to enhance online teaching; this, at best, had a mar- ginal positive impact among relatively better-off students. Given these limitations of remote learn- ing, particularly among disadvantaged students, the most efficient and equitable education strategy was to reopen schools as soon as it was safe. The second-best option would have been to comple- ment remote learning with interventions to improve the availability of family inputs, such as provid- ing disadvantaged households with technology devices with access to the internet. Shutting down schools meant closing one of the few opportunities for many households in low- and middle-income countries to escape poverty. Therefore, as shown by the simple framework described here, education authorities should implement remediation strate- 4 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS 3. The education response to COVID-19 3.1 The pre-pandemic context tive potential, due to lack of access to quality health and education services. In Azerbaijan, Georgia, and Five countries in Eastern Europe and the South Cau- Moldova, productivity and long-term growth are casus are analyzed in this study: Armenia, Azerbai- below 60 percent of full potential, due to a lack of jan, Georgia, Moldova, and Ukraine. These countries universal access to human capital. Most of this loss vary in their levels of development, per capita GDP, is accounted for by the relatively large share of the poverty rates, and pre-pandemic education indi- student population with low learning levels. cators. As shown in Table 1, the countries are mid- dle-income countries with per capita GDP levels in The following information sources were used in this purchasing power parity (PPP) of around $13,000, analysis to describe the various policies and initia- significantly below the average in Europe and Cen- tives implemented in the five countries to mitigate tral Asia (ECA). Three of the five countries have rel- the impact of the pandemic on student learning atively low poverty levels, measured as per capita outcomes: (i) human development reports prepared household incomes below $5.50 international dol- by World Bank staff in the five countries; (ii) global lars. In contrast, Armenia and Georgia have poverty survey data from ministries of education around the rates of 45 percent and 47 percent respectively. world collected by UNESCO, UNICEF and the World Bank in May 2020 (Round 1)8, July 2020 (Round The five countries have high levels of expected 2), and February 2021 (Round 3); (iii) phone survey years of schooling, but still considerable propor- data from parents in Georgia, collected by the tions of the student populations with low learning World Bank in August 2021; (iv) focus group discus- levels. In Armenia, and Ukraine, for instance, at least sions (FGDs) with parents, teachers, and principals 23 percent and 28 percent of children, respectively, in Georgia (July 2021), and Moldova and Armenia are unable to read proficiently by age 10. The World (December 2021); (v) the UNESCO global tracker of Bank’s Human Capital Index (HCI) captures how COVID-19-caused school closures and re-opening productive typical children born in each country will (UNESCO tracker,9 data retrieved on January 21, be when they grow up: all the countries included in 2022); (vi) and other data such as learning poverty,10 the analysis are well below the ECA average of 0.69. GDP per capita, Gini indices, and poverty levels, In other words, average newborns in Armenia are among others from the World Bank. expected to exploit just 58 percent of their produc- Table 1: Main socioeconomic indicators Poverty rate GDP per capita, at $5.50 a day Expected years Learning pov- Human Capital Country Gini Index3 PPP1 (2011 PPP) (% of schooling4 erty5 Index (HCI)6 population) ECA Average 35 345.09 11.5% 13.1 13% 0.69 Azerbaijan 14 442.04 7.0% --- 12.4 23% 0.58 Armenia 13 653.76 44.7% 29.9 11.3 26% 0.58 Georgia 14 989.26 46.6% 35.9 12.9 14% 0.57 Moldova 13 026.52 13.3% 26.0 11.8 11% 0.58 Ukraine 12 804.96 2.5% 26.6 12.9 28% 0.63 1 GDP per capita based on purchasing power parity (constant 2017 international $), 2019. 2 The most recent values for these countries are as follows: for Armenia (2020), Azerbaijan (2005), Georgia (2020), Moldova (2019), Ukraine (2020), ECA (2019). 3 Gini Index, 2019. 4 World Bank Human Capital Index, Component 2, 2020. 5 World Bank Learning Poverty Indicator, 2021. Note that although the whole dataset is from 2021 the most recent country level data is from 2019. 6 World Bank Human Capital Index, 2020. 8 UNESCO, UNICEF and World Bank, 2020 5 9 UNICEF, 2022 10 World Bank, 2022. COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS 3.2 Measures taken to mitigate the impact of school closures on learning outcomes can impact of the pandemic on learning be characterized by the effects they had on school inputs, which had to be allocated in an imperfect All five countries closed school institutions in March way (inefficiency), and the increased role played by 2020 due to the COVID-19 pandemic. National family inputs in the production of learning (ineq- education stakeholders responded by adopting uity). Therefore, the rest of this section describes the similar remote learning policies based on a com- efforts made by the governments to deploy school bination of online platforms and broadcast media inputs and to try to improve the availability of family delivery, to provide some continuity to the learning inputs among disadvantaged households. process while school institutions were closed. 3.2.1 Reallocation of school resources Figure 2 presents the timeline of government-man- dated closures of educational institutions between To provide some continuity in the learning process the emergence of the pandemic in March 2020 and while schools were closed, national education stake- October 2021. Between March 1, 2020 and October holders in the five countries adopted relatively sim- 31, 2021,11 Armenia fully closed school institutions ilar remote learning policies, based on a combina- due to COVID-19 (affecting at least 80 percent of tion of online platforms and television education on the student population) for 63 days, Moldova for 112, multiple channels. Online learning platforms were Georgia for 113, Ukraine for 125, and Azerbaijan for created by ministries of education or education 205. Azerbaijan closed school institutions for longer authorities, and included a variety of educational than any other country in ECA (the average for the resources for teachers and students (such as learn- region is 89 days). And when the number of days of ing materials and homework resources). Countries “academic breaks” and “extended academic breaks” also used a combination of synchronous instruc- are also included, Azerbaijan, Georgia, and Armenia tion (real-time interactions between students and are among the 10 ECA countries that closed school teachers while lessons and instructional content institutions for the longest period during the pan- are being shared) and asynchronous instruction demic. (students learning at their own pace either inter- acting or not with the teacher). A summary of these As described in the conceptual framework, the responses is included in Table 2. Figure 2: Timeline of school closures since March 2020 2020 2021 Country Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Jan Feb Mar. Apr. May June July Aug. Sept. Oct. Armenia Azerbaijan Georgia Moldova Ukraine Belgium Norway Denmark COVID-19 guidance for safe in-person learning released AAP ECDC Fully closed due to COVID-19: student population enrolled from pre-primary through to upper secondary levels. Most schools across the country are on scheduled academic breaks for periods of at least one week. All study Academic Break: during this period is suspended. Schools are (a) open/closed in certain areas only, and/or (b) open/closed for some grade levels/age groups Partially open: only; and/or (c) open but with reduced in-person class time, combine with distance learning (hybrid approach). For the majority of schools (at least 80%), classes are being held exclusively in person, noting that measures Fully open: to ensure safety and hygiene in schools vary considerably from context to context and/or by level of education. Source: Authors’ calculations using UNESCO Tracker on school closures caused by COVID-19. The American Academy of Pediatrics (AAP) released, on June 24, 2020, the first school guidance for safe in-person learning.12 The European Centre for Disease Prevention and Control released its recommendation on August 6, 2020.13 11 When data were retrieved from the UNESCO Global Tracker in January 2022, the cumulative data on the COVID-19 caused school closures and 6 re-opening were available until October 31, 2021. 12 AAP, 2020 13 ECDC, 2020 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Table 2: Distance education delivery systems deployed at different levels of education 2. Online Learning platform Commercial 3. Mobile Country Level 1. Television All subjects Commercial not for free (Black- Open source Available Modality for free (Teams, (Moodle, Can- phones covered? board, google etc.) vas, etc.) class, etc.) Pre-primary No No Yes, partially Armenia Primary Yes Yes Synchronous No subsidized Yes Yes Yes Yes, partially Secondary Yes Yes Synchronous Yes subsidized Yes Yes Yes Pre-primary Yes Yes Yes, free Azerbaijan Primary Yes Yes Hybrid Yes access for all Yes Yes, free Secondary Yes Yes Hybrid Yes access for all Yes Yes, free Pre-primary Yes No access for all Georgia Primary Yes Yes Yes Yes Secondary Yes Yes No Yes Pre-primary No No Moldova Primary No Yes Hybrid Yes Secondary Yes Yes Asynchronous Yes Source: UNESCO-UNICEF-WORLD BANK Surveys May and July 2020. Ukraine did not participate in any round of this survey. Note. None of these countries used radio or paper-based take-home materials to deploy distance education. However, in Georgia, parents with no access to preschool education for their children received take-home packages. Armenia launched television lessons broadcast on broadcasting classes for ethnic minorities in their the national public channel, new television channel languages. The country also enhanced the EL.ge Hybrid Edu, and the Armeduchannel YouTube and digital platform with digital educational resources cable channels. The broadcasts included more than and launched the I-school project, which provided 400 television lessons of up to one hour each in a primary, basic, and high school teachers and stu- limited number of subjects. These lessons targeted dents with homework sets (so-called “complex schoolchildren from primary through upper-sec- assignments”) and supporting materials. Geor- ondary levels, and children with disabilities (such gia rolled out the E-assessment journal, part of as hearing impairments). Through a public-private the Assessment for Development Project (ADP) partnership between the Ministry of Education and piloted before the pandemic. The ADP is intended telecommunications companies, broadcast cover- to implement a census-based e-assessment (fully age of these lessons was made available to users online), measure student achievements against regardless of their tariff plan or without extra cost. the national curriculum, and track their progress. Video lessons were also uploaded to “E-school Approximately 150,000 students in fourth, sixth, Armenia” digital platform and the official YouTube and tenth grades are expected to be assessed in “Armenian educational channel”. E-school Arme- literacy and numeracy every year. The assessment nia was launched in late spring 2020. The plat- results are available immediately at the end of each form includes digital resources, tele-lessons, online exam, and automated e-reports on the results are libraries, digital tools, and references to organiza- generated and distributed to teachers and parents tions with other digital resources. The platform was to incentivize data-driven decisions.14 Like in Azer- supported by the Ministry of Education and run by baijan, to address the fatigue students develop the National Center for Educational Technologies with online learning the Ministry of Education rec- (NCET): the national body responsible for provid- ommended that classes be reduced from 45 to 30 ing information and communications technology minutes. However, teachers reported that the Min- (ICT) resources to schools in Armenia. When schools istry did not provide explicit recommendations on started reopening in September 2020 and shifting how to adjust the content or which areas to prior- to new modes of remote and hybrid learning, the itize to comply with the adjusted schedule. Teach- leading digital platforms used for distance learning ers indicated that the reduced schedule only gave were Google Classroom and Microsoft Teams. them time to explain the content and explain pupils’ homework briefly. Georgia offered free access to Microsoft Office for all schools and introduced daily television educa- In Azerbaijan, the Ministry of Education, in cooper- tion for all grades. The government also started ation with two television channels, introduced the 14 Ministry of Education and Science of Georgia, 2021 7 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS ‘TV Lessons’ (‘TV Ders’) educational program. The used multiple online platforms and communication government recruited a group of qualified teachers mechanisms (such as Zoom, Viber, Google Class- to record television lessons, which were broadcast room, and Skype) for distance learning. Nonethe- nationwide to a pre-announced schedule in a 5–6- less, in March 2020, the government established hour period every day (From March 11 to June 12, a television show broadcast on all national chan- 2020). Television classes covered most subjects for nels, providing TV lessons for students in grades all grades following the school curriculum. All tele- five to eleven. A “timetable” for every school age vision lessons were then posted on the Ministry of was widely communicated through the MoES web- Education’s electronic portal,15 on its official Face- site and several platforms. In December 2020, the book page, and on its official YouTube channel. In government developed the All-Ukrainian School addition, the country launched ‘Virtual School’16 in Online website, in collaboration with NGO Osvito- collaboration with Microsoft, offering the Microsoft ria. This platform compiles television lessons, gath- Teams platform accessible to all public schools. Vir- ers learning materials in 18 basic subjects (including tual schooling was multifunctional, offering several lectures, tests, and materials for independent work) features: distribution of content (such as video les- for grades five to eleven, and offers methodolog- sons, digital education content, and e-books), online ical support for teachers. Nonetheless, the system classes, weekly assignments and assessments, and took time to launch. Moreover, in December 2020 collaboration (student and teacher groups, and the MoES, with support from UNICEF, launched correspondence between students and teachers). an online platform called “NUMO”, featuring edu- In addition, the Ministry offered two hours of live cational and development videos (such as games and interactive homework assignments each week, and exercises) for children aged three to six. The enabling teachers to explain some topics, provide government also supported public-private partner- feedback, and answer students’ questions submit- ships to integrate innovations in education. These ted via chat. Moreover, during the move to synchro- partnerships included companies such as Microsoft, nous online education, the government decided to Google and Zoom and provided over 100,000 MS adapt the curriculum, adjust daily lesson schedules, Office and Windows licenses to educational estab- decrease the number of teaching hours for each lishments at all levels, as well as access to Google topic, and minimize the length of the online classes Workspace for colleges and universities in Ukraine. for all grades from the traditional 45 minutes to 30 minutes, with 10–15-minute breaks between each While schools were closed, the most important class. challenge faced by the education systems was to ensure that teachers had the skills and ability they Moldova launched television lessons and online needed to deliver online learning. Countries offered platforms with live and recorded lessons to deliver free connectivity and some forms of professional free distance education.17 The policy of using multi- development training to support teachers' transition ple learning and communication platforms (such as to remote learning. Training included areas such as Zoom, Google, Viber and WhatsApp) offered several instruction on distance education to increase peda- options to teachers to provide distance education, gogical skills for remote learning, the adaptation of but at the same time led to challenges and confu- teaching content to remote teaching, and the use sion among teachers. In a FGD with teachers, some of open education resources, the use of ICT tools, reported that they would have preferred to use a and other topics (See Table 3). However, training single platform promoted by the Ministry of Edu- was not universal and often just supported a small cation to teach and assess students, like the expe- group of teachers. A survey of ministries of educa- rience of MS Teams introduced by the Ministry of tion found that at least one in three countries did Education in Armenia. not provide any training for teachers to use remote learning platforms.18 In Ukraine, schools were given autonomy and the responsibility to organize and deliver remote learn- As such, in Armenia, the distance education plat- ing activities. The Ministry of Education and Sci- form provides teachers and school management ence (MoES) provided limited guidance during this with links to educational platforms, and instructions process (school principals were offered some sani- on using ICT tools.19 Teachers were also offered tary recommendations and limited assistance). The special mobile and fixed internet tariff plans and decentralization process of the education system free access to the main applications and platforms created accountability gaps between the govern- used for distance learning. Besides, the Ministry of ment and local authorities, which contributed to Education of Armenia and the NCET began a cycle delays and problems implementing remote educa- of accelerated courses for teachers on e-learning tion. A consequence of this policy was that teachers platforms. In 2020, eight courses were developed 15 Edumedia-Azərbaycan, 2019 8 16 Ministry of Science and Education, 2020 17 Studii.md online platform, 2020 18 UNESCO, UNICEF and World Bank, 2020 19 Between 2017 and 2020 and under the World Bank’s Education Improvement Project (P130182), approximately 3,000 teachers and administrative staff were trained in ICT use. COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Table 3: Measures to support teachers to transition to remote learning 1. Instruction 2. Professional, 3. Teaching con- 4. ICT tools and 5. Guidelines 6. Professional 7. Training: 8. Materials to on distance psychosocial and tent adapted to free connectivity for reducing the development Percent trained support distance education (TV, emotional support remote teaching (PC, mobile amount of over- activities (e.g. learning: Percent radio, learning (e.g. chat groups, (e.g. use of open device, voucher time required to workshops and that received platforms, etc.) online forums educational for mobile prepare a virtual webinars) on materials Country to share ideas resources (OERs), broadband, etc.) classroom pedagogy and and educational sample lesson effective use of content) plans etc.) technologies with various peda- gogies 75% - but Armenia Yes Yes Yes No No Yes 50% -75% less than 100% Azerbaijan Yes Yes No No Yes Yes 25% - 50% 25% - 50% Unknown/not Georgia Yes Yes Yes Yes Yes Yes 25% - 50% monitored Moldova Yes Yes Yes Yes ––– ––– ––– ––– Source: UNESCO, UNICEF and World Bank Surveys July 2020 and February 2021. Ukraine did not participate in any round of this survey. on “Skills to organize distance learning” for 2,880 (for secondary and preschools) or the European teachers.20 At the beginning of 2021, the govern- Union (EU) through the EU4Skills program (for ment reported that between 50 and 75 percent of vocational education and training) to support the teachers had been trained. transition to online classes. However, these projects were often conducted on a small scale and in tar- In Moldova, with the support of development part- geted schools. ners, the Ministry of Education launched a cam- paign named “a computer for every teacher”, which Regarding access to electronic devices, the Ministry equipped teachers and students from disadvan- of Digital Transformation launched the “Laptop for taged groups with 12,500 laptops. It also offered Every Teacher" project in 2021 to equip secondary teachers 80 GB of free internet traffic data. The teachers. It was intended to procure 60,000 com- target population was identified using data from puters and distribute them to schools in 24 regions the Education Management Information System of Ukraine and the city of Kyiv. As of December (EMIS), which identified approximately 9,484 stu- 2021, over 27,000 computers had been delivered to dents from grades five to twelve in 35 districts who teachers. required computers to connect to online lessons. 3.2.2 Measures to support family inputs In Georgia, the National Center for Teacher Profes- sional Development21 offered webinars for teachers Many countries negotiated access to the internet to provide some pedagogical and methodological at subsidized or zero cost with local private com- recommendations. munication companies to facilitate student access to online distance learning infrastructure. “Zero-rat- The Ministry of Education of Azerbaijan also intro- ing educational content” is considered one of duced online training and webinars on distance the quickest ways to improve cost-free access to learning, e-teaching, e-resources, and tools. In online materials.23 For example, in Armenia, special partnership with a local in-service education center, mobile and fixed internet tariff plans were offered the ministry provided online training for 14,000 for schoolchildren and teachers, and free access educators to improve their ICT skills.22 Moreover, in was provided to the main distance learning appli- partnership with the Ministry of Transport, Commu- cations and platforms. Additionally, in Armenia, all nication, and Higher Technologies, the local mobile 181 general secondary and high school textbooks networks operator granted some teachers mobile were posted in electronic format free of cost. Geor- internet access free of charge. gia also introduced subsidized internet access to children in January 2021. Still, these benefits were In Ukraine, teacher training on distance learning introduced relatively late and were not widely dis- was often coordinated at school level. Since the seminated; therefore, families did not fully benefit government only provided limited guidance on how from them. In FGDs with Georgian teachers, some to deliver remote learning, several non-govern- added that the internet package was not enough ment organizations intervened to provide in-service to conduct classes, search, download, or upload training to teachers, prepare teaching materials, materials. Likewise, Moldova offered 80 GB of free and develop online classes. For instance, several ini- internet traffic per individual in need. Countries also tiatives were developed in partnership with UNICEF made access to distance learning platforms avail- 20 There are approximately 31,018 teachers in Armenia, according to the Statistical Committee of Armenia, 2020. 9 21 Ministry of Education and Science of Georgia, 2014. 22 There are 140,648 teachers in public and private secondary schools in Azerbaijan (State Statistical Committee of Azerbaijan, 2020). 23 Truncano, 2016 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS able through mobile phones. In Ukraine, national sive behavior among young students, panic, fear mobile operators provide non-tariffed access to the of admission exams, and so on. According to the All-Ukrainian School Online platform’s website and survey of ministries of education, Azerbaijan also its mobile application. offered support to counter interrupted school meal services. In Ukraine, the MoES and the Office of the Fewer countries subsidized or provided devices for Educational Ombudsman established several hot- free. For example, Armenia received donations of lines to support teachers and students in second- laptops24 that were distributed among students; in ary and higher education, and vulnerable students Georgia, since 2011, laptops have been distributed were prioritized and received free prepacked school for free to grade one students attending public meals at home during distance learning. schools. Identical laptops have also been distrib- uted to first-grade teachers. The laptops, referred On the other hand, Moldova introduced additional to as “bukis”, are a local version of the Intel classmate support to lower-income households and child pro- laptop produced by a Georgian computer firm,25 tection services (see Table 4). While Armenia, Azer- and are pre-loaded with digital textbooks and other baijan, and Moldova encouraged regular calls from educational content (such as education games teachers or principals to students, in Georgia, com- developed by the Ministry of Education). In Moldova, munications mainly took place via videoconference the Ministry of Education distributed 12,500 lap- technology or the e-school platform. None of these tops for teachers and students in 2021 to fully sat- countries reported implementing specific measures isfy the demand of secondary and lyceum students to support girls’ education. from all districts.26 Finally in Ukraine, while the MoES encouraged schools to provide electronic devices to students from disadvantaged socioeconomic 3.3 Main challenges related to the backgrounds, it also stressed that parents were rolling out of remote education responsible for ensuring access to online education. The information collected through FGDs in the dif- Some countries also introduced measures to ferent countries provided unique and deep under- include populations at risk of being excluded. These standing of how the pandemic affected the lives included flexible and self-paced platforms (asyn- of teachers, parents, and students in various ways. chronous learning), sign interpretation for students Teachers and principals reported how the pandemic with special needs, and psychosocial and mental changed their teaching strategies and interactions health support for learners (for example online coun- with students, shared many challenges encoun- seling) to minimize the impact of school closures on tered during this period, and revealed that many the well-being of students. All five countries offer elements needed for implementing a successful psychosocial and mental health support to learners remote strategy were not in place. Qualitative data by phone or online. For example, Azerbaijan created collected from parents exposed the limitations of a hotline to provide socio-psychological advice to the measures taken by ministries of education to students and teachers. Experienced school psy- support student learning – particularly among the chologists provided psychological counseling on disadvantaged – and revealed the struggles of fam- various psycho-emotional issues such as aggres- ilies during school closures. Table 4: Measures taken to support families A. To facilitate the access to B. To ensure inclusions of population at C. To minimize impact of school closure online learning risk of being excluded on the wellbeing of students 1. Negotiate 2. Make access 3.Subsidized 1. Special 2. Flexible and 3. Additional 4. Tailored 1. Psychosocial 2. Regular calls 3. Additional 4. Support access to to distance / free devices efforts to self-paced support to learning and mental from teachers child to counter internet at learning for access improve platforms lower-income materials to health support or principals protection interrupted subsidized or platforms access to households, learners with to learners services school meal zero cost available learners with including disability, services through disabilities, economic refugees, mobile phones refugees, support migrants, and migrants, and (e.g., (e.g., take-home minorities (e.g., distribution Asynchronous rations, cash (e.g., online of meals, food minorities Country learning) based transfers) counselling) banks, vouchers) Armenia Yes* Yes Yes* Yes Yes Yes Yes Yes Azerbaijan Yes Yes Yes Yes Yes Yes Yes Georgia Yes Yes Yes Yes Yes Yes Yes Moldova Yes Yes Yes Yes Yes Yes Yes Source: OECD, UNESCO, UNICEF and World Bank Surveys May 2020, July 2020, and February 2021. *School-by-school basis. Ukraine did not participate in any round of this survey. These donations came from commercial banks, IT companies and members of parliament. 24 10 Ministry of Education and Science of Georgia, 2011 25 ipn Press Agency, 2021 26 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Among teachers, the most common challenges lot of time was wasted on organizational issues were related to: (i) poor connectivity and availabil- before classes started.” (Rural teacher, Moldova) ity of equipment; (ii) difficulties adapting curricula to remote education and insufficient digital skills “School is the only environment for children to among teachers to deliver remote learning; (iii) communicate with each other … Every communi- increased workloads with little or no guidance from cation has a positive effect. No matter how hard ministries of education; and (iv) challenges grading we try, we cannot ensure the quality of offline students. For example, related to the first challenge, learning during remote learning, no matter what in Moldova, schools had to operate without equip- equipment we use.” (Male principal, Armenia) ment for long periods, and communicated with students and parents using text messages, social “In the beginning I was worried, to be honest, I media, or phones. Teachers would write text mes- was learning how to use online tools, and then I sages with the lessons and the homework without was teaching my students. It was hard emotion- any additional interaction with their students: ally, but slowly I got used to it. I had great moti- vation, which helped me keep calm.” (Female, “If other schools were somewhat prepared, we teacher, Georgia) were three steps below them in terms of com- puter equipment. We were confused, blocked, Third, teachers in all the countries reported an and at a disadvantage. It was torment. We were increase in their workloads, and little to no guidance only working through text messages; messaging from their ministries of education on how to face took us till 9 or 10 at night. We also had discus- the challenges of remote learning. The additional sions – mostly with parents – on landlines, and workload was usually related to the added task of used various other methods. It was no way to monitoring students and keeping them engaged work”. (Urban teacher, Moldova) during online education. These difficulties were compounded by inconsistent or insufficient guid- “We have very old equipment at our school, which ance by the ministries of education. For example, in we are trying to replace, but this is not enough. Moldova teachers had to report to the authorities We need a better internet connection for teach- on the status of students during distance education: ers to conduct online classes. Everything they have learned and achieved during this period “The problems were generally associated with needs to be supported to keep and develop the presence or connection of students from more in the future, but we are still struggling socially vulnerable families and monitoring them. with this.” (Male principal, Georgia) You had to be able to say where they were at that moment, what they were doing: that is, the A second challenge for teachers was to adapt cur- teacher had somehow become a social worker ricula for remote education with little or no guid- and a parent. We had to report to the ministry ance from ministries of education. Teaching on what their temperature was and how they were online platforms was particularly challenging for doing.” (Rural teacher, Moldova) older teachers, most of whom lack the necessary digital skills and, on the receiving end, among stu- “I taught many lessons outside school with dents with disabilities. In Georgia, some teachers pupils, when we knew that we didn’t have went to school to support students with disabilities, enough time and they needed more explana- who otherwise would have been wholly excluded tions of the lessons.” (Female teacher, Georgia) from learning. Teachers also found it challenging to keep students engaged in online learning. In Geor- Finally, another significant challenge for teach- gia, principals reported that pupils’ attitudes wors- ers was assessing students’ knowledge in remote ened: they did not attend classes, learn what they education. Teachers in Georgia felt they were lim- were supposed to, or behave as well in class as they ited in what they could do online, including how to did during face-to-face studying. Principals also approach activities that would typically be graded. reported that teachers struggled to control all their In Moldova, teachers concurred that assessing stu- students. In their opinion, students had more free- dents was more time-consuming and challenging dom at home, and they felt less responsible for their to enforce and validate: there were no interactive studies: assessment tools for testing. Students submitted their completed tests as pictures or scanned images “It was complicated for me when the students using communication tools. Teachers printed the didn’t want to join the chat room, I had to ask responses, marked them, scanned images (or took them nicely; they were hard to connect with. A pictures of the marked tests) and send them back 11 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS to each student. Additionally, not all students com- “When there is one phone in the home that the plied with the tests, reporting technical problems, child uses to join the class, and the phone rings, and it was difficult for teachers to determine if the it is already a problem, because the class dis- students were being helped by their parents. connects. Also, there are three children in my sis- ter’s family who had to join different classes at “Assessment suffers the most for three reasons; the same time, and there is a problem both with the first is our limited professional competence, the internet and the conditions at home to join the second is that we lack the equipment, skills the lesson”. (Male parent, Armenia) and resources to do it, and the third has to do with our generation.” (Teacher, Moldova) “For instance, there are four children in a family, but there is only one computer or mobile phone “I saw it was hard for some students to under- at home. It’s clear that the rest are cut off the stand various things I was explaining, and later lessons. In other words, it wasn’t possible to they sent back well-written homework, which ensure quality.” (Male principal, Armenia) I assume was done by their parents. And after we were back in face-to-face learning, I saw my Keeping students engaged in their learning was grades were baseless, because on the screen another challenge during remote education. Some everything seemed okay, but in reality we didn’t parents reported that their children developed have good outcomes from [distance learning]”. emotional and behavioral problems during this (Female teacher, Georgia) period. Parents in Armenia said in FGDs that their children were less motivated to prepare for the les- On the receiving end of the learning process, par- sons during this period. Moreover, in a 2021 tele- ents and students faced similar challenges: (i) poor phone survey in Georgia,27 parents reported that 61 or no connectivity at all at home and limited access percent of their children were anxious and 63 per- to adequate devices; (ii) little student interest and cent bored during distance education. Teachers in fatigue with online learning; (iii) deficient or incon- Moldova also reported behavioral changes in their sistent guidance on the remote education model students during this period. Some teachers said in from the ministry of education; and (iv) difficulties FGDs that the pandemic had adversely affected learning new topics, especially for low-performing students’ attitudes towards learning: primary pupils students. became less disciplined and organized, and second- ary students were less diligent. Poor connectivity or limited access to digital devices were recurrent problems among low-income fami- “The children agreed in the chat who would lies. In Azerbaijan, middle- and low-income families reply about each part of the lesson during the struggled to make full and effective use of online class, and they learned the lesson in turn, they classes due to the shortage of digital devices at got nimble: one of them wrote the homework home, a lack of uninterrupted high–speed internet, and sent it by chat, and everyone, for exam- and the overlapping class schedules of children. In ple, copied the solution of that math exercise.” Georgia, internet connection problems were prob- (Female parent, Armenia) lematic, especially outside Tbilisi, the capital. In rural Georgia, children mainly accessed distance learning “I noticed that my daughter’s efficiency dropped through mobile phones. In a household in Armenia during the online period. She had studied excel- with only one mobile phone and several children, lently before remote education. There were parents reported difficulties accessing remote edu- 40 people in their class and the teacher could cation. hardly manage. The children were also tired of constantly sitting in front of the screen; their “It is a major problem in villages; about a third of eyes were tired.” (Female parent, Armenia) all parents and children lacked the equipment, or could not afford it, or had several school-age “My two boys were badly affected emotionally; children in the family and could not allocate they became more aggressive in that period. ... It so much time for education, had little time for was difficult to explain to them why they should meetings, and could hardly ever be involved in not go to school and socialize with their friends.” school life. Contacts with parents are rare: not (Female parent, Armenia) because no meetings are held, but because the parents have worries of their own and their Learning new content was challenging during involvement is low.” (Rural teacher, FGD Mol- remote education, especially for low-performing dova) students. In Armenia, parents reported that low-per- 27 World Bank, 2021 12 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS forming children suffered from distance education, “It was interesting for us as parents to see how and that learning new content areas or subjects was the teacher conducted the lesson. Usually, you particularly difficult for them. Behavioral problems, take your child to school, and you do not know lack of focus, and even aggressive conduct among what is going on there, but here everything was students were reported in the FGDs. Parents in very well reflected, how caringly the teachers Armenia said children were becoming more aggres- conducted the classes process, how patient sive, losing interest, and were tired of remote learn- they were, because it was also a challenge for ing. In Moldova, one of the main challenges par- the teacher: different families, different children”. ents faced was leaving their children alone at home (Urban male parent, Armenia) without supervision while they went to work. These parents were concerned about their children’s per- sonal safety, and they were unable to ensure that 3.4 Lessons learned their children participated in distance education. The efforts of ministries of education to provide “The parents of primary school pupils had very services while schools were closed are commend- negative attitudes because they had to go to able, and it was undoubtedly much better than work, and their children were often left alone at nothing. However, the emerging evidence shows home”. (Urban teacher, Moldova). that it was impossible to introduce remote learning practically overnight without compromising learn- The diversity of online platforms (or learning man- ing. If the inefficiencies and inequities that char- agement systems) and digital communication used acterized remote education during the COVID-19 by different schools caused confusion among par- school closures had been acknowledged in the first ents. In Ukraine, since schools had the autonomy few months after the pandemic began, reopening and responsibility to deliver remote education, schools would have been the priority of education many online platforms and communication mech- strategies. As discussed in more detail in the sec- anisms (such as Zoom, Viber, Google Classroom and tion below, none of the countries analyzed in this Skype) were used by teachers for distance learn- report fully acknowledged the devastating effects ing. Families stressed their preference for a single of the pandemic on student learning, especially nationwide platform. among the most disadvantaged children. A few of them have designed and are currently implement- Despite all the challenges faced by parents during ing learning remediation strategies. This section is online learning, there were some positive outcomes. divided into two subsections, one discussing the For example, school directors in Georgia and Arme- lessons learned on improving remote education and nia reported that the pandemic helped to increase the second showing why the number of days that parental involvement in the learning process. In schools remained closed was excessive. Ukraine, school directors acknowledged strength- ening of the “triangle” (teachers-school-parents) 3.4.1 Lessons to improve remote education which is one of the foundations of the New Ukrain- ian School Reform. In Moldova, additionally, some No country or education system was prepared to teachers stated that students returned to school face a pandemic without disruption to learning. with positive perceptions of school, appreciating Schools and teachers needed extensive guidance the face-to-face learning experience and the envi- and skills to deliver remote education. Although in ronment it offers for socialization: some countries, like Georgia, legislation for online learning had been discussed for years, none of the “In the early stages of the pandemic, students countries were prepared nor had the enabling condi- perceived online studies as something inter- tions to transition to remote education successfully. esting, something unusual, something attrac- For instance, given the high levels of school auton- tive. But some fatigue was already noticea- omy in Ukraine, the system relied almost exclusively ble because they were in front of their screens, on principals and teachers for remote education, whether large or small, from morning to evening. and this policy likely contributed to increasing ine- After returning to school, they came to value qualities (as teachers or schools in urban areas and better the opportunity to socialize with other higher-income students were probably more likely students, social meetings and the need to com- to respond quicker and better). As a result of the municate. They saw the school with different policy implemented in Ukraine, many online plat- eyes, they perceived the importance of school forms and communication mechanisms were used and the need to be at school.” (Urban teacher, among teachers for distance learning, which on the Moldova) one hand helped parents to pick the most conven- 13 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS ient mechanisms for their children, but on the other enabled teachers to make the lessons more moti- hand contributed to the confusion among parents vating and interactive and created the possibility because of the lack of a unified approach to dis- of introducing new teaching methods. Armenian tance learning. principals also suggested that before the pandemic, most teachers did not know how to use ICT. The Although the design of some solutions was appro- need to deliver education remotely improved their priate, some countries failed to deliver or implement digital skills. In Georgia, teachers and principals par- them appropriately. For example, Georgia subsidizes ticipating in the FGDs indicated that teachers were access to the internet among school-aged chil- united during the pandemic: dren, which is an effective way to increase access to education resources. However, these benefits were “During the pandemic we saw professional introduced relatively late (in January 2021), and the unity. Some of our teachers used to go to others’ government failed to disseminate them widely to homes to teach them how to conduct classes families. In an August 2021 World Bank survey of online, and in addition even pupils were involved families with school-age children, only 7 percent in this [teaching] process”. (Female Teacher, reported receiving internet access for a reduced fee, Georgia) and 88 percent reported not benefiting from the reduced internet fee because they were unaware of “Elderly teachers didn’t know how to write text the subsidy or the price reduction. messages on their phones…We managed to organize online classes within five days. Glory The enormous challenges of switching from face- and honor to our teachers.” (Female principal, to-face to remote education came at a cost and Armenia) involved a wealth of knowledge and contributions. School closures accelerated the use of technology, 3.4.2 Lessons on the extent of school closures and showcased the potential of education tech- nology to support and complement the role of One of the most important lessons from the COVID- teachers for a more resilient education system. At 19 pandemic is the need to make decisions based on the system level, school closures encourage gov- existing evidence. As shown in Figure 2, as early as ernments to deploy learning management sys- June 2020 in the United States and August 2020 tems (LMS) which allow teachers to deliver distance in Europe, evidence suggested that most girls and education (synchronous or asynchronous), provide boys under the age of 12 who contracted COVID- a platform to communicate and interact between 19 did not develop any symptoms, and only a tiny students and teachers, support the submission proportion had severe symptoms. Soon after, it also of homework and quizzes, and facilitate access to became clear that schools were not the virus hot- courses and learning digital contents. These LMS spots that were initially thought.28 Evidence from could continue to be used and improved in future Germany,29 Spain,30 and the United States shows “hybrid” or “blended” learning educational models that the rate at which children transmitted the virus combining traditional face-to-face education with among themselves and adults was significantly remote education. For instance, in December 2020 lower than transmission rates among adults. Addi- the Ukrainian MoES developed e-journals to moni- tionally, in early 2021 evidence showed that primary tor student participation in online education and to school teachers did not have higher infection and record attendance, homework and grades. Moving mortality rates than the general population, sug- forward, the MoES is planning to couple e-journals gesting that teachers were not at greater risk.31 to existing data collection tools while enhancing their analytical functionalities in order to obtain Around the same time – between the start of the more accurate information on the education system pandemic and late 2020 – both theoretical and at the national, regional and local levels. empirical studies showed that school closures came at a considerable cost in terms of learning loss, par- The school closures also accelerated the testing ticularly for disadvantaged students. It was also and exploration of different software in education clear that children were not only losing learning in and digital libraries. They made evident the need core areas such as numeracy and literacy but were for fast-tracking investment in internet connectivity also suffering from depression, anxiety, and other and devices for teachers and students. Production mental health problems.32 of digital materials and video lessons also accel- erated during this period, and they were mainly Despite all the evidence available by early 2021, offered free of charge to boost access. Some prin- school closures remained part of the strategy to cipals from Armenia reported that online platforms contain the spread of the virus in many countries, 28 D. Lewis, 2020 14 29 Isphording et al., 2020 30 Catala et al., 2020 31 Lynda et al., 2021 32 Lewis et al., 2021 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS including those analyzed in this paper. Schools also where the student density in schools was signifi- remained closed, despite restrictions being lifted cantly lower. A strategy along these lines would have for other activities which, from the point of view of been a good reflection of the evidence showing that social welfare, could be considered of less impor- younger students and students in less agglomer- tance (such as restaurants and shopping centers). ated schools were at considerably lower risk. Apart Some European countries, such as Belgium, Norway, from a few countries where schools reopened with and Denmark (included in Figure 2 as a bench- students going on different days for only a few days mark), did take account of the emerging evidence per week to maintain social distancing and keep and prioritized school reopening over other eco- class sizes small, many countries decided to keep all nomic activities. For instance, Belgian schools were schools closed or open them all full time. closed between March and May 2020, reopened in June, closed again during July and August for The failure to base policy decisions on evidence, or the summer break, reopened again in September at least account for the evidence alongside other 2020, and have never closed since. Also, in Belgium, considerations, resulted in significant learning all non-essential activities closed during the fall of losses, particularly among disadvantaged students. 2020 as the second wave of the virus emerged, but Evidence from rich countries measuring student schools were kept open. learning before and after the pandemic shows sig- nificant learning losses despite short school clo- In contrast, schools in Azerbaijan remained closed sures and high internet penetration. For instance, even when most other countries started opening in the Netherlands, overall, students in grades four them, even partially, in the fall of 2020. Between to seven encountered an average 0.08 standard March 2020 and October 2021, Azerbaijan closed deviation (SD) learning loss in math, spelling, and school institutions for 438 days (due to COVID-19 reading, equivalent to half a year of formal school- and extended academic breaks) compared with ing.33 Losses were up to 60 percent larger among Denmark, which closed school institutions for only students from disadvantaged households. In Bel- 177 days in the same period. Countries that already gium, grade 6 students in the “COVID cohort” that had a learning crisis, as measured by the World suffered from school closures had 0.19 SD lower Bank’s learning poverty indicator, were also more scores in mathematics and 0.29 SD in Dutch than likely to close schools for more extended periods prior cohorts of students on standardized tests.34 (Figure 3). These are massive losses, equivalent to more than a full year of formal schooling.35 In Germany, authors Figure 3: COVID-19 school closure days and found learning losses of 0.07 SD in reading compre- learning poverty (all regions) hension, 0.09 in operations, and 0.03 in numeracy for grade 5 students, close to half a year of formal schooling.36 The learning losses in math were more severe in low-achieving students. A recent system- atic review (Patrinos et al, 2022),37 found that the average learning loss across a sample of 41 coun- tries with robust learning loss data is 0.23 SD, or almost one full year of formal schooling, with losses varying from zero in Sweden to 0.8 SD in Nepal. Figure 4 presents the microlevel data from this sys- tematic review. A third lesson is the lack of preparedness of educa- tion systems to address learning loss once schools reopen, in some cases returning to “business as usual” but ignoring the mounting evidence show- Source: Authors’ calculations using data from UNESCO Tracker ing significant learning loss.The evidence exam- and Learning Poverty (LP). The most recent learning poverty country data comes from 2019. ined for this report suggests that many countries are not implementing aggressive learning recovery A second important lesson is that education sys- plans with effective strategies and enough financial tems seem rigid regarding schedules, grades, and resources to remediate the learning lost during the types of schools that could remain open or closed pandemic. Many countries do not know the extent during the pandemic. Education ministries could of learning losses in their country and lost the oppor- have introduced differentiated policies regarding tunity to measure it. For instance, in response to the school closure by academic level or in rural areas, pandemic, Ukraine canceled the mandatory state 33 Engzell et al., 2021 15 34 Maldonado & De Witte, 2022 35 Evans & Yuan, 2019 36 Schult et al., 2022 37 Patrinos et al., 2022. The systematic review only includes studies using scores from school-based tests or assessments administered by researchers or international assessment programs, including EGRAs, and any study that can be scaled to the Harmonized Learning Outcomes. In addition, studies must have measured learning loss with pre-COVID control data and during or post-COVID treatment data. Studies using projections and simulations were excluded. The study includes microlevel data released in January 2023. COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Figure 4: Average learning losses across various countries Learning Loss (SD) in Countries with Robust Learning Loss Data 0.9 0.8 0.7 Standard Deviation 0.6 0.5 0.4 0.3 Average: 0.23 0.2 0.1 0.0 Sweden Australia Japan France Denmark Korea Spain Czech Indonesia Ethiopia Uganda Hungary Canada Netherlands England Russia Germany Italy Switzerland Finland United States South Africa China Greece Belgium Norway Colombia Ghana Chile Bangladesh Botswana Poland Brazil Kenya Mexico Turkey Argentina Cambodia Malawi India Nepal Source: Authors’ calculations using effect sizes from multiple studies with robust learning loss data obtained from Patrinos’ mi- crolevel data (2023). 0.23 SD is equivalent to almost a whole year’s worth of learning. final certification in grades four, nine, and eleven – school. These government inactions will enable the the only tool that could have been used to estimate costs of the pandemic to manifest themselves in the impact of school closures on students learning – the forms of higher poverty and inequality for sev- in 2020 and 2021. Failure to provide remediation to eral generations to come. help children catch up, or to make necessary adjust- ments to instruction to accommodate the learning Table 5 summarizes the results of a survey con- needs of poor and vulnerable children, means that ducted with ministries of education in selected these children will continue to fall further behind countries regarding the actions taken to address (accumulate learning losses) even after returning to the learning losses. Although countries have taken Table 5: Steps taken to assess learning losses because of COVID-related school closures Country Steps taken to assess students? Measures to address learning gaps Remedial measures for all students Yes, students were assessed at the Remedial measures with focus on students who Armenia classroom level (formative assessment were unable to access distance learning by teachers) Remedial measures with focus on immigrant, refugee, ethnic minorities, or indigenous students Assessment of gaps in student learning that may have accumulated during school closures Remedial measures with focus on students who Yes, students were assessed in a Azerbaijan were unable to access distance learning standardized way at the national level Remedial measures with focus on students in upper secondary with a national examination at the end of 2019 or 2020 calendar Assessment of gaps in student learning that may have accumulated during school closures Yes, students were assessed in a Georgia Remedial measures for all students standardized way at the national level Remedial measures with focus on students who were unable to access Moldova ––– ––– Source: UNESCO, UNICEF and World Bank Survey, February 2021 (Round 3). 16 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS some steps to measure students learning to iden- tify those lagging, the results of these assessments were not necessarily promptly communicated or published. In Georgia, 32 percent of parents said that no diagnostic test had been implemented to measure the learning and socio-emotional chal- lenges of students, and 49 percent indicated that they were unaware of these efforts. In the same country, 20 percent of parents reported that the school did not organize activities for students who needed to learn more, and 49 percent answered that they were unaware of such activities. Armenia was a clear exception, with parents of secondary school pupils in urban communities reporting that after-school classes and additional learning mate- rials were being offered to children falling behind. 17 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS 4. Policy options to construct more resilient education systems Acknowledging that school closures caused signif- with parents frequently. Frequent communication icant learning loss with a more considerable impact with parents significantly affects attendance, aca- among disadvantaged students is the starting point demic scores, and grade promotion rates,38 and for designing and implementing a “learning recovery information can be provided to parents by text plan.” Three of the five countries studied reported message. Interventions informing parents and stu- that they had developed learning recovery plans. dents about the high returns to schooling have also Nevertheless, these were poorly communicated to significantly increased attained years of schooling, parents, teachers, and principals or lacked imple- raised attendance rates, and had significant posi- mentation protocols and the additional budget tive effects on learning.39 needed to be impactful. Returning to business- as-usual once schools reopened, as seen in some The policy options discussed in this section focus countries, makes the implicit assumption that there on building back better, encouraging education was no forgone learning and, therefore, no need to systems to prioritize learning, and collecting stu- put a remediation strategy in place or increase edu- dent achievement data to identify students falling cation budgets to address the crisis. Some teachers behind. We put forward the elements of a simple, and principals surveyed for this study claimed that evidence-based learning recovery plan (LRP) that the curriculum was covered during school closures, could be implemented immediately to mitigate and did not see the need to implement remediation learning losses. But to implement an LRP, schools strategies. require technical and strong leadership from educa- tion authorities. Modifying the curriculum to prior- The immediate challenge faced by education sys- itize foundational skills is, in most cases, the respon- tems is to ensure that all students return to school. sibility of national ministries of education. Designing Many students became disengaged during the long and implementing large-scale student assessments school closures due to financial difficulties, a lack of is another task rarely implemented at subnational access to technology, sickness, or other challeng- level and is never done at school level. Implement- ing circumstances. Many secondary students might ing remediation strategies, such as tutoring, com- now be working, married, or out of the system. As puter-assisted learning (CAL), or grouping students schools reopen, many students may not come back. based on proficiency levels, can be overwhelming Therefore, education systems will exclude a large for schools. This section also identifies the impor- proportion of the student population, exacerbating tance of accompanying any LRP with financial sup- the inequalities that already exist in these countries. port that expands countries’ education budgets Countries need to put extra efforts into supporting to cope with the consequences of any crisis. Addi- re-engagement – especially of vulnerable groups tionally, the second part of this section distills the such as disadvantaged children, girls, refugees, and lessons from the COVID-19 school closure in coun- students with special educational needs – and pro- tries like Denmark and France that can contribute to vide the necessary conditions and incentives for constructing more resilient education systems. children to remain in schools. 4.1 Prioritizing, measuring, and Information campaigns have successfully improved enrollment, attendance, and learning outcomes, improving foundational skills and can be implemented at scale with limited finan- cial investments. The use of technology also ena- The learning loss caused by the pandemic could bles these information campaigns to reach parents be addressed through an LRP comprising at least through different media: television, radio, text mes- three elements: (i) prioritizing foundational skills in sages, social media platforms (such as Facebook, the curriculum; (ii) using standardized tests to iden- Instagram, and Twitter), email, and telephone. For tify students below the minimum learning standard; instance, in Armenia, parents reported that teach- and (iii) implementing compensatory or remediation ers used Viber and social platforms to communicate policies: Bettinger et al., 2021; Bergman & Chan, 2017; Berlinski et al., 2021; Rogers & Feller, 2018 38 18 Jensen, 2010; Nguyen, 2008; Avitabile & de Hoyos, 2018 39 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS 1 Prioritizing foundational skills. Careful consol- Countries should make inventories of existing idation of the curriculum ensures that all stu- assessments relevant to their context or use ready- dents acquire foundational skills – numeracy, made assessment frameworks applied in the region literacy, and socio-emotional skills – that will or similar countries. Well-designed classroom-spe- enable them to keep learning in the future. See cific assessments for diagnostic or formative pur- the World Bank’s Accelerating Learning Recov- poses can also help teachers to determine current ery Rapid Framework.40 student learning levels so that they can adjust their instruction appropriately. 2 Identify lagging students. Large-scale, cen- sus-based student assessments can measure Compensatory or remediation programs – such as foundational skills effectively and be a powerful tutoring, grouping, or CAL – have proven effective diagnostic tool to identify students below mini- when instruction is adjusted to students’ learning mum proficiency levels. level (to teach at the right level) rather than their age or grade. For instance, in India, working with volun- 3 Close gaps through compensatory or remedia- teers to provide instruction at the level of the stu- tion policies. Many education interventions have dent had significant learning gains after 40 days of proven to be effective at closing learning gaps. intervention.42 Also, hiring tutors to provide instruc- The third and final step of the LRP is to imple- tion to students lagging in basic literacy and numer- ment effective remediation interventions target- acy skills greatly impacted learning. The same study ing lagging students. shows that teaching at the right level using tutors is comparable to a CAL intervention.43 In Italy, a pro- Greater variation within classrooms is expected gram that provided free individual tutoring online to after the pandemic. Children with access to learn- disadvantaged middle school students during the ing support at home during remote learning may COVID-19 lockdown increased students’ academic maintain grade-level expectations. Nonetheless, performance by 0.26 SD on average (equivalent many students, especially the most disadvantaged, to more than a full year of formal schooling), and will likely fall behind, particularly if they require ade- enhanced their socio-emotional skills aspirations, quate remediation to help them catch up. In that and psychological well-being.44 case, they could continue to fall further behind even after returning to school. For instance, after the Another pedagogical strategy that has proven 2005 earthquake in Pakistan, children missed 14 effective for teaching at the right level in school weeks because of school closures. Four years later, systems with low learning levels is organizing stu- children affected by the earthquake were two years dents into different groups based on ability.45 Most behind similar children who had not been affected of the time, these pedagogical strategies do not in learning. The authors concluded that this deficit require a significantly greater teacher effort but was too large to be explained by the time out of instead rely on restructuring classes or providing school; instead, they argued that children learned remedial lessons for the lowest performers.46 CAL less every year after returning to school. One poten- programs enable adaptive and personalized learn- tial channel for their finding is that learning loss can ing, allowing students to go at their own pace after accumulate after children return if no remediation an initial screening and facilitating teaching at the or adjustments to instruction are implemented to right level. In India, a personalized technology-aided match the children’s level.41 afterschool instruction program implemented with middle school students improved math and Therefore, data on learning are critical for identify- Hindi test scores for all students, but particularly ing the competence level of each student, teach- for low-performing students.47 In a more resilient ing to the level of the student, designing remedial and inclusive education system, targeting teaching education programs tailored to students’ needs, instruction by learning level should be an institu- and, at the system level, monitoring where more tionalized and structural policy that teachers reg- investment is needed to ensure education is inclu- ularly implement to ensure that all students learn, sive. However, measuring learning is not politically and that no student is left behind. appealing because it has the potential to make the learning crisis visible to all, especially to parents Combining remediation with extended instruction who could demand better services. But all children time – achieved by expanding the school year, the must be supported after long periods of school clo- school day, or offering summer programs – can also sures, especially those who suffer the most from the result in a more resilient education system. Still, the remote learning policies of these countries, like vul- curriculum must be adjusted to prioritize founda- nerable populations. tional skills (literacy, numeracy, and socio-emotional 40 World Bank, 2022 44 Carlana & La Ferrara, 2021 19 41 Andrabi et al., 2020 45 Duflo et al., 2011 42 Banerjee et al., 2016 46 World Bank, 2018 43 Banerjee et al., 2007 47 Muralidharan et al., 2019 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS skills) and target poor students. As described above, learning proficiency levels, along with close to half remediation programs were difficult to incorporate of 15-year-old students in Moldova and more than when curriculums were not adjusted accordingly or 60 percent in Azerbaijan and Georgia. Half of the when sufficient financial resources were not avail- students in the five countries may know how to able. In several countries, the academic year was read but cannot comprehend simple texts; they shortened instead of expanded. And even when the may know how to solve basic math operations but school year was extended for two additional weeks, cannot use them to address everyday problems. as was the case in Azerbaijan, enrollment was vol- This is a massive crisis that was unfolding every untary, which could have exacerbated differences day before the pandemic. Hopefully, the pandemic among students instead of reducing them.48 made the situation visible and created the condi- tions to design and implement the interventions Overall, the countries included in this report made needed to address it. minor adjustments to the school calendars to make up for school days lost and reported needing to If a “learning recovery plan” can mitigate the loss of have plans to revise the regulation on the duration learning brought about by the pandemic, it could of instruction time and content of curriculum after also address the “learning crisis.” Prioritizing foun- the school year 2020/2021.49 In some countries, dational skills in the curriculum, identifying lagging ministries of education wanted to extend the school students, and implementing compensatory or reme- calendar – by shortening summer vacations – or diation policies should be part of an inclusive edu- introduce additional lessons to make up for the lost cation system. Literacy, numeracy, and socio-emo- classes. But these efforts were not accompanied by tional skills should be the core of the curriculum. the necessary financial incentives or adaptation of curricula to prioritize certain subject areas. There- The evaluation system should monitor attainment fore, it is not surprising that teachers pushed back levels in literacy, numeracy, and socio-emotional against these strategies, especially as they were skills. Well-designed large-scale student assess- already suffering from significant burnout because ments capture foundational skills, are linked with of the challenges engendered by remote learning. the national curricula, and allow for comparisons over time.52 A well-design testing scheme formed The learning recovery plan students need to recover by standardized tests is one of the essential fea- from long school closures must be accompanied by tures of a resilient education system. Countries with additional financial resources. Nonetheless, low and regular census-based student assessments could: lower-middle-income countries have been reduc- (i) identify lagging students at a higher risk of losing ing their education budgets while upper-middle learning or disengaging during school closure; (ii) and high-income countries are doing the contrary, quantify the overall learning loss engendered by increasing their budgets and education programs school closures; and (iii) identify students or groups to support students.50 If education is not prioritized of students who lost more learning during school in all government budgets, the gap between lower closures. Large-scale student assessments can also and higher-income countries in learning outcomes be effective at improving student learning. Pro- and human capital development will be amplified. viding schools with a report card summarizing the results of standardized tests and identifying the areas of the curriculum in which students struggled 4.2 Opportunities to construct a the most is enough to ignite changes in the class- more resilient education system room and improve student learning.53 The COVID-19 crisis has led to a significant learning With more than half of the students failing to loss but also created a unique opportunity to imple- achieve minimum learning standards in some of the ment long-awaited reforms capable of addressing countries analyzed in this paper, compensatory or structural constraints to educational systems. The remediation policies should have been in place even learning challenges did not start with the pandemic: before the pandemic. Providing tutoring in small they were there, and COVID-19 exacerbated them. groups, using CAL, or grouping students based on Before the pandemic, roughly half the students in their competency level are examples of the general low- and middle-income countries were not achiev- principle of “teaching at the right level.” Education ing basic foundational skills, and this had become systems, schools, and classrooms within schools known as the “learning crisis.”51 According to the operate under the highly restrictive assumption OECD’s 2018 Programme for International Student that there is minimum proficiency among all stu- Assessment (PISA) test, 36 percent of 15-year- dents within the classroom. Grade four teachers old students in Ukraine did not achieve minimum assume that all their students know how to read 48 These strategies are aligned with the World Bank’s framework for learning recovery and acceleration, referred by its acronym as R.A.P.I.D. 20 49 UNESCO, UNICEF and World Bank, 2021 50 World Bank and UNESCO, 2022 51 World Bank, 2018 52 de Hoyos et al., 2021 53 de Hoyos et al., 2017; de Hoyos, Ganimian et al., 2021 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS and write, count to 20, and make simple additions Creating resilient education systems that protect and subtractions. However, this is not the case for student learning from external shock is not only many students, especially those from poor or mar- desirable or ideal. Despite experiencing school ginalized households.54 For students who lag sig- closures, students in Denmark and France did not nificantly in competencies concerning the average suffer significant learning losses. Denmark closed student in the classroom, the time spent at school schools for a total of 22 school weeks, equivalent is highly inefficient. To bring them up to the right to more than half a school year – slightly more than level, lagging students need remedial interven- the OECD average.57 Nonetheless, Denmark has tions and more instructional time tailored to their mitigated the learning losses observed in other competencies. As mentioned above, this can be countries. The resilient education system of Den- achieved through tutoring, CAL, or grouping stu- mark had good preconditions for handling the edu- dents according to their level of proficiency. cational disruption – reliable digital infrastructure58 and broadband connectivity, high levels of digital Other, more complex, interventions can greatly skills among teachers and students,59 and online reduce the learning gap between poor and non- learning support platforms available for schools.60 poor students. For instance, high-quality early It also mobilized financial resources towards educa- childhood education is perhaps the most cost-ef- tion to support the delivery of compensatory inter- fective policy to close development gaps between ventions: additional teaching, remediation for stu- poor and non-poor children, creating the conditions dents lagging, improving student well-being, and for them to learn during their academic or forma- reducing inequalities among children from different tive years. Full-time school models can also improve socioeconomic backgrounds.61 learning outcomes among poor students.55 Denmark’s education system regularly measures COVID-19 not only brought the importance of students’ foundational skills using standardized guaranteeing learning for all to the center of the assessments implemented online, self-scoring, and education debate, but also forced stakeholders in adaptive programs.62 In response to the evidence, the education system to think about the conditions the country introduced differential return dates to required to use technology effectively in education. schools based on the age group of the student. The Teachers had to improve their digital skills, and edu- youngest grades (grades zero to four) were allowed cation ministries had to develop educational con- back in school after eight weeks of distance learn- tent and pedagogical plans to be delivered online. ing, and they were taught in small groups by a few All these efforts contribute to higher quality pro- teachers.63 On the other hand, older grades (grades vision of education services for rural, isolated, or six to eight) experience the full 22 school weeks of marginalized groups, including refugees – provided distance education. The country also changed its they are granted access to devices connected to response over time, and in the second lockdown, the internet. schools opened partially under the “emergency teaching” scheme for a shorter number of hours. The One technology stands out as transformational, curriculum focused on core, foundational skills.64 particularly for poor and lagging students. As men- tioned above, computer-assisted learning with France made its open school policy a priority, closing instruction and content that can be adjusted to schools for only 10 weeks, one of the shortest peri- the level of competency or proficiency of the stu- ods in Europe. France had a strong national assess- dent has been proven effective at reducing learn- ment system that measured student learning during ing gaps.56 The logic behind CAL that “teaches at the pandemic from students in multiple grades and the right level” is very simple. If lagging students cohorts and covered students in public and private have a level of proficiency far below the average schools. This generated robust data on academic student (the level at which teachers usually teach), progress before and during the pandemic.65 The the solution is to reduce the level of instruction. As country also collected student demographic data mentioned above, this can be done by tutoring a (such as socioeconomic background, ethnicity, and small group of lagging students or grouping stu- immigrant status) that enabled them to conduct dents according to their level of competence (as performance gap analysis, along with extensive opposed to their grades). However, none of these censuses or representative samples survey and lon- solutions individualize the learning experience. CAL gitudinal data on topics ranging from mental health, can do this. If education systems complement con- attitudes, and home and life situations during the ventional, face-to-face instruction with CAL that confinement. France established support for lag- targets lagging students, learning gaps can be sig- ging students to consolidate the aspects of their nificantly reduced. programs that were not covered during confine- 54 Muralidharan et al., 2019 60 Denmark also excelled in pre-pandemic online learning support: 91% of 55 Padilla-Romo, 2022 students attended schools with effective platforms, surpassing the OECD 21 56 Muralidharan et al., 2019 average of 54% (OECD Denmark Country Note, 2020). 57 OECD, 2021 61 European Commission, 2020; OECD Denmark Country Note, 2020 58 Denmark outperforms OECD average in student access to computers for school- 62 The data is collected by The National Agency for IT and Learning (Styrelsen for IT og work, with 98% overall and 96% for lower socioeconomic quartile (PISA 2018, OECD Læring) under the Ministry of Children and Education. Denmark Country Note, 2020). 63 The Danish Evaluation Institute, https://www.eva.dk/uk 59 Denmark excelled in pre-pandemic ICT use: 90% of secondary teachers frequently 64 Reimer et al., 2021. Another important feature of Denmark is that it accompanied its allowed student ICT use for projects or class work compared to 53% in other OECD education response with social protection measures that protected Danish workers countries, and 88% felt confident in supporting learning through digital technolo- from the economic costs of the lockdown. The generous social safety nets included gy, surpassing the OECD average (67%) (OECD Denmark Country Note, 2020). families’ income protection via the temporary wage compensation program, which COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS ment. The country halved class sizes for students in priority schools from 24 down to 12 students, which helped to rebound learning to pre-pandemic levels. COVID-19 exposed the vulnerabilities of educa- tion systems and made it clear that more unequal ones were less resilient. Education systems with a large proportion of the student population below the minimum learning level had more students at risk of falling further behind during school closures. Building inclusive education systems is central to the resilience agenda. Inclusion can be promoted by effectively using data to detect lagging students and design remediation strategies. However, more resources must be allocated to schools serving dis- advantaged students to implement these policies. The fiscal transfer rules should be more progressive, acknowledging that far more resources should be allocated to underprivileged students to produce a given level of learning. Carefully designing fiscal transfers can improve the system's efficiency (that is, how much enrollment and learning are delivered for every dollar spent) and reduce learning dispari- ties across regions and between poor and non-poor students.66 is believed to softer the economic impact of the pandemic (reducing layoffs). These measures likely translate into less stress on the home front compared to other coun- 22 tries (Greer, S., King, E., Massard da Fonseca, E., & Peralta-Santos, A. (2021)). 65 Thorn & Vincent-Lancrin, 2021 66 Al-Samarrai & Lewis, 2021 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Final Remarks COVID-19 was a massive shock for education sys- tutoring or remedial education could be provided tems around the world. In a matter of days, educa- remotely to lagging students needing individualized tion systems had to redeploy inputs typically used support. Using technology through online tutoring in schools to remote education. This shift reduced or CAL can, when designed appropriately, improve the efficiency of school inputs to produce learning. It students’ academic performance, socio-emotional also made relatively unequal household conditions skills, and psychological well-being.67 But these more critical for determining student learning. The solutions come together with heavy internet con- result was a significant reduction in average student nectivity and device investments. It is also evident learning, with disadvantaged students experiencing that digital literacy (the ability to use digital tech- a disproportionately large decline. nology) is insufficient for teaching and ensuring stu- dents’ learning. Teachers also need so-called “digital As learning is a cumulative process, while learning pedagogical skills”, or the ability to use digital tools today is partly the outcome of the stock of past and resources to facilitate learning, check for under- learning, not addressing the learning loss brought standing, provide feedback, and so on. Currently, about by COVID-19 would have long-lasting effects several skills and competency frameworks incorpo- on future learning outcomes, restricting productiv- rate digital pedagogical skills.68 ity and growth and dampening social mobility. To address learning loss, countries in the South Cau- Exploiting the opportunities engendered by COVID- casus, Eastern Europe, and elsewhere could design 19 will require evidence-based education policies and implement learning recovery plans that prior- and financial resources. More resources should go itize foundational skills, develop the tests needed where they are most needed: the schools serving to measure them, and close learning gaps through disadvantaged students. If finance ministries have remediation policies. long-term horizons when allocating resources, they should prioritize constructing equitable resilient But the pandemic also brought opportunities. Lack education systems. Well-designed policies with of learning among a significant proportion of the sufficient budgets are not a guarantee of success. student population, most of whom are from disad- In a large sector formed of many actors with inter- vantaged households, is not a challenge that began ests that are not necessarily well-aligned, imple- with COVID-19. The five Eastern European countries mentation can be as challenging as – if not more analyzed in this paper faced learning challenges well challenging than – design and budget allocation. before the pandemic. The pandemic exposed the Politicians and education policymakers have an vulnerability of unequal education systems. Per- enormous responsibility to create coalitions with haps the pandemic and the attention it is bringing relevant actors and find a consensus among stake- to students’ “learning loss” will create the political holders to improve learning opportunities among conditions to implement long-awaited education disadvantaged students and develop the basis for reforms to reduce the learning gaps and create constructing resilient education systems. better conditions for disadvantaged students, the core element of resilient education systems. Use of data to guide decisions, standardized tests as a diagnostic tool, and remediation policies should become permanent features of education systems. During school closures, remote education experi- enced a giant leap forward, with teachers, parents, and students becoming more familiar than ever with digital platforms and digital communication tools. This opens many opportunities to build more resilient education systems. For instance, online 67 Carlana & La Ferrera, 2021 23 68 World Bank, 2021 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS References Agostinelli, F., Doepke, M., Sorrenti, G., & Zilibotti, F. Asakawa, S., & Ohtake, F. (2021). Impact of (2020). When the Great Equalizer Shuts Down: Temporary School Closure Due to COVID-19 on Schools, Peers, and Parents in Pandemic Times. the Academic Achievement of Elementary School National Bureau of Economic Research, 28264. Students. Osaka University, Graduate School of https://doi.org/DOI 10.3386/w28264 Economics, Discussion Papers in Economics and Business (2114). https://ideas.repec.org/p/osk/ Al-Samarrai, S., & Lewis, B. (Eds.). (2021). The Role of wpaper/2114.html Intergovernmental Fiscal Transfers in Improving Education Outcomes. The World Bank. https:// Avitabile, C., & de Hoyos, R. (2018). The heteroge- doi.org/10.1596/978-1-4648-1693-2 neous effect of information on student perfor- mance: Evidence from a randomized control trial Amelia, R., Kadarisma, G., Fitriani, N., & Ahmadi, Y. in Mexico. Journal of Development Economics, (2020). The effect of online mathematics learning 135, 318–348. https://doi.org/10.1016/j. on junior high school mathematic resilience jdeveco.2018.07.008 during covid-19 pandemic. Journal of Physics: Conference Series, 1657(1), 012011. https://doi. Azevedo, J. P., Crawford, M., Nayar, R., Rogers, F. H., org/10.1088/1742-6596/1657/1/012011 Barron, M. R., Ding, E., Gutierrez, M., Dixon, A., Saavedra, J., & Arias, O. (2019). Ending Learning American Academy of Pediatrics. (2022). COVID-19 Poverty: What will it take? [Working Paper]. Guidance for Safe Schools and Promotion of http://documents.worldbank.org/curated/ In-Person Learning. https://www.aap.org/en/ en/395151571251399043/Ending-Learning- pages/2019-novel-coronavirus-covid-19-in- Poverty-What-Will-It-Take fections/clinical-guidance/covid-19-plan- ning-considerations-return-to-in-person-edu- Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, cation-in-schools/ H., Mukherji, S., Shotland, M., & Walton, M. (2016). Mainstreaming an Effective Interven- Andrabi, T., Daniels, B., & Das, J. (2020). Human tion: Evidence from Randomized Evaluations of Capital Accumulation and Disasters: Evidence “Teaching at the Right Level” in India. Working from the Pakistan Earthquake of 2005. https:// Paper, 22746. doi.org/10.35489/BSG-RISE- WP_2020/039 Banerjee, A. V., Cole, S., Duflo, E., & Linden, L. (2007). Angrist, N., de Barros, A., Bhula, R., Chakera, S., Remedying Education: Evidence from Two Cummiskey, C., DeStefano, J., Floretta, J., Kaffen- Randomized Experiments in India. The Quarterly berger, M., Piper, B., & Stern, J. (2021). Building Journal of Economics, 122(3), 1235–1264. https:// back better to avert a learning catastrophe: doi.org/10.1162/qjec.122.3.1235 Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term Bergman, P., & Chan, E. W. (2017). Leveraging Tech- and long-term learning recovery. International nology to Engage Parents at Scale: Evidence Journal of Educational Development, 84, 102397. from a Randomized Controlled Trial. Economics https://doi.org/10.1016/j.ijedudev.2021.102397 of Education, 6493. file:///C:/Users/wb373791/ Downloads/cesifo1_wp6493.pdf Ardington, C., Wills, G., & Kotze, J. (2021). COVID-19 learning losses: Early grade reading Berlinski, S., Busso, M., Dinkelman, T., & A., C. M. (2021). in South Africa. International Journal of Educa- Reducing Parent-School Information Gaps and tional Development, 86, 102480. https://doi. Improving Education Outcomes: Evidence from org/10.1016/j.ijedudev.2021.102480 High-Frequency Text Messages (No. w28581; p. w28581). National Bureau of Economic Research. https://doi.org/10.3386/w28581 24 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Bettinger, E., Cunha, N., Lichand, G., & Madeira, R. de Hoyos, R., Garcia-Moreno, V. A., & Patrinos, H. (2021). Are the Effects of Informational Interven- A. (2017). The impact of an accountability inter- tions Driven by Salience? Working Paper, 350. vention with diagnostic feedback: Evidence from Mexico. Economics of Education Review, Birkelund, J., & Karlson, K. (2021). No Evidence 58, 123–140. https://doi.org/10.1016/j.econe- of a Major Learning Slide 14 Months into the durev.2017.03.007 COVID-19 Pandemic in Denmark. SocArXiv. 10.31235/osf.io/md5zn De Paola, M., Gioia, F., & Scoppa, V. (2022). Online Teaching, Procrastination and Students’ Achieve- Carlana, M., & La Ferrara, E. (2021). Apart but ment: Evidence from COVID-19 Induced Remote Connected: Online Tutoring and Student Learning. 15031. file:///C:/Users/wb373791/ Outcomes during the COVID-19 Pandemic. Downloads/SSRN-id4114561.pdf Faculty Research Working Paper Series, RWP21- 001. https://www.hks.harvard.edu/publications/ Depping, D., Lücken, M., Musekamp, F., & Thonke, apart-connected-online-tutoring-and-stu- F. (2021). Kompetenzstände Hamburger dent-outcomes-during-covid-19-pandemic Schüler*innen vor und während der Coro- na-Pandemie. In D. Fickermann & B. Edelstein Catala, M., Prats, C., Alonso, S., Alvarez, E., Marchena, (Eds.), Schule während der Corona-Pan- M., Conesa, D., & Lopez, D. (2020). Analysis and demie. Neue Ergebnisse und Überblick prediction of COVID-19 for EU-EFTA-UK and über ein dynamisches Forschungsfeld (pp. other countries (Situation Report No. 136). https:// 51–79). Waxmann Verlag GmbH. https://doi. biocomsc.upc.edu/en/shared/20201002_ org/10.31244/9783830993315.03 report_136.pdf Domingue, B., Hough, H., Lang, D., & Yeatman, J. Clark, A. E., Nong, H., Zhu, H., & Zhu, R. (2020). (2021). Changing Patterns of Growth in Oral Compensating for Academic Loss: Online Reading Fluency During the COVID19 Pandemic. Learning and Student Performance during the h t t p s : //e d p o l i c y i n c a . o rg /s i t e s /d e f a u l t / COVID-19 Pandemic. halshs-02901505f. https:// files/2021-03/wp_domingue_mar21-0.pdf halshs.archives-ouvertes.fr/halshs-02901505/ document Dorn, E., Hancock, B., Sarakatsannis, J., & Viruleg, E. (2021). COVID-19 and education: The lingering Contini, D., Di Tommaso, M. L., Muratori, C., Piaz- effects of unfinished learning. https://www. zalunga, D., & Schiavon, L. (2021). The COVID-19 mckinsey.com/industries/education/our-in- Pandemic and School Closure: Learning Loss sights/covid-19-and-education-the-lingering- in Mathematics in Primary Education. 14785. effects-of-unfinished-learning https://docs.iza.org/dp14785.pdf Duflo, E., Dupas, P., & Kremer, M. (2011). Peer Effects, Das, J., Dercon, S., Habyarimana, J., Krishnan, P., Teacher Incentives, and the Impact of Tracking: Muralidharan, K., & Sundararaman, V. (2013). Evidence from a Randomized Evaluation in School Inputs, Household Substitution, and Test Kenya†. 101, 1739–1774. http://www.aeaweb.org/ Scores. American Economic Journal: Applied articles.php?doi= 10.1257/aer.101.5.1739 Economics, 5(2), 29–57. https://doi.org/10.1257/ app.5.2.29 Edumedia-Azərbaycan, 2019. Təhsil Nazirliyinin teledərsləri. www.video.edu.az de Hoyos, R., Estrada, R., & Vargas, M. J. (2021). What do test scores really capture? Evidence Engzell, P., Freya, A., & Verhagen, M. (2021). from a large-scale student assessment in Mexico. Learning loss due to school closures during the World Development, 146, 105524. https://doi. COVID-19 pandemic. PNAS, 118(17). https://doi. org/10.1016/j.worlddev.2021.105524 org/10.1073/pnas.2022376118 de Hoyos, R., Ganimian, A. J., & Holland, P. A. (2021). Evans, D., & Yuan, F. (2019). Equivalent Years of Teaching with the Test: Experimental Evidence Schooling: A Metric to Communicate Learning on Diagnostic Feedback and Capacity Building Gains in Concrete Terms. Policy Research Working for Public Schools in Argentina. The World Bank Paper, 8752. https://documents1.worldbank. Economic Review, 35(2), 499–520. https://doi. org/curated/en/123371550594320297/pdf/ org/10.1093/wber/lhz026 WPS8752.pdf 25 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS European Center for Disease Prevention and Kim, J., Rose, P., Tibebu, D., Sabates, R., & Wold- Control (ECDC) (2020). COVID-19 in children and ehanna, T. (2021, May). Learning Inequalities the role of school settings in COVID-19 transmis- Widen Following COVID-19 School Closures in sion. 6 August 2020. https://www.ecdc.europa. Ethiopia. https://riseprogramme.org/blog/learn- eu/sites/default/files/documents/COVID-19- ing-inequalities-widen-COVID-19-Ethiopia schools-transmission-August%202020.pdf Kofoed, M., Gebhart, L., Gilmore, D., & Moschitto, Gambi, L., & De Witte, K. (2021). The resiliency of R. (2021). Zooming to Class?: Experimental school outcomes after the COVID-19 pandemic. Evidence on College Students’ Online Learning Standardised test scores and inequality one year during COVID-19. 14356. https://docs.iza.org/ after long term school closures. file:///C:/Users/ dp14356.pdf wb373791/Downloads/dps%202112.pdf Kogan, V., & Lavertu, S. (2021). The COVID-19 Gonzalez, T., de la Rubia, M. A., Hincz, K. P., Comas- Pandemic and Student Achievement on Ohio’s Lopez, M., Subirats, L., Fort, S., & Sacha, G. M. Third-Grade English Language Arts Assess- (2020). Influence of COVID-19 confinement on ment. https://glenn.osu.edu/sites/default/ students’ performance in higher education. PLOS files/2021-09/ODE_ThirdGradeELA_KL_1-27- ONE, 15(10), e0239490. https://doi.org/10.1371/ 2021.pdf journal.pone.0239490 Korbel, V., & Prokop, D. (2021). Czech students lost 3 Haelermans, C., Korthals, R., Jacobs, M., de Leeuw, months of learning after a year of the COVID-19 S., Vermeulen, S., van Vugt, L., Aarts, B., pandemic. PAQ Research. https://88760faa- 4149-467c-8d6a-46e154cd4c14.usrfiles. Prokic-Breuer, T., van der Velden, R., van Wetten, S., com/ugd/88760f_c8f600c6cede4137a5b- & de Wolf, I. (2022). Sharp increase in inequality cb559319a8eb3.pdf in education in times of the COVID-19-pan- demic. PLOS ONE, 17(2), e0261114. https://doi. Kuhfeld, M., Tarasawa, B., Johnson, A., Ruzek, E., org/10.1371/journal.pone.0261114 & Lewis, K. (2020). Initial findings on students’ reading and math achievement and growth. 12. Hevia, F. J., Vergara-Lope, S., Velásquez-Durán, A., & Calderón, D. (2022). Estimation of the funda- Lewis, D. (2020). WHY SCHOOLS PROBABLY mental learning loss and learning poverty related AREN’T COVID HOTSPOTS. Nature, 587(5). to COVID-19 pandemic in Mexico. International https://media.nature.com/original/magazine-as- Journal of Educational Development, 88, 102515. sets/d41586-020-02973-3/d41586-020- https://doi.org/10.1016/j.ijedudev.2021.102515 02973-3.pdf ipn Press Agency. (2021, April 14). 10,000 laptops Lewis, S. J., Munro, A. P. S., Smith, G. D., & Pollock, to be distributed to students for online A. M. (2021). Closing schools is not evidence learning. https://www.ipn.md/en/10000-lap- based and harms children. BMJ, n521. https://doi. tops-to-be-distributed-to-students-for-on- org/10.1136/bmj.n521 line-7967_1081004.html Lichand, G., Dória, C. A., Neto, O. L., & Cossi, J. (2021). Isphording, I. E., Lipfert, M., & Pestel, N. (2020). The Impacts of Remote Learning in Secondary School Re-Openings after Summer Breaks in Education: Evidence from Brazil during the Germany Did Not Increase SARS-CoV-2 Cases. Pandemic [Preprint]. In Review. https://doi. IZA, 13790. https://docs.iza.org/dp13790 org/10.21203/rs.3.rs-568605/v1 Jensen, R. (2010). The (Perceived) Returns to Locke, V., Patarapichayatham, C., & Lewis, S. (2021). Education and the Demand for Schooling *. Learning Loss in Reading and Math in U.S. Schools Quarterly Journal of Economics, 125(2), 515–548. Due to the COVID-19 Pandemic. Istation. https:// https://doi.org/10.1162/qjec.2010.125.2.515 www.istation.com/Content/downloads/studies/ COVID-19_Learning_Loss_USA.pdf Jesper Fels Birkelund & Kristian Bernt Karlson (2022): No Evidence of a Major Learning Slide Lynda, F., Ciara, G., David, C., Sam, C., Jen, B., Martin, R., 14 Months Into the COVID-19 Pandemic Jane, W., Campbell, M., Hutchinson, S., Robertson, in Denmark, European Societies, DOI: C., Helen, M. C., Rachael, W., Paul, M. M., & David, 10.1080/14616696.2022.2129085 A. M. (2021). RISK OF HOSPITALISATION WITH 26 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS COVID-19 AMONG TEACHERS COMPARED TO OECD (2020). France Country Note. School HEALTHCARE WORKERS AND OTHER WORK- Education During COVID-19. Were Teachers ING-AGE ADULTS. A NATIONWIDE CASE-CON- and Students Ready? OECD Publishing, Paris. TROL STUDY [Preprint]. Epidemiology. https:// https://www.oecd.org/education/France-coro- doi.org/10.1101/2021.02.05.21251189 navirus-education-country-note.pdf Major, L., Eyles, A., & Machin, S. (2021). Learning OECD (2021) The State of Global Education: 18 loss since lockdown: Variation across the home Months Into the Pandemic, Paris: nations. Centre for Economic Performance, 23. https://files.eric.ed.gov/fulltext/ED614031.pdf OECD Publishing. doi:10.1787/1a23bb23-en. Maldonado, J. E., & De Witte, K. (2022). The effect Padilla-Romo, M. (2022). Full-time schools, poli- of school closures on standardised student test cy-induced school switching, and academic outcomes. British Educational Research Journal, performance. Journal of Economic Behavior 48(1), 49–94. https://doi.org/10.1002/berj.3754 & Organization, 196, 79–103. https://doi. org/10.1016/j.jebo.2022.01.025 Ministry of Education and Science of Georgia. (2011, August 26). The Process of Providing Schools with Patrinos, H. A., Vegas, E., & Carter-Rau, R. (2022). 60000 Net-Books Has Started. The Process of An Analysis of COVID-19 Student Learning Loss. Providing Schools with 60000 Net-Books Has The World Bank. https://doi.org/10.1596/1813- Started. https://mes.gov.ge/content.php?t=s- 9450-10033 rch&search=buki&id=2746&lang=eng Pier, L., Christian, M., Tymeson, H., & Meyer, R. Ministry of Education and Science of Georgia (2014). (2021). COVID-19 Impacts on Student Learning: National Center for Teacher Professional Devel- Evidence from Interim Assessments in California. opment, Programs. http://old.tpdc.ge/index. Policy Analysis for California Education. https:// php?action=page&p_id=68&lang=eng edpolicyinca.org/sites/default/files/2021-06/r_ pier_jun2021.pdf Ministry of Education and Science of Georgia (2021). National Assessment and Examinations Center. Reimer, D., Smith, E., Andersen, I. G., & Sortkaer, Category: eAssessment. https://naec.news/ B. (2021). What happens when schools shut category/eassessment/page/2/#_ftn1 down? Investigating inequality in students’ reading behavior during Covid-19 in Denmark. Ministry of Science and Education of Azerbaijan Research in Social Stratification and Mobility, (2020, 03 April). The Ministry of Education 71, Article 100568. https://doi.org/10.1016/j. presents the "Virtual School" project. https://edu. rssm.2020.100568 gov.az/en/news-and-updates/18178 Rogers, T., & Feller, A. (2018). Reducing student Muralidharan, K., Singh, A., & Ganimian, A. J. (2019). absences at scale by targeting parents’ misbe- Disrupting Education? Experimental Evidence on liefs. Nature Human Behaviour, 2(5), 335–342. Technology-Aided Instruction in India. American https://doi.org/10.1038/s41562-018-0328-1 Economic Review, 109(4), 1426–1460. https:// doi.org/10.1257/aer.20171112 Rose, S., Twist, L., Lord, P., Rutt, S., Badr, K., Hope, C., & Styles, B. (2021). Impact of school closures and Nguyen, T. (2008). Information, Role Models subsequent support strategies on attainment and Perceived Returns to Education: Exper- and socio-emotional wellbeing. National Foun- imental Evidence from Madagascar. http:// dation for Educational Research. https://www. citeseerx.ist.psu.edu/viewdoc/download?- nfer.ac.uk/impact-of-school-closures-and-sub- doi=10.1.1.565.1803&rep=rep1&type=pdf sequent-support-strategies-on-attainment- and-socio-emotional-wellbeing/ OECD (2020). Denmark Country Note. School Education During COVID-19. Were Teachers and Schult, J., Mahler, N., Fauth, B., & Lindner, M. A. Students Ready? OECD Publishing, Paris. https:// (2022). Did students learn less during the www.oecd.org/education/Denmark-coronavi- COVID-19 pandemic? Reading and mathe- rus-education-country-note.pdf matics competencies before and after the first pandemic wave. https://doi.org/10.1080/09243 453.2022.2061014 27 COVID-19 as an Opportunity to Build RESILIENT EDUCATION SYSTEMS Schuurman, T. M., Henrichs, L. F., Schuurman, N. World Bank and UNESCO (2022). Education Finance K., Polderdijk, S., & Hornstra, L. (2021). Learning Watch 2022. Washington D.C., Paris: The Loss in Vulnerable Student Populations After the First Covid-19 School Closure in the Netherlands. World Bank and UNESCO. https://thedocs. Scandinavian Journal of Educational Research, worldbank.org/en/doc/5c5cdd4c96799335e- 1–18. https://doi.org/10.1080/00313831.2021.2 263023fa96db454-0200022022/related/ 006307 EFW-2022-Dec21.pdf Skar, G. B. U., Graham, S., & Huebner, A. (2021). World Bank. (2018). World Development Report Learning loss during the COVID-19 pandemic 2018: Learning to Realize Education’s Promise. and the impact of emergency remote instruc- Washington, DC: World Bank. https://doi. tion on first grade students’ writing: A natural org/10.1596/978-1-4648-1096-1 experiment. Journal of Educational Psychology. https://doi.org/10.1037/edu0000701 World Bank Group. (2021). Teachers’ Skills & Skills Frameworks for Remote and Blended Learning Studii.md online platform. (2020). https://studii.md/ [Knowledge Pack]. https://documents1.worldbank. org/curated/en/099118004132238882/pdf/ Thorn, W., & Vincent-Lancrin, S. (2021). Schooling P1742520615db3006094220dcbb0af52257.pdf During a Pandemic: The Experience and Outcomes of Schoolchildren During the First World Bank. (2022, April). Accelerating Learning Round of COVID-19 Lockdowns. OECD. https:// Recovery: A Guidance Note for Addressing doi.org/10.1787/1c78681e-en Learning Losses in the Context of COVID-19 and Building Back Better. Tomasik, M. J., Helbling, L. A., & Moser, U. (2021). Educational gains of IN-PERSON vs. Distance learning in primary and secondary schools: A natural experiment during the COVID-19 pandemic school closures in Switzerland. Inter- national Journal of Psychology, 56(4), 566–576. https://doi.org/10.1002/ijop.12728 Truncano, M. (2016, March 16). Zero-rating educa- tional content on the Internet. Zero-Rating Educational Content on the Internet. https:// blogs.worldbank.org/edutech/zero-rating-edu- cational-content-internet UNESCO. (2022) COVID-19 recovery. Education: from school closure to recovery. https:// en.unesco.org/covid19/educationresponse UNESCO-UNICEF-World Bank Survey on National Education Responses to COVID-19 School Closures – Key Results (1st Iteration) (2020). https://uis.unesco.org/en/news/unes- co-unicef-world-bank-survey-national-educa- tion-responses-covid-19-school-closures-key UNESCO-UNICEF-World Bank Survey on National Education Responses to COVID-19 School Closures – Key Results (3rd Iteration) (2021). https://covid19.uis.unesco.org/joint-covid-r3/ Whizz. (2021). Measuring The Impact of COVID-19 on Learning in Rural Kenya. https://www.whizzedu- cation.com/wp-content/uploads/Kenya-Covid- Impact-SCREEN.pdf 28 © 2023 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org