LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS Koen Geven and Amer Hasan OCTOBER 2020 i LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS 1 Introduction Pakistan was among the first countries in the world to institute widespread school closures as a result of COVID-19. Schools in Sindh were closed starting from February 27, 2020. School closures in the rest of the country started from the weekend of March 14, 2020. As of this writing, schools have begun a staggered re-opening starting with Classes 9–12 on September 15. Classes 6–8 started on September 23. Nursery to class 5 started on September 30. All children—regardless of whether they attend private or public schools—will attend on alternate days. What are the expected levels of learning that teachers will have to deal with in the class? Will children have lost learning while schools were closed? What should teachers, parents and children expect from the first few weeks of schooling? How can parents, teachers and the school system as a whole help children catch up? We have learned from earlier crises that the effects of school closures can persist for many years. An entire cohort of students aged 3–15 at the time of the 2005 earthquake in Pakistan had lower academic scores four years later, despite substantial remediation efforts.1 While school closures have been effective in supporting efforts at social distancing, they may well have serious consequences for schooling and learning. This note presents results from a series of simulations that aim to capture the impacts that school closures in Pakistan might have on the learning levels, enrollment and future earnings of children and students.2 In this note, we present an overview of how these numbers are calculated and how to interpret them. This note draws on a simulation exercise for all countries on which data is available, including Pakistan, conducted by researchers at the World Bank (Azevedo et. al, 2020). 2 Analytical approach The simulations consider a number of pathways of learning loss.3 These pathways start with the under- standing that in the absence of COVID-19 children would go to school and learn. When schools are closed, new learning does not take place and what has already been learned is likely to be forgotten. In addition, families across the spectrum are losing income. Furthermore, any mitigation of school closures is likely to be a function of what remote learning modalities governments are able to supply, who is able to access these modalities, and how effective these modalities are. On balance school closures, mitigation efforts and their effectiveness as well as the effect on household incomes will all come together in impacting not only the amount of schooling children receive but also their learning outcomes. 1 Andrabi, Daniels and Das, 2020. 2 The estimates reported in this note are subject to change as new data becomes available. The simulations presented in this note build on the concept referred to as ‘Learning-Adjusted Years of Schooling’ or LAYS — 3  a metric that combines both quantity and quality of schooling. For details cf Filmer et al, 2020 Acknowledgments: The authors would like to thank Cristian Aedo, Joao Pedro Azevedo, Diana Goldemberg, Syedah Aroob Iqbal and members of the World Bank’s Pakistan Education Team for comments on earlier drafts. 1 Conceptually, learning loss materializes in two ways: (1) as learning that will not take place while schools are closed, which is directly linked to schooling adjusted for quality, and (2) as learning that will be lost or forgotten when students disengage from schools. The framework also captures the impact of household income shocks on school dropouts. (See Figure 1) Consider the current cohort of students as a panel observed just before the crisis, and who can be observed again the moment schools reopen. Figure 1 shows the learning trajectory of this cohort of students for whom we assume that for a given level of education quality, learning (l), is a linear function of time t spent in school. The length of school closures (s), assuming no mitigation, will reduce the amount of time students will be exposed to learning opportunities. If schools close between t1 and t2, we do not expect any new learning to take place, and at t2, the student will be at l2’. However, this is not the whole effect. As students disengage from education, part of their stock of learning (l1) will be forgotten such as has been found for summer breaks in various settings. This will bring students from l2’ to l2’’. The area of triangle A (bounded by l1, l2 to l2’) corresponds to the learning that will not take place while schools are closed, while triangle B (bounded by l1, l2’, and l2’’) corresponds to the learning that will be lost due to school disengagement and dropout. The learning loss due to each of these mechanisms will be a function of how effective mitigation strategies might be. Figure 1: Analytical framework for an individual student learning Source: based on Azevedo et al (2020). 2 LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS 3 Scenarios considered The simulations described below consider the effects of school closures of various lengths, mitigation strate- gies which may have different effectiveness for different groups in the population and examine both the likely impact on average as well as the impact over the entire distribution of learning outcomes. The simulations are based on three scenarios4 described in Table 1:  Optimistic—schools are closed only for 3 months of a 10-month school year, and the effectiveness of mitigation measures put in place by governments (such as remote learning) is relatively high.5  Intermediate—schools are closed for 5 months, and mitigation measures have a middle level of effec- tiveness. This is the most realistic scenario that we see, given that schools have recently re-opened in the country.  Pessimistic—schools are closed for 7 months, and mitigation measures have low levels of effective- ness. This may still become a reality even with schools having re-opened, as it is unclear what the path of the pandemic will be during the rest of the school year. Table 1: Parameters for Pakistan LAYS estimates and scenarios A. Learning gains or school productivity (in HLO points/year) 40 Optimistic Scenario B1. School closure (share of a school year) 30% C1. Mitigation effectiveness (0 to 100%) 26% D1. HLO decrease (points) = A*B1*(1-C1) 4.8 Intermediate Scenario B2. School closure (share of a school year) 50% C2. Mitigation effectiveness (0 to 100%) 13% D2. HLO decrease (points) = A*B2*(1-C2) 9.0 Pessimistic Scenario B3. School closure (share of a school year) 70% C3. Mitigation effectiveness (0 to 100%) 6% D3. HLO decrease (points) = A*B3*(1-C3) 13.3 Gross Domestic Product (GDP) per capita growth % -4.4 Notes: based on Azevedo (2020). 4 These scenarios are the same as in Azevedo et al (2020). 5  This is not high in absolute terms. 3 4 Data The simulations reported in this note rely on the following data sources: (1) The Learning-Adjusted Years of Schooling (LAYS) component of the World Bank’s Human Capital Index (HCI) database released in September 2020. (2) Economic forecasts from the World Bank Macro Poverty Outlook June 2020. (3) The Global Monitoring Database which contains the latest household survey data which is used to estimate country-specific dropout-income elasticities using observed cross-sectional variation between educational enrollment and welfare. (4) Earnings information from ILOSTAT database (ILO 2020), complemented by the Global Jobs Indicators database (JoIn) (World Bank, 2020). (5) The 2014 National Achievement Test conducted by the National Education Assessment System (NEAS) under the Federal Ministry of Education. (6) An ongoing impact evaluation on the drivers of school enrollment for girls in Pakistan that focuses on the country’s most populous province—Punjab. 5 Key findings (1) Income losses due to COVID-19 could lead to a sharp increase in dropouts An estimated 930,000 additional children are expected to drop out from both primary and secondary educa- tion. Given that 22 million are already out of school, this represents an increase of almost 4.2 percent. Pakistan is globally the country where we expect the highest dropouts due to the COVID crisis in relative terms. This estimate is based on the observed income elasticity of education for various socio-economic quintiles and is based on the June 2020 growth estimates for Pakistan which were estimated to be -4.4 percent.6 Income elasticity in Pakistan is high for two main reasons, (1) high poverty levels, which lead families to push their children into labor or marriage from an early age, and (2) the cost of private schooling in which 38 percent of school-going children aged 6–10 were enrolled before the crisis (PSLM 2018/19). These numbers may be considered a lower bound of dropouts, given that estimates of growth are being continually revised downward and we are still learning about the scale of income loss at the household level. In one survey, as many as 54 percent of households in a random digit-dialing study reported a substantial income loss.7 A Gallup survey from early in the crisis found that as many as 27 percent of families were considering not to return their children to school. 6  Using data from the Macro Poverty Outlook. 7  Asad, S., Qureshi, J., Shah, T., Zafar, B. (2020) Economic Vulnerability Assessment. 4 LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS (2) Even the most optimistic scenario suggests an overall loss of learning for every child enrolled in school We estimate that school closures during the pandemic will result in a loss of a between 0.3 and 0.8 years of learning-adjusted schooling for the average student. Pakistani children currently spend 9.1 years in school on average, but they only accumulate 5.1 years of learning due to low quality of the school system.8 The simulations suggest that this number could fall to between 4.8 and 4.3 years by the time schools have fully reopened. In the intermediate scenario, this is expected to fall by 0.6 years of learning-adjusted schooling (Figure 2). These numbers range between an optimistic scenario and a pessimistic scenario, in which we make different assumptions about the length of school closures and the effectiveness of distance learning. The scenarios take into account mitigation being offered through distance learning programs on television and on the internet (for example, the Teleschool initiative launched by the Federal government). Figure 2: COVID-19 could reduce Pakistan’s stock of Learning-Adjusted Years of Schooling (LAYS) Learning-Adjusted Years of School (LAYS) Learning loss Source: Authors calculations based on Azevedo et al (2020). Note: Learning-adjusted years of school (Filmer et al, 2018) are calculated by multiplying the estimates of expected years of school by the ratio of most recent harmonized test scores to 625. 8  Numbers from the World Bank’s Human Capital Index, 2020. 5 (3) Learning Poverty will go up to 79 percent ‘Learning Poverty’ is defined as the share of children who do not learn to read and understand a simple text by age 10.9 In Pakistan, the level of learning poor was already high at 75 percent, based on official government data.10 Learning poverty is calculated as the simple addition of the share of children out of school (27.3 percent in 2014) and the share of children who do not learn to read by age 10 (47.3 percent in 201411), even if they are attending school. The simulations suggest that the share of children who are learning poor will increase from 75 percent to 79 percent. The loss of 930,000 children from the school system translates into an increase of 1.3 percentage points in the share of children out of school—given that there are 65 million school-age children in the country. This means that the share of children out of school will go up from 27.3 percent to 28.6 percent. The share of children who do not learn to read by age 10 (even if they are in school) is expected to increase by 2.7 percentage points from 47.3 percent to 50 percent. This results from a shock to the distribution of learning outcomes with a reduction of about half a year of learning (5 months out of a 10-month school year), representing 0.2 standard deviations of learning. (4) The COVID-19 pandemic is not just a health crisis, it is poised to translate into a substantial economic cost on the long run If we quantify this loss of learning in terms of labor market returns, the average student will face a reduc- tion between US$193 and US$445 in yearly earnings (2017 PPP) once s/he enters the labor market, which represents between 2.8 percent and 6.6 percent of annual income. Aggregated for all students in Pakistan and projected twenty years into the future (when all graduates have entered the labor market, this would cost the Pakistani economy between US$67 billion and US$155 billion in GDP at Net Present Value (2017 PPP). These numbers are based on an assumption that returns to schooling are 8 percent, which reflect the global literature on returns to schooling.12 When aggregating for the country as a whole, we correct for adult mortality and labor force participation. This is important, given the low female labor force participation. (5) Take-up of remote learning (data from joint surveys and SIEF and Gallup) Since the start of the COVID-19 pandemic, Pakistan has put in place an impressive infrastructure to support remote learning.13 Despite its universal appeal, remote learning is not accessible to everyone. In Pakistan, television is available in the majority of households, for instance, but far from universally accessible. Even low-tech tools such as radios are not regularly used.14 9  See https://www.worldbank.org/en/news/immersive-story/2019/11/06/a-learning-target-for-a-learning-revolution 10 The learning poverty measure was based on data reported by Pakistan to the UNESCO Institute of Statistics. This is the share of children who do not reach ‘proficiency’ or ‘advanced’ level on the Grade 4 English Reading Assessment 11  conducted by NAT in 2014, corrected for the share of children in school. Note that the NAT is based on a sample of public schools only, so this only covers about 62 percent of the children who are in school. The ASER household data from 2014 showed that only 30.5 percent of all children (an even lower share of children) aged 9–11 was able to read a simple paragraph out loud. Note that ASER is conducted in the language of the household and represents a lower bar than the NEAS assessment. We recognize that this is somewhat higher than values found for Pakistan such as the 6.2 percent return found in 12  Psacharopoulos and Patrinos (2018) Returns to investment in education: a decennial review of the global literature. In Education Economics 26(5), as well as the 5 percent found in Aslam, Bari and Kingdon (2010) Returns to schooling, ability and cognitive skills in Pakistan. Education Economics, 20(2). However we note that for the world as a whole returns to schooling have been rising, and are expected to continue to rise while the current cohort of students will enter the labor market. This includes for instance the much-lauded ‘Teleschool’ initiative by the Federal government, as well as some provincial 13  initiatives such as the ‘Taleem Ghar’ program in Punjab. Although mobile phones do have radios, many people do not know that it is possible to listen to the radio with a regular cellphone. 14  6 LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS In an ongoing telephone survey in Punjab15, we have found that only 30 percent of households are aware of remote learning opportunities. The figure of families where children actually make use of remote learning is much lower, however, at 10 percent of all households (in other words, only 1 in 3 families who know about remote learning actually use remote learning). An array of factors are behind this: (1) It is hard to keep children engaged cognitively with all the distractions in the household, (2) Even if families have devices to participate in remote learning, they need to be shared between siblings, (3) It can be hard for families to decipher instructional programming (i.e. when to watch Teleschool for which lesson). (4) The household may not understand the language used in remote learning; (5) Access to a television or internet (the main channels of delivering distance learning) is highly unequal. An estimated 98 percent of households from the wealthiest quintile in Pakistan has access to internet or television, compared to only 15 percent of households from the poorest quintile (see Table 2). Table 2: Asset ownership in Pakistan Percentage of households that have asset Across all groups Poorest 20 20–40 40–60 60–80 Richest 20 Any remote learning technology 63% 15% 45% 74% 90% 98% Television 62% 15% 45% 73% 88% 96% Computer 15% s0% 3% 7% 15% 56% Internet 12% 0% 0% 2% 9% 50% Radio 8% 7% 10% 8% 6% 8% Internet and Computer 8% 0% 0% 1% 4% 40% Source: DHS 2017/18 What is missing from this simulation exercise? The simulations described above do not include estimates of other potentially important pathways of learning loss or disengagement from school. The literature suggests an array of pathways—such as gender—based violence, gendered patterns of spending or time-use, closures of privateschools, and the perception of schools as sites of health risks. Therefore, we are currently not able to break these numbers down by gender, but historical evidence suggests that girls may drop out at particularly high rates. These data have been collected as part of an ongoing impact evaluation that is trying to increase enrollment and learning 15  among girls in Punjab, Pakistan. This impact evaluation is funded by the Strategic Impact Evaluation Fund of the World Bank. 7 6 Conclusions These estimates are not cast in stone. In collaboration with the government, development partners can influence these numbers by taking appropriate action, particularly now that schools have reopened: (1) Make sure that dropouts do not materialize. Organize an enrollment drive and leverage cash transfers to encourage enrollment or re-enrollment of children. (2) Use student assessments to gauge the true size of the problem, help teachers strengthen teaching to the level of the student and to facilitate planning. (3) Improve access to remote learning by expanding connectivity, device ownership and by ensuring families know when programming is available. (4) Improve the quality of remote learning by further developing the content, its sequencing, and by making the content more interactive. (5) Plan the first few months after schools reopen carefully. Strengthen curricula and support teachers to facilitate rapid catch-up with learning losses. Now is the time to build back better. While there have been some improvements in both access and learning levels in recent years the pandemic is a substantial setback against hard-won gains. Collectively, we need to step up our support to the school system, protect education as an essential service, and preserve the budget for education. COVID-19 affects everyone, but we cannot let the youngest and most vulnerable members of society suffer from a crisis that threatens their present and their future. 8 LEARNING LOSSES IN PAKISTAN DUE TO COVID-19 SCHOOL CLOSURES: A TECHNICAL NOTE ON SIMULATION RESULTS 7 References Andrabi, Tahir, Benjamin Daniels and Jishnu Das. 2020. Human Capital Accumulation and Disasters: Evidence from the Pakistan Earthquake of 2005. RISE Working Paper 20/039. Azevedo, Joao Pedro Wagner De, Amer Hasan, Diana Goldemberg, Syedah Aroob Iqbal and Koen Martijn Geven. Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates. Policy Research Working Paper Series No. 9284. Filmer, D. Rogers, H., Angrist, N., Sabarwal, S. (2020) Learning-adjusted years of schooling (LAYS): Defining a new macro measure of education. Economics of Education Review, 101971. https://doi.org/10.1016/j. econedurev.2020.101971 Human Capital Variables (World Bank): LAYS, EYS; HLO, Adult Survival https://datacatalog.worldbank.org/ dataset/human-capital-index “Macro Poverty Outlook, June 8th 2020 update with GDP projections aligned with World Bank,” (World Bank, 2020, https://www.worldbank.org/en/publication/macro-poverty-outlook) ILO. 2020. ILOSTAT database. https://ilostat.ilo.org/ . Retrieved on May, 2020. National Education Assessment System (2014). Stakeholder Report NAT 2014. Islamabad: Pakistan. Available on http://www.neas.gov.pk/Detail/M2Q0NWExYzUtOTRiMS00NDZlLWE2NGUtY2QxMzJkYTdhYzY5 World Bank. 2020. Global Jobs Indicators Database (JoIn), Retrieved on May 2020. https://datacatalog. worldbank.org/dataset/global-jobs-indicators-database. Cover photo credit: Feed My Starving Children (CC BY-ND 2.0) The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/ World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. worldbank.org 9