Report No: AUS0003055 Cambodia LEARNING LOSS IN CAMBODIA AND THE USE OF EDTECH DURING COVID-19 . November 2022 . . Document of the World Bank © 2022 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: Bhatta, S. D., S. Katwal, T. Pfutze, V. Ros, Y.N. Wong, F. No. 2022. Learning Loss in Cambodia and the Use of EdTech during COVID-19. Washington, DC: World Bank. All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Learning Loss in Cambodia and the Use of EdTech during COVID-19 Saurav Dev Bhatta, Saurav Katwal, Tobias Pfutze, Vutha Ros, Yi Ning Wong, and Fata No November 2022 Education Global Practice East Asia Pacific Region Table of Contents Acknowledgement ....................................................................................................................................... iv Acronyms ...................................................................................................................................................... v Executive Summary ..................................................................................................................................... vi 1. Introduction ............................................................................................................................................... 1 2. Current state of learning outcomes and learning loss due to COVID-19................................................ 10 2.1 Learning loss in the global context................................................................................................... 10 2.2 Current state of student learning outcomes in Cambodia ................................................................. 11 2.3 Impact of COVID-19: learning loss as reflected in student learning outcomes ............................... 21 2.4 Impact of COVID-19: changes in LAYS, learning poverty, and earnings....................................... 31 3. EdTech-based policy response to COVID-19 and Cambodia’s EdTech readiness ................................ 35 3.1 Government response to COVID-19 ................................................................................................ 35 3.2 Policy provisions for supporting EdTech-based teaching and learning ........................................... 36 3.3 Translating policy into practice: state of EdTech at the school level ............................................... 42 4. EdTech-based school-level responses to COVID-19 and their effect on learning ................................. 55 4.1 Overview .......................................................................................................................................... 55 4.2 Use of EdTech during school closures ............................................................................................. 56 4.3 EdTech and learning outcomes: descriptive evidence...................................................................... 66 5. Conclusions and policy implications ...................................................................................................... 82 5.1 Summary of findings ........................................................................................................................ 82 5.2 Policy implications ........................................................................................................................... 88 References ................................................................................................................................................... 93 Annex .......................................................................................................................................................... 97 List of Tables Table E.1: Key learning loss estimates from the regression models (equated scaled scores) ........ x Table 2.1: Summary statistics ....................................................................................................... 26 Table 2.2: Aggregate national-level estimates of learning loss using regression models ............ 27 Table 2.3: Estimates of learning loss for rural and urban areas, and male and females ............... 27 Table 2.4: Estimates of learning loss for different ecological zones ............................................ 28 Table 2.5: Learning loss for lowest wealth quintile ...................................................................... 29 Table 2.6: Learning loss for public and private schools ............................................................... 30 Table 2.7: Consolidated overview of learning loss findings from the regression models ............ 30 Table 3.1: Streaming and broadcasting channels .......................................................................... 36 Table 3.2: Strategic dimensions and key questions of interest ..................................................... 37 Table 4.1: Determinants of student Khmer test scores ................................................................. 78 Table A.1: Timeline of policy actions and related developments affecting schools and learning, 2020/21 ......................................................................................................................................... 97 Table A.2: Remote learning channels, target grades, and their reach ........................................... 98 Table A.3: Government policies, plans, and strategies supporting EdTech ................................. 99 Table A.4: Determinants of student Khmer test scores .............................................................. 107 List of Figures Figure E.1: Percentage correct scores in Khmer and math in NLA 2021..................................... vii Figure E.2: Student test scores across wealth quartiles ............................................................... viii Figure E.3: Average equated scaled scores by gender and location, 2013–2021 .......................... ix Figure E.4: Share of schools that provided various types of orientation/training to teachers ....... xi Figure E.5: Student test scores by teachers’ digital literacy/competency ..................................... xii Figure 1.1: Human Capital Index, Cambodia and other ASEAN countries (2020) ....................... 3 Figure 1.2: Shares of the households with school-age children engaged in different learning activities during COVID-19 in ASEAN countries, 2021................................................................ 5 Figure 2.1: Student test scores in Khmer and math (2021)........................................................... 12 Figure 2.2: Student test scores by student demographic characteristics ....................................... 13 Figure 2.3: Percentage of students by preschool attendance, grade repetition, and annual absent days ............................................................................................................................................... 14 Figure 2.4: Student test scores across student schooling experiences .......................................... 14 Figure 2.5: Student test scores across household characteristics .................................................. 15 Figure 2.6: Student test scores across geographic regions ............................................................ 16 Figure 2.7: Percentage correct in Khmer and math subdomains .................................................. 16 Figure 2.8: Percentage correct in Khmer and math subdomains by gender and location ............. 18 Figure 2.9: Percentage correct by content difficulty level ............................................................ 19 Figure 2.10: Percentage correct by content difficulty level by gender and location .................... 20 Figure 2.11: Student test scores at the national level, 2013–2021 ................................................ 21 Figure 2.12: Student test scores by gender, 2013–2021 ............................................................... 22 Figure 2.13: Student test scores by rural versus urban residents, 2013–2021 .............................. 22 Figure 2.14: Student test scores by gender and rural versus urban residents, 2013–2021 ........... 23 Figure 2.15: Student test scores by geographical regions, 2013–2021 ......................................... 23 Figure 2.16: Student test scores by wealth quintile, 2013–2021 .................................................. 24 Figure 2.17: Student test scores by school type (public versus private), 2016–2021 ................... 25 Figure 2.18: Number of days of school closure in EAP due to COVID-19 (February 19, 2020, to February 23, 2022) ........................................................................................................................ 32 Figure 2.19: Loss in LAYS and increase in learning poverty....................................................... 34 Figure 2.20: Loss in earnings ........................................................................................................ 34 Figure 3.1: Do government policies on ICT access exist? (national level) .................................. 42 Figure 3.2: Do government policies on ICT access exist? (by region) ......................................... 43 Figure 3.3: Teacher competency in using digital technology (teacher self-rating) ...................... 44 Figure 3.4: Teacher access to ICT infrastructure and devices at home ........................................ 45 Figure 3.5: Teacher competency in using digital technology for teaching (teacher self-rating) .. 46 Figure 3.6: Does the typical initial teacher training program for primary education include ICT training? (national level) ............................................................................................................... 46 Figure 3.7: Use of desktops, laptops, and tablets in class by grade 6 students before COVID-19 (principals’ opinions) .................................................................................................................... 47 Figure 3.8: Importance of using ICT and developing students’ ICT skills (principals’ opinions) 47 Figure 3. 9: Student access to digital infrastructure and devices (self-reporting by students) ..... 48 Figure 3.10: Student digital competency and effectiveness of online learning (teachers’ opinions) ....................................................................................................................................................... 49 Figure 3.11: Share of schools with access to ICT infrastructure (percentage) ............................. 50 Figure 3.12: Number of digital devices in school ......................................................................... 51 Figure 3.13: Principals’ perceptions of sufficiency of ICT infrastructure and resources ............. 52 Figure 3.14: Principals’ perceptions of the sufficiency of access to digital devices with internet connectivity in school (by region) ................................................................................................ 52 Figure 3.15: Principals’ perceptions of the sufficiency of internet stability in school (by region) ....................................................................................................................................................... 53 Figure 3.16: Challenges in effective implementation of distance learning using digital technology perceived by principals and teachers ............................................................................................ 54 Figure 4.1: Share of teachers who provided school material and physical packages during COVID-19..................................................................................................................................... 56 Figure 4.2: Share of digital and physical materials in the total schoolwork ................................. 57 Figure 4. 3: Platforms (digital and non-digital) used by teachers to distribute and collect homework ..................................................................................................................................... 57 Figure 4.4: Total hours and weeks of online classes during COVID-19 ...................................... 58 Figure 4.5: Share of schoolwork completed during school closure .............................................. 59 Figure 4.6: Distance teaching/learning platforms used by teachers to teach new materials to students during the pandemic........................................................................................................ 60 Figure 4.7: Distance teaching/learning platforms used by teachers during the pandemic (not including messaging apps) ............................................................................................................ 60 Figure 4.8: Frequency with which school conducted different activities during school closure .. 62 Figure 4.9: Frequency of use of learning resources by students during school closure ............... 63 Figure 4.10: Share of schools that provided various types of orientation/training to teachers ..... 65 Figure 4.11: Share of schools that provided guidance to teachers on different aspects of online teaching ......................................................................................................................................... 65 Figure 4.12: Share of schools that provided parent orientation .................................................... 66 Figure 4.13: Student test scores by student access to ICT equipment at home ............................ 67 Figure 4.14: Student test scores by student use of digital resources provided by school ............. 68 Figure 4.15: Student test scores by caregiver support at home..................................................... 69 Figure 4.16: Student test scores by access to ICT infrastructure at school ................................... 70 Figure 4.17: Student test scores by access to ICT devices at school ............................................ 70 Figure 4.18: Student test scores by teacher ICT access at home .................................................. 71 Figure 4.19: Student test scores by teachers’ digital literacy/competency ................................... 72 Figure 4.20: Student test scores by orientation/training for teachers ............................................ 73 Figure 4.21: Student test scores by the number of ICT training modules provided to teachers ... 74 Figure 4. 22: Student test scores by activities implemented by schools (as reported by students) during school closure .................................................................................................................... 74 Figure 4.23: Student test scores by total hours of online classes, share of schoolwork covered through online classes, and teacher preparation ........................................................................... 75 Figure 4.24: Loss in LAYS for Cambodia in the intermediate scenario under different mitigation effectiveness assumptions ............................................................................................................. 81 Figure 4.25: Present value loss to Cambodian economy lifetime earnings under different mitigation effectiveness assumptions ........................................................................................... 81 List of Boxes Box 1.1: Regression model for estimating learning loss ................................................................ 8 Box 3.1: Open Educational Resources Cambodia ........................................................................ 40 Box 4.1: Distance learning during COVID-19: Experience of high-performing urban schools .. 60 Box 4.2: Distance learning during COVID-19: Experience of low-performing rural schools ..... 63 Box 5.1: RAPID framework ......................................................................................................... 88 Box A.1: Cambodia National Learning Assessment .................................................................... 97 Acknowledgment This report was authored by a team led by Saurav Dev Bhatta (Senior Economist and Task Team Leader, HEAED) and comprising Saurav Katwal (Economist, Consultant, HEAED), Tobias Pfutze (Senior Economist, EAPCE), Vutha Ros (Education Consultant, HEAED), Yi Ning Wong (Junior Professional Associate, HEDGE), and Fata No (Education Specialist, HEAED). It was prepared under the guidance of Mariam Sherman (Country Director for Myanmar, Cambodia, and Lao PDR), Daniel Dulitzky (Regional Director, HEADR), Maryam Salim (Country Manager for Cambodia), Toby Linden (Practice Manager, HEAED), and Cristian Aedo (Practice Manger, HEAED). The peer reviewers for this study were Venkatesh Sundararaman (Lead Economist, HMNED), Tsuyoshi Fukao (Senior Education Specialist, HSAED) and Cristobal Cobo (Senior Education Specialist, HEDGE) at the concept stage and Tsuyoshi Fukao (Senior Education Specialist, HSAED) and Marie-Helene Cloutier (Senior Economist, HEDGE) at the quality enhancement review stage. The team would like to thank the peer reviewers as well as Joao Pedro Wagner De Azevedo (Lead Economist, HEDGE), Simeth Beng (Senior Education Specialist, HEAED), Tara Beteille (Program Leader, HEADR), and Jeffery Marshall (Senior Education Expert) for their helpful feedback and suggestions and Maly Prak (Program Assistant, EACSF) and Mary Dowling (Program Assistant, HEAED) for providing administrative and coordination support for the study. The team also thanks Jeffery Marshall for providing detailed clarifications on the technical aspects of the national learning assessment (NLA) datasets used in this report. Finally, we want to express special thanks to Ung Chinna, Director of EQAD/MoEYS, and his team for sharing the NLA datasets, facilitating the implementation of the World Bank’s principal and teacher surveys, and providing feedback on the preliminary findings of the study. iv Acronyms ASEAN Association of Southeast Asian Nations CCUN Cambodian Cyber University Network EAP East Asia and Pacific EdTech Education Technology EMIS Education Management Information System EQAD Education Quality Assurance Department ETRI EdTech Readiness Index EYS Expected Years of Schooling GDP Gross Domestic Product GPI Gender Parity Index HCI Human Capital Index HEIP Higher Education Improvement Project HLO Harmonized Learning Outcome IATA Item and Test Analysis ICT Information and Communication Technology ICT-CFT ICT Competency Framework for Teachers IDA International Development Association IRT Item Response Theory LAYS Learning-Adjusted Years of Schooling LMIC Lower-Middle-Income Country LMS Learning Management System LSS Lower Secondary School MoEYS Ministry of Education, Youth, and Sports MoI Ministry of Information NER Net Enrollment Rate NLA National Learning Assessment OER Open Education Resources PISA-D Programme for International Student Assessment for Development PPP Purchasing Power Parity RTI Research Triangle Institute SD Standard Deviation SDGs Sustainable Development Goals SEA-PLM Southeast Asia Primary Learning Metrics SES Socioeconomic Status STS Student Tracking System UNESCO United Nations Educational, Scientific, and Cultural Organization UNICEF United Nations Children’s Fund USS Upper Secondary School WHO World Health Organization v Executive Summary This report estimates the effects of COVID-19 on learning and earnings in Cambodia, analyzes the country’s education technology (EdTech) readiness and the extent to which EdTech access and use are correlated with learning, and discusses the policy implications of the study findings for enhancing learning and improving system resilience through EdTech-based teaching and learning. More specifically, it first analyzes the state of learning outcomes in Cambodia in the immediate post-COVID period (November 2021) using the government’s national learning assessment (NLA) data for grade 6 students and estimates the decline in learning outcomes experienced by these students between 2016 and 2021 in Khmer and mathematics. Additionally, using a learning loss simulation model developed at the World Bank, it also estimates losses in learning-adjusted years of schooling (LAYS) and future earnings of students resulting from the pandemic. Second, it analyzes the relationship between the EdTech-based distance learning measures implemented at the school level and learning outcomes as well as the extent to which the country is prepared to systematically integrate and expand the use of EdTech in the education system. Third, it provides recommendations for enhancing learning recovery and learning outcomes and addressing gaps in policy provision and implementation to support the scaling-up of EdTech for improving system resilience. Current state of learning outcomes While Cambodia made impressive progress in enhancing access to school education in the first two decades of this century, the quality of basic education in the country remained relatively poor. The NLA of grade 3 students conducted in 2015 showed relatively poor student performance, with approximately 40 percent correct answers in math and 41.5 percent correct answers in Khmer, a situation virtually unchanged from results measured a decade earlier. Similarly, the percentages of correct answers in the 2007 NLA of grade 6 students were only 52 percent in math and 66 percent in Khmer, with little change a decade later in 2017.1 Furthermore, findings from the 2019 Southeast Asia Primary Learning Metrics (SEA-PLM) showed that only 11 percent of Cambodian fifth graders had reading proficiency at the level Sustainable Development Goal (SDG) 4.1.1b expects children to achieve by the end of primary school (UNICEF and SEAMEO 2020).2 These findings indicate that the country was facing a learning crisis even before the onset of the pandemic. Findings from the most recent grade 6 NLA—conducted in November 2021 (that is, at the beginning of the post-COVID period)—show that student learning outcomes in both Khmer and math continue to remain low. The students perform relatively poorly in this assessment, especially in math, where the overall score is only 38 percent (Figure E.1a). In Khmer, their performance is particularly low in dictation, with an average student getting only 24 percent of the answers correct. In math, the average student performance in the subdomain with the lowest score (geometry) is somewhat higher but is nevertheless quite low (35 percent correct). 1 While the percentages of correct answers for the two years cannot be directly compared, a comparison of corresponding equated scaled scores derived using Item Response Theory (IRT) indicates that there was only a small increase in student performance during this period in both subjects (EQAD 2017). 2 Compared to the other five Association of Southeast Asian Nations (ASEAN) countries that participated in the assessment, Cambodia was a median performer. Its performance was similar to that of Myanmar (with 11 percent of fifth graders achieving Level 6 or higher) and the Philippines (10 percent) and better than that of the Lao People’s Democratic Republic (2 percent). But it significantly lagged behind Vietnam and Malaysia, where the shares of children proficient in reading were 82 percent and 58 percent, respectively. vi There are significant disparities in student learning outcomes across different student populations, with females and urban students outperforming males and rural students, respectively, in both Khmer and math. While there are gender gaps in student performance in both subjects in favor of females, the magnitude of the gap in math performance is small. In the case of Khmer, however, females not only have a much higher aggregate percentage correct score than males (11 percentage points higher) but also substantially outperform males in all three subdomains—dictation, writing, and reading. Compared to the gender gap, the urban-rural gap in student performance is much more pronounced in both subjects. Urban school students have consistently higher scores than rural school students in each subdomain (Figure E.1b) and at all levels of difficulty. There is also strong evidence that students who are at the right age for grade 6 (that is, 12 years) perform better than older or younger students. Figure E.1: Percentage correct scores in Khmer and math in NLA 2021 (a) Overall scores (b) Scores for urban and rural students 100% Reading 100% Math (overall) 80% 68 Dictation 80% 60% 44 40% 57 60% Algebra 37 34 Writing 60 20% 60 47 51 22 44 0% 38 44 44 40% 33 36 57 Statistics 59 40 Khmer (overall) 41 43 20% 47 Geometry Numbers 0% Measurement Khmer Math Urban Rural Source: Authors’ calculations using NLA 2021. Furthermore, students from wealthier households have significantly better learning outcomes than students from poorer households. Looking at equated scaled scores in NLA 2021, it is observed that student performance in both Khmer and math is progressively higher for the richer quartiles of the wealth index, and the gaps are quite substantial (Figure E.2). In particular, the gap in performance between children from the wealthiest quartile and the poorest households is large—71 points or 0.75 standard deviation (SD) in Khmer and 57 points or 0.65 SD in math. vii Figure E.2: Student test scores across wealth quartiles Source: Authors’ calculations using NLA 2021. Student schooling experiences, namely, preschool attendance, grade repetition, and regular school attendance, are also strongly correlated with test scores. Students who have attended preschool consistently outperform those who have not in both subjects. Similarly, students who have never repeated any grade have higher scores than those who have repeated at least one grade. Furthermore, there is strong evidence that regular school attendance makes a big difference in learning outcomes. More specifically, students who were never absent during the academic year outperform those who were absent at least seven days by 60 points or approximately 0.63 SD in Khmer and 42 points or about 0.42 SD in math.3 Learning loss due to the pandemic In response to the COVID-19 pandemic, like most other East Asia and Pacific (EAP) countries, Cambodia also closed schools either fully or partially for extended periods between February 2020 and February 2022. School closure data from the United Nations Educational, Scientific, and Cultural Organization (UNESCO) show that 16 EAP countries experienced school closures (full or partial) of 100 days or more, and except for Japan, the countries where schools were closed for shorter periods all belonged to the Pacific region. All Southeast Asian countries apart from Singapore kept their schools closed for particularly long periods. With schools closed for 532 days (fully closed for 280 days and partially closed for 253 days), Cambodia stands out as the country experiencing the third highest number of school closure days in EAP during these two years. Globally it ranks among the top 10 percent of countries that experienced the highest number of school closure days. It is reasonable to assume that these extended school closures had a significant negative effect on the learning of students. A comparison of equated scaled scores of grade 6 students over the years shows that the low student learning outcomes in 2021 summarized above reflect substantial learning losses between 2016 and 2021. A simple descriptive analysis of the NLA data shows that while student learning outcomes improved slightly between 2013 and 2016, there was a large decline in student performance between 2016 and 2021 in the nation as a whole, especially in math. Student assessment scores decreased by 42.8 points in Khmer and 56 points in math, representing substantial losses amounting to 8.4 percent and 11.3 percent, respectively. While many factors could affect the decline in learning outcomes, much of it can be attributed to the disruptions caused by the pandemic, especially considering that there was a mild upward trend in learning outcomes between 2013 and 2016. 3 Student absence denotes the number of days the students were absent when the schools were in session for the three months at the beginning of the year and at the end of the year. viii But the loss in learning was not experienced equally by all students; rather, males and students from urban schools experienced greater losses than females and rural students, respectively (Figure E.3). In general, learning losses were higher for males than females, especially in Khmer. The relatively lower loss in Khmer for females seems to have been driven mainly by rural females who experienced the least amount of loss in this domain. The urban-rural gap in learning loss, however, was more prominent in math than in Khmer, with urban students suffering a significantly higher learning loss. There was little variation in learning loss across Cambodia’s four ecological zones. Figure E.3: Average equated scaled scores by gender and location, 2013–2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). There were also substantial disparities in learning loss across students from different socioeconomic backgrounds, with students from the wealthiest households experiencing the highest levels of learning loss in both Khmer and math. Although students from better-off households had progressively higher average scores for all years, they also suffered the highest losses (Figure E.4). More specifically, the losses were relatively low for the bottom two wealth quintiles (that is, the bottom 40 percent of the wealth distribution), somewhat higher for the third quintile, and highest for the top two quintiles (that is, the top 40 percent of the wealth distribution). Estimates of learning loss derived using regression analysis support these findings. These regression models provide more rigorous estimates of the pandemic-induced learning losses by examining changes in learning outcomes between 2016 and 2021 relative to the changes in outcomes between 2013 and 2016 while controlling for a host of other student, household, and locational factors. Consistent with the findings from the descriptive analysis, the regression results show substantial declines in average equated scaled scores between 2016 and 2021 at the aggregate national level as well as across different population groups. As summarized in Table E.1, both males and females, urban and rural dwellers, and students from all wealth quintiles experienced substantial learning losses. For example, the regressions with control variables show that the estimated mean learning loss was around 28 points for Khmer and 41 points for math for the nation as a whole and ranged from a low of 19 points in Khmer for the poorest quintile to a high of 68 points in math for urban males. ix Table E.1: Key learning loss estimates from the regression models (equated scaled scores) Khmer Math No controls Controls No controls Controls Overall −42.76 −27.73 −55.96 −41.00 Rural and male −43.99 −27.63 −50.94 −35.35 Rural and female −33.04 −24.00 −43.85 −33.61 Urban and male −51.05 −30.83 −86.88 −67.87 Urban and female −47.16 −37.63 −77.75 −60.94 2nd to 5th Quintile −40.97 −30.11 −56.62 −44.77 1st Quintile −28.22 −19.08 −32.69 −26.25 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Furthermore, findings from learning loss simulations based on the model of Azevedo et al. (2022) indicate that the pandemic is expected to result in substantial reductions in LAYS, an increase in learning poverty, and losses in individual and economy wide earnings. Between February 19, 2020, and February 23, 2022, schools in Cambodia were fully closed for 280 days and partially closed for 253 days. As a result of these school closures (and to some extent income shocks to families), it is estimated that Cambodia will experience a decrease in the average LAYS for the current cohort of school-age children by 1.46 to 1.76 years or 21 to 26 percent. These figures compare unfavorably with the LAYS loss estimates for the average of EAP LMICs, which range from 0.86 to 1.03 years. Similarly, the learning poverty rate is expected to have risen to 99.7 percent from an already severe pre-COVID-19 baseline of 90 percent, even in the optimistic scenario.4 As a result of the loss in LAYS, the average annual earnings per student is expected to decline by 8.8 to 10.6 percent compared to the baseline earnings of US$6,077 per year in 2017 purchasing power parity [PPP] US$.5 In present value terms, this loss in earnings per student translates to an estimated loss of US$23–28 billion or 32.3–38.8 percent of gross domestic product (GDP) to the economy as a whole. Government and school-level EdTech-based responses to the pandemic: activities and effects In response to the pandemic, the government prepared a response plan that emphasized the use of EdTech-based distance learning to mitigate the adverse effects of prolonged school closures. After schools closed in March 2020 in response to the pandemic, the Ministry of Education, Youth, and Sports (MoEYS) developed key instructions, guidelines, and directives to immediately offer distance learning and teaching to students from the preprimary to the secondary levels. These documents, along with the experience gained by MoEYS from the initial implementation of distance learning measures, informed the development of the ‘Cambodia Education Response Plan to the COVID-19 Pandemic’, which was formally endorsed in July 2020. The use of EdTech to ensure continuity of education for children was one of the key elements of the plan. Following the closure of schools, the government made a range of resources available to students to support their learning, including paper-based learning materials as well as TV and radio programs, digitalized documents, multimedia content, and e-learning programs. To disseminate these resources, they used both traditional channels, namely TV and radio broadcasts, as well as modern channels such as Facebook pages, YouTube channels, websites, and mobile applications. They also promoted the 4 The average learning poverty rate before the pandemic for the EAP LMICs was 70.9 percent, and this is expected to have risen to 79 percent in the optimistic scenario. 5 In dollar terms, the expected reduction in average annual earnings per student for Cambodia ranges from US$536 to US$642. These losses are substantially greater than the corresponding declines for the EAP LMICs (US$407 – 496). x use of messaging apps as crucial tools to facilitate student-teacher interaction and engagement during school closures. At the school level, findings from the surveys of principals and teachers indicate that the vast majority of schools provided some form of study materials to students for distance learning during school closure. The schools used both digital and non-digital (printed) materials to support home-based learning by students. While digital and non-digital materials were used in equal proportions in the nation as a whole, digital materials were used more widely in urban schools, and non-digital materials were more prevalent in rural schools. Messaging apps such as WhatsApp and Telegram were widely used as a medium not only for sharing learning materials and homework assignments but also for conducting online classes. Specifically, about two-thirds of the teachers reported using messaging apps at least once a week to conduct virtual classes, and almost half of them used these tools daily. Among the different MoEYS digital platforms available, Facebook and YouTube channels were most widely used by teachers for delivering distance learning. About two-thirds (62–63 percent) of the teachers used these channels at least once a week to teach their students new materials. The MoEYS e- learning platform was also used quite frequently by the teachers, with about 56 percent of the teachers using them at least once a week. Broad access to smartphones and mobile data among the teachers likely made the use of these platforms possible, as almost 80 percent of the grade 6 teachers reported having regular access to smartphones and mobile data at home. Some of the other platforms used regularly but somewhat less frequently include the MoEYS Podcast and MoEYS TV, TVK, and a mobile app from the Ministry of Information (MoI). Over 90 percent of the schools provided their grade 6 teachers with orientation/training to support distance learning during the pandemic, but training on specific EdTech-based teaching skills was relatively limited. In general, training on specific skills required for effective EdTech-based teaching provided by the majority of the schools was inadequate. More specifically, only about 43 percent of schools trained their teachers on using EdTech to teach specific subjects to students connected to the internet from home, and only 48 percent provided training on sustaining the attention and engagement of students during online classes (Figure E.4). Smaller percentages of schools provided training on preparing lessons for EdTech-based teaching, supporting students with poor digital literacy, and managing the classroom. Training on other dimensions of online education delivery, such as classroom management and designing and conducting of assessments, was even more limited. Figure E.4: Share of schools that provided various types of orientation/training to teachers Any kind 93 Student engagement 48 Teach (distance) 43 Prepare lessons 38 Support students with poor digital literacy 36 Teach (in class) 34 Psychological support 29 Classroom management 23 Assessment 23 0% 20% 40% 60% 80% 100% Source: Authors’ calculations using supplemental teacher survey 2021. xi Figure E.5: Student test scores by teachers’ digital literacy/competency Source: Authors’ calculations using NLA 2021 and supplemental teacher survey 2021. Student learning outcomes were significantly related to teaching-related interventions on the part of the school, teacher access to EdTech at home, and teacher digital competency. The teaching-related interventions by schools that were positively associated with learning outcomes included sending homework to students and asking them to submit completed assignments and the share of total schoolwork covered through online classes. Students’ use of various resources provided by schools during the pandemic was also positively related to their test scores. These resources included physical (printed) and digital textbooks, workbooks, worksheets developed by schools, worksheets provided by MoEYS for self-study, real-time virtual lessons delivered by their teachers, and recorded lessons or other digital materials from other sources. For instance, students who used the materials provided via messaging apps every day outperformed those who never used them by 49 points (or 0.52 SD) in Khmer and by 34 points (or 0.39 SD) in math. Students whose teachers had better digital competency in the use of the internet and different EdTech devices and applications also had higher learning outcomes (Figure E.5), as did students whose teachers had access to the internet and EdTech devices such as laptops, tablets, and smartphones at home. Similarly, student scores were positively associated with student access to EdTech at home during COVID- 19. While it is not possible to estimate the impact of EdTech-based distance learning interventions on changes in student assessment scores due to data limitations, findings from learning loss simulations suggest that the provision of such measures did contribute to mitigating learning loss. They indicate that in the absence of these EdTech-based interventions, the loss in LAYS experienced by Cambodia would have been 7 percent larger than the loss estimated under the default intermediate scenario. This would have translated to an increase in economy lifetime earnings loss of around US$2 billion in 2017 PPP US$. EdTech readiness of Cambodia: policy and practice Cambodia has a largely supportive policy environment for expanding EdTech-based teaching and learning. During the past two decades, the government has approved a number of policies, plans, and strategies that support the expansion and use of EdTech to improve the quality and resilience of the xii education system. Furthermore, they seek to build capacity in information and communication technology (ICT) use and utilize ICT as an alternative platform to increase access to education. They also highlight the need to train both pre-service and in-service teachers on ICT skills to improve their teaching and administration work. This EdTech supportive policy environment was most likely one of the factors that enabled the government to quickly implement distance learning interventions in response to the pandemic. There are, nevertheless, a number of policy gaps that limit the EdTech readiness of Cambodia. First, these policies do not delegate responsibilities to schools for systematically integrating EdTech in teaching and learning, a gap that limits the effective use and expansion of EdTech. Second, although they recognize the importance of building the digital competency and EdTech-based teaching capacity of teachers, Cambodia has yet to establish a digital standards/competency framework for teachers and a system for evaluating their digital competencies. Furthermore, as EdTech training is not included systematically in in- service teacher training programs, opportunities available to teachers for receiving such trainings are limited. Third, the country also does not have a student digital standards/competency framework, which is relevant for guiding the design of concrete teaching-learning activities for building students’ digital skills. Fourth, the government does not yet have a clear financing strategy and plan for systematically expanding EdTech infrastructure and internet connectivity in the coming years in line with the country’s Educatio n 2030 Roadmap. At the same time, data from the principal and teacher surveys indicate that, in practice, the EdTech supportive policies on paper have not translated into greater EdTech readiness among teachers. Most teachers have low EdTech competency levels and limited and inequitable access to EdTech devices and internet connectivity in school. The vast majority of teachers perceive their own EdTech competency (in general use of EdTech as well as use of EdTech for teaching) to be poor and believe that inadequate digital literacy among teachers is a major challenge. Only 29 percent of the schools have internet access, and on average, schools have about four working devices (per 300 students) available for student use. Furthermore, the lack of access to connectivity and digital devices is particularly severe in rural schools where, on average, only one working device (per 300 students) is available for student use. Expansion of EdTech use is further constrained by the students’ limited EdTech competency and inadequate access to devices and connectivity in their homes, especially in rural areas and poorer households. Less than one-third of the teachers believe that students were able to use EdTech for learning effectively and understand online lectures during the pandemic. According to the NLA 2021 survey, only 10 percent of rural students and 4 percent of students from the poorest quartile had access to computers at home. Policy implications  The discussion above provides evidence that Cambodia experienced substantial losses in learning due to the pandemic, as reflected in decreases in student assessment scores and LAYS, and an increase in learning poverty. These findings point to an urgent need for short-term interventions aimed at learning recovery and accelerated learning. Learning recovery and accelerated learning will require a number of systematic interventions such as bringing all students back to school, providing adequate support especially to academically lagging students, making appropriate modifications to the curriculum, and training teachers to teach effectively in an environment where students are at different levels of learning. While some of these interventions might already be under way, it will be useful for policy makers to consider using approaches such as the RAPID framework proposed by UNESCO, United xiii Nations Children’s Fund (UNICEF), and the World Bank to systematically guide the development of learning recovery and accelerated learning programs in the Cambodian context.6  The evidence provided on low learning outcomes among grade repeaters, children who are frequently absent from school, and poorer children suggests that these groups of students need special attention. The first two groups likely represent students who are struggling the most academically and have not been able to engage effectively with the learning process. Providing a more supportive learning environment for these vulnerable learners may require different interventions, such as providing remedial classes, training teachers to identify students in need of specialized attention, and teaching at the right level. Support for students from low-income families is equally important and may be provided in the form of cash transfers and free meals.  While longer-term improvement of education quality, in general, has been a vital theme of the different policies and plans in the education sector, it would be relevant to pursue additional system-level reforms to address the learning crisis. These could include, among others, interventions for incentivizing reforms by linking education service targets with financing, upgrading the education management information system (EMIS) to support results-based performance monitoring, strengthening school- based management, and increasing the engagement of parents and communities in the education process.  The extended school closures due to the pandemic and the resulting severe disruptions in learning point to the need for improving system resilience. To make the system more resilient to future shocks in the longer term, it would be important for Cambodia to significantly strengthen and expand its EdTech- based remote learning infrastructure and scale up EdTech use in teaching and learning. Given that different aspects of EdTech are positively correlated with learning outcomes, systematically integrating EdTech in the teaching-learning process also has the potential to improve student learning outcomes. This will require a holistic EdTech expansion strategy that involves increasing ICT access for educators and pupils in schools, enhancing teacher and student digital competency, and improving access to ICT infrastructure and digital devices in teachers’ and students’ homes. Furthermore, it will be important to engage the larger ecosystem of EdTech partners (including the government, students, teachers, school leaders, parents/caregivers, nongovernmental organizations, donor agencies, academia, and private sector companies) in developing and implementing such strategies and policies.  To better utilize EdTech to strengthen system resilience and enhance education quality, policy gaps that limit the country’s EdTech readiness must be addressed. The first such gap is the absence of national policies that delegate responsibility to schools for systematically integrating EdTech in teaching and learning. The second gap is the absence of a digital standards/competency framework for teachers and a system for evaluating their digital competencies. Cambodia would need to develop EdTech training modules aligned with this type of framework and make them an integral part of initial in-service teacher training programs. The absence of a digital competency framework for students is the third gap, which can be readily addressed by adapting existing frameworks to the Cambodian context. Fourth, there is a need for a clear financing strategy and a costed plan for scaling up EdTech-based teaching-learning in line with the Education 2030 Roadmap. 6 The RAPID framework proposed by UNESCO, UNICEF, and the World Bank for establishing a learning recovery program focuses on five policy actions: (1) reaching every child and retaining them in school, (2) assessing learning levels regularly, (3) prioritizing teaching the fundamentals, (4) increasing catch-up learning, and (5) developing psychosocial health and well-being (World Bank et al. 2022a) (see Box A.1 for details). xiv  The surveys of principals and teachers also point to major gaps in policy implementation that need to be addressed. One of these gaps is the proper dissemination of policy provisions to schools, principals, and teachers, as the vast majority of principals and teachers are not aware of existing policies or have differing opinions regarding the existence of these policies. Another is the implementation of teacher training focused on EdTech. Despite the emphasis on providing EdTech training to teachers in the different plans and policies, most teachers received little ICT-related training/orientation even during the pandemic. Incorporating EdTech training as a mandatory component of the initial teacher training program for primary education could be a crucial first step toward improving digital literacy among both teachers and students and scaling up the integration of EdTech in teaching-learning. Finally, it should be noted that while MoEYS has made a substantial body of e-learning resources available to students and teachers as envisioned by the different ICT-related policies and plans, most of these resources became available only because of the urgent need to respond to the disruptions caused by COVID-19. Now that the pandemic has subsided, it will be important to make adequate provisions for maintaining and expanding these resources and ensuring that they continue to be available for use in the classroom and beyond. It is also important to assess the quality of the existing EdTech resources to make required upgrades and establish quality assurance standards and procedures to guide future expansion of new resources.  The extensive use of non-digital learning resources during the pandemic suggests the public sector should also invest in upgrading the quality of such resources and training teachers in using digital as well as non-digital tools for distance learning. This would be beneficial from a system resilience perspective, especially in the short to medium term, as it will take some time for Cambodia to make EdTech infrastructure and devices accessible to all children. This is also equally important from an equity perspective since poor students and rural students have significantly lower access to EdTech. xv 1. Introduction Background Cambodia has made impressive progress during the past two decades in enhancing access to school education. The national net enrollment rate (NER) for primary education increased from 84 percent in 2001 to 97 percent in 2019. The increase in access at the post-primary levels during this period was even more impressive. The NER increased almost fourfold from 17 to 60 percent for lower secondary education and from 8 to 31 percent for upper secondary education (MoEYS 2005a; World Bank 2021a). Furthermore, Cambodia has achieved gender parity in access to education at the primary level, as indicated by a gender parity index (GPI) in NER of 1.0 in 2019/20. The GPI for lower secondary and upper secondary education in 2019/20 was 1.27 and 1.41, respectively, suggesting that females have an advantage over males in terms of access at these levels. Despite these improvements, Cambodia’s education sector continues to face some broader supply- side challenges which have constrained the expansion of education service delivery. These include, among others, inadequate infrastructure and low teacher capacity. Due to an insufficient number of school buildings, many schools operate in double shifts, which reduces the instruction time in school. This is of particular concern at the primary level, as over two-thirds of primary schools operate more than one shift per day compared to just about one-quarter of the lower secondary schools (LSSs) (World Bank 2018). As a result, the instruction hours in primary schools are limited to just three to four hours per day. Teacher capacity is also relatively low in Cambodia as a relatively small share of teachers have bachelor’s degrees or higher: for example, at the primary level, only 8 percent of female teachers and 15 percent of male teachers have bachelor’s degrees. Over the past two decades, the Cambodian government has implemented several interventions to address these issues. These include the expansion of school construction and the development of a comprehensive teacher policy to conduct quality teacher training and improve teacher working conditions. But these efforts have yet to be reflected in significant improvements in student learning outcomes. The quality of basic education remains relatively poor and has improved only marginally during the past decades. National learning assessments (NLAs) of students conducted by the government during this period show persistently low scores in literacy and numeracy tests. The most recent grade 3 NLA, conducted in 2015, showed relatively poor student learning outcomes with approximately 40 percent correct answers in mathematics and 41.5 percent in Khmer, a situation virtually unchanged from results measured a decade earlier.7 The national assessments of grade 6 students conducted in 2007 showed higher scores in both mathematics (52 percent) and Khmer reading (66 percent). But as with grade 3, there was little change in the percentage of correct answers a decade later. In 2017, the grade 6 average scores in mathematics and Khmer reading were 48 percent and 62 percent, respectively. While the percentage of correct answers across the different years cannot be directly compared, a comparison of corresponding equated scaled scores 7 In the 2006 NLA of grade 3 students in mathematics, on average, the percentage of questions answered correctly was only 38 percent. Similarly, the average score in the Khmer language assessment was 41 percent. Although the scores in the 2006 and 2015 assessments cannot be directly compared, students’ poor performance in both sets of assessments paints a picture of lower than expected performance over an extended period. 1 derived using the item response theory (IRT) indicates that there was only a slight increase in student performance during this period in both subjects (EQAD 2017).8 International assessments conducted before the COVID-19 pandemic also indicated that there is an urgent need to improve the learning outcomes of Cambodian children. According to the results from the 2018 Programme for International Student Assessment for Development (PISA-D), only 8 percent of 15-year-old students achieved the minimum level of proficiency in reading and 10 percent of students achieved the minimum level of proficiency in mathematics (MoEYS 2018a). Similarly, findings from the 2019 Southeast Asia Primary Learning Metrics (SEA-PLM) showed that only 11 percent of Cambodian fifth graders had reading proficiency at the level Sustainable Development Goal (SDG) 4.1.1b expects children to achieve by the end of primary school (Level 6 or higher on SEA-PLM) (UNICEF and SEAMEO 2020). Compared to the other five Association of Southeast Asian Nations (ASEAN) countries that participated in the assessment, Cambodia was a median performer. Its performance was similar to that of Myanmar (with 11 percent of fifth graders achieving Level 6 or higher) and the Philippines (10 percent), and better than that of Lao PDR (2 percent). But it significantly lagged behind Vietnam and Malaysia, where the percentages of children proficient in reading were 82 percent and 58 percent, respectively. Utilizing the SEA-PLM assessment data and information on out-of-school children, the World Bank estimates that around 90 percent of 10-year-olds in Cambodia in 2019 were learning poor—that is, they could not read and understand a simple text by the end of primary school (World Bank et al. 2022b). The SEA-PLM results for mathematics are also discouraging—only 19 percent of Cambodian children were found to perform at or above the requirement of SDG 4.1.1b, reflecting the poor quality of primary education in the country. In addition, the SEA-PLM data showed significant disparities in learning outcomes across students from different income groups and geographical locations. The shares of rural students and poorer students who achieved minimum proficiency in both math and reading were much lower than the corresponding shares of urban students and students from wealthier households. The low learning outcomes of Cambodian children have contributed to a deficit in the country’s human capital. With a Human Capital Index (HCI) of 0.49, Cambodia is one of the lowest HCI countries in the Southeast Asian region (Figure 1.1). Its HCI of 0.49 indicates that an average Cambodian child born in 2020 would be only 49 percent as productive by age 18 as a child who had a complete education and full health (World Bank 2020a). The HCI has six components, three of which—harmonized learning outcomes (HLOs), expected years of schooling (EYS), and learning-adjusted years of schooling (LAYS)—are directly related to education. It is estimated that, in Cambodia, children who start school at age four can expect to complete 9.5 years of schooling by their 18th birthday, but when what children actually learn is considered, the resulting LAYS is only 6.8 years. This low value of LAYS (which reflects poor education quality) is a key contributor to the low HCI score of Cambodia. 8 As noted above, the overall percentage scores for different years cannot be directly compared. However, it is possible to compare equated scaled scores—obtained using IRT modeling techniques—across the years to understand the trends in learning outcomes. The equated scaled scores for both subjects in both grades indicate that there was a small increase in student performance during this period. 2 Figure 1.1: Human Capital Index, Cambodia and other ASEAN countries (2020) Source: World Bank 2020a. COVID-19 resulted in extended periods of school closure, severely disrupting children's education across the country. In response to the World Health Organization (WHO) declaration of the COVID-19 outbreak as a global pandemic on March 11, 2020, the Government of Cambodia closed all schools in the country from March 16, 2020, onward (World Bank 2021b).9 Some schools, including 20 private schools with high safety standards and all schools in four low-risk provinces, were allowed to reopen in September and October 2020, and the remaining schools were given permission to reopen on November 3, 2020. The academic year, adjusted for the disruptions caused by the pandemic, was completed at the end of November 2020, and schools reopened again on January 11, 2021, for the new (delayed) academic year. However, because of an outbreak of COVID-19 in February 2021, all schools were again closed for around six months from mid-March to mid-September 2021, at which point some schools reopened again in certain parts of the country. All schools finally reopened for regular operations on November 1, 2021, and there have been no further COVID-induced closures since then.10 It is estimated that these periodic extended disruptions in schooling resulted in a loss of around 280 days of face-to-face instruction between March 2020 and December 2021 (see Figure 2.18), which in turn is expected to have severely affected the learning outcomes of students. To help compensate for the loss of regular school days and maintain continuity of learning to the extent possible, the government implemented a number of remote learning measures, including interventions based on education technology (EdTech), to provide educational opportunities to children. These measures included the provision of tools for online teaching-learning using various platforms, delivery of educational programs through television and radio, and provision of home-based self- learning instructions and materials (MoEYS and UNICEF 2022). For example, school students were provided access to learning content on the Ministry of Education, Youth, and Sports (MoEYS) YouTube Channel, Facebook page, some education applications, and the TVK2 television channel, which broadcasts video lessons for different grades. With support from the IDA-financed Higher Education Improvement 9 The dates of school closures mentioned here align with the data available from UNESCO’s Dashboards on the Global Monitoring of School Closures Caused by the COVID-19 Pandemic (https://covid19.uis.unesco.org/global- monitoring-school-closures-covid19/). 10 See Table A.1 for a detailed timeline of education-related policy actions taken by the government in response to the pandemic. 3 Project (HEIP) and the GPE COVID-19 Accelerated Fund managed by the United Nations Children’s Fund (UNICEF), MoEYS also produced video lessons and student worksheets to be distributed to the students. However, distance learning, especially through digital technology, is only a partial and imperfect substitute for face-to-face classroom instruction. Accumulating evidence shows that the capacity of remote learning efforts to substitute for in-person learning is very low (World Bank et al. 2022a). Not only is it challenging to keep children engaged in learning when they cannot go to school, but in the case of developing countries, the need to share devices among multiple users in the household, unreliable connections to the internet, electricity interruptions, unequal access to the internet and television, and limited student-teacher real-time interactions during remote delivery of lessons further reduce the effectiveness of distance learning. Evidence from a study on learning loss due to COVID-19 in the Netherlands suggests that even when the quality and accessibility of information and communication technology (ICT) infrastructure are not major issues, technology-based remote learning may have a limited impact on mitigating learning loss (Engzell, Frey, and Verhagen 2021). According to the World Bank’s COVID-19 Household Monitoring Dashboard data,11 in Cambodia, children in less than 23 percent of the households with school-age children who attended school before the pandemic used mobile learning apps since school closure, and children from only around 13 percent and 1 percent of the households watched educational TV programs and listened to educational radio program, respectively (Figure 1.2). Hence, it can be assumed that the effectiveness of remote learning in Cambodia was also limited. The use of mobile learning apps, TV, and radio for learning was relatively low in other East Asian lower-middle-income countries (LMICs) as well. 11 https://www.worldbank.org/en/data/interactive/2020/11/11/covid-19-high-frequency-monitoring-dashboard 4 Figure 1.2: Shares of the households with school-age children engaged in different learning activities during COVID-19 in ASEAN countries, 2021 Source: Authors’ presentation using data from https://www.worldbank.org/en/data/interactive/2020/11/11/covid-19- high-frequency-monitoring-dashboard (Round 5, March 1–21, 2021). Note: (a) Sample includes households with school-age children who attended school before the pandemic. (b) Data not available for other ASEAN countries. A review of the literature by UNICEF, United Nations Educational, Scientific, and Cultural Organization (UNESCO), and the World Bank shows that the vast majority of countries experienced substantial learning losses as a result of the COVID-19-induced school closures (UNICEF, UNESCO, and World Bank 2022). Among the 104 countries covered by their review, 87 countries report learning losses, while the remaining indicate mixed results (5), non-significant gains or losses (10), or gains (2). The authors note that for most of these countries, the reported findings are based on simulations, 12 and information on actual declines in learning outcomes is available only for 35 countries. Nevertheless, the 12 Simulation models estimate potential learning losses based on different assumptions and are also useful for showing how estimated losses are related to aspects beyond the education sector (for example, future earnings of students). Hence, simulation-based estimates are distinctly different from learning loss estimates based on learning outcomes data from learning assessments which show the actual declines in learning outcomes during the period under consideration. It is important to be cognizant of the difference between these two types of estimates. 5 evidence provided by this review makes it clear that the pandemic has had a severe negative impact on the learning of children globally. Learning loss simulation results based on school closure data up to the end of February 2022 suggest that, in the absence of concrete measures to support learning recovery, the learning loss suffered by the current cohort of students globally would translate into an aggregate lifetime earnings loss of around US$21 trillion in present value terms or approximately 17 percent of the global gross domestic product (GDP) (World Bank et al. 2022a). Several studies have been conducted to assess the impact of COVID-19 on the education sector of Cambodia. Most of these studies mainly look at school closures; access to the internet, smart phones, TVs, and radios by students and teachers; access to learning content or platforms provided by MoEYS; and possible adverse impacts on student dropout and mentality. Examples of key studies of this type are the Cambodia COVID-19 Joint Education Assessment (MoEYS 2021a), the Socioeconomic Impacts of COVID-19 on Households in Cambodia (Karamba, Salcher, and Tong 2021), and Rapid Assessment of COVID-19 Impacts on Girl Education (CARE International and PLAN International 2020). These studies provide evidence on the extensive school closures experienced by Cambodian students and the low levels of access to and use of EdTech among students but do not look at loss in learning or the correlation between distance learning opportunities and student learning outcomes. Except for a recent study by MoEYS and UNICEF (2022), studies that do analyze learning loss in Cambodia have derived estimates using variations of a simulation model developed at the World Bank by Azevedo (2020). The most recent analysis of this type conducted by the World Bank in 2021 estimates that Cambodia’s current cohort of students will experience a decline in LAYS of 1.5 years and a corresponding annual expected earnings loss of US$738 (in 2017 purchasing power parity [PPP] US$). The MoEYS and UNICEF (2022) study provides direct evidence of learning loss experienced by Cambodian students due to the pandemic-induced school closures by estimating the actual decline in learning assessment scores of grade 6 students between 2016 and 2021. More specifically, it finds that equated scaled scores in Khmer reading and math declined by 9 percent and 15 percent, respectively, between 2016 and 2021. It also finds that males and urban students experienced greater learning loss than females and rural students, respectively, but there were no significant differences in learning loss across socioeconomic status (SES) quintiles. One limitation of this study is that the learning loss estimates are based on a simple comparison of equated scaled scores across time. The study does not take advantage of earlier rounds of national assessment data to more rigorously derive learning loss estimates. Another limitation is that, similar to other studies, its analysis of the links between ICT measures and learning is limited. Study objectives This report estimates the effects of COVID-19 on learning and earnings in Cambodia, analyzes the country’s EdTech readiness and the extent to which EdTech access and use are correlated with learning, and discusses the policy implications of the study findings for enhancing learning and improving system resilience through EdTech-based teaching and learning. First, complementing the findings of the study conducted by MoEYS and UNICEF (2022), it presents more rigorously derived estimates of learning loss experienced by grade 6 students in Khmer and mathematics using a regression model that makes use of learning assessment data from 2013 in addition to data from 2016 and 2021. It also analyzes the disparities in learning loss across genders, locations, and income groups. Additionally, it presents simulations of expected learning and earnings losses due to the pandemic using the latest, updated version13 of the World Bank simulation model. These simulations also estimate what the impact of COVID- 13 This updated model is discussed in Azevedo et al. (2022). 6 19 would have been in the absence of mitigation measures implemented by the government to provide indicative information on the contribution of these measures toward mitigating learning loss. Second, it documents the EdTech-based distance learning measures taken by Cambodia to mitigate learning loss and analyzes the extent to which such measures at the school level are correlated with student learning outcomes. Complementing the documentation of EdTech-based measures implemented in schools in practice with an analysis of policy provisions in support of EdTech, the report also discusses the state of EdTech in Cambodia and its EdTech readiness in terms of policy provisions as well as policy implementation at the school level. Third, it discusses the policy implications of these findings, providing suggestions for enhancing learning recovery and learning outcomes and addressing gaps in policy provision and implementation to support the scaling-up of EdTech for improving system resilience. Data sources and analytical approach The analyses of learning loss due to COVID-19 are based on secondary data from the 2013, 2016, and 2021 grade 6 NLAs implemented by the Education Quality Assurance Department (EQAD) of MoEYS. In each of the three rounds, schools and students were selected using a two-stage random sampling process where schools were randomly sampled in the first stage and students were randomly selected from each sample school in the second stage to create nationally representative samples of schools and students. The numbers of schools included in the NLA samples in 2013, 2016, and 2021 were 209, 228, and 230, respectively. Data on student learning outcomes and student and teacher characteristics available in the NLA datasets are used to estimate learning loss as represented by declines in learning outcomes. As the NLA 2021 was implemented by EQAD in November 2021, the student assessment data from this NLA round represent post-COVID-19 learning outcomes, while the assessment scores from earlier rounds are a proxy for pre-COVID-19 outcomes. The simulated estimates of learning and earnings losses, on the other hand, are based on school closure data (from UNESCO) and other data included in the simulation model of Azevedo et al. (2022). The analyses of the EdTech-based distance learning measures taken by Cambodia and the relationship between such measures and student learning outcomes utilize the 2021 NLA data as well as some primary data collected from school teachers and principals. These supplemental primary data, which include, among others, information on EdTech measures implemented at the school level, were collected in late November 2021 through an online survey of principals and teachers in the same schools included in the 2021 NLA sample.14 Hence, it is possible to link the principal and teacher responses from these surveys with student learning outcomes in their respective schools. In addition, information on EdTech readiness at the national and subnational levels was collected through a review of official documents and discussions with relevant government officials. Simple descriptive statistics and regression methods are used to analyze declines in student learning outcomes and the relationship between EdTech measures and student learning. A rigorous evaluation of the impact of COVID-19 on learning would require a comparison of changes in learning assessment scores between students randomly assigned to a treatment group (affected by COVID-19) and students assigned to a control group (not affected by COVID-19). Given the absence of such treatment and control 14 As noted above, both the supplemental principal survey and the teacher survey include the same sample of schools used in NLA 2021. However, while the principal survey includes information from all 230 schools in the NLA 2021 sample, the teacher survey includes responses from 462 teachers in only 222 out of the 230 sample schools (or 96.4 percent). The school-level characteristics of the full sample of 230 schools and the sample of 222 schools are largely similar: for example, there are no significant differences between these two groups of schools in the SES of students, the proportion of female students, and school type (private/public), and location (urban/rural). 7 groups, however, it is not possible to establish a causal relationship between COVID-19 and learning outcomes. As a second-best alternative, this study estimates the COVID-19-induced declines in learning outcomes using a regression model that compares the post-COVID outcomes with outcomes in 2016 while controlling for baseline outcomes in 2013. This model attributes all the observed changes between 2016 and 2021 to COVID-19. Details of the model are presented in Box 1.1. Report structure The rest of the report is organized as follows. Section 2 discusses the current state of learning outcomes in Cambodia, followed by a detailed analysis of the learning loss experienced by the country due to COVID- 19, as evidenced by the decline in student assessment scores in NLA 2021 compared to scores in 2016. This analysis is supplemented by learning and earnings loss estimates derived from the World Bank simulation model. Section 3 first briefly discusses the government’s response to the pandemic at the policy level, focusing on EdTech-based distance learning provisions to support learning at home. To provide an understanding of how well EdTech is integrated in the education system, this section then analyzes the EdTech-related policies and plans and their recent evolution and the extent to which they are reflected in the state of EdTech at the school level. Moving from the policy level to the service delivery level, Section 4 discusses the responses of schools to COVID-19 and the extent to which student learning outcomes in the 2021 NLA are correlated with EdTech-based distance learning interventions implemented by schools. Furthermore, using the simulation model, it also attempts to provide indicative estimates of the contribution of distance learning interventions toward mitigating learning loss. Section 5, the final section, summarizes the findings of this study and discusses their policy implications. Box 1.1: Regression model for estimating learning loss The assessment data analyzed in this study are pooled cross-section data, that is, data that come from a different representative sample of observations (schools and students) in each of the three assessment years. Thus, unlike panel data, which comprise observations taken over multiple periods for the same subjects, it is not possible to look at the changes in learning outcomes for the same set of schools and students between the pre-COVID-19 and post- COVID-19 periods. Furthermore, as separate treatment (COVID-affected) and control (non-affected) groups of randomly assigned schools and students do not exist, it is not possible to make a before-after comparison of affected and non-affected groups. In this context, a regression model which captures the average difference in outcomes between 2016 (pre-COVID) and 2021 (post-COVID) while controlling for past trends in average learning outcomes using 2013 data can be used to estimate learning loss. This estimate cannot, however, be interpreted as a causal effect of COVID-19 alone. In the simplest version of the model for the pooled cross-section data used in this study, the assessment score, ,, , for student i in school j and year t is modeled as ,, = 0 + 1 2016 + 2 2021 + ,, + ,, , where 2016 and 2021 are year-specific dummies, ,, is a matrix of control variables representing different student and household characteristics, and ,, is a random error term. The average change in score (or learning loss if the change is negative) between 2016 and 2021 is captured by 2 − 1 . The change in score between 2013 and 2016—or past trend in learning outcomes before the onset of COVID- 19—is given by the coefficient 1 , while the change in score between 2013 and 2021 is given by the coefficient 2 . The coefficient vector represents the average effects of the characteristics ,, on score. This model can be extended to estimate the average changes in scores for different groups of students by interacting the year-specific dummies 2016 and 2021 with any discrete characteristic ,. that defines the student groups. The extended model is as follows: ,, = 0 + 1 2016 + 2 2021 + 1 2016 × ,. + 2 2021 × ,, + ,, + ,, , 8 where ,. is a discrete characteristic in ,, such as gender, location, or income group; 1 is the difference in the effect of ,. in 2016 relative to 2013; and 2 is the difference in the effect of ,. in 2021 relative to 2013. For example, if ,. is a dichotomous variable representing gender (,. = 1 if student is a female and ,. = 0 if male), then (2 − 1 + 2 − 1 ) represents the average change in score for females between 2016 and 2021 and (2 − 1 ) represents the average change in score for males during this period. Ideally, a variable measuring the length of school closures as the treatment of interest would be used as this would give a concrete measure of the effects of learning interruptions. There are, however, two problems with this approach in the present context. First, even though self-reported data on the length of school closures in the 2021 round are available, this variable has a large incidence of omitted values (that is, no reporting) and little variance across schools. Second, and more importantly, school closure data were not collected for the 2013 and 2016 school samples, rendering the estimation approach just described unfeasible. Such an estimation would only be possible if panel data at the school level were available. In this paper, regression results will be presented for (a) rural/urban and male/female students, (b) the different wealth quintiles, (c) Cambodia’s four ecological zones (plain, Tonle Sap Lake, mountainous, and coastal areas), and (d) public versus private schools. 9 2. Current state of learning outcomes and learning loss due to COVID-19 2.1 Learning loss in the global context The estimation of COVID-19-induced learning losses is a dynamic field of research that is continuing to produce new evidence on the negative impact of the pandemic on student learning across the globe. The number of studies on this topic is rapidly increasing as new and better data become available. The first systematic studies on learning loss appeared in high-income countries using data from 2020. In one of these early studies, Maldonado and De Witte (2020) found statistically significant losses in learning outcomes— 0.19 standard deviation (SD) in math and 0.29 SD in Dutch—among six graders in the Dutch-speaking Flemish Region of Belgium. While the actual numbers differ, qualitatively similar results have been found for the Netherlands (Engzell, Frey, and Verhagen 2021), Switzerland (Tomasik, Helbling, and Moser, 2020), Germany (Schult and Lindner 2022), Italy (Contini et al. 2021), the United Kingdom (Juniper Education 2021, among others), and the United States (Kuhfeld, Soland, and Lewis 2022). More recently, in a review of 35 methodologically robust studies based on data from 20 mostly high- or higher-middle- income countries, Patrinos, Vegas, and Carter-Rau (2022) found evidence of learning losses ranging from 0.25 to 0.12 SD in 90 percent of the studies. Studies from lower-income countries are fewer and more recent and often come with more caveats given the lower quality of the data employed, but most of these too show that children experienced significant learning losses during the pandemic period. For example, a study from Mexico found losses of 0.34–0.45 SD in reading and 0.62–0.82 SD in math, although the results are based on two household surveys that are not strictly comparable (Hevia et al. 2022). Similarly, a study from Brazil also found large learning losses amounting to 0.32 SD (Lichand et al. 2022), and significant learning losses have been reported for South Africa as well (Ardington, Wills, and Kotze 2021; Shepherd et al. 2021). The MoEYS and UNICEF (2022) study on Cambodia also found learning losses of 0.3 SD and 0.60 SD in Khmer and math, respectively, among grade 6 students. Amin, Hossain, and Ainul (2021) report small learning losses (5–6 percent) among grade 7 and 8 students in Bangladesh when they were tested on grade 3 content. One almost universal finding in these studies as well as in studies from high-income countries discussed above is that learning losses are much more pronounced among poorer students. There is also some evidence in the literature that the pandemic has contributed to greater polarization of students into high- and low- performing groups. In Uganda, for example, Sandefur (2022) finds that during the pandemic, there was an increase in the proportion of students fully proficient in reading as well as those without any reading ability. Similar findings of polarization have been reported by Yarrow et al. (2022) in a study covering two provinces in the Republic of Korea (Seoul and Gyeongsangnam-Do), where pronounced increases were observed in the share of students in the highest and lowest-performing groups in math, Korean, and English. The methodology used for estimating learning loss in most of the learning loss studies can be grouped into two broad categories: (a) approaches based on a comparison of pre- and post-pandemic learning assessment scores and (b) projection or simulation-based approaches. As noted in Section 1, most of the studies use simulations or projection models to estimate learning loss as measured by changes in learning outcomes or in LAYS. Studies that estimate learning loss by comparing pre- and post-pandemic learning assessment data are fewer in number and vary in terms of methodological sophistication. The simplest estimates are based on a straightforward comparison of pre-post means using data collected without random sampling and are thus not generalizable to the population of interest. More rigorous pre-post mean comparisons utilize data collected using random sampling techniques. Other studies go beyond a simple comparison of means when estimating learning loss by using advanced statistical techniques to control for different factors that affect learning outcomes (Patrinos, Vegas, and Carter-Rau 2022). 10 The following section summarizes the post-COVID-19 state of learning outcomes, followed by an analysis of the learning loss experienced by Cambodian children due to the pandemic. Descriptive statistics are used to analyze the post-COVID-19 state of learning outcomes, including disparities in learning outcomes, based on the NLA 2021 data. The analysis of learning loss first discusses findings based on a simple comparison of mean equated scaled scores from the November 2021 NLA (that is, post- pandemic data) with the scores from 2016 (pre-pandemic data). This is followed by more rigorous estimates of learning loss using regression models that include controls for the baseline score in 2013 as well as other confounding factors (as elaborated in Box 1.1). While multiple factors might have contributed to the changes in learning outcomes between 2016 and 2021, it is assumed that the decline in learning outcomes during this period can primarily be attributed to the school closures induced by COVID-19. This analysis is supplemented by a discussion on the changes in estimates of two other measures of learning, namely LAYS and learning poverty rate, and two measures of earnings, using the learning loss simulation model from Azevedo et al. (2022). 2.2 Current state of student learning outcomes in Cambodia This analysis of the current state of learning outcomes in Cambodia is based on data from the NLA 2021, which was conducted by EQAD during a two-week period in November 2021. The NLA 2021 consisted of a nationally representative sample of over 6,000 grade 6 students in 230 schools across Cambodia. The sample was selected using a two-stage cluster sampling approach. In the first stage, the probability proportional to size method was used to select 200 public schools and 30 private schools. In the second stage, 30 grade 6 students were randomly selected from each school selected in the first stage. Paper- based tests were administered to these students in their respective schools to assess their performance in Khmer and math.15 This was the first national-level student assessment conducted in Cambodia after the onset of the COVID-19 pandemic. The assessment was accompanied by a student questionnaire, which was designed to collect information not only on the characteristics of the student but also on a rich array of household attributes (such as household assets, parental education, and occupation) and student learning activities during school closures, including student use of learning resources provided by schools during the pandemic. The student learning outcomes data for both Khmer and math in the NLA 2021 dataset include equated scaled scores16 as well as the percentage of questions answered correctly by the students. In addition, the dataset also includes percentage correct scores for each of the content areas in Khmer (reading, writing, and dictation) and math (numbers, measurement, algebra, geometry, and statistics). Overall, the learning outcomes of grade 6 students in both Khmer and math are relatively low. As shown in Figure 2.1, the average scaled score of these students in the NLA 2021 is 464 in Khmer and 440 in math. While these figures show that students performed better in Khmer than in math, they do not provide 15 Information on how the NLA assessment instruments were constructed, how they are aligned to the curriculum or country-level standards, and how their psychometric properties (reliability and validity) were reviewed and validated are presented in Annex. 16 Equating is a statistical process which generates comparable scores across multiple tests that share at least some common test questions. EQAD used a sequential approach to equate the scores. First, the 2007 and 2013 tests were equated by setting the baseline (2007) mean to 500. Then, the 2013 and 2016 tests were equated using the same procedure but with a baseline (2013) mean set at the level obtained from the 2007–2013 equating. Finally, the 2016 and 2021 data were equated, which does not rely on standard equating procedures since the tests are identical but uses the 2016 baseline mean (from the 2013–2016 equating) as the reference point (MoEYS and UNICEF 2022). It should be noted that the link between the 2013 and the later tests is weak since there are only a small number of test items common to both 2013 and 2016 rounds. The equating was done with technical assistance from UNICEF. 11 clarity on how well the students performed with respect to any benchmarks. Hence, to better understand the student performance levels, it is helpful to also look at the percentages of questions students answered correctly in the tests. According to this measure of performance, the students performed relatively poorly, scoring below 50 percent in each subject. More specifically, the average percentage correct scores for Khmer and math were only 47 and 38 percent, respectively. Student proficiency levels also tell the same story, with only about one-fifth of the students performing at the proficient or advanced levels in both Khmer and math. There is, however, a notable difference in Khmer and math proficiency levels. Almost three-quarters of the students (73 percent) perform below the basic level in math, whereas the corresponding share is much smaller for Khmer (44 percent).17 These relatively low performances in both domains are also consistent with the findings from the SEA-PLM 2019 regional learning assessment summarized in Section 1, which show that grade 5 students in Cambodia have low proficiency in reading and math. Figure 2.1: Student test scores in Khmer and math (2021) (a) Equated scaled (b) Percentage correct (c) Shares of students achieving scores different proficiency levels 500 100% 100% 1 8 19 13 480 80% 80% 6 464 460 60% 60% 36 47 440 38 440 40% 40% 73 420 20% 20% 44 400 0% 0% Khmer Math Khmer Math Khmer Math Below Basic Basic Proficient Advanced Source: Authors’ calculations using NLA 2021. There are significant disparities in learning outcomes between males and females and across age groups (Figure 2.2). Female students outperform male students in both Khmer and math, which is consistent with findings from earlier rounds of the NLA as well as findings from other countries in the region (Bhatta and Katwal 2022; UNICEF and SEAMEO 2020). The gender gap in performance is particularly large in Khmer, 47 points or about 0.5 SD.18 Similarly, significant differences in scaled scores are observed across age groups. Both Khmer and math scores are the highest for students who are at the ‘right’ age (12 years old) for grade 6, followed by underage students (11 years or younger). Overage students (13 years or older)—who make up the majority of students who have repeated at least one grade— have the lowest scores.19 The existing global literature suggests that older students tend to perform better than younger ones (Crawford, Dearden, and Meghir 2010; Dhuey et al. 2018); however, this does not seem 17 As the proficiency levels are not available for the subdomains, the subsequent analysis will focus on percentage correct and scaled scores. 18 The gender gap in math scores is also significant but much smaller —11 points or 0.13 SD. 19 Around 52 percent of the students in the sample are of the right age, that is, 12 years old; a relatively small share are underage (11 years or younger); and about one-third of the students are overage (13 years or older). It seems that while the majority of the overage children might have started schooling at a relatively older age, a significant share of them are overage because they repeated at least one grade. 12 to be true in case of the grade 6 students in Cambodia. The evidence rather suggests that the older age among these students is potentially accompanied by some learning deficiency or is a result of grade repetition.20 Figure 2.2: Student test scores by student demographic characteristics Source: Authors’ calculations using NLA 2021. Student schooling experiences—namely, preschool attendance, grade repetition, and regular attendance—can potentially have an impact on learning outcomes. Over 60 percent of the students reported having attended preschool before starting primary education (Figure 2.3). Female students have a slightly higher rate of preschool attendance than male students, and at-age students have a significantly higher preschool attendance rate compared to underage and overage students. Overall, more than one- quarter of the students had repeated grades at least once. As noted earlier, the incidence of grade repetition is slightly higher among male students than female students and significantly higher among overage students. Student absence21 among the grade 6 students is quite high, with half of the students missing class between one to six days and about 16 percent absent seven or more days during an academic year. 20 The incidence of grade repetition is significantly higher among overage students (43 percent versus 17–18 percent). The overage students who did not repeat any grade do somewhat better than grade repeating overage students. However, even when only those students who never repeated any grade are considered, the ‘at age’ students significantly outperform the overage children and the gap is even larger than the corresponding gap between grade repeaters and students who did not repeat any grade. 21 Student absence denotes the number of days the students were absent when the schools were in session for the three months at the beginning of the year and at the end of the year. 13 Figure 2.3: Percentage of students by preschool attendance, grade repetition, and annual absent days repetition attendance Grade Preschool Attended preschool 62 Did not attend preschool 38 Did not repeat any grade 73 Repeated grade (at least once) 27 Annual absent Never 33 days 1 to 6 days 50 7 or more 16 0% 20% 40% 60% 80% 100% Source: Authors’ calculations using NLA 2021. As expected, these schooling experiences are strongly correlated with the test scores in both subjects. Consistent with the recent regional findings, Cambodian grade 6 students who have attended preschool consistently outperform those who have not across both cognitive domains, and similarly, students who have never repeated any grade outperform those who have repeated at least one grade (Figure 2.4). Furthermore, there is strong evidence that regular school attendance makes a big difference in learning outcomes. Students who were never absent during the academic year outperform those who were absent at least seven days in the academic year by 60 points or approximately 0.63 SD in Khmer and 42 points or about 0.42 SD in math. Figure 2.4: Student test scores across student schooling experiences Source: Authors’ calculations using NLA 2021. There are large disparities in student performance across socioeconomic groups as well, with students from wealthier households significantly outperforming students from poorer households. As shown in Figure 2.5, student scaled scores—in both Khmer and math—are higher for each successive quartile of 14 the wealth index,22 and the gaps are quite substantial. The gap in performance between children from the wealthiest (top quartile) and the poorest households is quite large—71 points or 0.75 SD in Khmer and 57 points or 0.65 SD in math. Moreover, the household environment also seems to play a role in student performance. In particular, the test scores are significantly higher among students from households with books compared to students from households without any books (who constitute 40 percent of the students). Figure 2.5: Student test scores across household characteristics Source: Authors’ calculations using NLA 2021. Significant disparities in student performance are observed between urban and rural areas. The scaled scores are substantially higher among urban students, who make up about one-fifth of all grade 6 students, than among rural students (Figure 2.6). While the gap in both domains is sizeable, it is larger in Khmer (61 SD) than in math (43 SD). On the other hand, results on the differences in student test scores across the four ecological zones are mixed. In Khmer, students from the plains, Tonle Sap, and coastal regions have similar scores on average and outperform students from the mountainous areas. In math, however, students from the plains and the coastal regions have the highest scores, followed by Tonle Sap students, while students from the mountainous areas have the lowest scores. 22 The wealth index constructed for this study is based on the first principal components of an array of observable dwelling characteristics and ownership of consumer durables. These include if the home is a cement house, if the household owns a car, a refrigerator, a television, a radio, an air conditioning unit, and an electric or gas stove, whether the household access to clean water, whether the dwelling is connected to the electrical grid, and if the household has an exclusive toilet. By construction, the mean of the index is equal to zero in each year. 15 Figure 2.6: Student test scores across geographic regions Source: Authors’ calculations using NLA 2021. There are significant differences in student performance across the subdomains of each subject, with students performing particularly poorly in dictation and geometry. As shown in Figure 2.7, on average, students tend to perform better in Khmer than in math, as indicated by the higher percentage correct score in Khmer (47 percent) compared to math (38 percent). In the case of Khmer, student performance in reading is relatively good (67 percent correct) but is low in dictation (only 24 percent). While the variation in average student performance across subdomains is somewhat smaller in math, the percentage correct scores for geometry (35) and numbers (37), in particular, are significantly lower than for algebra (53). Figure 2.7: Percentage correct in Khmer and math subdomains Reading 59 Khmer Writing 47 Dictation 24 Khmer (overall) 47 Algebra 53 Statistics 47 Measurement 41 Math Numbers 37 Geometry 35 Math (overall) 38 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Source: Authors’ calculations using NLA 2021. While gender gaps in the different math subdomains are insignificant, females outperform males in all subdomains in Khmer, and urban students outperform rural students in all subdomains in both subjects. Among the Khmer subdomains, both gender and urban-rural gaps are the largest in writing compared to the other two subdomains. For instance, the urban-rural gap is 16 percentage points in writing, 11 percentage points in reading, and 12 percentage points in dictation (Figure 2.8). However, in relative 16 terms, the differences are most prominent in dictation, followed by writing, and lowest in reading.23 In the case of math, while the gender gaps in the overall score and various subdomains scores are relatively small, the gender gap in performance in Khmer is sizeable. Furthermore, there are significant urban-rural disparities in all subdomains. Notably, the urban-rural gap is much higher in statistics (15 percentage points) compared to the other four domains (7–9 percentage points). 23 It should be noted that the learning gaps are likely to be underestimated in math and reading because the tests are based mostly on multiple choice questions (on the other hand, writing and dictation are based on open-ended questions). Multiple choice questions can underestimate the true learning gap by creating a smaller range of achieved scores as the children can guess the answers (MoEYS and UNICEF 2022). 17 Figure 2.8: Percentage correct in Khmer and math subdomains by gender and location (a) Gender (b) Location (urban/rural) Reading Reading 100% 100% Math (overall) 80% Dictation Math (overall) 80% 68 Dictation 64 60% 60% 39 40% 54 44 40% 57 Algebra 28 Writing Algebra 34 Writing 37 60 37 20% 60 53 20% 53 51 52 22 44 19 40 0% 0% 44 45 41 44 34 33 57 52 59 36 48 41 36 Statistics 40 Khmer (overall) Statistics 35 Khmer (overall) 41 43 39 42 47 Geometry Numbers Geometry Numbers Measurement Measurement Male Female Urban Rural Source: Authors’ calculations using NLA 2021. 18 Student performance also varies by the difficulty of the test content and is better for urban students regardless of the test content difficulty level. The NLA test questions are rank ordered, from the easiest to the most difficult, into three categories: basic, proficient, and advanced. For each student, the NLA dataset also has information on the percentage of questions answered correctly in each of these categories. As expected, students perform relatively well (above 50 percent correct) in the basic content and struggle to perform well in content categorized as proficient and advanced. More specifically, they get only 25 percent of the answers correct in the advanced-level content in both Khmer and math, while the corresponding scores for basic content are at least two times higher (Figure 2.9). As in the case of subject subdomains, there are significant gaps in performance between males and females and between urban and rural areas, with urban students outperforming rural students at all levels of content difficulty in both subjects (Figure 2.10). In Khmer, gender and urban-rural gaps are the largest at the basic and proficient levels of test content difficulty. As seen earlier, while no significant gender gaps exist in math, urban-rural math gaps are substantial at all difficulty levels and are the largest at the basic and proficient levels. Figure 2.9: Percentage correct by content difficulty level Basic 65 Khmer Proficient 46 Advanced 25 Basic 53 Math Proficient 35 Advanced 25 0% 20% 40% 60% 80% 100% Source: Authors’ calculations using NLA 2021. 19 Figure 2.10: Percentage correct by content difficulty level by gender and location (a) Gender (b) Location (urban/rural) Khmer (Basic) Khmer (Basic) 100% 100% 76 80% 71 80% 60% Khmer 60% Math (Advanced) Math (Advanced) Khmer (Proficient) 40% 58 (Proficient) 40% 62 57 40 51 29 20% 43 25 20% 25 24 0% 0% 34 22 34 23 31 36 27 42 51 51 Khmer Math (Proficient) Khmer (Advanced) Math (Proficient) (Advanced) 60 54 Math (Basic) Math (Basic) Male Female Urban Rural Source: Authors’ calculations using NLA 2021. 20 2.3 Impact of COVID-19: learning loss as reflected in student learning outcomes Descriptive evidence The analysis below is primarily based on a simple comparison of mean student scores for grade 6 students from the 2021 NLA with NLA scores from the 2016 round. However, to put the results in perspective, the corresponding results for 2013 are also presented. Students were assessed on math and Khmer knowledge, and a subset of the questions was consistent across all three rounds, allowing for comparability in performance. As noted previously in footnote 10, the 2013 and 2016 NLAs only have a small set of common questions that could be used to create the equated scores. The results for the pre-trends are thus not perfectly comparable. It is striking, however, that the estimated mean scores show only a relatively small increase in student learning outcomes between 2013 and 2016, followed by a large drop between 2016 and 2021. All the declines in performance between 2016 and 2021 are highly statistically significant. Student performance declined significantly between 2016 and 2021, with losses in math larger than those in Khmer. Figure 2.11 presents the average national-level equated scaled scores in Khmer and math for the three years under study. It shows that, between 2013 and 2016, there were slight improvements in student performance in both subjects, with average scores increasing by 12.9 points in Khmer and 10.8 points in math. This slight upward trend in student performance was, however, sharply reversed between 2013 and 2021. During this period, student assessment scores declined by 42.8 points in Khmer and 56 points in math, representing substantial losses amounting to 8.4 percent and 11.3 percent, respectively. Figure 2.11: Student test scores at the national level, 2013–2021 520 506 494 496 500 485 Equated scaled score 480 464 460 440 440 420 400 Khmer Math 2013 2016 2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Learning losses in both subjects were more significant for males compared to females, and while there was relatively little urban-rural difference in learning loss in Khmer, losses in math were higher in urban areas relative to rural areas. Figures 2.12, 2.13, and 2.14 present the average scaled scores for both subjects across time for male and female students, rural and urban areas, and the four resulting subgroups separately. Figure 2.12 shows that the learning loss, on average, was higher for males than for females both in absolute and percentage terms—particularly in Khmer. Considering that males had lower learning outcomes in 2016, the larger learning losses they experienced have put them at a further disadvantage compared to females. Figure 2.13 shows a substantial decline in math scores in urban areas relative to rural ones—albeit from a much higher starting point. 21 Figure 2.12: Student test scores by gender, 2013–2021 (a) Khmer (b) Math 540 540 523 520 520 Equated scaled score 505 494 497 500 487 486 500 490 480 479 480 480 460 460 445 439 440 440 434 420 420 400 400 Male Female Male Female 2013 2016 2021 2013 2016 2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Figure 2.13: Student test scores by rural versus urban residents, 2013–2021 (a) Khmer (b) Math 580 561 580 560 546 552 560 540 537 540 Equated scaled score 520 511 520 500 491 485 500 480 476 480 480 469 460 453 460 440 440 432 420 420 400 400 Rural Urban Rural Urban 2013 2016 2021 2013 2016 2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Looking at the four subgroups separately, it is clear that learning losses in Khmer are lower for rural females compared to the other groups. For the first three groups—urban males, urban females, and rural males—the absolute loss in scaled score ranges between 44 and 51 points, which translates to losses of between 8.2 and 9.4 percent relative to the 2016 scores (Figure 2.14). The corresponding numbers are slightly lower for rural females at 33 points and 6.5 percent, respectively. This finding suggests that it is mainly rural females who were behind the lower learning loss in Khmer experienced by females as a whole. For math, the urban-rural divide in learning loss was substantial for both genders. The learning losses experienced by urban males and urban females were 86.9 points and 77.6 points, respectively. These translate to 15.7 and 14.1 percent, respectively. On the other hand, rural males lost on average 51 points, or 22 10.7 percent, and rural females lost 43.9 points, or 9.1 percent, on average. While in both cases, the losses for males are slightly higher than those for females, these differences are dwarfed by the urban-rural divide. Figure 2.14: Student test scores by gender and rural versus urban residents, 2013–2021 (a) Khmer (b) Math 574 580 566 580 554 Equated scaled score 560 546 560 550 550 540 527 526 540 525 520 509 520 495 496 500 500 481 483 472 471 476 472 470 476 480 480 467 460 460 439 440 427 440 425 420 420 400 400 Urban & Urban & Rural & Rural & Urban & Urban & Rural & Rural & Male Female Male Female Male Female Male Female 2013 2016 2021 2013 2016 2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Learning losses did not vary significantly across Cambodia’s four ecological regions. As shown in Figure 2.15, for Khmer, the average learning loss ranges from 43.2 to 49.2 points in the plain, mountainous, and coastal regions and was slightly lower in the Tonle Sap Lake region at 34.2 points. In relative terms, these losses fall within a relatively narrow range (between 8.5 and 9.6 percent) for the three regions, with the loss in the Tonle Sap Lake region (6.8 percent) falling a little outside this range. A similar pattern emerges for math scores, but the estimated losses are even less dispersed. More specifically, the loss for the Tonle Sap Lake region is equal to 48.3 points, which corresponds to almost exactly 10 percent. In the other three regions, learning loss ranges from 57.5 to 60.5 points or 11.7 to 12.0 percent. Figure 2.15: Student test scores by geographical regions, 2013–2021 (a) Khmer (b) Math 520 512 511 520 501 501 505 505 495500 Equated scaled score 500 488 500 491 492 485 484 485 480 480 473 471 467 466 467 460 452 460 446 445 435 440 440 428 420 420 400 400 Lake Plain Mountain Coast Lake Plain Mountain Coast 2013 2016 2013 2016 2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). While test scores are higher for wealthier households, students from better-off households suffered the highest learning losses in both absolute and relative terms. Figure 2.16 shows learning outcomes 23 over time by the wealth quintile of the student’s household. Consistent with the findings presented in Section 2.2, students from better-off households have progressively higher average learning scores for all years. However, students from wealthier households are also the ones who suffered the highest losses. The pattern that emerges for both math and Khmer are comparatively low losses for the lowest two quintiles, an intermediate loss for quintile 3, and the highest losses for quintiles 4 and 5. For Khmer, the two lowest quintiles lost on average 28.2 and 26.3 points, or 6.0 and 5.4 percent, respectively. The corresponding numbers for quintiles 4 and 5 are 47.2 points (9.3 percent) and 47.2 points (8.3 percent), respectively. For math scores, this pattern is still more salient: quintiles 1 and 2 lost 32.7 and 40 points (7.1 and 8.3 percent), respectively, while quintiles 4 and 5 lost 66.1 and 68.4 points (12.8 and 12.3 percent), respectively. This finding is surprising considering that studies from several other countries either show greater losses for lower SES students or do not show significant differences across SES groups (Schady et al. 2021). These results should, however, be interpreted with caution for two reasons. First, it is not possible to rule out the existence of floor effects since all questions in math and many questions in Khmer are of the multiple choice type, which limits the minimum number of points a student is expected to score when answering questions randomly. Second, as can also be seen in the figure, the larger loss for better-off children relative to poorer ones was preceded by larger gains for the former between 2013 and 2016. The results may thus be caused by a regression to the mean effect. That said, not finding larger learning losses for poorer children, especially in a non-high-income country, is still noteworthy. Figure 2.16: Student test scores by wealth quintile, 2013–2021 (a) Khmer (b) Math 580 566 580 557 Equated scaled score 560 533 542 560 540 519 540 530 511 516 520 507 520 489 493 495 496 489 500 467 483 500 461 480 481 464 470 462 469 464 480 480 457 450 460 439 460 440 442 440 440 425 420 420 400 400 Quintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 Quintile Quintile Quintile Quintile Quintile 1 2 3 4 5 2013 2016 2021 2013 2016 2021 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Learning losses were higher in private schools compared to public schools. Figure 5 presents the learning outcomes of students in public and private schools in the 2016 and 2021 national learning assessments.24 Assuming that better-off students are more likely to attend private schools, the results mirror the findings for wealth quintiles. Students in private schools have, on average, higher scores than those in public schools in both years, but they also suffered a higher learning loss during the pandemic. In Khmer, private school students lost 70.1 points, or 11.8 percent, while private school students lost 42.7 points, amounting to 8.5 percent. For math, the numbers are 84.6 and 55.8 points, corresponding to 14.4 and 11.3 percent, respectively. 24 Learning outcomes data for 2013 are not available since no private schools were sampled that year. 24 Figure 2.17: Student test scores by school type (public versus private), 2016–2021 (a) Khmer (b) Math 620 600 587 596 600 580 580 560 560 Equated scaled score 540 540 526 520 502 520 503 492 500 500 480 480 460 460 460 436 440 440 420 420 400 400 Public Private Public Private 2016 2021 2016 2021 Source: Authors’ calculations using NLA (2016 and 2021). Regression findings While the above discussion provides clear evidence of a decline in learning outcomes between 2016 and 2021, it is not possible to fully attribute the observed learning loss to the pandemic in the absence of properly constructed treatment and control groups as explained in Section 1. To partially mitigate this problem, the analysis below utilizes learning assessment data from the 2013 NLA to account for pre- 2016 trends relative to the change in learning outcomes between 2016 and 2021 to provide more rigorous estimates of learning loss. The regression model used to do this analysis is summarized in Box 1.1. The 2016 and 2021 rounds of the survey followed a common sampling scheme but constituted two independently drawn samples. The 2013 round employed a different sampling frame but is still representative at the national level. This implies that a larger, yet unknown, proportion of the changes in outcomes between 2013 and 2016 can be expected to be the result of sampling variance compared to those between 2016 and 2021. However, the sheer size of the observed effects over 2016–2021 compared to the 2013–2016 period suggests that the bulk of the observed decline in learning outcomes can be attributed to the disruptions caused by the pandemic. Summary statistics of the different variables used in the regression model for the three years show some changes in these variables between the 2013 and the 2016 and 2021 rounds. Table 2.1 presents unweighted summary statistics for the variables included in the regressions. Consistent with the graphs discussed above, it shows that the average test scores increased slightly between 2013 and 2016 and dropped sharply in 2021. The remaining variables are used either as pure control variables or also as interaction terms in the regressions. There were some notable changes in the relative shares of students living in urban and rural areas and in the different ecological zones over these three years. While less than 17 percent of students lived in urban areas in 2013, around 24 percent did so in 2016 and 2021—reflecting the different sampling frame in the first year. All three samples were almost evenly split between male and female students. As no private schools were sampled in 2013, there are no data for this variable in 2013, but the percentage of private school students increased from around 7 percent in 2016 to around 13 percent in 2021. The coastal region 25 was the least represented zone, with between 6 and slightly over 8 percent of the sample. The region around the Tonle Sap Lake accounted for a stable proportion of close to 30 percent of the sample. There were, however, some changes in the representation of the central plains and the mountainous areas between 2013 and the following two rounds. Students from the mountainous area account for 36 percent in 2013 and only for 13 to less than 15 percent in 2016 and 2021. Similarly, students from the central plains represented only 26 percent in 2013 but were the largest group, accounting for about half the sample in 2016 and 2021. There were also some changes in student characteristics and household demographic and socioeconomic characteristics between 2013 and the latter rounds. The average annual student absentee days increased slightly in 2021 compared to the first two rounds, from a little over two to three days on average. The percentage of students living with both parents decreased over time from 85 to 80 percent, possibly reflecting broader social changes. There was a sharp decline in the percentage of students whose mother or father or both were working as farmers, from 65–80 percent in 2013 and 2016 to around 25–26 percent in 2021. Grade repetition increased drastically from less than 5 percent in the first two rounds to about one-quarter (25 percent) of students in the 2021 round. The percentage of households that have at least one book was very small in 2013 (around 7 percent) but had increased to 57 and 63 percent in 2016 and 2021, respectively. Lastly, as explained earlier in footnote 22, the mean for the wealth index is the same (zero) for all years by design. Table 2.1: Summary statistics 2013 2016 2021 Obs. Mean SD Obs. Mean SD Obs. Mean SD Khmer score 5,967 495.52103.86 6,379 509.66 102.29 6,007 470.88 95.59 math score 5,967 486.57101.11 6,379 498.99 103.71 6,007 445.79 91.10 Urban 5,932 0.16760.3735 6,341 0.2438 0.4294 5,974 0.2375 0.4256 Female 5,751 0.54980.4976 6,396 0.5441 0.4981 5,934 0.5340 0.4989 Private 0 6,412 0.0689 0.2534 6,013 0.1287 0.3349 Lake 6,006 0.2867 0.4523 6,412 0.2941 0.4557 6,013 0.2890 0.4534 Plain 6,006 0.2616 0.4395 6,412 0.4652 0.4988 6,013 0.5112 0.4999 Mountain 6,006 0.3613 0.4804 6,412 0.1463 0.3534 6,013 0.1334 0.3400 Coast 6,006 0.0783 0.2686 6,412 0.0833 0.2763 6,013 0.0599 0.2373 Age 5,822 12.7743 1.2055 6,297 12.4909 1.2347 5,934 12.3089 0.9937 Annual absentee days 5,542 2.1474 3.4617 6,200 2.2044 3.6793 5,931 3.2655 4.6992 Lives with both parents 5,945 0.8506 0.3565 6,263 0.8293 0.3763 5,692 0.8011 0.3992 Mother is farmer 5,801 0.8000 0.4000 5,971 0.7361 0.4408 5,340 0.2427 0.4288 Father is farmer 5,635 0.7285 0.4448 5,994 0.6508 0.4768 5,206 0.2655 0.4416 Has repeated grades 5,831 0.0352 0.1842 6,253 0.0472 0.2120 5,467 0.2532 0.4349 Household owns books 5,866 0.0733 0.2607 6,187 0.5662 0.4956 5,677 0.6318 0.4823 Wealth index 5,643 0.0000 1.7840 5,644 0.0000 1.8261 4,292 0.0000 1.7528 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Regression results show a sharp decline in test scores between 2016 and 2021 for both Khmer and math at the national level. Table 2.2 presents the aggregate national-level trends in Khmer and math test scores over 2013–2021. It shows that the average test scores increased slightly between 2013 and 2016 only to be followed by a sharp drop in 2021.25 As the baseline year is 2013, the drop in score between 2016 and 2021—or the learning loss during this period—is computed as the difference between the coefficient for Dummy 2021 and the coefficient for Dummy 2016. Thus, the estimated learning loss between 2016 and 2021 is 42.76 points in Khmer when control variables are not used in the regression and 27.73 points when control variables are included. The estimate without control variables is the same as the Khmer learning 25 It should, however, be kept in mind that a different sampling frame was used for 2013. 26 loss estimate obtained from Figure 2.11. For math, the estimated learning loss is 55.96 points without control variables and 41 with control variables. The inclusion of control variables thus attenuates the effects slightly. This can be either because the inclusion of control variables corrects for some of the sampling variation between the 2016 and 2021 rounds or it may reflect some selection effects in the sample due to the reduced number of observations.26 Table 2.2: Aggregate national-level estimates of learning loss using regression models Khmer Math Dummy 2016 12.88955** 12.87816** 10.75947 9.15560 6.033 5.717 6.748 6.803 Dummy 2021 −29.87420*** −14.85170** −45.20437*** −31.84656*** 5.879 6.105 6.050 6.829 Controls No Yes No Yes Observations 18,353 12,837 18,353 12,837 R-squared 0.033 0.187 0.062 0.145 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Note: Results show parameter estimates of linear regressions. Robust standard errors are shown below the parameter estimates. Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. There are few systematic differences in the effect of the pandemic on test scores between urban and rural and male and female students. The regression results in Table 2.3 show learning loss estimates for the following four groups of students: urban males, rural males, urban females, and rural females. The baseline category is the rural male group. The learning loss estimates for this baseline group (rural males) are simply the differences between the coefficients for Dummy 2021 and Dummy 2016 and are similar in magnitude to the estimates obtained for the whole sample. For example, the estimated learning loss for rural males in Khmer is 43.99 points without controls and 27.63 points with controls. Comparing the various parameter point estimates in 2016 and 2021 for the different interaction terms, it is striking that they are quite similar in magnitude. This implies that learning loss does not differ much across groups. Table 2.3, which summarizes the estimated learning losses for the various groups, shows that the learning losses for the four groups were indeed quite similar and ranged from 0.25 to 0.3 SD for Khmer and a little more for math in the specifications with control variables. The sole exceptions are the urban interaction terms for math scores. In this case, for both specifications (with and without control variables), urban students lost over 30 additional points between 2016 and 2021. As shown in Table 2.7, this additional loss doubles the math learning loss estimates for both males and females in urban areas to over 0.6 SD in the specification with control variables. Table 2.3: Estimates of learning loss for rural and urban areas, and male and females Khmer Math Dummy 2016 −1.85172 2.84309 6.13997 9.53216 6.585 6.633 7.554 7.974 Dummy 2021 −45.83937*** −24.79015*** −44.80211*** −25.82199*** 6.704 7.686 6.641 7.97 Urban x Female x Dummy 2016 −25.56605** −28.37286** −24.46096** −28.83620** 11.988 11.806 12.364 12.515 Urban x Female x Dummy 2021 −32.62975*** −38.80469*** −22.42419** −23.66020* 26 When including control variables, a total of 5,516 observations were lost (because they do not contain information on at least one of them). If this loss of observations is close to random, the results are still consistent and changes can be attributed to controls previously acting as omitted variables. However, if the loss of observations is determined by unobserved factors that also have an effect on the outcome, the results will be biased. 27 Khmer Math 11.586 11.783 11.368 12.777 Urban x Dummy 2016 20.70054 14.83641 23.11186 19.93699 15.773 15.044 17.034 15.728 Urban x Dummy 2021 13.64269 11.638 −12.82393 −12.5759 14.653 14.025 15.008 14.477 Female x Dummy 2016 14.57312*** 17.91929*** −4.20454 −3.50861 4.803 5.278 5.38 5.569 Female x Dummy 2021 25.51842*** 21.55488*** 2.8835 −1.76198 5.143 6.134 4.794 5.903 Urban x Female 15.28974 21.18325** 14.68842 18.39232* 9.978 9.63 9.642 9.608 Urban 54.54744*** 22.14262* 54.22117*** 22.21584* 12.561 12.51 12.526 11.801 Female 23.56081*** 20.99795*** 10.30483*** 10.49629** 3.718 3.919 3.801 4.122 Dummy 2016 −1.85172 2.84309 6.13997 9.53216 Controls No Yes No Yes Observation 17,862 12,837 17,862 12,837 R-squared 0.129 0.189 0.117 0.149 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Note: Results show parameter estimates of linear regressions. Robust standard errors are shown below the parameter estimates. Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. Comparing Cambodia’s four ecological zones, there seems to be some degree of convergence in the math scores in the sense that the originally more disadvantaged regions suffered a smaller learning loss during the pandemic. In Table 2.4, the omitted category is the Tonle Sap Lake area, which includes the capital Phnom Penh. In the regression results, we find that students in the plains and the coast suffered larger learning losses than those in the Tonle Sap Lake area. The results for mountains are mixed and point to slightly larger learning losses. Also, at baseline (in 2013), the Tonle Sap Lake area had the highest overall test scores. Table 2.4: Estimates of learning loss for different ecological zones Khmer Math Dummy 2016 0.06281 5.09905 −7.17285** −6.31836* 3.447 3.754 3.338 3.826 Dummy 2021 −34.24463*** −15.69672*** −55.30400*** −38.69651*** 3.561 4.351 3.253 4.324 Plain x Dummy 2016 24.46100*** 13.61004*** 38.94574*** 29.23868*** 4.683 5.017 4.419 4.915 Plain x Dummy 2021 12.60429*** 1.55277 27.92622*** 16.13587*** 4.759 5.329 4.279 5.209 Mountain x Dummy 2016 4.74708 8.22878 0.98340 12.82528** 5.202 5.459 5.274 5.904 Mountain x Dummy 2021 −8.81031 5.20089 −8.37995* 6.13707 5.529 6.418 5.043 6.389 Coast x Dummy 2016 25.87329*** 9.24626 41.71455*** 25.21196*** 6.949 7.168 7.235 7.777 Coast x Dummy 2021 17.01363** −8.39654 29.32237*** 4.36536 7.500 8.097 7.536 8.505 Plain −11.75911*** −16.33365*** −18.08833*** −19.11479*** 3.562 3.956 3.329 3.810 Mountain −4.64080 −12.48792*** 0.57128 −8.68295** 28 Khmer Math 3.366 3.659 3.393 3.769 Coast −14.79641*** −7.59448 −20.51385*** −11.58632** 5.181 5.402 5.235 5.540 Controls No Yes No Yes Observations 18,353 12,837 18,353 12,837 R-squared 0.036 0.190 0.070 0.148 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Note: Results show parameter estimates of linear regressions. Robust standard errors are shown below the parameter estimates. Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. Students in the lowest wealth quintile suffered less from learning loss compared to students in the top four quintiles. It is of particular concern to assess how learning losses affected different socioeconomic groups as represented by wealth quintiles. To keep Table 2.5 to a manageable size, only the dummy variable for living in a household in the lowest quintile is included and interacted with the two year-specific dummy variables.27 The table shows a large positive and statistically significant difference between 2016 and 2021 for the lowest quintile relative to the top four quintiles in math scores. The differences for Khmer are somewhat lower but still positive. The upshot is that children from households in the lowest quintile suffered lower learning losses than those from the other households though their test scores are lower overall. However, as noted above, this may be an artifact of either floor effects or a regression to the mean. On the latter point, the table also shows that lower loss between 2016 and 2021 is driven by lower gains between 2013 and 2016. Table 2.5: Learning loss for lowest wealth quintile Khmer Math Dummy 2016 20.85630*** 14.71347** 18.96147*** 12.63144* 6.161 5.737 7.055 6.836 Dummy 2021 −20.11468*** −15.40027** −37.65774*** −32.14272*** 6.101 6.199 6.542 7.012 Quintile 1 x Dummy 2016 −17.97602* -8.89793 −22.04186** −15.49092 9.403 9.462 9.959 10.037 Quintile 1 x Dummy 2021 −5.22134 2.13354 1.88364 3.03103 8.889 9.043 8.907 9.464 Quintile 1 −39.04269*** −0.19582 −32.34980*** 10.29454 6.744 6.847 7.091 7.005 Controls No Yes No Yes Observations 15,520 12,837 15,520 12,837 R-squared 0.063 0.187 0.075 0.147 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Note: Results show parameter estimates of linear regressions. Robust standard errors are shown below the parameter estimates. Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. Private school students suffered larger learning losses than students at public schools. The results for public and private schools reflect the findings for the wealth quintiles. As shown in table 2.6, it is estimated 27 A more detailed analysis for all five quintiles shows qualitatively similar results: similar, comparatively low, learning losses in the two bottom quintiles; similar larger learning losses in the two highest quintiles; and intermediate learning losses in the third quintile. 29 that students in private schools lost an additional 17 points in Khmer and 21 points in math (after controlling for student characteristics).28 Table 2.6: Learning loss for public and private schools Khmer Math Dummy 2021 −42.66401*** −25.04131*** −55.81841*** −38.02021*** 5.384 4.999 5.459 5.411 Private x Dummy 2021 −27.47328* −18.97921 −28.74964 −23.99791 15.063 12.002 21.096 19.792 Private 93.09075*** 34.39877*** 94.63307*** 48.06999*** 12.376 10.150 18.035 16.468 Controls No Yes No Yes Observations 12,386 8,311 12,386 8,311 R-squared 0.072 0.229 0.109 0.175 Source: Authors’ calculations using NLA (2016 and 2021). Note: Results show parameter estimates of linear regressions. Robust standard errors are shown below the parameter estimates. Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. Learning losses between 2016 and 2021 were large in absolute terms as well as relative to standard deviations of scores in 2016. Table 2.7 provides an overview of the different results discussed above. It shows for each subgroup and specification the estimated learning loss between 2016 and 2021 in terms of average scores and relative to the standard deviation of the score for each group in 2016. The 2016 figures, as opposed to a pooled measure comprising outcomes from 2016 and 2021, are used because the standard deviation of test scores itself was likely affected by the pandemic. As can be gleaned from the previous tables, the differences in scaled scores between the two years are very large. Typical losses are between 30 and 40 points. More strikingly, these losses correspond to 0.3 to 0.5 SD of the respective 2016 score. In some cases, the estimated learning losses are even higher. Another result is that the size of the learning loss is reduced when control variables are included.29 Table 2.7: Consolidated overview of learning loss findings from the regression models Khmer Math No Controls Controls No Controls Controls Aggregate −42.76 −27.73 −55.96 −41.00 −0.4201 −0.2803 −0.5476 −0.4046 Rural and male −43.99 −27.63 −50.94 −35.35 −0.4488 −0.2823 −0.5168 −0.3518 Rural and female −33.04 −24.00 −43.85 −33.61 −0.5609 −0.3493 −0.7935 −0.6300 Urban and male −51.05 −30.83 −86.88 −67.87 −0.3407 −0.2564 −0.4652 −0.3612 Urban and female −47.16 −37.63 −77.75 −60.94 −0.5391 −0.4554 −0.7664 −0.6317 Lake −34.31 −20.80 −48.13 −32.38 28 With the caveat that the sample size for private schools is small and pre-2016 trend data from 2013 are not available for private schools, 29 Statistical significance is not shown in Table 2.7. Almost all results are highly statistically significant at levels much below the 1 percent threshold. Only results for the mountainous region with control variables are not statistically significant at the 5 percent level. A handful of estimates fall between the one and 5 percent significance levels. 30 Khmer Math No Controls Controls No Controls Controls −0.3284 −0.2110 −0.4842 −0.3373 Plain −46.16 −32.85 −59.15 −45.48 −0.4499 −0.3216 −0.5718 −0.4441 Mountain −47.86 −23.82 −57.49 −39.07 −0.4958 −0.2518 −0.5531 −0.3671 Coast −43.17 −38.44 −60.52 −53.22 −0.4771 −0.4473 −0.5909 −0.5302 2nd to 5th quintile −40.97 −30.11 −56.62 −44.77 −0.4189 −0.3113 −0.5509 −0.4405 1st quintile −28.22 −19.08 −32.69 −26.25 −0.2922 −0.1983 −0.3668 −0.2922 Public −42.66 −25.04 −55.82 −38.02 −0.4223 −0.2546 −0.5569 −0.3827 Private −70.14 −44.02 −84.57 −62.02 −0.9178 −0.5785 −0.7816 −0.5717 Source: Authors’ calculations using NLA (2013, 2016, and 2021). Note: For each group in the first column, the numbers in the first row show differences in assessment scores. The numbers in the second row show the differences in terms of standard deviations relative to 2016 outcomes. 2.4 Impact of COVID-19: changes in LAYS, learning poverty, and earnings The above evidence on declines in learning outcomes is also supported by retrospective simulations of learning and earnings losses obtained from the simulation model of Azevedo et al. (2022), which utilizes observed data on school closure as a critical input. Responding to the COVID-19 pandemic, most East Asia and Pacific (EAP) countries closed schools either fully or partially for extended periods between February 2020 and February 2022. School closure data from UNESCO show that schools were either fully or partially closed for more than 75 days in 11 countries and over 50 days in 15 countries during this period (Figure 2.18). In total, 16 countries experienced school closures (full or partial) of 100 days or more, and except for Japan, the countries where schools were closed for shorter periods all belonged to the Pacific region. All Southeast Asian countries apart from Singapore kept their schools closed for particularly long periods. With schools closed for 532 days (fully closed for 280 days and partially closed for 253 days), Cambodia stands out as the country experiencing the third highest number of school closure days in EAP during these two years. Globally it ranks among the top 10 percent of countries that experienced the highest number of school closure days. 31 Figure 2.18: Number of days of school closure in EAP due to COVID-19 (February 19, 2020, to February 23, 2022) Australia 0 434 308 Marshall Islands 77 728 Tonga 0 14 728 Vanuatu 217 714 Kiribati 0 21 721 Pacific Solomon Islands 28 21 693 Samoa 280 714 Tuvalu 35 49 658 Papua New Guinea 42 0 700 Palau 56 0 686 New Zealand 56 203 483 Fiji 364 7 371 Japan 28 56 658 East asia (other) China 63 126 553 Mongolia 168 322 252 Singapore 28 84 630 Vietnam 49 245 448 Timor-Leste 77 203 462 Lao PDR 308 336 South-east asia 98 Thailand 126 224 392 Indonesia 147 483 112 Brunei Darussalam 259 56 427 Cambodia 280 252 210 Malaysia 301 224 217 Philippines 511 7 224 Myanmar 532 77 133 0 100 200 300 400 500 600 700 800 Number of days Fully closed due to Covid-19 Partially closed due to Covid-19 Open, including scheduled breaks Source: UNESCO estimates included in the simulation model by Azevedo et al. (2022). Based on evidence from the literature, the simulation model assumes that school closures result in learning losses in two ways. First, in the absence of face-to-face instruction at school, the quality of education deteriorate, and children learn less, and second, the break in the continuity of learning causes children to forget what they have learned in the past. This loss in learning is reflected in reduced LAYS for the current cohort of school children. In addition, the model assumes that income shocks to households can result in an increase in school dropouts as children disengage from studies to work or because schooling becomes unaffordable, which in turn contributes to a further reduction in LAYS. Apart from estimates of losses in LAYS, the simulation also allows for the estimation of the impact of the pandemic on the learning poverty rate, that is, the percentage of 10-year-olds who cannot read and understand a short passage of age- appropriate material.30 Using the estimated loss in LAYS and information on returns to education, life expectancy, labor market earnings, and people’s ability to utilize their skills through paid employment, the 30 The World Bank defines learning poverty rate as “the percentage of 10 -year-olds who cannot read and understand a short passage of age-appropriate material—in other words, those who are below a ‘minimum proficiency’ threshold for reading” (Azevedo et al. 2021, 5). 32 model also estimates the loss in earnings (Azevedo et al., 2020). It should be noted that while the pandemic is expected to have resulted in learning loss through school closures as well as income shocks, it is assumed that school closures are, by far, the more important factor. The simulation results discussed below include the impacts of the pandemic on two indicators of learning (LAYS and learning poverty) and two indicators of earnings (individual earnings and economy wide earnings) under three scenarios. The scenarios, which are based on assumptions regarding partial school closures and the effectiveness of measures implemented by the country to mitigate learning loss during the pandemic, are as follows: (a) optimistic scenario (75 percent of the schools are closed and mitigation effectiveness is 14 percent), (b) intermediate scenario (85 percent of the schools are closed and mitigation effectiveness is 7 percent), and (c) pessimistic scenario (100 percent of schools are closed and mitigation effectiveness is 7 percent). The above assumptions for mitigation effectiveness are based on the assumptions used by Azevedo et al. (2022) for LMICs.31 For the estimation of earnings loss, a work life of 45 years and a discount rate of 3 percent are assumed under all three scenarios.32 The simulation results indicate that, as a result of the pandemic, Cambodia will experience a decrease in average LAYS for the current cohort of school-age children by 1.40 to 1.76 years (Figures 2.19a). Considering that the pre-COVID-19 baseline LAYS estimate for Cambodia was 6.8 years, the LAYS losses represented by these figures are quite significant in percentage terms, ranging from 21 percent in the optimistic scenario to 24 percent in the intermediate scenario to 26 percent in the pessimistic scenario. They also compare unfavorably with the LAYS loss estimates for the average of EAP LMICs (0.86 to 1.03 years). It should also be noted that an average loss in LAYS of 1.46 to 1.76 years implies that the children learned almost nothing during the period of school closure and that many children may, in fact, have regressed (that is, forgotten some of what they learned previously). The simulation also shows that, as a result of the pandemic, Cambodia is expected to experience a jump in the learning poverty rate from an already severe pre-COVID-19 baseline of 90 percent to 99.7 percent, even in the optimistic scenario (Figure 2.19b). The above loss in LAYS translates to an expected average annual learning loss of US$536–642 per student and lifetime earnings loss of US$23–28 billion (in 2017 PPP US$) in present value terms to the economy (Figures 2.20a and 2.20b). This represents a decline in average annual earnings per student of 8.8 to 10.6 percent compared to a baseline of US$6,077 per year. The corresponding present value of individual lifetime earnings loss per student ranges from a low of US$9,770 for the optimistic scenario to a high of US$11,720 for the pessimistic scenario. As shown in Figure 2.3a, the above average annual earnings loss per student for Cambodia is significantly higher than that for the average of EAP LMICs. Using a baseline student enrollment of 3.3 million and a factor of 0.70 to adjust for labor force participation and adult survival rates to aggregate the individual lifetime earnings losses across the students, the model estimates that the economy wide earnings loss will range from US$23 billion to US$28 billion, which is equivalent to 32.3 percent to 38.8 percent of the GDP of Cambodia in 2019 (in 2017 PPP US$).33 Hence, 31 The Azevedo et al. (2022) model assumes that mitigation effectiveness varies by income level and scenario. The assumptions for mitigation effectiveness used in the simulations done for this study are further discussed in Section 4.3. 32 In addition, the model also assumes that expected learning gains vary by income level, where LMICs, including Cambodia, gain 30 HLO points per year. The corresponding assumed HLO gains per year for higher-income countries, upper-middle-income countries, and lower-income countries are 50, 40, and 20 HLO points, respectively. HLO is a learning outcomes index based on international learning assessments that is comparable across countries. It should be noted that the simulation model also estimates the percentage of the school system closed during the period under consideration. These estimates exclude scheduled breaks as a part of the school year. 33 The GDP of Cambodia in 2019 was 72.36 billion in 2017 PPP US$ (World Development Indicators database). 33 in the absence of concerted efforts to help students recover their lost learnings, the long-term impacts on the economy could be quite severe. Figure 2.19: Loss in learning adjusted years of schooling and increase in learning poverty (a) Loss in LAYS under different scenarios (b) Learning poverty in the optimistic scenario 99.7 100.0 100.0 2.18 100.0 Learning poverty rate (%) 2.30 97.7 79.0 1.76 2.10 80.0 90.0 89.5 1.80 Years lost 1.91 70.9 1.65 1.30 60.0 1.30 1.46 1.03 1.16 40.0 0.96 0.80 0.98 20.0 0.86 0.30 0.0 Cambodia Lao PDR Myanmar EAP LMIC Cambodia Lao PDR Myanmar EAP LMIC Average Average Intermediate Optimistic Pessimistic Optimistic Pre-COVID learning poverty Source: Simulation model from Azevedo et al. (2022). Figure 2.20: Loss in earnings (a) Loss in average annual earnings per student (b) Present value loss to economy lifetime earnings under different scenarios (2017 PPP US$) under different scenarios (2017 PPP US$ billion) 750 60 B 53 B Decrease in earinings ($ bil.) Decrease in earinings ($) 700 698 50 B 51 B 650 642 623 46 B 600 602 40 B 572 550 549 536 530 28 B 500 501 496 30 B 26 B 450 461 20 B 23 B 400 407 350 7B 10 B 6B 300 5B Cambodia Lao PDR Myanmar EAP LMIC 0B Average Cambodia Lao PDR Myanmar Intermediate Optimistic Pessimistic Intermediate Optimistic Pessimistic Source: Simulation model from Azevedo et al. (2022). 34 3. EdTech-based policy response to COVID-19 and Cambodia’s EdTech readiness 3.1 Government response to COVID-19 Following the closure of schools on March 16, 2020, in response to the pandemic, the Cambodian government prepared a systematic response plan for the education sector to mitigate the impacts of disruptions caused by COVID-19. Initially, however, MoEYS developed key instructions, guidelines, and directives to immediately offer distance learning and teaching to students from the preprimary to the secondary levels. These included a directive on distance learning and e-learning for preprimary, primary, and secondary school education issued in April 2020, an operational guideline for distance learning implementation issued in June 2020, and guidelines on e-training for teacher education also issued in June 2020 (MoEYS 2020a). At the same time, to increase the availability of distance learning resources, the government also initiated the development of online lessons for students in grades 9 to 12 and other digital content for lower grades, including preschool. By June 2020, MoEYS had produced over 1,303 video lectures and 270 radio spots in three languages. MoEYS also developed paper-based student worksheets for grades 1 to 12 and instructed schools to print and distribute them to support students’ remote learning. The response plan, entitled Cambodia ‘Education Response Plan to the COVID-19 Pandemic’, was finalized and formally endorsed by the Minister of Education, Youth, and Sports in July 2020. The response plan aimed to ensure the continuity of teaching and learning and increase system resilience. The plan’s specific objectives were to (a) ensure that staff and students are able to continue remote teaching and learning safely, return to education institutions safely, and teach and learn in an adaptable learning environment and (b) increase the resilience of the MoEYS system. Distance learning using EdTech is one of the key elements of the activities proposed in the plan to meet the objective of ensuring continuity of remote teaching and learning. Strengthening distance learning was also at the core of enhancing the education system’s resilience. One of the sub-objectives of the plan was to ensure that students have access to distance learning through digital technology, radio, and TV. The activities proposed in the plan to support this sub- objective included the development of educational videos, TV programs, and radio programs for various grades and subjects and their delivery through the internet (online streaming, internet platforms, and so on) as well as TV and radio broadcasts. The proposed activities also included the procurement of necessary equipment for these purposes and the development of an MoEYS website through which the different distance learning resources could be readily accessed. Consistent with the proposed activities in the response plan, MoEYS used both traditional channels (TV and radio broadcast) as well as modern web/app- based channels (Facebook pages, YouTube channels, websites, and mobile applications) to broadcast distance learning content during the pandemic (MoEYS 2020b, 2021a). Table 3.1 lists some examples of the channels used by MoEYS for this purpose (see Table A.2 for a comprehensive list of remote learning channels, including their target grades and reach). Video lectures, which targeted all grades from preschool to the secondary level, were initially accessible only through Facebook pages, YouTube channels, and websites but were later broadcast on TV channels as well (MoEYS 2021a). Radio programs were targeted toward parents and students at the preschool level and in multilingual education programs for grades 1 to 3 (MoEYS 2020a). In addition to video lectures, live question and answer sessions were conducted on Facebook to support grade 9 and 12 students to prepare for the national exams (MoEYS 2020a). 35 Table 3.1: Streaming and broadcasting channels Channel type Examples Traditional: TV and radio TVK2, Decho DTV, Apsara TV, Wiki TV, One TV Web/app based: Facebook pages MoEYS, Early Childhood Care and Development (ECCD) Cambodia, Komar Rien Komar Ches, Krou Cambodia YouTube MoEYS Cambodia Websites elearning.moeys.gov.kh, krou.moeys.gov.kh Mobile applications MoEYS E-Learning App, Open Education Resources (OER) Cambodia Source: MoEYS 2020b. Telegram and other mobile messaging apps were promoted as the key tools for interaction among schools, teachers, and students during school closure. MoEYS instructed schools and teachers to create messaging groups on Telegram or other mobile messaging applications to communicate with students and parents about distance learning and e-learning and facilitate these learning activities (MoEYS 2020b). Teachers were also directed to send video lectures, lesson summaries, worksheets, and responses to questions about lessons from students via Telegram (MoEYS 2021b). Thus, teachers were expected to communicate regularly with students to support their distance learning through these messaging groups (MoEYS 2021a). Moreover, teachers were encouraged to use Zoom and Google Meet to teach online (MoEYS 2021b). While these ICT-based responses to the pandemic have helped to partially mitigate the impacts of COVID-19-induced school closures, the extent to which EdTech can be used to enhance system resilience and education quality in Cambodia depends largely on the policy provisions supporting EdTech and their implementation at the school level. The following sections provide an overview of the evolution and coverage of plans and policies related to EdTech, areas where there is strong policy support for EdTech and areas where there are gaps, and the extent to which these plans and policies are being reflected in the state of EdTech at the school level. They also briefly discuss the challenges perceived by schools in meeting the outcomes desired by these policies and plans. Thus, this discussion sheds light on the EdTech readiness of Cambodia, that is, the extent to which the country is prepared to systematically integrate and expand the use of EdTech in the education system. 3.2 Policy provisions for supporting EdTech-based teaching and learning Different policies and plans prepared by MoEYS during the past two decades have highlighted the importance of EdTech in increasing access to education, enhancing teaching and learning quality, developing students’ digital skills for the labor market, and improving productivity within the ministry. In 2003, the Cambodian government prepared the ‘Education for All: National Plan 2003–2015’, which clearly acknowledged the importance of EdTech in enabling Cambodia to compete in the knowledge- based global economy. This plan highlighted the need to expand the use of EdTech in teaching and learning, information management, and distance learning (Royal Government of Cambodia 2003). Similarly, the ‘Education Strategic Plan: 2004–2008’ also emphasized the inclusion of ICT in teacher training and professional development, the use of ICT in teaching and learning and school management, and the expansion of ICT facilities (MoEYS 2010a). In line with these plans, MoEYS published the ‘Policy and Strategies on Information and Communication Technology in Education in Cambodia’ in 2005 as the first stand-alone ICT policy for the education sector to address the limited utilization of ICT34 in education. This 34 This policy identified computer technology, computer networks, email, internet, TV, and radios as ICT tools and infrastructure. 36 policy aims to increase access to ICT among teachers and students to promote learner autonomy and lifelong learning and ICT-competent graduates (MoEYS 2005b). The policy also corresponds to Article 28 of the Education Law, which requires MoEYS to formulate policies on technology for education (MoEYS 2010b). Subsequent plans and policies have continued to further promote and support the expansion and utilization of EdTech. The Education Strategic Plan 2009–2013 included a subprogram to increase the use of ICT to strengthen the ICT capacity of staff to effectively decentralize administration from the ministry level to the school level (MoEYS 2010a). In 2010, MoEYS published the ‘Master Plan for Information and Communication Technology in Education: 2009–2013’ in alignment with the above strategic plan. This master plan, which also provides guidelines for implementing the 2005 ‘Policy and Strategies on ICT’, specifically aims to increase the use of ICT for improving teaching and learning quality across all levels of education, including at the tertiary level, and as an alternative education platform for increasing access to education (MoEYS 2010b). In 2018, MoEYS published a new version of the stand-alone ICT in education policy document ‘Policy and Strategies on Information and Communication Technology in Education’. This policy provides guidelines on how to increase the use of ICT in teaching and learning to ensure students are technologically capable and ready for the workforce in the twenty-first century (MoEYS 2018). Subsequent policies, including the most recent policies, Cambodia’s Education 2030 Roadmap and the Education Strategic Plan 2019–2030 (both published in 2019), also articulate aims of increasing the use of ICT for similar purposes. A summary of the policies, plans, and strategies supporting EdTech development in Cambodia is presented in Table A.3. An understanding of the EdTech readiness of Cambodia at the policy level can be obtained by looking at how these policies and plans support and promote the use of EdTech along different strategic dimensions. The strategic dimensions discussed below draw on the framework used by the World Bank’s EdTech Readiness Index (ETRI), which focuses on six strategic ‘axes’: school management, teachers, students, ICT devices in schools, connectivity in schools, and digital resources (World Bank 2021c). The key questions of interest in each of these dimensions are summarized in Table 3.2. As there are no clear policies giving schools responsibility for integrating EdTech in teaching-learning, the discussion below focuses only on dimensions 2 to 6. It should also be noted that this discussion aims only to provide an overview of Cambodia’s EdTech readiness and does not delve into the details of the individual dimensions. Table 3.2: Strategic dimensions and key questions of interest Dimension Questions of interest 1. School management Do national policies give schools the responsibility for integrating EdTech in teaching-learning, and are the schools provided guidance in this regard? 2. Teachers Do a standards/competency framework and evaluation system for teachers exist, and is there a system of training and professional development to build teacher ICT competencies? 3. Students Do a standards/competency framework and evaluation requirement for students exist? And is ICT integrated in the curriculum? 4. ICT devices in schools Do standards, monitoring tools, and technical support exist? 5. Connectivity in schools Do connectivity plans, monitoring tools, and technical support exist? 6. Digital resources Do digital resources exist, and is there support for their creation? Teachers The ICT-related policies and plans outlined above highlight the need to train both pre-service and in-service teachers on ICT skills to improve their teaching and administration work. Specifically, the Master Plan for Information and Communication Technology in Education: 2009–2013 aims to ensure that 37 “…all pre-service teacher trainees and a significant number of in-service teachers (especially in upper secondary schools) not only are equipped with ICT literacy but also are trained in the pedagogical principles and appropriate teaching methodologies for using ICT to improve the quality of education” (MoEYS 2010b, 15). Moreover, according to this plan, MoEYS aimed to graduate over 300 ICT teachers—who could train both students and teachers—from the National Institute of Education 35 by 2010 (MoEYS 2010b). The ‘Teacher Professional Standards’ of MoEYS also specifies that teachers must be able to use ICT to enhance their teaching and motivate students to maximize their learning (MoEYS 2016). Accordingly, ICT training has been included in preservice teacher training programs offered by the government’s Teacher Training College. As part of these programs, teacher trainees are expected to be trained on using ICT to support their teaching, preparing lessons and materials, and conducting research for professional learning (Teacher Training College 2019). Moreover, to prepare teachers for immediate implementation of distance learning during the pandemic, MoEYS (2020a) included training for improving teachers’ pedagogical skills related to digital education in the COVID-19 response plan. Although it is difficult to estimate the extent to which the trainings in the response plan took place in practice, the Education Congress of 2021 reported that some of these trainings had been conducted, including distance teaching training for 400 teachers in nine provinces and Google Classroom training for 253 participants in nine provinces (MoEYS 2021c). Despite the continuing emphasis on providing EdTech training to teachers in the different plans and policies, Cambodia does not yet have a digital standards/competency framework for teachers, and neither does it have a system for evaluating their digital competencies. The development and implementation of such a framework and its utilization for evaluation purposes are important for ensuring that teachers have the required digital skills to integrate EdTech effectively in their teaching activities. Students Increasing the ICT competencies of students has been one of the aims of the different ICT plans and policies in Cambodia. All the policies and plans on ICT in education emphasize the expansion and use of ICT not only to enhance students’ learning experience but also to increase their ICT competence to support their lifelong learning and employment. For example, the 2018 Policy and Strategy on Information and Communication Technology in Education states that ICT will be used in education “to equip students with ICT knowledge and skills to transition to the 21st-century world of work” (MoEYS 2018a, 2). In line with this policy direction, in 2015, MoEYS also developed the Curriculum Framework for General Education and Technical Education, which explicitly specified ICT skills as one of the core competencies for students. It proposed that students should “acquire knowledge and skills in basics of computer literacy” and “possess competencies in the use of information, communication and technology for communication, collaboration, collection and analysis of data for further study and research effectively” (MoEYS 2015, 7). The Curriculum Framework for General Education and Technical Education also has components that include ICT training for students. The proposed components are as follows: (a) basics of computer literacy, (b) daily usage of the computer, (c) computer graphics, (d) sending documents and information through a computer system, and (e) presentation technology. However, the expectations regarding the intensity of the proposed ICT training were relatively low. At the primary level, only grade 4 to 6 students were expected to study a computer subject and that too for only one hour per week. While students were 35 MoEYS (2010b) classified ICT teachers into two categories: pure ICT teachers who taught only ICT and teachers teaching ICT as a secondary subject in addition to their specialized subjects (such as mathematics or physics). At least 30 pure ICT teachers were expected to graduate by 2010. 38 expected to study a computer subject for two hours per week at the lower secondary level, the proposed time allocation for upper secondary students was only one hour per week. ICT devices and connectivity Enhancing ICT infrastructure, device availability, and connectivity at the school level has always been a part of MoEYS’ policies and plans. One of the strategies outlined in the 2005 Policy and Strategies on Information and Communication Technology in Education in Cambodia is to provide power supply and the necessary hardware to all secondary schools to enable students to gain access to computers, TV, and radios (MoEYS 2005b) 36 Complementing this strategy, the Master Plan for Information and Communication Technology in Education: 2009–2013 gives the specifications of computers to be deployed in schools, the targeted number of computers per computer lab, and the targeted number of students and teachers per computer.37 It also aims to provide internet connections to all schools that have computers (MoEYS 2010b). The 2018 Policy and Strategy on Information and Communication Technology in Education goes a step further and proposes as a strategy the use of standardized equipment and network design in upper secondary schools and teacher training centers to strengthen the national EdTech infrastructure (MoEYS 2018b). These different policy goals and strategies are reflected in Cambodia’s Education 2030 Roadmap (MoEYS 2019), which envisions 50 percent of preschools, 70 percent of primary schools, and 98 percent of secondary schools having access to electricity, internet, and computers for pedagogical purposes by 2030.38 Given these ambitious targets, adequate provisions will be necessary to monitor the conditions and usage of the ICT infrastructure and provide technical support as necessary to schools in particular. Recognizing this need, the 2018 Policy and Strategy on Information and Communication Technology in Education lists as a short-term strategy the monitoring of ongoing IT operations and the provision of a help desk in the Department of Information Technology for providing technical support at the national, subnational, and school levels. However, further monitoring and technical support details are not provided in any of the policy or plan documents. Digital resources The provision of e-learning resources to support teaching and learning has been a visible theme in the different policy documents. The 2005 Policy and Strategies on Information and Communication Technology in Education in Cambodia proposed to expand access to primary and secondary education by providing distance learning opportunities39 using printed materials as well as other educational resources delivered through technology-based media such as radio, TV, mobile phone, internet, and email (MoEYS 2005b). Giving continuity to this thinking, in 2018, MoEYS prepared strategies for developing (by identifying and localizing free e-learning resources) and distributing e-learning resources through a central MoEYS portal and promoting the use of learning management systems (LMSs) at all levels (MoEYS 2018b). In higher education, the 2009–2013 master plan discussed above put forward the idea of conducting a feasibility study to create a ‘National Open University’ for offering a small number of degrees through 36 However, no timeline is proposed for achieving this goal. 37 For example, according to these specifications, desktop computers should have screens of at least 16 inches, a computer lab should have 26 computers to accommodate between 40 and 60 students, and one computer should be used by five or ten teachers (MoEYS 2010b). 38 This is an ambitious target considering that, in 2018, the percentages of schools (preschools, primary schools, and secondary schools) that had access to electricity, internet, and computers were only 20.4, 36.5, and 63.4 percent, respectively. 39 The policy document also used the term ‘open school’ to refer to distance learning programs. 39 distance learning and allow at least 5 percent of the universities to offer distance learning programs. It also proposed to develop a web-based repository to store research findings and training materials for university lecturers and teachers in general education to use for their teaching and research (MoEYS 2010b). However, there are no publicly available reports that show whether these activities were implemented. In January 2022, the government also issued a sub-decree on the establishment of the Cambodian Cyber University Network (CCUN), which is supposed to offer distance learning programs through digital means from 2022 onward (Royal Government of Cambodia 2022). Six universities are participating in the establishment of the CCUN with the goal of making online learning available for 66 subjects through 858 online lessons for 12 bachelor’s degree programs by 2023 (MoEYS 2022). Within the context of this supportive policy environment, Cambodia has developed a substantial body of digital resources to support students’ learning, especially to support distance learning and teaching during school closure. As discussed in Section 3.1, digital resources are being delivered through a range of channels, including, among others, social media, websites, and mobile applications. These resources include a variety of materials such as video lectures, video clips, and radio programs for preschool children and students in grades 1 to 3 in multilingual education programs, digitalized books and documents, lesson plans, teaching manuals for supplementary lessons to students affected by COVID-19, and student worksheets. The Open Educational Resources Cambodia (OER Cambodia), launched on March 1, 2022, is the latest web-based platform for accessing digital resources available from MoEYS in alignment with the ministry’s strategy of creating a central, one-stop portal for this purpose. Box 3.1 gives an overview of this platform. It should be noted the development and dissemination of e-resources only picked up momentum when students and teachers needed them for distance teaching and learning during the COVID-19-induced school closures. Hence, the quantity of available e-resources is still far from adequate to support regular EdTech-integrated teaching and learning. Furthermore, many of the resources are not necessarily aligned with the national curriculum, and the materials developed during school closures were limited in scope as they were designed to focus on prioritized areas of the curriculum. Ensuring uniform quality of newly developed e-learnings resources also presents a challenge as quality assurance mechanisms for this purpose have yet to be developed. Box 3.1: Open Educational Resources (OER) Cambodia Launched on March 1, 2022, OER Cambodia is an MoEYS dual-language (Khmer and English) website for disseminating e-resources to students and teachers as well as to the general public. The site states that the materials in its repository “include interactive multimedia, posters, digital lesson plans and video clips, as well as background documents from MoEYS and links to other websites to support access to knowledge.” As of August 2022, OER contained 1,832 e-learning resources, classified into 13 categories: presentations, case studies, videos and audios, lesson summaries, manuals, exercises and worksheets, tutorials, research reports, lesson plans, posters and images, animations and stimulations, games, and standard operating procedures. These materials are also classified according to education levels and subject areas. Users do not need to register to get access to these materials, but they can share e-resources only if they have user accounts. As of August 2022, there were 847 registered users of OER. 40 Source: OER website (https://oer.moeys.gov.kh/). Administration MoEYS is strengthening its education management information system (EMIS) by increasing the use of ICT. While EMIS is not one of the six dimensions of EdTech readiness outlined above, it is a critical element of the monitoring and evaluation of any education system. In the Master Plan for Information and Communication Technology in Education: 2009–2013, MoEYS acknowledges the importance of quick access to information to support educational planning and management and make evidence-based decisions for policy formulation (MoEYS 2010b). As the EMIS is the main repository of data for MoEYS, the timeliness, quality, and consistency of EMIS data are of paramount importance. Hence, MoEYS is promoting the use of standardized ICT-based applications and software for data collection and analysis in the national and subnational offices and is improving the management of document archives in all MoEYS national-level offices (MoEYS 2018b). Moreover, the ministry has also been making efforts to upgrade its capacity for ensuring real-time data sharing, analysis, and reporting through ICT-based systems (MoEYS 2019). For example, it implemented a pilot for a new student tracking system (STS)40 in 126 schools in Puok District, Siem Reap Province, in July 2020 (UNESCO 2020). Following the completion of the pilot, MoEYS has begun preparing guidelines for implementing the STS at scale (MoEYS 2021c). Financing Budgets allocated to support the expansion of EdTech are not clearly specified in the different policies and plans, and the available limited information on EdTech-related public investments indicates that the implementation of EdTech policies and plans is not being prioritized by the government. The stand-alone EdTech policies and plans do not present the budgets required for implementing proposed activities. EdTech-specific budgets are also not included in most of the education strategic plans. The exceptions are the Education Strategic Plan 2006–2010 and the Education Strategic Plan 2009–2013. The 2006–2010 plan shows a projected total capital investment budget of 4.6 percent for ICT, and the 2009– 2013 plan has one budget line entitled ‘ICT’ under the MoEYS Program Budget showing an allocation of 40 The STS is an electronic system for collecting and storing individual student data, including bio data, academic achievement by subject and year, attendance, and so on at the school level. Schools, district offices of education, provincial offices of education, and the Department of Education Management Information System can produce reports based on the data in the STS (UNESCO 2020). 41 only 0.1 percent annually between 2011 and 2013 (MoEYS 2010a). These relatively low figures suggest that Cambodia has not been giving high priority to investing in EdTech despite having policies and plans to support the expansion and promotion of EdTech. 3.3 Translating policy into practice: state of EdTech at the school level Awareness of government policies at the school level Data from the principal survey conducted under this study indicate that the vast majority of school principals are either not aware of any policies governing access to ICT infrastructure, including digital devices and internet connectivity, or believe that such policies do not exist. For the EdTech- related policies and strategies to be implemented effectively, they need to be disseminated widely, and schools also need to be aware of them. But as shown in Figure 3.1, there is limited awareness among school principals of these policies. Almost one-third (31 percent) of the principals reported that they did not know whether policies governing access to digital devices, such as desktops, laptops, and tablets, in public schools existed. Close to half (48 percent) of them were not aware if the government has any strategy or plan to provide or facilitate internet connectivity to public schools. Similarly, almost half (42 –46 percent) of the principals were unaware of any government institution or mechanism that monitors whether all public schools have access to functioning ICT devices and internet connectivity and whether the students have access to ICT devices. Furthermore, there is no uniformity of understanding among those who did clearly answer whether or not such policies and strategies existed. While less than half of them understand that the government has policies requiring students to have access to digital devices and strategies for providing internet to schools, the remaining principals stated that such policies and strategies did not exist. The findings are similar with regard to the existence of government institutions or mechanisms for monitoring access to devices and connectivity at the school level—most respondents either have no knowledge of such institutions or mechanisms or believe that they do not exist. Figure 3.1: Do government policies on ICT access exist? (national level) Policy requiring public school students access to digital 30 39 31 devices Strategy to provide or facilitate Internet connectivity to all 21 30 48 public schools Government institution or mechanism that monitors 27 27 46 whether all public schools have access to functioning ICT Government institution or mechanism that monitors 25 32 44 whether all public schools have an Internet connection Government institution or mechanism that monitors if ICT 22 36 42 is used by students 0% 20% 40% 60% 80% 100% Yes No Don't Know Source: Authors’ calculations using the supplemental principal survey 2021. There are, however, significant regional disparities—across urban/rural areas and the four ecological zones—in school management’s awareness of the government’s ICT policies and strategies. While there is no significant urban-rural gap in their awareness of the existence of government policy regarding internet connectivity in public schools, a much larger share of urban school principals (39 percent) do not 42 know whether policies on access to digital devices exist compared to their rural counterparts (28 percent). It is also interesting to note that significantly higher percentages of rural school principals believe that device access and connectivity policies do not exist.41 Looking at the four ecological zones, principals in the mountainous and coastal regions have significantly higher rates of unawareness of ICT policies regarding access to digital devices. More specifically, 42 percent of principals in the mountainous region and 50 percent in the coastal region do not know if policies on digital devices exist compared to 29 percent in the plain area and 25 percent in Tonle Sap (Figure 3.2). In terms of strategies for connectivity, the coastal region stands out in two ways: it is, by far, the region with the largest percentage of principals who do not know whether the government has strategies regarding internet connectivity (79 percent), and it is also the only region where all the remaining respondents (21 percent) state that government strategies for providing connectivity exist (Figure 3.3). Figure 3.2: Do government policies on ICT access exist? (by region) (a) Access to digital devices (b) Internet connectivity Urban/ Urban/ Urban 32 29 39 Urban 27 23 50 rall Ecological zones Rural rall Ecological zones Rural Rural 29 42 28 Rural 19 33 48 Plain area 32 38 29 Plain area 18 34 48 Around Tonle Sap Lake 26 49 25 Around Tonle Sap Lake 22 37 41 Mountainous 30 27 42 Mountainous 30 18 52 Coastal 29 21 50 Coastal 21 79 Ove Ove 30 39 31 21 30 48 0% 50% 100% 0% 50% 100% Yes No Don't Know Yes No Don't Know Source: Authors’ calculations using the supplemental principal survey 2021. Teachers While the competency of grade 6 teachers in using digital devices and tools varies by technology, almost half the teachers either do not know how to use or are poor users of the core devices and tools used for EdTech-based distance learning. For EdTech to be used effectively to support teaching and learning, it is essential for teachers to be relatively comfortable with using different EdTech devices and tools, especially computers/laptops and tablets and tools used for delivering learning virtually. However, the shares of teachers reporting good or very good skills in using these devices and tools are very low—14 percent and 9 percent for laptops and tablets, respectively. Notably, one-quarter (25 percent) of the teachers do not know how to use laptops, and almost half (45 percent) do not know how to use tablets. Although it is concerning, the low level of teacher competency in using these core devices for virtual teaching-learning is hardly surprising given that the vast majority of grade 6 teachers have poor access to these devices at home. Almost one-third (29 percent) of the teachers say they do not have a computer at home, and 35 percent say they have only irregular access to the device. Teacher access to tablets is even more limited: almost half (48 percent) of the teachers do not have tablets at home, and a relatively small share (13 percent) have regular access. Furthermore, teacher competency in using LMSs (Google Classroom, Microsoft 41 There is a possibility that the responses of principals are influenced by the level of digital access experienced by their own schools in practice (see Section 4). 43 Teams, and so on)—core tools that are essential for both synchronous and asynchronous lesson delivery— is also low. Almost one-quarter (22 percent) of the teachers do not know how to use LMSs, and only 18 percent are proficient users. However, larger shares of teachers report being proficient in using the internet and other internet- based tools and applications. About 40 percent of the teachers assess their general internet skills as either good or very good, and only 12 percent consider themselves poor users or nonusers. Furthermore, a significant proportion of teachers report being proficient in using various internet-based tools and applications: 45 percent consider themselves proficient in using social media and messaging apps and 36 percent are proficient in downloading and using apps. An impressive 90 percent say they have at least average proficiency in using smartphones, indicating the viability of utilizing this device for virtual teaching and learning. Teacher competency in using videoconferencing apps (Zoom, Teams, WebEx, and so on), a key component in any online education delivery system, however, is severely lacking, with only 16 percent of the teachers categorizing themselves as proficient users. Interestingly, compared to the self-ratings of the teachers themselves, the school principals’ ratings of the digital competencies of their teachers are somewhat more positive—for instance, 26 percent of principals believe that the teachers in their schools are competent users of videoconferencing apps. Similar patterns of higher ratings by principals are observed for other online tools and ICT devices as well. Figure 3.3: Teacher competency in using digital technology (teacher self-rating) Laptops 25 22 39 11 3 Tablets 45 16 30 8 1 Smartphones 4 6 46 35 9 Internet 4 8 48 33 6 Social media and messaging app 3 7 45 37 8 Downloading and using app 5 9 49 30 6 LMS 22 21 40 15 3 Video conferencing app 27 19 38 13 3 0% 20% 40% 60% 80% 100% Don't know Poor Average Good Very good Source: Authors’ calculations using the supplemental teacher survey 2021. 44 Figure 3.4: Teacher access to ICT infrastructure and devices at home Electricity 73 16 8 2 2 Internet 27 9 23 9 32 Mobile data 64 15 17 2 2 Computer 28 8 26 9 29 Tablet 8 5 26 12 48 Smartphone 67 10 12 3 8 0% 20% 40% 60% 80% 100% Available all the time Available most of the time Available some of the time Almost never available Don't have this device/service at home Source: Authors’ calculations using the supplemental teacher survey 2021. Teachers report having a relatively low competency in conducting ICT-integrated classes, which mirrors their low levels of perceived competency in the use of ICT tools and poor access to digital devices. Only about 40 percent of the teachers say they have adequate expertise in using digital technology in teaching, and the impression they have of their fellow teachers’ expertise is even less encouraging—just 27 percent believe that most of their fellow teachers have adequate expertise (Figure 3.5). The teachers’ self-assessments of their ICT expertise are largely consistent with their perception of the effectiveness of their online classes during school closure—37 percent say they were able to conduct online classes effectively, and 32 percent reported being able to get students in online classes to participate fully and stay engaged. Geographical variation in perceived competency in using EdTech for teaching is similar to the variations for the other aspects discussed above: compared to rural teachers and teachers from mountainous and Tonle Sap regions, larger percentages of urban teachers and teachers in plain and coastal areas see themselves as adequately competent. Hence, there is clearly a need to provide substantial training and support to enhance the teacher capacity so that they can effectively conduct EdTech-integrated classes in their schools. Unfortunately, there seems to be little evidence that the continuing policy emphasis on enhancing teachers’ digital skills has translated to greater ICT training opportunities for them. The majority of the school principals surveyed either said that ICT training—whether on using ICT generally or on using ICT for teaching—was not included in the initial teacher training programs for primary students or that they did not know if ICT was included in these trainings (Figures 3.6). This suggests the need for upgrading initial teacher training programs to enhance their ICT-related content substantially. 45 Figure 3.5: Teacher competency in using digital technology for teaching (teacher self-rating) 53 Urban/Rural Urban 37 52 45 29 Rural 18 24 21 44 Plain area 30 43 36 Ecological Zones 31 Around Tonle Sap Lake 21 29 21 31 Mountainous 23 17 27 50 Coastal 33 38 42 40 Overall 27 37 32 0% 20% 40% 60% 80% 100% Adequate expertise in using digital technology in teaching (self) Adequate expertise in using digital technology in teaching (Most of the fellow teachers) Able to conduct online classes effectively Able to get students in online classes to fully participate and stay engaged Source: Authors’ calculations using the supplemental teacher survey 2021. Figure 3.6: Does the typical initial teacher training program for primary education include ICT training? (national level) Training on how to use ICT generally 44 22 35 Training on how to use ICT for teaching 39 25 36 0% 20% 40% 60% 80% 100% Yes No Don't Know Source: Authors’ calculations using the supplemental principal survey 2021. Students The use of ICT devices by students in Cambodian primary schools was quite limited before the pandemic. Before the pandemic, ICT devices, such as desktops, laptops, and tablets, were never used by grade 6 students in 81 percent of the schools, and students in only 15 percent of the schools used them regularly (Figure 3.7). As might be expected, urban schools used ICT devices much more frequently than rural schools—about one-third (31 percent) of urban schools used them regularly compared to only 9 percent of rural schools. In general, the use of ICT was fairly uniform across the four ecological zones; 46 however, coastal schools used them somewhat more frequently, with 21 percent of the schools using them regularly compared to about 15 percent in the other three regions. Figure 3.7: Use of desktops, laptops, and tablets in class by grade 6 students before COVID-19 (principals’ opinions) Urban/R Urban 66 2 23 8 2 ural Rural 86 1 6 3 4 Plain area 80 3 11 4 3 rall Ecological zones Around Tonle Sap Lake 81 10 4 4 Mountainous 82 12 3 3 Coastal 79 7 14 Ove 81 1 11 4 3 0% 20% 40% 60% 80% 100% Never Once or twice Once or twice a week Everyday or almost everyday I don't know Source: Authors’ calculations using the supplemental principal survey 2021. Despite the low usage of ICT devices by students in schools, the vast majority of school principals recognize the importance of enhancing students’ competency in using digital technology and ICT to augment student learning. Apart from a tiny minority (less than 4 percent), all teachers agree that it is important to improve students’ basic computer skills and ability to access and use information with ICT and create an appropriate environment for safe and proper use of ICT (Figure 3.8). Similar patterns are observed for the principals’ opinions on the use of ICT for student learning. About 96 percent agree that it is important to use ICT to augment and improve students’ learning, though there is some variation in the degree of importance they give to this task. Figure 3.8: Importance of using ICT and developing students’ ICT skills (principals’ opinions) Basic computer skills 3 40 19 39 Proficiency in accessing and using information with ICT 2 41 19 38 Safe and appropriate use of ICT 4 40 17 40 Use of ICT to augment and improve students' learning 4 39 15 42 0% 20% 40% 60% 80% 100% Not important Little important Moderately important Very important Source: Authors’ calculations using the supplemental principal survey 2021. Effective utilization of ICT in teaching-learning also depends on the level of access students have to digital infrastructure and devices at home, which is currently quite low, especially among poorer and rural students. Data from the NLA 2021 survey show that, overall, only 29 percent of students have access to the internet at home, and only 15 percent have access to a computer (Figure 3.9). While no gender gap in access to the internet or computers is observed, there are substantial differences across students from 47 different socioeconomic backgrounds. Not surprisingly, students from wealthier households have progressively higher levels of access. For example, the percentage of students with access to the internet is seven times higher for the wealthiest quintile (Q5) compared to the poorest quintile (Q1), and the corresponding percentage for access to computers is 12 times higher for the richest quintile. There are also significant disparities in access across the different regions. Students from urban locations are about 2.5 times more likely than rural students to have access to the internet at home and over three times more likely to have access to a computer. Differences in access are also observed across the four ecological zones, albeit to a much smaller extent, with higher access among students from the plain and coastal regions. Figure 3. 9: Student access to digital infrastructure and devices (self-reporting by students) (a) Gender and socioeconomic status (b) Region 29 58 Urban/Rur all Wealth Index Quintiles Gender Male 72 Urban 90 16 33 al 29 22 Female 71 Rural 67 14 9 10 Q1 (lowest) 56 34 4 Plain area 75 Ecological Zones 14 19 Q2 66 25 5 Around Tonle Sap Lake 67 16 11 Q3 69 21 9 32 Mountainous 66 Q4 79 11 14 39 65 Coastal 79 Q5 (highest) 91 14 47 29 Over 29 Over all 71 71 15 15 0% 50% 100% 0% 50% 100% Internet (at home) Internet (anywhere) Internet (at home) Internet (anywhere) Computer Computer Source: Authors’ calculations using NLA 2021. While no assessment data are available on the ICT competency levels of students, only a relatively small share of teachers believe that their students have adequate expertise in using digital technology for learning. Less than one-third of the teachers surveyed say that students can use EdTech for learning effectively and are able to understand and enjoy online classes. The teachers’ perceptions of the competency levels of their students vary substantially across urban and rural areas and are consistent with the patterns of urban-rural differences (in favor of the urban regions) discussed above. More specifically, compared to teachers in rural areas, urban school teachers are twice as likely to say that their students have sufficient competency in using digital technology for learning and are able to follow and enjoy online classes. There are also some differences across regions, with larger shares of teachers in the plain and coastal regions believing that students are adequately competent in using digital technology for learning compared to teachers in Tonle Sap and mountainous areas. 48 Figure 3.10: Student digital competency and effectiveness of online learning (teachers’ opinions) 42 Urban/Rural Urban 42 42 18 Rural 18 17 33 Plain area 32 31 Ecological Zones 22 Around Tonle Sap Lake 21 20 21 Mountainous 23 33 33 Coastal 38 33 Overall 29 29 29 0% 20% 40% 60% 80% 100% Students have adequate expertise in using digital technology for learning Students able to follow and understand online lectures without any problem Students enjoyed online classes and learning using digital technology Source: Authors’ calculations using the supplemental teacher survey 2021. Electricity, connectivity, and ICT devices in schools While almost all schools have access to electricity, their access to the internet is limited and varies significantly between urban and rural areas. Nearly 95 percent of the schools across the country have access to electricity—an essential prerequisite for implementing EdTech—and there are relatively small differences in electricity access across geographical locations (Figure 3.11). In particular, electricity access is almost universal in urban schools (97 percent), and 92 percent of schools have electricity access even in rural areas. The disparity in electricity access is somewhat greater across ecological regions, with the mountainous area lagging behind the other three regions. On the other hand, schools have limited access to the internet, with only 29 percent of the schools reporting that they have internet connectivity. Furthermore, the urban-rural disparity in internet access is staggering: the percentage of urban schools with access to the internet (57 percent) is over three times higher than that of schools in rural areas (17 percent). Looking at the four regions, schools in the mountainous area have significantly lower rates of internet access (22 percent) compared to the plain area (33 percent), which is the best-performing region. 49 Figure 3.11: Share of schools with access to ICT infrastructure (percentage) 97 Urban/Rural Urban 57 92 Rural 17 97 Plain area 33 Ecological Zones 94 Around Tonle Sap Lake 26 78 Mountainous 22 100 Coastal 29 Overall 94 29 0% 20% 40% 60% 80% 100% Electricity Internet Source: Authors’ calculations using the supplemental teacher survey 2021. Access to digital devices, namely desktops, laptops, and tablets, is also limited in Cambodian schools and varies widely across urban and rural areas. On average, schools have eight working devices, of which only four are available for students, and only three of the devices available to students are connected to the internet (Figure 3.12). These average figures, however, mask significant disparities in access to digital devices—in terms of both the absolute number of devices and the number of devices relative to school size—across urban and rural areas. For instance, while there are, on average, about 11 digital devices per 300 students connected to the internet and available for student use in urban schools, rural schools do not have even one digital device available per 300 students.42 This falls far short of the goal of 26 computers for 40 to 60 students stated in the ICT master plan. There are large disparities in access to digital devices across the four ecological regions as well. In absolute terms, the number of digital devices is the largest in the plain region. For instance, on average, six internet-connected working devices per school are available for student use in the plain region, compared to one each in Tonle Sap and mountainous areas and two in coastal schools. However, in relative terms, access to digital devices is significantly higher in mountainous and plain regions compared to the Tonle Sap and coastal regions. For example, about six connected working devices are available per 300 students in mountainous schools, compared to about four in plain schools and one each in Tonle Sap and coastal schools. Moreover, the relative number of working devices—that are not necessarily available to the students or connected to the internet—is significantly higher in the mountainous region (25) than in the remaining three regions: 8 in plain, 2 in Tonle Sap, and 3 in coastal regions. 42 This indicator shows the availability of devices relative to school size. Here, ‘per 300 students’ is used to get figures that are reasonably close to the actual total numbers and to avoid getting extremely small (less than 1) number of devices. 50 Figure 3.12: Number of digital devices in school (a) Total number of devices (b) Number of devices per 300 students 24 26 Urban/Rural Urban/Rural Urban 13 Urban 12 11 11 3 2 Rural 1 Rural 1 1 0 13 8 Plain area 7 Plain area 5 6 4 Ecological Zones Ecological Zones 3 2 Around Tonle Sap Lake 1 Around Tonle Sap Lake 1 1 1 6 25 Mountainous 3 Mountainous 7 1 6 5 3 Coastal 2 Coastal 1 2 1 Overall 9 Overall 8 5 4 3 3 0 5 10 15 20 25 30 0 20 40 Working devices Working devices working devices available for students working devices available for students Working devices (available and connected to internet) Working devices (available and connected to internet) Source: Authors’ calculations using the supplemental principal survey 2021. The relatively limited access to digital devices (observed above) is reflected in the school management’s perception of the sufficiency of ICT-based distance learning infrastructure and devices in their schools. In general, over half of the school principals report that the available ICT infrastructure and devices are insufficient to support effective teaching and learning (Figures 3.13). In particular, 55 percent of school principals disagree or strongly disagree that their schools have sufficient digital devices, and 57 percent believe that the number of digital devices with internet connectivity available in their schools is insufficient. Furthermore, around half of the principals think the internet bandwidth, speed, and stability are insufficient to support effective teaching and learning. Similarly, most principals do not feel they have adequate technical support to properly maintain the ICT resources in their schools. 51 Figure 3.13: Principals’ perceptions of sufficiency of ICT infrastructure and resources The number of digital devices for instruction is sufficient 14 41 40 5 to support teaching and learning effectively The number of digital devices connected to the Internet is 15 42 39 4 sufficient to support teaching and learning effectively The school’s Internet bandwidth or speed is sufficient to 9 38 49 4 support teaching and learning effectively The school's Internet stability (i.e. connection without 8 42 45 4 service interruption) is sufficient to support teaching and… There is sufficient technical support to maintain ICT 12 39 44 4 resources so that they are fully functional 0% 20% 40% 60% 80% 100% Strongly disagree Disagree Agree Strongly agree Source: Authors’ calculations using the supplemental principal survey 2021. There are substantial regional variations in the principals’ perceptions of ICT resources/infrastructure sufficiency. The principals’ perceptions of the adequacy of digital devices with internet connectivity are significantly different for rural areas than for urban areas. Only 35 percent of rural school principals think their schools have sufficient access compared to 63 percent of urban school principals (Figure 3.14). Similarly, the urban-rural gap in terms of internet stability is quite significant (Figure 3.15). Furthermore, there are substantial differences in the principals’ views across the four ecological zones with regard to the sufficiency of access to internet-connected devices. Compared to the other three regions, a larger share of the Tonle Sap school principals disagree that there is sufficient access to such devices in their schools. On the other hand, there is relatively little variation across the four ecological zones in principals’ perceptions of the sufficiency of internet stability. Figure 3.14: Principals’ perceptions of the sufficiency of access to digital devices with internet connectivity in school (by region) Urban/Rura Urban 16 21 55 8 l Rural 15 50 33 2 Plain area 16 38 43 4 Ecological zones Around Tonle Sap Lake 19 47 31 3 Mountainous 3 48 45 3 Coastal 14 43 29 14 Over all 15 42 39 4 0% 20% 40% 60% 80% 100% Strongly disagree Disagree Agree Strongly agree Source: Authors’ calculations using the supplemental principal survey 2021. 52 Figure 3.15: Principals’ perceptions of the sufficiency of internet stability in school (by region) Urban/Ru Urban 11 23 58 8 ral Rural 7 50 41 3 Plain area 8 40 47 4 all Ecological zones Around Tonle Sap Lake 9 43 44 4 Mountainous 3 52 45 Coastal 14 36 36 14 Over 8 42 45 4 0% 20% 40% 60% 80% 100% Strongly disagree Disagree Agree Strongly agree Source: Authors’ calculations using the supplemental principal survey 2021. Key challenges identified by principals and teachers The most critical challenges identified by principals and teachers in ensuring effective implementation of distance learning using digital technology are the high cost of digital infrastructure and resources, inadequate digital literacy and experience with online teaching, and poor and uneven access to ICT infrastructure. About 90 percent of principals and teachers cite the high costs of digital devices and the internet as major challenges (Figure 3.16).43 Insufficient online teaching experience among teachers is another key challenge highlighted by both principals and teachers. They also see inadequate digital literacy—among teachers as well as parents—as a critical issue of concern. For example, the vast majority of principals (81 percent) and teachers (85 percent) believe that inadequate digital literacy among teachers is a major challenge. This view is consistent with their perception of low teacher competency in using digital technology in general and for teaching in particular. Similarly, reflecting concerns about the inadequacy of the ICT infrastructure, about 89 percent of teachers cite poor internet connection, and more than 80 percent of teachers and principals cite unequal access to internet connection and digital devices as significant impediments to the effective implementation of distance learning. Other key challenges identified by principals and teachers include lack of supporting materials and unreliable electricity supply. In particular, a relatively large share of the teachers (82 percent) see a lack of supporting materials and guidelines as an impediment to the effective implementation of distance learning. Though lack of reliable electricity supply is cited as a critical challenge by fewer respondents, it is considered a major issue of concern by substantial shares of both principals (74 percent) and teachers (64 percent). It is relevant to note that the percentages of teachers viewing these different issues as challenges are greater than the corresponding percentages of principals. These higher figures for teachers may reflect the fact that since teachers are the ones directly engaged in delivering education in the classroom, they have a better understanding of the challenges faced in the classroom. 43 The average self-reported monthly internet expenditure for teachers before the onset of the pandemic was about KHR 32,000, which increased by over 40 percent to KHR 45,000 during school closure. 53 Figure 3.16: Challenges in effective implementation of distance learning using digital technology perceived by principals and teachers 91 Digital devices cost 90 86 Internet cost 89 84 Insufficient experience in online teaching (among… 88 81 Digital literacy (teachers) 85 82 Digital literacy (parents) 86 84 Unequal internet access 86 81 Unequal access to digital devices 84 81 Poor internet connection 89 78 Lack of supporting learning materials and guidelines 82 64 Lack of reliable electricity supply 74 0% 20% 40% 60% 80% 100% Principals Teachers Source: Authors’ calculations using the supplemental principal and teacher surveys 2021. These findings indicate that although MoEYS has made notable progress in developing policies, plans, and strategies to support EdTech-based education, the country still has a long way to go in terms of EdTech readiness, especially at the school level. Student and teacher access to essential EdTech devices is still quite limited both in school and at home, and the ICT infrastructure, especially connectivity, is relatively poor. Furthermore, there are significant disparities in access to devices and ICT infrastructure between urban and rural areas and across ecological regions. School teachers have poor EdTech skills, their experience with and capacity to conduct EdTech-based teaching is limited, and students too do not have adequate expertise in using digital technology for learning. Strengthening teacher EdTech capacity to address these challenges is difficult as mechanisms for assessing teachers’ digital competencies have yet to be developed, and there are no requirements for in-service teachers to participate in ICT training. All these factors severely constrain the effective and expanded use of EdTech for teaching-learning in Cambodia. 54 4. EdTech-based school-level responses to COVID-19 and their effect on learning 4.1 Overview During the COVID-19-induced school closures, schools attempted to maintain the continuity of learning of students by implementing distance learning using both EdTech-based tools and learning content as well as traditional paper-based materials. Student worksheets—both in soft and hard copy forms—were the primary means through which most teachers taught students and checked their learning progress while their students were at home. In some cases, they used worksheets developed by MoEYS, while in other cases, they developed their own worksheets or adapted existing worksheets to address their students’ learning requirements. Regardless of whether soft or hard copies were used, teachers would need to explain to the students the lessons related to the worksheets and provide instructions on how to complete them when the worksheets were distributed. They would review the completed worksheets submitted by the students, return the marked versions along with relevant feedback, and distribute new worksheets for new lessons when students picked up their marked worksheets. Teachers provided printed worksheets to students who did not have access to smartphones or the internet. In such cases, teachers would set up meeting time slots for different groups of students and parents to come to school to pick up printed worksheets. During such meetings, they would also spend some time explaining lessons, answering questions, and clarifying the instructions given for completing the worksheets. In situations where some parents or students could not come to school during the specified time slots, teachers would deliver worksheets directly to the students’ homes or arrange a different meeting time and place for these students and parents with the help of local authorities. In areas where teachers and students had access to the internet, teachers utilized EdTech tools for distance learning. In such cases, instead of sharing printed copies, teachers distributed soft copies or digital pictures of worksheets to students and collected answers through messaging applications. Mobile messaging applications, which constituted the most common EdTech tool used, were utilized by teachers to communicate with students to teach asynchronously by sharing worksheets, short voice recordings of lesson explanations and feedback on assignments, and, in some cases, video recordings of lessons. Synchronous teaching using videoconferencing applications such as Zoom and Google Meet was far less common. Smartphones were the primary ICT device teachers used for distance learning. The use of computers for distance learning was low among teachers, perhaps because of their limited capacity and restricted access to computers. According to the Cambodia COVID-19: Joint Education Needs Assessment, only 34 percent of 3,570 teachers surveyed in August/September 2020 reported having access to computers, while 86 percent said they had smartphones (MoEYS 2021a). Using their smartphones, teachers were able to perform a variety of teaching-related tasks such as sharing video recordings of lessons, sharing audio recordings of lesson explanations and feedback to students, distributing worksheets, sending announcements and instructions to students and parents, interacting with students and parents, and making conference calls with students as necessary. The discussion below summarizes the key findings from the surveys of principals and teachers on EdTech-based measures taken by schools in response to the school closures induced by COVID-19. It focuses on two aspects: (a) the extent to which different types of EdTech-based measures were used during school closure to support distance learning and the kinds of support provided to teachers and parents and (b) the correlation between these measures and student learning outcomes in 2021. The discussion on 55 the correlation between EdTech measures and learning outcomes is supplemented by an analysis of the determinants of learning outcomes using regression methods. 4.2 Use of EdTech during school closures EdTech-based delivery of distance learning by schools In response to the pandemic, the vast majority (around 90 percent) of Cambodian grade 6 teachers provided school materials to their students for home-based study when the schools were closed. These school materials included both digital and paper-based study materials for lessons, classwork, and homework. As shown in Figure 4.1, though there was little difference between urban and rural areas in the provision of school materials in general, a significantly larger percentage of rural teachers (97 percent) reported that they provided physical packages—that is, printed materials—to students compared to urban teachers (82 percent). Looking at the four ecological zones, the Tone Sap Lake region was distinctly ahead of the other three regions, where almost 97 percent of the teachers provided school materials (both any type as well as physical packages). On the other hand, the coastal region significantly lagged behind the others regarding the provision of physical packages. Figure 4.1: Share of teachers who provided school material and physical packages during COVID-19 97 97 100% 90 93 90 89 90 87 88 90 89 82 83 80% 75 60% 40% 20% 0% Urban Rural Plain area Around Tonle Mountainous Coastal Sap Lake Urban/Rural Ecological Zones Overall Provided study material Provided physical packages Source: Authors’ calculations using the supplemental teacher survey 2021. Overall, schools utilized digital and non-digital materials in equal proportions to support distance learning during school closures. Evidence from the teacher survey shows that, overall, half of the total study materials provided by schools constituted digital materials and half were physical or printed materials (Figure 4.2). However, the use of digital materials was much more prevalent in urban schools compared to their rural counterparts. While digital materials accounted for over two-thirds (68 percent) of the total study materials provided to students in urban schools, the opposite was true for rural schools, where printed materials made up the bulk (64 percent) of the study materials. Although the differences are not as prominent across ecological zones, the plain region had a larger overall share of digital materials compared to the other regions. 56 Figure 4.2: Share of digital and physical materials in the total schoolwork 100% 80% 32 47 54 53 50 66 58 60% 40% 68 53 46 48 50 20% 34 42 0% Urban Rural Plain area Around Tonle Mountainous Coastal Sap Lake Urban/Rural Ecological Zones Overall Digital material Physical material Source: Authors’ calculations using the supplemental teacher survey 2021. Messaging apps played a crucial role in the dissemination of digital content to students when the schools were closed. Messaging apps such as Telegram, Facebook Messenger, and WhatsApp, as well as email, were the most commonly used platforms for sharing digital content with students (Figure 4.3). Over half of grade 6 teachers (52 percent) used these messaging tools and email daily to distribute and collect homework, and about 87 percent used them at least a few times a month. In contrast, the use of e-learning platforms, such as Google Classroom, to digitally share assignments was much less common, with 41 percent of the teachers reporting that they never used these platforms during school closures and only 18 percent reporting that they used them daily. Interestingly, though teachers used both digital and non-digital materials to support their students’ learning, in-person distribution and collection of assignments (that is, in hard copy form) was not a common practice for most teachers. Figure 4. 3: Platforms (digital and non-digital) used by teachers to distribute and collect homework Messaging apps and email 52 20 13 22 11 E-learning 18 16 15 6 3 41 In-person 11 22 28 7 7 25 0% 20% 40% 60% 80% 100% Daily 2-3 times weeek Once a week 2-3 times a month Only once or twice during school closure Never Source: Authors’ calculations using the supplemental teacher survey 2021. 57 Though schools also tried to support their students’ learning through online classes,44 these classes were offered only for a limited number of hours during the pandemic. On average, online classes were held around 11 hours per week during school closures, which was less than half the number of hours (30) students spend in class each week during normal times (Figure 4.4). Furthermore, on average, schools offered only about 21 weeks of online classes or a total of around 290 hours of instruction during the pandemic. Considering that classes are expected to be held for 38 weeks per year according to the national curriculum, the contact hours available through online classes were limited. As might be expected, however, there was a substantial urban-rural gap in the number of hours of online classes offered by schools: on average, urban schools were able to provide over two times more hours of online instruction than rural schools. Figure 4.4: Total hours and weeks of online classes during COVID-19 (a) Hours per week (b) Total weeks Urban/ Urban/ Urban 14 Urban 27 Rural Rural Rural 9 Rural 16 Plain area 12 Plain area 23 Ecological Ecological Around Tonle Sap Lake Around Tonle Sap Lake 17 Zones 10 Zones Mountainous 11 Mountainous 17 Coastal 9 Coastal 22 Ove Ove rall rall 11 21 0 5 10 15 0 10 20 30 (c) Total hours Urban/ Urban 403 rall Ecological Zones Rural Rural 185 Plain area 330 Around Tonle Sap Lake 197 Mountainous 236 Coastal 240 Ove 286 0 200 400 600 Source: Authors’ calculations using the supplemental teacher survey 2021. Despite the limited availability of online classes, survey findings suggest that teachers were able to cover a relatively large share of the regular schoolwork during the pandemic using EdTech-based distance learning approaches. Teachers reported being able to cover almost three-quarters (71 percent) 44 Online classes in this context refer to any teacher interaction with students through digital means during set class periods. Hence, they do not necessarily refer only to the delivery of online lectures; they can also include activities such as sending and collecting homework during the online class period. 58 of the total regular schoolwork (lessons, classwork, homework, and so on) through online classes45 during the period of school closure, though the coverage varied significantly across the country. In particular, the percentage of schoolwork covered is much higher for urban areas (83 percent) compared to rural areas (60 percent), reflecting the urban-rural differences in the total weeks and hours of online classes held during the pandemic (Figure 4.5). It is also consistent with the general pattern of better outcomes seen for urban areas in Sections 2 and 3 and is not surprising considering that urban areas have much better access to EdTech. Although the differences across the ecological zones are far less pronounced, there is an 8 percentage point difference in coverage between the best-performing (plain) and worst-performing (mountainous) regions. Figure 4.5: Share of schoolwork completed during school closure 100% 83 80% 73 71 66 65 67 60 60% 40% 20% 0% Urban Rural Plain area Around Tonle Mountainous Coastal Sap Lake Urban/Rural Ecological Zones Overall Source: Authors’ calculations using the supplemental teacher survey 2021. When conducting online classes, teachers relied heavily on messaging apps as the preferred medium of lesson delivery. These apps were used extensively not just for assigning and collecting homework but also for teaching students new materials. Over 40 percent of the teachers reported using messaging apps daily for this purpose, and another 30 percent indicated that they used them at least once a week (Figure 4.6). By comparison, videoconferencing apps and MoEYS platforms were used much less frequently—26 percent of the teachers used videoconferencing apps, and only 18 percent used MoEYS platforms daily. Furthermore, significant percentages of teachers did not use videoconferencing apps and MoEYS platforms at all. Among the different MoEYS digital platforms available, Facebook and YouTube channels were the most widely used platforms for delivering distance learning during the pandemic. About two-thirds (62–63 percent) of the teachers used these channels at least once a week to teach their students new materials (Figure 4.7). The MoEYS e-learning platform was also used quite frequently by the teachers, with about 56 percent of the teachers using them at least once a week. Wide access to smartphones and mobile data among the teachers likely made the use of these platforms possible, as almost 80 percent of the grade 6 teachers reported having regular access to smartphone and mobile data at home. Some other platforms used regularly but somewhat less frequently include the MoEYS Podcast and MoEYS TV, TVK, and a mobile app from the Ministry of Information (MoI). On the other hand, radio channels were rarely used by the teachers 45 This surprisingly larger share of schoolwork coverage reported by teachers could perhaps be related to teach ers’ understanding of what total schoolwork means. Findings from focus group discussions with teachers suggest that, in the COVID context, teachers seem to view total regular schoolwork as consisting of only the most important lessons and topics. 59 during the pandemic, a finding fully consistent with the low usage of radio by children for learning as reported by households (see Figure 4.7 and Figure 1.2). Figure 4.6: Distance teaching/learning platforms used by teachers to teach new materials to students during the pandemic Video conferencing 26 13 11 4 6 40 MoEYS platforms 18 24 15 8 9 26 Messaging apps 41 18 14 6 7 14 0% 20% 40% 60% 80% 100% Daily 2-3 times weeek Once a week 2-3 times a month Only once or twice during school closure Never Source: Authors’ calculations using the supplemental teacher survey 2021. Figure 4.7: Distance teaching/learning platforms used by teachers during the pandemic (not including messaging apps) MoEYS Facebook 20 28 15 24 10 2 MoEYS Youtube 19 27 16 24 12 3 MoEYS E-learning 13 22 21 21 18 6 MoEYS TV 12 18 18 21 26 5 MoEYS Podcast 12 21 16 20 22 8 TVK 13 16 18 23 26 4 MoI Mobile App 11 17 16 19 27 10 Local radio 4 6 23 11 44 13 0% 20% 40% 60% 80% 100% Daily 2-3 times weeek Once a week Only sometimes Never Don't know Source: Authors’ calculations using the supplemental teacher survey 2021. Box 4.1: Distance learning during COVID-19: Experience of high-performing urban schools Wat Sansom Kosal Primary School, located in the capital city of Phnom Penh, and Banteay Chas Primary School located in the provincial town of Siem Reap were among the high-performing schools in NLA 2021. As of June 2021, Wat Sansom Kosal had a total of 2,018 students and 39 teachers. Among them, 354 students and six teachers were from grade 6. Banteay Chas was slightly smaller, with a total of 1,750 students and 59 teachers. The school had six grade 6 teachers and 259 grade 6 students. Both schools had Wi-Fi internet connections for the school management and teachers. Choice of technology and platforms. During school closure, most of the teachers in these two schools used mobile messaging apps as the primary means for conducting remote teaching. Telegram and Facebook Messenger were the most familiar applications for both teachers and students. To facilitate remote teaching, the teachers set up Telegram and Messenger groups comprising the parents of the students as most of the students used their parents ’ smartphones. These groups were used by teachers to communicate with both parents and students and to teach students remotely. 60 Asynchronous teaching-learning using EdTech. Most of the teachers used EdTech for sharing announcements, delivering printable worksheets, and sending feedback. Their focus was on giving assignments and providing feedback to the students rather than on delivering online lectures. Using smartphones, the teachers sent images of assignment worksheets (produced by MoEYS or designed by themselves) to their students via Telegram or Facebook Messenger, along with voice recordings or videos explaining the relevant lessons in the coursebooks and giving instructions on how to complete the assignments. The students completed the worksheets at their own pace, using the coursebooks and obtaining help from their caregivers or parents. While some students could afford to print the worksheets, others wrote their answers in their notebooks. After they finished, they sent the images of their answers in their notebooks or worksheets to their teachers using smartphones. In the case of reading lessons, they also sent voice recordings to their teachers. The teachers sent feedback to their students in the form of scores and answer keys. The teachers would need one or two days to complete a typical lesson. Most teachers in these two schools were not able to use laptops for remote teaching. Hence, they prepared hand-written worksheets and used smartphones to take their photos and send them to students. Synchronous teaching-learning using ICT. The use of EdTech devices, especially computers, for synchronous teaching and learning was limited among these teachers due to their low capacity and lack of access to laptops. There were only a few teachers who had their own laptops and the necessary digital skills to use videoconferencing apps such as Zoom and Google Meet for real-time online (that is, synchronous) teaching-learning. For example, at Banteay Chas, some grade 6 teachers who were pursuing post-graduate programs had laptops and used Zoom with free accounts to teach two 45-minute online classes each day. They also recorded their Zoom sessions and shared them with their students in Telegram groups. Thus, students who missed these Zoom sessions could watch the recordings to understand how to complete the worksheets. Source: Field visit by authors. Student access to digital learning materials and resources Student survey data from NLA 2021 largely support the teacher survey findings regarding the provision of learning resources for students during the pandemic. The NLA student questionnaire asked students how often their schools provided the following: study materials for self-study, schoolwork or homework, uploaded material on an LMS, and distance learning through live virtual classes on videoconferencing platforms. The vast majority (80–86 percent) of the students responded that their schools sent them assignments and materials to study on their own on a weekly or daily basis (Figure 4.8). However, their responses indicate that the provision of uploaded materials and live virtual classes was much less prevalent: in fact, 47 percent of the students reported that they did not receive any uploaded materials and 42 percent said that they never attended live virtual classes during school closures. Students also reported receiving regular follow-up support from their schools to help them in their studies during school closures. As shown in Figure 4.8, almost 90 percent of the students reported that their teachers asked them to submit completed assignments, and around 80 percent said that their teachers regularly checked in on them to ensure that they were completing the assignments. Most of the students also said their teachers provided them with helpful self-study tips. Student responses also suggest that most schools recognized the potential psycho-social impacts of the pandemic on students and provided them with some psychosocial support by regularly checking in on them to find out how they were feeling. 61 Figure 4.8: Frequency with which school conducted different activities during school closure Learning materials for self-study 20 46 34 School assignments 14 44 42 Uploaded material on a learning management system 47 26 27 Live virtual classes on a video communication program 42 24 34 Check-in to ensure completing assignments 22 40 38 Asked to submit completed assignments 12 43 45 Helpful tips on self study 19 40 41 Check-in to ask how I was feeling 38 37 25 0% 20% 40% 60% 80% 100% Never Weekly Daily Source: Authors’ calculations using NLA 2021. The extent to which students used different learning resources during school closures varied by the type of resource (Figure 4.9). Students reported using both digital and non-digital (paper) forms of textbooks, workbooks, or worksheets, though the use of hard copies was more frequent, which is consistent with the findings from MoEYS (2021a). Consistent with the wide use of messaging apps, as reported by teachers, almost three-quarters (73 percent) of the students said that they used learning materials provided by their teachers via messaging apps, with about 37 percent using them daily. Students used self-study worksheets prepared by MoEYS somewhat less frequently, and over one-third (36 percent) of the students reported that they never used these resources during the pandemic. Synchronous (that is, real-time) teaching-learning using videoconferencing technology was even less common, with 51 percent of the students reporting that they never accessed any real-time videoconferencing lessons conducted by their teachers. Similarly, almost half of the students had never accessed recorded lessons or other digital materials. It should be noted that some students also reported receiving real-time private tutoring (extra classes in Figure 4.8) from their teachers via videoconference —however, the share of such students is relatively small. These findings suggest that, despite the progress Cambodia has made in expanding internet access to the population, it is not yet at a stage where distance learning using videoconferencing technology (whether for synchronous or asynchronous teaching-learning) can be expanded at scale. 62 Figure 4.9: Frequency of use of learning resources by students during school closure Paper textbooks, workbooks or worksheets 10 55 35 Digital textbooks, workbooks or worksheets 46 35 19 Learning material my teachers sent via Facebook 27 36 37 Messenger or Telegram Self-study worksheets (prepared by MoEYS team) 36 46 18 Real-time lessons by a teacher from my school on a video 51 23 27 communication program Real-time lessons by your extra-class teacher from your 72 16 12 school on a video commun Recorded lessons or other digital material provided by 49 30 20 teachers from my school Recorded lessons or other digital material from other 49 34 16 sources 0% 20% 40% 60% 80% 100% Never Some days Every day Source: Authors’ calculations using NLA 2021. Box 4.2: Distance learning during COVID-19: Experience of low-performing rural schools Serei Chot Voan Primary School and Hun Sen Prey Chuo Primary School were among the lowest-performing schools in NLA 2021. As of June 2022, Serei Chot Voan had 501 students and 12 teachers, and Hun Sen Prey Chuo had 689 students and 18 teachers. In both schools, the student-teacher ratio in grade 6 was very high (119 students and only one teacher in Serei Chot Voan and 87 students and one teacher in Hun Sen Prey Chuo), reflecting the severe human resource constraints faced by these rural schools. EdTech usage. During school closure, there was limited usage of ICT devices and applications for distance teaching and learning in these two schools. The teachers and parents cited three main reasons for the low usage of EdTech. The first was connectivity issues: both teachers and students had limited access to the internet and mobile data in their neighborhoods due to their locations (Serei Chot Voan and Hun Sen Prey Chuo Prey are located 41 km and 24 km, respectively, from Takeo provincial town and even further from Phnom Penh City). The second was inadequate digital competency on the part of teachers and their limited access to devices. More specifically, most of the teachers in these schools said that they were not adequately skilled in using smartphones for distance teaching. Furthermore, most of them did not possess laptops for designing student worksheets. The third was the parents’ reluctance to let their children use smartphones. As they were concerned about ‘misuse’ of smartphones, including playing mobile games and using social media apps, they gave their children relatively limited access to these devices. Asynchronous teaching-learning using printed materials. Distance learning for students in these schools relied heavily on the distribution of printed worksheets. At Serei Chot Voan, worksheets were distributed to the students three times a week—on Mondays, Wednesdays, and Fridays. The students would come to school in small groups to meet with their teachers and collect the worksheets. During these meetings, the teachers would also spend around 15 minutes explaining the lessons related to the worksheets. The students were required to return the completed worksheets to the teachers for feedback two days after receiving them and would receive their marked worksheets, answer keys, and feedback during their next visit. Distance teaching-learning was more challenging at Hun Sen Prey Chuo Primary School. As the school had been converted to a COVID-19 quarantine center, it was not possible to use the school premises as the meeting point for teachers and students. Instead, teachers collected worksheets from the Vice-Principal’s house and then distributed them to their students in their respective villages. At first, the teachers delivered the worksheets directly to the homes of individual students, but later, they arranged for the 63 students to meet with them in small groups in a pagoda compound in their respective villages, where they distributed the worksheets and provided instructions and lesson explanations. As with students from Hun Sen Prey Chuo, the students from this school were also required to submit their completed worksheets for feedback two days after receiving the assignment. Challenges and solutions. The main challenge cited by teachers was the difficulty level of worksheets developed by MoEYS. The teachers from both schools said that the ready-made worksheets from MoEYS were too difficult for their students, and their use tended to demotivate the learners. Hence, the teachers either designed their own worksheets from scratch or adapted the MoEYS worksheets to fit their students ’ learning levels. Source: Field visit by authors. EdTech support to teachers and parents Most schools provided their grade 6 teachers with orientation/training to support distance learning during the pandemic, but training on specific EdTech-based teaching skills was relatively limited. As noted earlier, the use of ICT for teaching/learning was quite limited in Cambodian elementary schools before the pandemic; hence, initial support in the form of orientation/training and clear guidelines was imperative. Recognizing this, the government/schools provided teacher training/orientation on different aspects of technology-enabled distance teaching/learning. About 93 percent of schools offered some form of distance-teaching training to their teachers (Figure 4.10). However, the training provided by most schools was inadequate in terms of training focused on specific skills required for effective EdTech-based teaching. More specifically, only about 43 percent of schools trained their teachers on teaching specific subjects using EdTech when students are connected to the internet from home, and only 48 percent provided training on sustaining the attention and engagement of students during online classes. Even smaller percentages of schools provided training to the teachers on preparing lessons for EdTech-based teaching, supporting students with poor digital literacy, and managing the classroom. Training on other dimensions of online education delivery was even more limited. While training on managing the classroom, preparing lessons, and designing and conducting assessments for EdTech-based teaching was also provided, the share of schools providing these forms of training was relatively small. Furthermore, less than half of the schools provided their teachers with ready-made teaching materials and practical guidelines or manuals to support online teaching. 64 Figure 4.10: Share of schools that provided various types of orientation/training to teachers Any kind 93 Student engagement 48 Teach (distance) 43 Orientation / Training Prepare lessons 38 Support students with poor digital literacy 36 Teach (in class) 34 Psychological support 29 Classroom management 23 Assessment 23 guidelines Materials Ready-made teaching materials 44 and Practical guidelines or manuals 31 0% 20% 40% 60% 80% 100% Source: Authors’ calculations using the supplemental teacher survey 2021. Schools provided some guidance to their teachers on various aspects of remote teaching to support them during school closures. Most schools (88 percent) provided teachers guidance on the amount and frequency of work, at least to a moderate extent. A relatively smaller share of schools provided at least a moderate degree of guidance to teachers on other elements of remote teaching—technology platforms (69 percent), materials and resources available (64 percent), and student assessments (71 percent). Figure 4.11: Share of schools that provided guidance to teachers on different aspects of online teaching Amount of work 12 59 28 1 Frequency of work 12 50 37 1 Technology platforms 3 28 45 24 Provide materials/resources 4 31 45 19 Assessing students 4 24 49 21 1 0% 20% 40% 60% 80% 100% Not at all To a small extent To a moderate extent To a large extent To a very large extent Source: Authors’ calculations using the supplemental principal survey 2021. Most schools provided orientation to parents on how to support their children’s learning at home during school closure, but there were significant variations across geographical areas. As discussed in Section 3, a relatively small share of households in Cambodia have access to the internet and modern digital devices such as desktops and laptops. Even in households with children who do have access to EdTech devices and platforms, most parents are likely to not have experience in using ICT to support their children’s learning. Principals in most schools perceived inadequate digital literacy among parents as one of the key challenges facing the effective implementation of distance learning. When asked if the 65 caregivers/parents would be available and able to support their children’s education at home, the majority (83 percent) of school principals said that at most 50 percent of the parents would be able to do so. Perhaps because of this perception on the part of the school principals, a relatively large share (83 percent) of schools provided orientation to parents (Figure 4.12). However, there are substantial regional variations in the percentages of schools providing such support. A significantly larger share of urban schools (95 percent) compared to rural schools (79 percent) provided parental orientation. This urban-rural gap is significant and particularly troubling since the use of ICT before the pandemic was significantly lower in rural schools. Lower initial use of ICT combined with a lack of adequate training could have further disadvantaged the most vulnerable learners and prevented them from taking full advantage of technology-enabled education. The disparity in the provision of parental orientation across the four ecological zones is also quite significant, with a 15 percentage point difference between the worst-performing (coastal) and best- performing (plain) regions. Figure 4.12: Share of schools that provided parent orientation Urban/R Urban 95 rall Ecological Zones ural Rural 79 Plain area 86 Around Tonle Sap Lake 81 Mountainous 85 Coastal 71 Ove 83 0% 20% 40% 60% 80% 100% Source: Authors’ calculations using the supplemental principal survey 2021. 4.3 EdTech and learning outcomes: descriptive evidence While EdTech can potentially improve learning outcomes for children, there are no guarantees that it will have a positive impact on learning. Some experts argue that well-implemented EdTech can complement traditional teaching/learning and potentially raise children’s learning outcomes (Rodriguez- Segura 2022). However, there is also an argument that—if not implemented properly—EdTech can be a distraction and might actually have a negative impact on children’s learning ( Staton 2022). In the context of the pandemic-induced school closure, however, online learning was the norm in many countries rather than a complementary tool. Hence, the role of EdTech was undoubtedly much more significant than what would be expected under normal circumstances, and it is likely that better access to and use of EdTech was positively correlated with student learning during the pandemic. The analyses below, which link the data on student learning outcomes from NLA 2021 with information on ICT use, show that access to ICT infrastructure and devices, the extent to which technology was used by schools and students, and the effectiveness of ICT-based learning were all correlated with children’s learning outcomes. It should, however, be noted that these analyses simply look at the relationships between EdTech factors and learning outcomes in 2021 and are not meant to establish a causal relationship between EdTech and learning loss. 66 EdTech and learning outcomes: roles of student and home factors The 2021 NLA student scores indicate that access to ICT infrastructure at home plays a significant role in student performance. As shown in Figure 4.13, students who have access to ICT infrastructure at home significantly outperform those who do not. Students with access to a desktop or laptop computer at home have significantly higher scores than those without access to these digital devices. Similarly, students with access to the internet outperform those without internet access. For instance, differences in test scores between students who have internet access at home and those who do not are 46 scaled score points or 0.49 SD in Khmer and 37 scaled score points or 0.43 SD in math. Similarly, students with access to the internet anywhere—which primarily represents students with mobile data access—also perform better than students without access. Figure 4.13: Student test scores by student access to ICT equipment at home Source: Authors’ calculations using NLA 2021. Students’ use of various digital resources provided by the school has a positive relationship with their learning outcomes. The use of school-provided resources, such as learning material shared via messaging apps, real-time virtual lessons, and recorded lessons and digital materials, is significantly positively associated with student test scores. The test scores are the highest among students who used these resources every day during the pandemic, followed by students who use them some days, while students who never used these resources have the lowest scores on average. For instance, students who used the materials provided via messaging apps every day outperformed those who never used them by 49 points (or 0.52 SD) in Khmer and by 34 points (or 0.39 SD) in math. There is also evidence that frequent use of digital or non- digital (paper) textbooks, workbooks, or worksheets also makes a significant difference in learning outcomes. 67 Figure 4.14: Student test scores by student use of digital resources provided by school Source: Authors’ calculations using NLA 2021. An analysis of the relationship between the home environment or household/caregiver interventions during the pandemic and student learning outcomes in 2021 gives mixed results. Some forms of parental support, such as allowing children to borrow digital devices, asking them what they are learning, and checking whether they complete school assignments regularly, are positively correlated with student performance. In contrast, other forms of parental interventions, such as helping children with schoolwork or helping them create learning schedules, seem to be negatively associated with learning outcomes. The observed negative association might be because students who are already struggling, academically, are the ones who are helped by their caregivers in these direct ways more frequently. On the other hand, there is a clear negative relationship between learning outcomes and disruptions in student learning activities caused by parents. In particular, children who are never asked by their parents to do other tasks when they are learning significantly outperform (0.34 to 0.41 SD difference) those who are asked by their parents, on a daily basis, to do something else during their study time. 68 Figure 4.15: Student test scores by caregiver support at home Source: Authors’ calculations using NLA 2021. EdTech and learning outcomes: roles of teachers and schools Access to digital technology in schools and at teachers’ homes and how effectively the educators utilized EdTech are all likely to have played an important role in student outcomes. It is, therefore, relevant to look at the differences in 2021 learning outcomes across schools in terms of ICT access, digital resource utilization by teachers, and the ability of teachers to use ICT tools and conduct online classes effectively. There are indeed significant differences in learning outcomes across schools in terms of access to ICT infrastructure and devices in the schools. Schools with access to electricity and internet have significantly higher average test scores than those without access (Figure 4.16). The differences in average scores are particularly striking when comparing schools with and without internet access. The average scores of schools that have internet access are higher compared to schools without internet access by 0.27 SD in Khmer and 0.26 SD in math. Similarly, the availability of digital devices in the school is also associated with student performance. For instance, when schools are ranked according to the number of working devices (per 300 students) connected to the internet that are available for students, the top 25 percent of the schools have significantly higher average scores than schools in the bottom 25 percent—approximately 0.37 SD higher in Khmer and 0.26 SD higher in math. 69 Figure 4.16: Student test scores by access to ICT infrastructure at school Source: Authors’ calculations using NLA 2021 and the supplemental teacher survey 2021. Figure 4.17: Student test scores by access to ICT devices at school Source: Authors’ calculations using NLA 2021 and the supplemental principal survey 2021. Average performance is also greater for students whose teachers have better access to ICT at home. While the differences in test scores across teacher access to most of the ICT infrastructure and devices are small, teachers’ access to the internet at home seems to make a big difference (Figure 4.18). Students whose teachers have home internet access most or all of the time substantially outperform those whose teachers have low levels of access or no access—0.35 and 0.38 SD higher in Khmer and math scaled scores, respectively. Teachers’ access to a computer at home is also positively correlated with student performance, though the difference is relatively small. Test scores are 0.14 and 0.15 SD higher in Khmer and math, respectively, among students whose teachers have access to a desktop or laptop compared to students whose teachers lack access. Differences in test scores in terms of teacher access to tablets and smartphones are also observed, albeit small. 70 Figure 4.18: Student test scores by teacher ICT access at home Source: Authors’ calculations using NLA 2021 and the supplemental teacher survey 2021. Another key teacher attribute positively associated with student learning outcomes is teacher digital literacy. Students whose teachers have high levels of competencies in using various digital devices, such as laptops, tablets, and computers, have substantially higher scores in both Khmer and math compared to students of teachers with low levels of digital competencies (Figure 4.19). For instance, students of teachers who are very good at using tablets outperform students whose teachers are poor users of tablets by about 0.79 and 1.06 SD in Khmer and math, respectively. Furthermore, the gaps vary across the two domains depending on the device. The student test score gap is much higher in math than in Khmer in terms of teacher competency in using a laptop. On the other hand, when it comes to teacher competency in using smartphones, the gap is significantly higher in Khmer compared to math. Higher levels of teacher skills in using the internet and various internet-based tools are also positively correlated with student test scores. These include using social media and messaging apps, downloading and using other apps, and using LMSs and videoconferencing apps. In contrast to the performance gaps observed across teachers with different competency levels in using digital devices, however, the test score gaps are consistently higher in Khmer than in math when comparing teachers with different competencies in using internet-based tools. 71 Figure 4.19: Student test scores by teachers’ digital literacy/competency Source: Authors’ calculations using NLA 2021 and the supplemental teacher survey 2021. Student performance is also expected to have been influenced by the different interventions implemented by schools, including the utilization of EdTech, for effectively delivering distance learning in response to the pandemic. As discussed earlier, many schools provided teacher orientation/training to support educators in effectively teaching their students in a remote learning environment. This initial training would be expected to have some positive effects on student learning outcomes. Furthermore, student performance would also be expected to be positively correlated with the frequency of teaching activities such as giving homework assignments, conducting real-time online classes, and providing other learning materials for home-based learning. Similarly, student support activities such as frequent follow-ups to encourage students to complete their assignments and check-ins to assess how students are generally feeling would also be expected positively affect student performance. Even more importantly, the total hours of online classes and the amount of schoolwork covered through these classes also likely have influenced children’s learning outcomes. EdTech-related trainings for teachers to support distance teaching during the pandemic are associated with higher student scores in NLA 2021. While students of ICT-trained teachers generally outperform students whose teachers were not trained, the results vary across training topics (Figure 4.20). Trainings on designing and conducting assessments and sustaining student engagement are associated with relatively large score differences for both Khmer (8–13 SD) and math (10–18 SD). For most of the other training topics, however, the average scores are only marginally higher among students of ICT-trained teachers compared to students whose teachers did not receive training in that particular topic. Furthermore, the breadth of the ICT training or orientation also seems to matter. Among the schools that provided ICT training to teachers, schools that provided training on more topics have progressively 72 higher average scores (Figure 4.21). For example, schools that provided teacher training on seven or eight topics substantially outperform students from schools that provided training on fewer topics.46 This finding suggests the need for a more holistic training strategy that covers a broad set of topics. Figure 4.20: Student test scores by orientation/training for teachers Source: Authors’ calculations using NLA 2021 and the supplemental teacher survey 2021. 46 The share of teachers who received training on seven or eight modules is relatively low (14.7 percent), and about 61 percent of the teachers received training on only three or fewer modules. 73 Figure 4.21: Student test scores by the number of ICT training modules provided to teachers 500 488 480 472 465 460 460 455 447 444 436 440 420 400 None 1 to 3 Modules 4 to 6 Modules 7 to 8 Modules Khmer Math Source: Authors’ calculations using NLA 2021 and the supplemental teacher survey 2021. The teaching activities implemented by the schools during the pandemic are associated with better student performance in NLA 2021. Students from schools that implemented different teaching activities during the pandemic have better learning outcomes (Figure 4.22). These activities include not just the core teaching activities such as sending assignments, sharing learning materials on LMS, offering live virtual classes, and asking students to submit completed school assignments but also those that support students such as checking in with them to ensure that they are completing assignments and giving them helpful study tips. Furthermore, the frequency of these teaching and support activities also matters. Students performed better—across both domains—in schools that provided these teaching resources more frequently. For instance, the average scores are higher (by 0.25–0.29 SD) in schools that offered daily live virtual classes compared to schools that provided only weekly classes. Figure 4. 22: Student test scores by activities implemented by schools (as reported by students) during school closure Source: Authors’ calculations using NLA 2021. 74 More broadly, student performance is better in schools with higher total hours of online classes and schools that were able to cover a relatively large share of the schoolwork during the pandemic (Figure 4.23). Higher scores among students from schools that conducted more hours of online classes do not suggest that online classes were more effective than in-person classes. Instead, as student learning predominantly took place online during the pandemic, the better outcomes for students who had access to more online classes most likely reflect the effects of more hours devoted to studying. Not surprisingly, the scores positively correlate with the share of the schoolwork completed. This suggests that schools that were the most effective in delivering technology-enabled distance learning to their students during the pandemic were able to cover a larger share of the schoolwork and consequently produce better learning outcomes for the children. Figure 4.23: Student test scores by total hours of online classes, share of schoolwork covered through online classes, and teacher preparation Source: Authors’ calculations using NLA 2021 and the supplemental teacher survey 2021. EdTech and learning outcomes: regression findings The above findings on the positive association between EdTech and student learning outcomes are largely supported by multiple regression results that consider the effects of different student and household characteristics, student learning experiences, and school characteristics on student performance. Table 4.1 presents results from the regression analysis for the determinants of student Khmer test scores in NLA 2021. Model 1 examines how student access to and use of ICT and household interventions are related to student Khmer test scores. Model 2 focuses on the teaching and support activities provided by schools to students during the pandemic and the availability of ICT infrastructure and devices in schools. Model 3 looks at the relationship between teacher digital competency and teacher access to ICT at home. Model 4 is a comprehensive model that incorporates all the explanatory variables in Models 1 –3. All four models control for student and household characteristics (including household location), student learning experiences, and school characteristics.47 Given the absence of pre-COVID learning outcomes data for the students tested in NLA 2021, these models simply look at the relationships between EdTech factors and learning outcomes in 2021 and are not meant to analyze the relationship between EdTech and learning loss. Table 4.1 also presents Model 5 which is an extension of Model 4 that includes an additional control variable representing an assumed score in 2016 for each sampled student included in NLA 2021. These assumed scores were obtained by matching students in 2021 with students in 2016 along various 47 Locational characteristics are accounted for in the regression models by including fixed effects for ecological zones and provinces. 75 dimensions.48 As these matched student scores help to partially control for some of the unobserved student characteristics (such as latent student academic ability), the regression results in Model 5 serve to check the robustness of the findings from Model 4 and provide a potentially causal interpretation of the results. Analyzing the relationship between student learning outcomes and student access to ICT at home, student use of school-provided resources, and caregiver support during school closure, a number of variables can be identified as significant determinants of student test scores. As shown in Model 1, the identified positive determinants include student access to the internet, use of self-study worksheets, and access to real-time virtual lessons. More precisely, students who had access to the internet, which for most students was via mobile data, have significantly higher Khmer scores than other students. Similarly, students who used self-study worksheets provided by MoEYS and other printed workbooks and worksheets, at least some of the time, significantly outperform those who never used them. Students who had access (sometimes or daily) to real-time virtual lessons from their teachers have significantly higher Khmer scores than students who never had access to these learning resources. On the other hand, access to real-time lessons from extra-class teachers (that is, private tutoring) is associated with significantly lower Khmer test scores. This might suggest that it is usually the academically weaker students who take private tutoring lessons. The lack of statistical significance of this factor in Models 4 and 5, however, indicates that there is no causal link between extra-class lessons and Khmer test scores. Student use of paper-based workbooks and worksheets, self-study worksheets provided by the MoEYS, and real-time virtual lessons remain significant in Model 4. The regression results indicate that caregiver/parental interventions—both positive and negative— have a significant relationship with student outcomes. For instance, students whose parents regularly checked whether they were completing the assignments outperform those whose parents never checked their assignments, though the statistical significance disappears once the prior (matched) test score is considered. On the other hand, students who were frequently asked to do something else during their study time have significantly lower scores, and the effect is robust across models. It is statistically significant even when accounting for prior test score, suggesting a potential causal effect. The results on the effect of ICT availability in school and various forms of school interventions on learning outcomes are mixed. While the availability of ICT infrastructures such as electricity and the internet does not have a statistically significant relationship with student learning outcomes, there is some evidence that the availability of digital devices for students in schools is associated with better student scores. However, the positive association between digital device access in schools and student outcomes is not consistent across the models. Among the school teaching and support activities, asking students to submit completed assignments seems to be a positive determinant of student test scores, a finding that is consistent across the models. In contrast, learning materials and tips on self-study are negatively associated with Khmer test scores. This might suggest that schools that provided more materials on self-study perhaps provided less of other more effective forms of instruction. As might be expected, the share of schoolwork 48 The matching was done using the nearest neighbor method. First, students in 2021 were matched with those in the same district in 2016. In cases where matches could be found in the same district, they were matched with students from the same province. The matching was restricted to public school students as the sample of private school students was too small to allow for credible matching. Furthermore, cross-gender matching was not allowed, meaning male students were matched with other male students and females with other female students. The students were matched on 20 different variables (all categorical variables) related to household asset ownership, access to basic services in the household (electricity, water, and sanitation), principal construction material of the house, presence of books in the household, presence of both parents at home, parents’ occupations and education, student age, number of siblings, and preschool attendance. 76 completed is highly statistically significant with a relatively large coefficient. However, the coefficient is not statistically significant in the more comprehensive models. Teacher access to ICT at home and teacher competency in some elements of the digital skill set are significantly positively associated with student learning outcomes. Teacher access to the internet at home seems to be a positive correlate of Khmer test scores. The coefficients remain statistically significant in Models 4 and 5, which provide added credibility to its positive association with students’ Khmer test scores. Similarly, teacher competencies in digital devices, including laptops, tablets, and smartphones, are also positive determinants of student test scores. Results from similar regressions on math test scores provide additional insights into the association between ICT access and use and student learning outcomes. For instance, there is some evidence on the effectiveness of other school-provided resources, including digital textbooks, workbooks, or worksheets, and recorded lessons or other digital materials from other sources (Table A.4). Furthermore, the positive relationship between these resources and Khmer test scores—including worksheets provided by MoEYS and other printed workbooks and worksheets—is more pronounced and consistent in the case of math scores. In terms of the potential association between household interventions and student learning during the pandemic, students whose parents allow them to borrow digital devices for online learning have significantly higher scores on average. Apart from the EdTech and distance learning factors discussed above, several student characteristics and household attributes are significant determinants of student performance. As shown in Table 4.1, on average, female students have a significantly higher score than males in all models. The number of siblings is negatively associated with learning outcomes, though the relationship is significant in only two (out of five) models. Test scores in Khmer are generally higher among students from wealthier households. In particular, students from the top 40 percent of the wealth distribution outperform the rest, and this result is consistent across all the models. Even after accounting for household SES, having a father who has a university education seems to matter, but the effect of mother’s education is not statistically significant. Furthermore, households’ poverty status—as measured by hunger among children and whether or not children had to work during the pandemic to support their families—has a significant negative relationship with learning outcomes. The effect of work, in particular, is substantial and consistent across the models. Rural location seems to have a negative association, though the result is not consistent across models. Student absentee days and grade repetition—indicators that reflect the student’s schooling experiences— are negatively associated with student test scores, with the relationship remaining consistent across the models. While the evidence on the school and class size is mixed, school type is a significant determinant of student outcomes, with private school students outperforming public school students. 77 Table 4.1: Determinants of student Khmer test scores Model 1 Model 2 Model 3 Model 4 Model 5 Student ICT access Internet access (anywhere) 11.5559** 7.6907 7.7848 (4.6221) (4.9532) (5.5830) Student use of school resources Paper textbooks, workbooks, or worksheets 7.9200 15.6127** 14.1495 (6.6188) (7.5381) (8.8150) Self-study worksheets (prepared by MoEYS team) 10.9805*** 7.1575* 7.7298 (3.8219) (4.2186) (5.1287) Real-time lessons by a teacher from school on a video communication program 9.0165** 13.1554** 8.3619 (4.4000) (5.3283) (5.9766) Real-time lessons by extra-class teacher from school on a video communication program −10.7952*** −3.9090 −2.2050 (4.0549) (4.4509) (5.5553) Caregiver support Help with schoolwork −3.7177 −7.7181* −10.5005** (4.0727) (4.3367) (5.0920) Check whether completing school assignments regularly 9.1181** 8.1363* 9.0031 (4.6132) (4.9233) (5.5086) Explain new content −10.3958** −8.0716 −8.1881 (4.7665) (5.1180) (5.9161) Ask student do something else when learning −15.6736*** −14.7286*** −12.9328*** (3.8562) (4.1838) (4.9707) School ICT access Working devices available for students (per 300 students) 0.4004 0.4322* −0.8235 (0.2505) (0.2578) (1.6538) School teaching activities and support Learning materials for self-study −8.1066* −7.8921 −12.5167* (4.6621) (5.0681) (6.4566) Asked to submit completed assignments 18.5553*** 14.6687** 16.7221** (6.8987) (7.4759) (8.3869) Helpful tips on self-study −8.4035* −1.6803 −4.8949 (4.7270) (5.1122) (6.0426) Coverage of schoolwork Share of schoolwork completed 19.8987* 13.9812 17.8420 (11.0463) (12.6666) (15.5678) Teacher ICT access at home Internet 9.5260** 11.8077** 10.3536* (4.4515) (4.9249) (5.7776) Teacher digital competency Laptop 12.5936* 14.7900** 16.0381 (6.5791) (7.3304) (10.1888) Tablet 14.3239** 22.6860*** 38.3876*** (7.2789) (8.3105) (10.9252) Smartphone 9.8151* −1.5448 −5.4423 78 Model 1 Model 2 Model 3 Model 4 Model 5 (5.6068) (7.3059) (8.9852) Downloading and using app −19.8306*** −20.4010*** −20.3770** (6.4769) (7.9007) (10.2917) Khmer score 2016 (matched) 0.0504* (0.0268) Student/household characteristics Student is female 35.0811*** 35.5758*** 35.7582*** 33.9178*** 32.1237*** (3.5773) (3.5871) (3.3835) (3.8488) (4.5905) Number of siblings −2.2872* −1.8030 −2.2944** −1.6522 −2.1181 (1.1712) (1.2282) (1.1009) (1.3249) (1.5732) Wealth quintile (reference: bottom quintile) Second quintile 11.0692* 8.7657 15.5892*** 7.5706 8.7537 (6.0534) (5.9910) (5.6714) (6.4719) (7.2281) Third quintile −0.8569 6.0042 4.5641 −0.8359 −2.9089 (6.4264) (6.2170) (6.0723) (6.9090) (7.6081) Fourth quintile 20.1288*** 24.3988*** 27.2214*** 17.7874*** 15.3652** (6.2185) (6.1756) (5.9091) (6.7950) (7.6160) Top quintile 25.8526*** 36.7032*** 34.8752*** 26.2287*** 22.7118** (7.1016) (6.8538) (6.4838) (7.8141) (8.9069) Father has university education 6.6020 13.6772* 13.9674** 12.1833* 10.8681 (5.9873) (7.2518) (6.4192) (7.2338) (8.8705) Household owns books 5.9854 6.7977* 9.4146** 7.0960* 4.5306 (3.8964) (3.9280) (3.6959) (4.2031) (4.7781) Child is hungry −11.5218** −6.4202 −9.3539* −9.7788* −10.8158 (5.2188) (5.3231) (4.9518) (5.5444) (6.8396) Child worked during school closure (reference: did not work) Worked some days −28.1638*** −32.8513*** −33.7778*** −26.1291*** −23.3452*** (5.7635) (5.4189) (5.3316) (5.8731) (6.4800) Worked every day −41.4675*** −52.2984*** −53.2817*** −41.5569*** −42.9965*** (9.7943) (9.6841) (8.8084) (10.0688) (11.5982) Rural location −3.4421 −9.1985* −6.5005 −8.3426 −7.2995 (4.8239) (5.2404) (5.0014) (5.8803) (9.0944) Student experience/engagement Annual absentee days −2.5750*** −3.5026*** −2.4950*** −3.4406*** −4.0408*** (0.4271) (0.5399) (0.4245) (0.5892) (0.7636) Has repeated grade −17.8222*** −13.5385*** −12.9206*** −16.1277*** −14.2538** (4.4644) (4.4456) (4.2137) (4.9080) (5.6501) School characteristics School enrollment 0.0110 0.0168 0.0264** 0.0184 0.0185 (0.0116) (0.0119) (0.0116) (0.0138) (0.0178) Private school 16.6211*** 16.0154** 16.6422*** 8.8636 — (5.9372) (6.9867) (5.5578) (7.8074) — Constant 480.4129*** 448.9789*** 439.8651*** 504.8478*** 429.1242*** (29.3420) (33.7024) (26.6416) (37.2408) (48.5602) Observations 2,120 2,124 2,476 1,796 1,306 R-squared 0.3475 0.3322 0.3368 0.3637 0.3565 Source: Authors’ calculations using NLA 2021, the supplemental principal and teacher surveys 2021, and NLA 2016. Note: Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. 79 While the above regression findings indicate a positive relationship between certain EdTech measures and learning outcomes, they do not provide any evidence on the impact of EdTech measures on learning loss during the pandemic. However, it is possible to use the learning loss simulation model discussed in Section 2 to provide some indicative estimates of the contribution of the EdTech measures toward mitigating learning loss. The learning loss estimates in Section 2 assume that the effectiveness of mitigation measures is 14 percent in the optimistic scenario and 7 percent in both the intermediate and pessimistic scenarios. These values are, in turn, based on the assumption that mitigation effectiveness is a product of three factors: the government supply or coverage of remote learning modalities (G), the share of learners who have access to remote learning (A), and the expected effectiveness of remote learning (E). Hence, some understanding of the contribution of the EdTech-based measures implemented in the country can be obtained by asking how the learning loss estimates would have changed in the absence of remote learning measures or in cases where the measures were limited. Figure 4.24 presents simulation estimates of LAYS in the intermediate scenario for a range of assumed mitigation effectiveness assumptions.49 As expected, the estimate for LAYS decreases, and the percentage loss in LAYs increases progressively with smaller values of mitigation effectiveness. In particular, if the supply of remote learning measures was nonexistent (resulting in zero mitigation effectiveness), then the loss in LAYS would be around 25.8 percent—which is 1.7 percentage points larger (7 percent greater) than the loss in LAYS estimated in Section 2 for the default intermediate scenario. As shown in Figure 4.25, the economy’s lifetime earnings loss also increases substantially as mitigation effectiveness decreases, with the loss peaking at US$28.1 billion (or 38.9 percent of GDP) in the absence of remote learning measures. These figures suggest that the provision of the EdTech-based distance learning measures in the country contributed to reducing the loss in LAYS by around 7 percent and loss to the economy lifetime earnings by US$2 billion compared to the default intermediate scenario. 49 The range of mitigation effectiveness reflects possible combinations of learning effectiveness and student use. As noted in in the Cambodia Economic Update: Living With COVID report (World Bank 2021a), it is difficult to estimate country-specific mitigation effectiveness value for Cambodia because of lack of data. Utilizing data from five rounds of household surveys, the economic update notes that, during the pandemic, while almost all students had access to at least one resource that could allow them to participate in remote learning programs, a relatively small percentage of students participated in remote learning. Accordingly, it assumes that the use of remote learning alternatives was only 19.7 percent in the pessimistic and intermediate scenarios and 30.2 percent in the optimistic scenario. It further assumes that the effectiveness of remote learning alternatives is 40, 30, and 20 percent for the optimistic, intermediate, and pessimistic scenarios, respectively. Based on these assumptions, the mitigation effectiveness value used for optimistic, intermediate, and pessimistic scenarios in the economic update are 12, 6, and 4 percent, respectively. Figures 4.24 and 4.25 include simulation results for these effectiveness values as well. 80 Figure 4.24: Loss in LAYS for Cambodia in the intermediate scenario under different mitigation effectiveness assumptions 5.4 5.28 5.31 27 5.20 5.21 5.25 5.08 5.15 5.18 5.2 5.12 26 Loss in LAYS (%) LAYS (years) 5.0 25.77 25 25.30 4.8 24.83 24 24.36 24.12 4.6 23.88 23 23.41 4.4 22.93 22 22.45 4.2 21 4.0 20 0 2 4 6 7 8 10 12 14 Mitigation effectiveness (%) LAYS (years) Loss in LAYS (%) Source: Simulation model from Azevedo et al. (2022). Figure 4.25: Present value loss to Cambodian economy lifetime earnings under different mitigation effectiveness assumptions 34.0 40 Earnings decrease (% of GDP) Earnings decrease ($ bil.) 28.1 27.6 27.1 39 29.0 26.6 26.3 26.1 25.5 38.9 25.0 24.5 38 24.0 38.2 37 37.4 36.7 36 19.0 36.3 36.0 35 14.0 35.3 34 34.6 33.8 33 9.0 32 4.0 31 0 2 4 6 7 8 10 12 14 Mitigation effectiveness (%) PV of loss to economy (2107 $PPP bil.) PV of loss to economy (%) Source: Simulation model from Azevedo et al. (2022). 81 5. Conclusions and policy implications This report has analyzed the learning and future earnings losses experienced by Cambodia as a result of the COVID-19 pandemic, summarized the EdTech-based distance learning responses implemented in the country and their relationship with learning outcomes, and discussed the country’s EdTech readiness at the policy and school levels. To analyze learning loss, it has estimated the actual decline in learning assessment scores of grade 6 students between 2016 and 2021, utilizing student learning outcome data from the government’s 2016 and 2019 national learni ng assessments. Supplementing this analysis, it has also provided estimates of the pandemic’s impact on learning poverty, LAYS, expected annual earnings of individuals, and expected economy wide earnings using a simulation model that links past school closures caused by the pandemic to losses in student learning outcomes. The EdTech-based responses discussed include activities implemented by the government as well as actions taken by schools. The report has attempted to explore the role of EdTech in supporting student learning by linking school-level responses with the NLA 2021 student assessment data. Furthermore, it has taken stock of the state of EdTech in schools and policy provisions for supporting EdTech development and expansion to provide preliminary insights into EdTech readiness of Cambodia. 5.1 Summary of findings Current state of learning outcomes Student assessment data from NLA 2021 show that the learning outcomes of grade 6 students are relatively low in both Khmer and math. Students, on average, score below 50 percent in both domains in terms of the percentage of questions answered correctly. In Khmer, student performance is particularly low in dictation, with an average student getting only 24 percent of the questions correct. In math, students perform relatively poorly in numbers, measurement, and geometry and relatively well in algebra and statistics. The average score in geometry—the math subdomain showing the lowest student performance— is only 35 percent. As might be expected, in both Khmer and math, scores get progressively worse as the content difficulty level increases, with students getting only 25 percent of the answers correct on questions from the most difficult or advanced levels. Significant differences in learning outcomes are observed across gender and age groups, with girls and ‘at age’ students performing better than boys and other age groups. While female students outperform male students in both Khmer and math, the gender performance gap in Khmer is especially striking. Females consistently outperform males across all subdomains and content difficulty levels in Khmer, and among the Khmer subdomains, the gender gap is largest for dictation. While student performance is the worst in the advanced-level content area, the gender gap in performance is the smallest for this difficulty level. Regarding learning outcomes across age groups, the ‘at age’ students have the best performance in both subjects, followed by underage children, while overage children have the worst performance. Furthermore, there are significant differences in learning outcomes across student learning experiences as well as household SES and home environment. In terms of students’ learning experiences, students who have attended preschool, students who have never repeated a grade, and those who attend classes more regularly consistently outperform—across both domains—other children. Students from wealthier households outperform those from poorer households in both subjects, and the performance gap between students in the extremes of the household wealth distribution is staggering. Similarly, learning 82 environment at home, as measured by the availability of books at home, is also associated with better learning outcomes. Urban students outperform rural students in both Khmer and math scaled scores. While the urban- rural gap in performance is substantial for both subjects, the gap is particularly large in Khmer. The urban school students have consistently higher scores than rural students in each content area and at all levels of difficulty in both Khmer and math. There are differences in outcomes across the four ecological zones, with students from the mountainous regions lagging behind students in the other three areas. Regression analyses of the determinants of student learning outcomes support the above findings (which are based on a descriptive analysis of the NLA 2021 data) and provide insights into other factors that could influence learning. Consistent with the above findings and those from other countries, the regression results indicate that female students significantly outperform males, especially in Khmer. Similarly, household SES is a significant determinant of learning outcomes, with students in the wealthiest (top 40 percent of household wealth distribution) households significantly outperforming the students from the less wealthy households. Furthermore, the regression results suggest that economic hardships during the pandemic—which led to inadequate food for some families and required some students to work to support their families financially—had significant adverse effects on learning outcomes. Other important factors found to be negatively correlated with student performance include grade repetition and absence from school. Learning loss due to the pandemic The low student learning outcomes in 2021, summarized above, reflect substantial learning losses experienced by students due to the periodic extended school closures induced by the pandemic. A simple descriptive analysis of the NLA data shows that while student learning outcomes improved slightly between 2013 and 2016, there was a massive decline in student performance between 2016 and 2021, with substantial losses in both math (56 points) and Khmer (43 points) scores. But the loss in learning was not experienced equally by all students; instead, it varied significantly between males and females and across geographical locations. In general, learning losses were higher for males than females—especially in Khmer. The relatively lower loss in Khmer for females seems to have been driven mainly by rural females who experienced the least amount of learning loss in this domain. The urban-rural gap in learning loss, however, was more prominent in math than in Khmer, with urban students suffering a significantly higher learning loss. There was also some variation in learning loss across Cambodia’s four ecological zones. In general, the losses were somewhat smaller in the Tonle Sap region compared to the other three regions. There was also a substantial disparity in learning loss across students from different socioeconomic backgrounds, with students from the wealthiest households suffering the highest levels of learning loss in both Khmer and math. More specifically, the losses were relatively low for the bottom two wealth quintiles (that is, the bottom 40 percent of the wealth distribution), somewhat higher for the third quintile, and highest for the top two quintiles (that is, the top 40 percent of the wealth distribution). Given the more considerable learning losses experienced by students from wealthier households, it is not surprising that learning losses were also higher among students from private schools—which presumably have a higher concentration of children from wealthier students—than from public schools. Estimates of learning loss derived using regression techniques also generally support these findings. These regression models control for trends before the pandemic by including data from NLA 2013 in addition to data from NLA 2016 and NLA 2021. They essentially examine the changes in learning outcomes 83 between 2016 and 2021 relative to the changes in outcomes between 2013 and 2016 while controlling for a host of other student, household, and locational factors that could potentially influence student outcomes. Consistent with the findings from the descriptive analysis, the regression results show substantial declines in average scaled scores between 2016 and 2021 at the aggregate national level—28 points for Khmer and 41 points for math when control variables are used in the model. The regression results also confirm that significant learning losses were experienced by students across the board. Both males and females, urban as well as rural dwellers, residents of all four ecological zones, and students from all SES groups experienced substantial learning losses. For example, using the regression model with controls, the estimated learning losses in Khmer for rural males, rural females, urban males, and urban females were around 28, 24, 31, and 38 points, respectively. Disparities in learning loss across the four gender-location subgroups (rural males, rural females, urban males, and urban females) were generally small in the regression results. However, urban students—both males and females—experienced significantly larger learning losses in math. Among the four ecological zones, math learning loss was lower in Tonle Sap compared to the plain and coastal regions, while the result for the mountainous region was inconclusive. Consistent with findings from the descriptive analysis, the regressions also show substantial disparities in learning loss across income groups. Children from the bottom wealth quintile experienced significantly lower losses in both domains than children from wealthier households. This indicates a convergence in learning outcomes between the children from the poorest families and the rest of the children. Furthermore, confirming the findings from the descriptive analyses, the regression results show that private school students experienced higher learning losses than public school students. Findings from learning loss simulations based on the model of Azevedo et al. (2022) indicate that the pandemic is expected to result in substantial reductions in LAYS and an increase in learning poverty. Cambodia was facing a learning crisis even before the pandemic, as indicated by its 90 percent learning poverty rate in 2019. The combined impacts of school closures and income shocks to families are expected to have significantly worsened this crisis: it is estimated that Cambodia will experience a decrease in the average LAYS for the current cohort of school-age children by 21–26 percent. Similarly, the learning poverty rate is expected to have risen to 99.7 percent. The corresponding expected impacts on individual and economy wide earnings are large. As a result of the loss in LAYS, the average annual earning per student is expected decline by 8.8 percent to 10.6 percent compared to the baseline earning of US$6,077 per year in 2017 PPP US$. In present value terms, this loss in earnings per student translates to an estimated loss of US$23–28 billion or 32.3–38.8 percent of GDP to the economy as a whole. Government- and school-level EdTech-based responses to the pandemic: activities and effects In response to the pandemic, the government prepared a systematic response plan that emphasized the use of EdTech-based distance learning to mitigate the adverse effects of prolonged school closures. Following the closure of schools in March 2020 (within a span of two to three months), MoEYS developed several guidelines and directives to offer distance learning to students. These included a directive on distance learning and e-learning, an operational guideline for distance learning implementation, and guidelines on e-training for teacher education. The government also took initiatives for the development of online lessons and other digital content to increase the availability of e-learning resources for students. Furthermore, MoEYS developed paper-based student worksheets for grades 1 to 12 and instructed schools 84 to print and distribute them to support students’ remote learning. MoEYS endorsed the formal response plan, entitled ‘Cambodia Education Response Plan to the COVID-19 Pandemic’, in July 2020. The use of EdTech to ensure continuity of education for children was one of the plan’s key elements. The government used both modern digital channels and traditional mediums to disseminate educational content to students. The conventional channels included TV and radio broadcasts, while the contemporary channels included various web- and app-based platforms, including Facebook pages, YouTube channels, websites, and mobile applications. The government also promoted messaging apps as crucial tools to facilitate student-teacher interaction and engagement during school closures. Toward this, MoEYS instructed schools and teachers to create messaging groups on Telegram or other mobile messaging applications to communicate with students and parents about distance learning and e-learning and facilitate these learning activities. Teachers were also advised to send video lectures, lesson summaries, worksheets, and responses to questions about lessons from students via Telegram. At the school level, findings from the survey of principals and teachers indicate that most schools provided some form of study materials to students for distance learning during school closure. The schools used both digital and non-digital (printed) materials to support home-based learning by students. While digital and non-digital materials were used in equal proportions in the nation as a whole, digital materials were used more widely in urban schools, and non-digital materials were more prevalent in rural schools. Messaging apps emerged as critical channels through which schools delivered online education during the pandemic. Since messaging apps such as WhatsApp and Telegram were readily available and accessible throughout the country, they were widely used as a medium not only for sharing learning materials and homework assignments but also for conducting online classes. Specifically, about two-thirds of the teachers reported using messaging apps at least once a week to teach virtual classes and almost half of them used the tools daily. Smartphones were the EdTech device used most frequently by both teachers and students. In line with the trends in other low-income countries, access to smartphones and mobile data was quite high in Cambodia during the pandemic. Most teachers had regular access to smartphones and mobile data, enabling them to communicate with students and their families easily. Almost two-thirds of the students reported that they used smartphones, the EdTech device used most frequently by students for schoolwork and homework. Student learning outcomes in 2021 were associated with student access to ICT at home and household interventions on student learning activities during COVID-19. With regard to ICT infrastructure and devices, students who had access to the internet at home had higher learning outcomes. Household interventions that had significant positive relationships with student learning outcomes included asking the student what they were learning, regularly checking whether they completed their assignments, and allowing them to borrow digital devices for online learning. On the other hand, asking students to do something else during their study time negatively affected the outcomes for the children. Similarly, student learning outcomes were also significantly related to teaching-related interventions on the part of the school (including the provision of learning resources), teacher access to ICT at home, and teachers’ digital competency. The teaching-related interventions by schools that had a positive effect included sending homework to students and asking them to submit completed assignments and the total share of total schoolwork covered through online classes. Students’ use of various resources provided by schools during the pandemic was also positively associated with their test scores. These resources included physical (printed) and digital textbooks, workbooks, worksheets developed by schools, worksheets 85 provided by MoEYS for self-study, real-time virtual lessons delivered by their teachers, and recorded lessons or other digital materials from other sources. Students whose teachers had better digital competency and access to the internet and EdTech devices such as laptops, tablets, and smartphones at home also had higher learning outcomes. While it is not possible to estimate the impact of EdTech-based distance learning interventions on student scores, findings from the learning loss simulations suggest that the provision of such measures did contribute to mitigating learning loss. They indicate that if these interventions had not existed, the loss in LAYS experienced by Cambodia would have been 7 percent larger than that estimated under the default intermediate scenario, which assumes that the mitigation effectiveness of distance learning is 7 percent. This would have translated to an increase in economy lifetime earnings loss of around US$2 billion in 2017 PPP US$. EdTech readiness of Cambodia: policy and practice Cambodia has a largely supportive policy environment for expanding EdTech-based teaching and learning. During the past two decades, the government has approved several policies, plans, and strategies that support the expansion and use of EdTech to improve the quality and resilience of the education system. These policies highlight the importance of EdTech in increasing access to education, enhancing education quality, developing students’ digital literacy for the labor market, and increasing productivity within MoEYS. The Education for All: National Plan 2003–2015, for example, emphasizes the need to expand the use of EdTech in teaching and learning, information management, and distance learning. The government’s first EdTech specific policy document (published in 2005)—the Policy and Strategies on Information and Communication Technology in Education in Cambodia—aims to increase ICT access among educators and students to promote autonomy and lifelong learning and produce ICT-competent graduates. Subsequent plans and policies have further supported the integration of technology in education. For example, the Education Strategic Plan 2009–2013 and Master Plan for Information and Communication Technology in Education: 2009–2013 aim to not only increase access to ICT but also build capacity in ICT use and utilize ICT as an alternative platform to increase access to education. These aims are further supported by the latest version (dated 2018) of the Policy and Strategies on ICT in Education, which emphasizes increasing ICT use in education and enhancing digital literacy and competency among students. The expansion of ICT use for teaching and learning is also emphasized by the Education 2030 Roadmap and the Education Strategic Plan 2019–2030. This EdTech supportive policy environment was one of the factors that enabled the government to implement distance learning interventions in response to the pandemic quickly. These policies also highlight the need to train preservice and in-service teachers on ICT skills to improve their teaching and administration work. For example, the Master Plan for Information and Communication Technology in Education: 2009–2013 and the Teacher Professional Standards emphasize that teachers should be well trained in ICT use to teach the students competently. Furthermore, they indicate that teachers should also be able to help their fellow teachers and students become competent users of ICT in learning. Similarly, the Education Strategic Plan: 2004–2008 emphasizes the inclusion of ICT in teacher training and professional development, the use of ICT in teaching and learning and school management, and the expansion of ICT facilities. Despite the continuing emphasis of the different policies on promoting EdTech for teaching and learning, there remain a number of policy gaps that limit the EdTech readiness of Cambodia. First, it should be emphasized that effective use of EdTech requires schools to integrate EdTech in teaching and learning systematically. Currently, however, there are no national policies that delegate this responsibility to schools. Second, although the policies recognize the importance of building digital competency and 86 EdTech-based teaching capacity of teachers, Cambodia has yet to establish a digital standards/competency framework for teachers and a system for evaluating their digital competencies. These are critical for assessing the training needs of teachers and devising training programs accordingly. Furthermore, while ICT training is included in preservice training programs, they are not included systematically in in-service teacher training programs. Third, while it is encouraging that the Curriculum Framework for General Education and Technical Education specifies ICT skills as one of the core competencies for students, the translation of this specification to concrete teaching learning activities will require Cambodia to first develop a student digital standards/competency framework, which is currently missing. Fourth, as the plans outlined for the expansion of EdTech infrastructure and devices and internet connectivity in the Education 2030 Roadmap are quite ambitious, there is also a need for a clear financing strategy and plan to achieve the stated targets. Furthermore, these plans and strategies would also need to specify the monitoring mechanisms and approaches to providing technical support to schools. Going from the policy level to the school level, data from the principal and teacher surveys indicate that the EdTech supportive policies on paper have not translated into greater EdTech readiness among teachers. In general, the EdTech competency levels of teachers are low, and most schools are not aware of policy provisions for ICT training of teachers. Most of the school principals surveyed for this study said that ICT training was not included in the initial teacher training programs for primary students or they did not know if ICT was included in these trainings. Most teachers reported receiving little orientation or training even during the pandemic, which was when there was a real and urgent need to properly orient them for EdTech-based distance teaching. Furthermore, the vast majority of the teachers perceive their digital competency (in both general use of ICT as well as the use of ICT for teaching) to be poor and believe that inadequate digital literacy among teachers is a significant challenge. These findings suggest that the policy aims of enhancing the EdTech capacity of teachers were being translated into practice at the school level at a minimal scale. Access to ICT infrastructure and digital devices in schools is also limited and falls short of what is envisioned in the policies. For instance, only 29 percent of the schools have internet access, which falls significantly short of the vision—as outlined in Cambodia’s Education 2030 Roadmap—to have internet connectivity in 70 percent of primary schools. Furthermore, Cambodian primary schools have limited provisions for digital devices for teacher and student use. On average, schools have about four working devices (per 300 students) available for student use. Again, this falls significantly short of the plans (as outlined in the ICT master plan) for providing 26 computers to accommodate between 40 and 60 students. The lack of access to digital devices is reflected in school principals’ perception of the sufficiency of ICT- based distance learning infrastructure and devices in their schools. While this is a concern in schools across the country, the lack of access to digital devices is particularly severe in rural schools. Enhancing ICT use among students and increasing their competency in using them are also at the core of the EdTech policies and plans, but EdTech competency among students continues to be limited. While the vast majority of the school principals recognize the importance of enhancing students’ competency in using digital technology and using ICT to augment student learning, use of ICT devices by students in Cambodian primary schools was quite limited before the pandemic. This is also reflected in the low levels of student competency in using digital technology for learning as assessed or perceived by their teachers. Moreover, inadequate access to ICT infrastructure and devices in the students’ homes— particularly among poor and rural students—is also a contributing factor when it comes to low competency in ICT use among the students. 87 5.2 Policy implications Addressing learning loss and low learning outcomes The substantial learning and earnings losses due to the pandemic clearly point to an urgent need for short-term interventions aimed at learning recovery and accelerated learning. As noted by UNICEF, UNESCO, and the World Bank (2022), in the absence of swift and focused actions aimed at learning recovery, learning losses could increase over time, even for students who return to school, thereby making mitigating measures increasingly more difficult. Hence, the most critical short-term interventions necessary in the current post-COVID-19 context are those designed to help students accelerate and recover learning. Learning recovery and accelerated learning will require a number of systematic sub-interventions such as bringing all students back to school; providing adequate academic and psychosocial support, especially to academically lagging students, to ensure that they do not drop out due to difficulties in catching up with their peers; modifying the curriculum appropriately to allow for the coverage of prioritized topics within a compressed time frame; and training teachers to teach effectively and efficiently in an environment where students are at different levels of learning. This may be particularly challenging in Cambodia, given that classes are conducted in double shifts and operate for only four hours during each school day. While some of these interventions might already be under way, it will be useful for policy makers to consider using approaches such as the RAPID framework proposed by UNESCO, UNICEF, and the World Bank to systematically guide the development of learning recovery and accelerated learning programs in the Cambodian context (see Box 5.1). Box 5.1: RAPID framework Formulated by UNICEF, UNESCO, and the World Bank, the RAPID framework for establishing programs for learning recovery and accelerated learning includes the following short-term interventions: 1. Reaching every child and retaining them in school: (a) reopening schools safely and keeping them open; (b) conducting re-enrollment campaigns, strengthening early warning systems to identify potential dropouts, and involving families; and (c) providing free schooling, meals, and cash transfers 2. Assessing learning levels regularly: (a) providing classroom-level assessment tools and conducting diagnostic/formative assessments to inform instruction and (b) conducting system-level (national, subnational) assessments of learning levels and losses to inform policy 3. Prioritizing teaching the fundamentals: prioritizing (a) foundational learning (mainly numeracy and literacy but also socio-emotional skills and digital skills) and (b) prerequisites for future learning by adjusting the curricula and time allocations and training teachers accordingly 4. Increasing the efficiency of instruction, including through catch-up learning: (a) using approaches such as targeted instruction, structured pedagogy, self-guided learning, tutoring, and increase in learning time and (b) supporting teachers by providing continuous training, coaching, access to technology, and counseling/peer support 5. Developing psychosocial health and well-being: (a) building teachers’ capacity to support students’ well- being and identify students in need of specialized services; (b) supporting teacher well-being and resilience; and (c) investing in students’ safety, nutrition, and water, sanitation and hygiene (WASH). Source: World Bank et al. 2022a. Along with the learning loss findings, the evidence on continuing low levels of learning outcomes and high levels of learning poverty indicates that there is also a great need for longer-term interventions aimed at enhancing education quality and system resilience. The longer-term improvement of education quality, in general, has been a key theme of the different policies and plans in the education sector, but more needs to be done to address the learning crisis. Apart from implementing the interventions proposed in these 88 documents, it would, therefore, be relevant to pursue a number of additional longer-term system-level reforms to improve the learning of both current and future students. These could include, among others, incentivizing reforms by linking education service targets with financing, upgrading the EMIS to support results-based performance monitoring, strengthening school-based management, and increasing the engagement of parents and communities in the education process (World Bank 2021b). As students from poor households have substantially lower learning outcomes than students from wealthier households, improving the overall quality of education both in the long run and in the short run will require interventions targeted specifically at poor students.50 It is encouraging to note that MoEYS has been providing support to students from poor households through key programs for more than a decade, including through scholarship programs for primary and lower secondary students and school feeding programs for primary and preprimary students. There is a need to further strengthen and expand such programs. Targeted support to poor students is particularly relevant in the short run as well in the current post-COVID-19 context as they are in immediate need of assistance, being the ones most at risk of dropping out of school. Accordingly, the first element of the RAPID framework gives special attention to reaching and retaining economically disadvantaged children. In the context of improving the quality of education, the evidence of poor learning outcomes among grade repeaters and children who are frequently absent from school suggests that these groups of students need special attention, especially to make long-term improvements in education quality. These two groups likely represent students who are struggling the most academically and have not been able to engage with the curriculum or, more broadly, with the learning process. Providing a more conducive and supportive learning environment for these vulnerable learners will help improve their performance and reduce grade repetition and absenteeism. This may require different interventions such as providing remedial classes, training teachers to identify students needing specialized attention and teach at the right level, and making provisions for greater peer support from academically better performing students. Such interventions would also be relevant in the short run for supporting the learning recovery of students in the post-COVID context. Strengthening system resilience To make the system more resilient in the longer term, it would be important for Cambodia to significantly strengthen and expand its EdTech-based remote learning infrastructure. The evidence presented in this report indicates that Cambodia has an EdTech supportive policy environment, and the government did make concerted efforts to maintain the students’ continuity of learning during school closure through distance education. However, there was only so much that could be accomplished given the relatively low levels of EdTech availability, access, utilization, and expertise among both teachers and students. As discussed in Section 4, different aspects of EdTech are positively correlated with learning outcomes suggesting that systematically integrating EdTech in the teaching-learning process has the potential to improve learning outcomes and make the education system more resilient against future shocks. This, however, requires a holistic EdTech expansion strategy that involves increasing ICT access for educators and pupils in schools, enhancing teacher and student digital competency, and improving access 50 As discussed above, poor students experienced less learning loss than others during the pandemic. But this finding does not negate the need for providing targeted support to the poor since these students have significantly lower learning outcomes than other students. Furthermore, as noted in Section 2, the smaller decline in learning experienced by the students from the poorest quintile between 2016 and 2021 could be an artifact of the smaller increase in learning they experienced between 2013 and 2016. 89 to ICT infrastructure and digital devices in teachers’ and students’ homes. Furthermore, it should be recognized that along with the government, it is important to engage the larger ecosystem of EdTech-related partners, providers, and users in the development and implementation of such strategies and supporting policies. This ecosystem of key stakeholders would include the government as well as students, teachers, school leaders, parents/caregivers, nongovernmental organizations, donor agencies, academia, and private sector companies (World Bank 2020b).51 To enable the system to cope better with possible disruptions in schooling in the future and to better utilize EdTech to improve the quality of education, existing policy gaps that limit the country’s EdTech readiness must also be addressed. As noted earlier, the first gap is the absence of national policies that delegate responsibility to schools for systematically integrating EdTech in teaching and learning. The approval of such policies would need to be followed by the development of guidelines, appropriate training to school management on EdTech integration, and gradual scaling up of financial or material support to schools. While such policies will not enable schools to immediately integrate EdTech in teaching-learning, given the resource limitations they face, they can encourage schools to plan for gradual scale-up, considering their specific constraints. To address the second gap identified earlier—the absence of a digital standards/competency framework for teachers and a system for evaluating their digital competencies— Cambodia could readily develop its competency framework based on the UNESCO ICT Competency Framework for Teachers (ICT-CFT) or similar frameworks used by other countries. In addition, Cambodia would also need to develop EdTech training modules aligned with this framework and make them an integral part of initial in-service teacher training programs. A digital competency framework for students— the absence of which was identified as the third gap—can also be developed by adapting existing frameworks to the Cambodian context. Finally, a clear financing strategy and a costed plan for EdTech expansion are critical for scaling up EdTech-based teaching-learning in line with the Education 2030 Roadmap. As the cost of expansion is likely to be substantial, the government would need to develop a phased expansion plan that considers equity. Within this context, it would also be important for the government to collaborate with the private sector to extend network coverage across the country (including rural areas) at a low cost. It should be noted that while these policies, frameworks, and strategies will have long-term impacts on the sector, their development should be among the short-term priorities of MoEYS. The surveys of principals and teachers also point to some gaps in policy implementation that need to be addressed. One of these gaps is the proper dissemination of policy provisions to schools, principals, and teachers. The fact that the vast majority of principals and teachers were not aware of existing policies, or had differing opinions regarding the existence of these policies, highlights the importance of systematically sharing this information with stakeholders. Another is the implementation of teacher training focused on EdTech. Despite the emphasis on providing EdTech training to teachers in the different plans and policies, most teachers had received little ICT-related training/orientation even during the pandemic and perceived their digital competency and expertise in conducting EdTech-based teaching to be low. Hence, incorporating ICT-based training as a mandatory component of the initial teacher training program for primary education could be a crucial first step toward improving digital literacy in schools, among both teachers and students. Finally, it should be noted that while MoEYS has made a substantial body of digital learning resources available to students and teachers as envisioned by the different ICT-related policies and plans, most of these resources became available only because of the urgent need to respond to the disruptions caused by COVID-19. Now that the pandemic has subsided, it will be essential to make 51 Engaging the larger EdTech ecosystem is identified as one of the five key principles in the World Bank's EdTech strategy (World Bank 2020b). 90 adequate provisions for maintaining and expanding these resources and ensuring that they continue to be available for use in the classroom and beyond.52 Furthermore, it will also be important to assess the quality of these existing resources, make quality upgrades accordingly where feasible, and define standards and procedures for quality assurance to guide the development and expansion of new resources in the future.53 The extensive use of non-digital learning resources during the pandemic suggests that the government should also invest in upgrading the quality of such resources and training teachers in using both digital and non-digital materials for distance learning. Given that it will take some time before access to digital infrastructure and devices becomes universal, it is likely that the provision and use of non-digital resources such as worksheets will continue to be essential for coping with future potential schooling disruptions in the short to medium term. Furthermore, since poorer students and rural students have lower access to EdTech, investment in increasing the quality and availability of non-digital educational resources for self-learning and distance learning is relevant from an equity perspective as well. In addition, the higher learning losses experienced by the wealthiest students and urban students—who are likely to be most advantaged in terms of EdTech readiness and most likely to have benefitted from EdTech interventions—suggests that it is important to gain a better understanding of the relative effectiveness of distance teaching-learning approaches that utilize digital versus non-digital resources. More specifically, it may be relevant to ask whether wealthier students and urban students in Cambodia experienced larger learning losses during the pandemic because digital resources were used less effectively than non-digital resources for distance learning. This could be an interesting question for future research. 52 To immediately start implementing shorter-term EdTech interventions, including the development and utilization of new digital resources, MoEYS should consider using different existing projects in the ministry. For example, the World Bank-supported General Education Improvement Project (GEIP) already includes provisions for piloting EdTech for math and digital education. Accordingly, it is developing a mobile phone application for grade 7 math for piloting in academic year 2023/24. 53 Some key aspects that could be considered in such assessments include, among others, (a) technical quality (namely, with regard to functionality and usability), (b) inclusiveness (extent to which they are free of cultural, gender, or other forms of bias), (c) responsiveness (extent to which they can be used in multiple devices), and (d) curriculum alignment. 91 92 References Amin, S., M. Hossain, and S. Ainul. 2021. Learning Loss among Adolescent Girls during the COVID-19 Pandemic in Rural Bangladesh. New York: Population Council. Ardington, C., G. Wills, and J. Kotze. 2021. “COVID-19 Learning Losses: Early Grade Reading in South Africa.” International Journal of Educational Development 86: 102480. Azevedo, J.P., A. Hasan, D. Goldemberg, S.A. Iqbal, and K. M. Geven (2020). “Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates” Policy Research working paper 9284, World Bank Group, Washington, DC. Azevedo, J. P., D. Goldemberg, S. Montoya, R. Nayar, H. Rogers, J. Saavedra, and B. W. Stacy. 2021. “Will Every Child Be Able to Read by 2030? Defining Learning Poverty and Mapping the Dimensions of the Challenge.” Policy Research Working Paper 9588, World Bank Group, Washington, DC. Azevedo, J. P., M. Akmal, M-H. Cloutier, H. Rogers and Y. N. Wong. 2022. “Learning Losses during COVID-19: Global Estimates of an Invisible and Unequal Crisis.” Policy Research Working Paper 10218, World Bank Group, Washington, DC. Bhatta, S. D., and S. Katwal. 2022. Learning in Myanmar, Pre and Post-Covid-19. Washington, DC: World Bank Group. CARE (CARE International Cambodia) and PLAN (PLAN International Cambodia). 2020. Rapid Assessment of COVID-19 Impacts on Girls Education in Northeast Provinces (Ratanakiri, Mondulkiri and Stung Treng). Phnom Penh: CARE International Cambodia and PLAN International Cambodia. Dalit Contini, D., M.L. Di Tommaso, C. Muratori, D. Piazzalunga, L. Schiavon. 2021. “The COVID-19 Pandemic and School Closure: Learning Loss in Mathematics in Primary Education.” Bonn: IZA Institute of Labor Economics DP No. 14785. Crawford, C., L. Dearden, and C. Meghir. 2010. When You Are Born Matters: The Impact of Date of Birth on Educational Outcomes in England. DoQSS Working Paper No. 10-09, Institute of Education, University of London. Dhuey, E., D. Figlio, K. Karbownik, and J. Roth. 2018. School Starting Age and Cognitive Development. Working Paper No. WP23660, National Bureau of Economic Research. Engzell, P., A. Frey, and M. D. Verhagen. 2021. “Learning Loss Due to School Closures during the COVID- 19 Pandemic.” Proceedings of the National Academy of Sciences 118 (17). EQAD (Education Quality Assurance Department). 2017. Results of Grade Six Student Achievement from the National Assessment in 2016. Phnom Penh: Education Quality Assurance Department, Ministry of Education, Youth and Sport, Kingdom of Cambodia. Hevia, F. J., S. Vergara-Lope, A. Velásquez-Durán, and D. Calderón. 2022. “Estimation of the Fundamental Learning Loss and Learning Poverty Related to COVID-19 Pandemic in Mexico.” International Journal of Educational Development 88: 102515. Juniper Education. 2021. The Impact of the Covid-19 Pandemic on Primary School Children’s Learning. https://21e8jl3324au2z28ej2uho3t-wpengine.netdna- ssl.com/wpcontent/uploads/juniper_folder/Juniper-Education-National-Benchmark-Dataset- 93 Report.pdf. Karamba, W., I. Salcher, and K. Tong. 2021. The Socioeconomic Impacts of COVID-19 on Households in Cambodia, Report No. 4: Results from the High-Frequency Phone Survey of Households Round 4 (17 December 2020–12 January 2021). Washington, DC: World Bank Group. https://openknowledge.worldbank.org/handle/10986/35383. Kuhfeld, M., J. Soland, and K. Lewis. 2022. “Test Score Patterns across Three COVID-19-Impacted School Years.” Educational Researcher 0 (0). https://doi.org/10.3102/0013189X221109178. Lichand, G., C. A. Doria, J. P. C. Fernandes, and O. Leal-Neto. 2022. “Association of COVID-19 Incidence and Mortality Rates with School Reopening in Brazil during the COVID-19 Pandemic.” JAMA Health Forum 3 (2): e215032. Maldonado, J. E., and K. D. De Witte. 2020. “The Effect of School Closures on Standardized Student Test Outcomes.” Leuven: Leuven Economics of Education Research (LEER) Discussion Paper Series DPS20.17. MoEYS. 2005a. Education Strategic Plan 2006–2010. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2005b. Policy and Strategies on Information and Communication Technology in Education in Cambodia. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2010a. Education Strategic Plan 2009–2013. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2010b. Master Plan for Information and Communication Technology in Education: 2009–2013. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2015. Curriculum Framework of General Education and Technical Education. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2016. Teacher Professional Standards. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2018a. Education in Cambodia: Findings from Cambodia’s Experience in PISA for Development. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2018b. Policy and Strategy on Information and Communication Technology in Education. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2019. Cambodia’s Education 2030 Roadmap. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2020a. Cambodia Education Response Plan to COVID 19 Pandemic. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2020b. Directive No. 23 on Distance Learning and E-learning Program for Students from Pre- Primary, Primary and Secondary Education. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2021a. Cambodia COVID-19: Joint Education Needs Assessment. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2021b. Directive No. 27 on Distance Teaching and Learning and E-learning for Public Education Institutions: Academic Year 2020–2021. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2021c. Education Congress: The Education, Youth and Sport Performance in the Academic Year 94 2019–2020 and Goals for the Academic Year 2020–2021. Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS. 2022. Public Investment Program for Higher Education Sub-Sector 2022–2023 on Cambodian Cyber University Network (CCUN). Phnom Penh: MoEYS, Kingdom of Cambodia. MoEYS (Ministry of Education, Youth and Sports) and UNICEF (United Nations Children’s Fund). 2022. Learning Loss in the Covid-19 Pandemic Era: Evidence from the 2016–2021 Grade Six National Learning Assessment in Cambodia. Phnom Penh: Education Quality Assurance Department, MoEYS, Kingdom of Cambodia. Patrinos, H. A., E. Vegas, and R. Carter-Rau. 2022. “An Analysis of COVID-19 Student Learning Loss.” World Bank Policy Research Working Paper 10033, World Bank, Washington, DC. Rodriguez-Segura, D. 2022. “EdTech in Developing Countries: A Review of the Evidence.” The World Bank Research Observer 37 (2): 171–203. Royal Government of Cambodia. 2003. The Education for All: National Plan 2003–2015. Phnom Penh: Royal Government of Cambodia, Kingdom of Cambodia. Royal Government of Cambodia. 2022. Sub-Decree on the Establishment of Cambodian Cyber University Network. Phnom Penh: Royal Government of Cambodia, Kingdom of Cambodia. Sandefur, J. 2022. “Uganda’s Record-Breaking Two-Year School Closure Led to… No Decline in the Number of Kids Who Can Read?” Center for Global Development, February 3, 2022. https://www.cgdev.org/blog/ugandas-record-breaking-two-year-school-closure-led-to-no-decline- number-kids-who-can-read. Schady, N., A. Holla, S. Sabarwal, J.Silva, and A.Y. Chang. 2021. “Collapse & Recovery: How COVID- 19 Eroded Human Capital and What to Do about It”; The World Bank, forthcoming. Schult, J., and M. A. Lindner. 2022. Did Students Learn Less during the COVID-19 Pandemic? Reading and Mathematics Competencies before and after the First Pandemic Wave. https://doi.org/10.1080/09243453.2022.2061014. Shepherd, D., N. Mohohlwane, S. Taylor, and J. Kotzé. 2021. Changes in Education: A Reflection on COVID-19 Effects over a Year. National Income Dynamics Study (NIDS) - Coronavirus Rapid Mobile Survey (CRAM). Staton, B. 2022. “Edtech Flops Provide Lessons for Future Success: Educational Technology Can Improve Children’s Prospects - But Only If We Get It Right.” Financial Times, June 11, 2021. https://www.ft.com/content/16d29631-6212-47d6-8346-b623e578479d. Teacher Training College. 2019. Framework Bachelor of Arts (Education). Phnom Penh: Teacher Training College, Kingdom of Cambodia. Tomasik, M. J., L. A. Helbling, and U. Moser. 2021. “Educational Gains of In-Person vs. Distance Learning in Primary and Secondary Schools: A Natural Experiment during the COVID-19 Pandemic School Closures in Switzerland.” Int J Psychol. 56 (4): 566–576. https://doi.org/10.1002/ijop.12728. UNESCO (United Nations Educational, Scientific and Cultural Organization). 2020. MoEYS Test Student Tracking System. https://www.unesco.org/en/articles/moeys-test-student-tracking-system. 95 UNICEF and SEAMEO (Southeast Asia Ministers of Education Organization). 2020. SEA-PLM 2019 Main Regional Report, Children’s Learning in 6 Southeast Asian Countries. Bangkok, Thailand: UNICEF and SEAMEO - SEA-PLM Secretariat. UNICEF, UNESCO, and World Bank. 2022. Where Are We on Education Recovery? New York: UNICEF. World Bank, the Bill & Melinda Gates Foundation, FCDO, UNESCO, UNICEF, and USAID. 2022a. Guide for Learning Recovery and Acceleration: Using the RAPID Framework to Address COVID-19 Learning Losses and Build Forward Better. Washington, DC: World Bank Group. https://www.worldbank.org/en/topic/education/publication/the-rapid-framework-and-a-guide-for- learning-recovery-and-acceleration. World Bank, UNESCO, UNICEF, FCDO, USAID, and BMFG. 2022b. The State of Global Learning Poverty: 2022 Update. Washington, DC: World Bank Group. https://www.worldbank.org/en/topic/education/publication/state-of-global-learning-poverty. World Bank. 2018. Cambodia Education Sector - Public Expenditure Tracking and Quality of Service Delivery Survey. Washington, DC: World Bank Group. World Bank. 2020a. Cambodia: Human Capital Index 2020. Washington, DC: World Bank Group. Accessed March 8, 2022. https://www.worldbank.org/en/publication/human-capital#Data. World Bank. 2020b. Reimagining Human Connections: Technology and Innovation in Education at the World Bank. Washington, DC: World Bank Group. World Bank. 2021a. Cambodia General Education Improvement Project (English). Washington, DC: World Bank Group. World Bank. 2021b. Cambodia Economic Update: Living with Covid. Washington, DC: World Bank Group. World Bank. 2021c. World Bank EdTech Readiness Index (ETRI) (English). Washington, DC: World Bank Group. http://documents.worldbank.org/curated/en/811011628250703800/World-Bank-EdTech- Readiness-Index-ETRI. World Bank 2021d. The Socioeconomic Impacts of COVD-19 on Households in Cambodia: Results from the High-Frequency Phone Survey of Households Round 5 (1–21 March 2021). Yarrow, N. B., H. Kim, J. Yoo, and T. Pfutze. 2022. EdTech in COVID Korea: Learning with Inequality. Washington, DC: World Bank Group. https://documentsinternal.worldbank.org/search/33782409. 96 Annex Box A.1: Cambodia National Learning Assessment Cambodia has been conducting the NLA since 2005 in line with international best practice with some form of technical assistance from the World Bank, Research Triangle Institute (RTI), and others. The NLA is conducted by EQAD of MoEYS, using the following procedure. Initially, a curriculum blueprint is created by MoEYS experts that maps out the curriculum across the main domains and assigns an intended number (or share) of test questions based on the number of hours of instruction in different curriculum or content areas. This process is carried out to ensure alignment of the tests with the curriculum. Then, an item bank is developed by item writers from the same group of experts who prepared the curriculum blueprint. Once the item bank is developed, the items are trialed in a sample of about 20–25 schools. The items are then analyzed using a computer program that fits IRT models (in most years the program used was Item and Test Analysis (IATA), in other years Conquest was the program of choice) to select the best items based on standard psychometric properties or parameters, including reliability and validity. The final instrument is then divided into multiple forms or booklets, which are linked with a set of anchor items, that is, common items. After the data are collected, the items are analyzed again using the IRT software, and scaled scores are created in addition to simple measures of percentage correct. Proficiency scales, which are relatively new in the NLA (they have been used only in the last three to four rounds of assessment), are also created. These scales are usually based on formulas set by the same experts who were involved in earlier stages. The samples are large enough to provide national estimates with some stratification across public and private schools (with oversampling of private schools) and urban and rural areas. With the help of external experts, EQAD has also made an effort over the years to equate scaled scores across different rounds of assessment in common grades, but this work has been limited by the relatively small number of common items that have been used. Technical evaluations are conducted by outside/third-party experts to ensure the reliability and validity of the assessment. For example, experts from the RTI conducted the technical evaluation of NLA 2016, in which they carefully reviewed the test item bank, as well as assessment and sampling processes. Their expert opinion was that the assessment instruments met international standards and that EQAD demonstrated the instruments’ reliability and validity in a statistically rigorous way. The RTI experts also validated the sampling process and deemed the sample to be of high quality and suitable for producing statistically valid results to make inferences for the school and student population in Cambodia. In addition, an expert review was conducted by EQAD to evaluate the difficulty levels of test items. This was done by asking 41 grade 6 teachers in 20 primary schools around Cambodia to review and assess some test items in which students performed relatively poorly. The EQAD review concluded that the test items were valid and were at an appropriate level of difficulty. Source: Primarily based on EQAD (2017). Table A.1: Timeline of policy actions and related developments affecting schools and learning, 2020/21 Date Action January 27, 2020 First confirmed case of COVID-19 in Cambodia March 7, 2020 First local transmission of COVID-19 in Cambodia confirmed in Siem Reap; officials announce that schools in the province will close for 14 days. March 11, 2020 The WHO declares COVID-19 outbreak a global pandemic. March 16, 2020 Government of Cambodia closes all education institutions, including public and private schools; MoEYS announces teleworking arrangements. April 2020 MoEYS issues directive and operational guidelines for remote learning, elaborates management tasks, and requests funding for contract teachers. April 8, 2020 MoEYS postpones national exams at lower and upper secondary. May–June 2020 MoEYS suspends new teacher recruitment for 2020 and issues guidelines to teacher education institutes for e-learning training and capacity development. July 10, 2020 MoEYS issues guidance on health promotion to prepare for school reopening, with requirements for monitoring, case management, and reporting. 97 Date Action July 15, 2020 MoEYS formally endorses the Cambodia Education Response Plan to the COVID-19 Pandemic. September 2020 20 private schools with high safety standards allowed to open in Phnom Penh, Siem Reap, and Battambang. October 2020 Four low-risk provinces—Kratie, Stung Treng, Ratanakiri, and Mondulkiri—allowed to reopen all grades; other provinces to reopen grades 9 –12. November 2, 2020 Remaining schools reopen throughout the country. November 8, 2020 Schools in Phnom Penh and Kandal close for two weeks. November 29, Private schools closed for two weeks 2020 January 11, 2021 Schools open for the delayed 2020/21 academic year. February 4, 2021 Cambodia’s Ministry of Health approves use of Sinopharm COVID-19 vaccine. February 20, 2021 Cambodia begins vaccinations with Sinopharm, Sinovac, and AstraZenaca. February 20, 2021 A large COVID-19 outbreak in the capital—known as the ‘February 20 incident’—results in school closures in Phnom Penh and Kandal. March 20, 2021 School closures are extended to the rest of the country. September 15, More than 200 schools opened in Phnom Penh. 2021 November 1, 2021 Schools across the country reopen; classrooms are limited to 15 students, and they are required to sit 1.5 meters apart to maintain social distancing. Source: World Bank (2021b), Table S.1, page 44. Based on WHO, Cambodia: Coronavirus Disease 2019 (COVID-19) Situation Report, various dates, 2020–2021; MoEYS 2021a; World Bank 2021d. Table A.2: Remote learning channels, target grades, and their reach Channel Type Target Reach MoEYS e-learning website Online/mobile app High school and below 100,000 installations MoEYS Facebook Social media Highschool and below 3,669,000 followers MoEYS YouTube Online Highschool and below 147,000 subscribers Krou Cambodia Social media Highschool and below 928,000 followers Komar Rien Koma Cheh Social media Early grades 29,600 followers ‘Distance Education Mobile app Grades 9–12 Application’ on Google Play and Apple Store TVK2 Television/mobile app Highschool and below 10,000 installs Apsara TV Television/social media Grade 12 and early grades 185,000 followers Sky One TV-35 Television/mobile app Grade 12 1,000 installs Digital One TV-6 Television/mobile app Grade 12 500,000 installs Splus TV/Wiki TV Television/mobile app Grades 9–12 10,000 installs Singmeng TV/Wiki TV Television/mobile app Grades 9–12 50,000 installs Zoom Communication tools All grades Google Meet Communication tools All grades Telegram Communication tools All grades WhatsApp Communication tools All grades BEEP platform Online/social media Lower secondary dropouts 13,000 followers Multilingual education Radio Preschool and early Ratanakiri, Mondulkiri, and radio-based program grades (ethnic minorities) Kratie Provinces Source: World Bank (2021b), Table S.2, page 50. 98 Table A.3: Government policies, plans, and strategies supporting EdTech Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech Education for All: Education 2003 ICT policies will Specific strategies for ICT in Policy and Not Not specified National Plan 2003– Plan include (a) expansion education are still being finalized. Strategies on specified 2015 of ICT as a teaching Key strategies are likely to include Information and and learning tool; (b) (a) inclusion of ICT education and Communication as a means of training, initially in selected Technology in improving education secondary schools, teacher training Education in service productivity institutions, and technical and higher Cambodia and management education institute; (b) selective through improved curriculum enrichment programs, information sharing, especially where the small numbers communication and involved do not justify significant knowledge investment in dedicated physical management; and (c) facilities, staff and equipment; and expansion of distance (c) ICT-based targeted programs for learning opportunities a diverse range of socially especially for disadvantaged groups to ensure disadvantaged groups reentry to mainstream formal in remote areas. The education. overarching goal will be to ensure Cambodia’s international competitiveness in an increasingly global and interconnected knowledge-based economy. Education Strategic Plan 2005 Strengthening teaching Train 5,000 teachers at all levels per Education for US$5.5 This policy includes Plan 2006–2010 and learning quality to year, with ICT training included in All: National million a few EdTech- ensure primary and their training from 2006. Plan 2003–2015 (4.6% of related indicators: secondary teachers the  No. of The program will provide these with pedagogy and projected schools/teacher facilities in upper secondary schools ICT awareness. capital training centers and higher education institutions. investment 99 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech The science, budget) with new science technology, and ICT from 2006 and ICT facilities facilities expansion to 2010)  Results not program has the main available objective of enhancing  Results not quality education specified. outcomes through provision of appropriate specialist teaching and learning facilities. Policy and Strategies Standalone 2005 Increased access to Developing/updating curriculum for Education for Not Not specified on Information and ICT Policy basic education for all, training teachers to use ICT for All: National available Communication both formal and non- administration, professional Plan 2003–2015 Technology in formal, using ICT as development, teaching, supporting Education Law Education in one of the major tools their teaching and learning Cambodia for learning, teaching, Developing/updating secondary searching, and sharing curriculum to include the use of ICT information. teaching students Improved quality of Increasing hardware and basic education and infrastructure in schools to allow promote independent teachers and students to have access and lifelong learning, to computers and the internet especially for post- primary education. Developing local contents in Khmer for both formal schools and distance Availability of education workforce with the ICT skills needed for Promoting distance education employment and use through open schools. in knowledge-based society; to ensure that Cambodia can compete and cooperate 100 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech in an increasingly interconnected world. Education Law Law 2007 The ministry in charge Policy and Not Not available of education shall Strategies on available determine the policies Information and on science and Communication technology for Technology in education at all Education in educational levels of Cambodia the Cambodian education system in compliance with the policy of the Royal Government of Cambodia. Education Strategic Plan 2010 The objective of this Provide training to officials at Master Plan for 2010: KHR Plan 2009–2013 sub-program central, provincial and district levels Information and 185.8 [Development of ICT on how to use school management Communication million usage] is to ensure that software and productivity software Technology in 2011: KHR the technology used or developed by MoEYS. Education: 209.2 developed by MoEYS 2009–2013 Prepare and pilot school million will be appropriate to management by using computer- address the needs to 2012: KHR based automation software and data update the ministry’s 2,230.3 exchange software among schools administration and million and MoEYS’ database. flexible with its 2013: KHR decentralization 256.6 policy. Another million objective is to promote and conduct These monitoring on the budget standardization of allocation information and accounts computers used in the for roughly ministry. 101 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech 0.1% of the total budget Master Plan for Standalone 2010 To increase access to Developing ICT-based professional Education Not Not specified Information and ICT Plan basic education, skills for students Strategic Plan available Communication tertiary education and 2009–2013 Developing new curricular to Technology in life-long learning, both increase ICT literacy for teachers Education: 2009– formal and non- and school directors 2013 formal, by using ICT as alternative Issuing specifications for computer education delivery labs in schools media. Ensuring internet access in all To improve the teacher training centers and schools relevance and that have computers for education effectiveness of basic Deploying ICT master trainers to education by support the implementation of ICT- harnessing the based curriculum at teacher training potential of ICT as a centers major tool to enhance the quality of teaching Developing video training materials and learning. for teacher trainees To develop the ICT- based Professional skills needed by graduates for employment in a knowledge-based society and to ensure that Cambodia can compete and cooperate in an increasingly interconnected world. To increase the effectiveness and efficiency of the 102 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech ministry and school management. Education Strategic National 2014 Ensure equitable and Develop infrastructure through Not Not specified Plan 2014–2018 Education expanded access to construct new schools and expand specified Plan general secondary the grades as well as equipping the education schools physical facilities (library, which meet all quality laboratory, ICT, audiovisual room, standards in general sport field, toilets, clean water, hand and technical washing space, the first aid room, education and inspire workshop and dormitory for physical and sports students). activity including the Improve the quality of teaching and enhancement of school learning (science and mathematics, health. ICT and foreign language, health Ensure graduates education, technical education, students from Effective Vocational Education secondary schools and Program (EVEP), and life skills). general and technical secondary education high schools are equipped with right and employable skills or to enter into technical training and higher education. Policy and Strategies Standalone 2018 MoEYS will adopt Equip Upper Secondary Schools Education Law Not Not specified on Information and new management and (USSs) and Teacher Training available Communication administrative Centers (TTCs) using standardized Technology in processes to equipment and institutional network Education modernize practices design. and increase the Equip sub-national MoEYS offices efficiency, with standardized equipment and transparency, and systems design. effectiveness of education sector 103 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech governance and Modernize the education curriculum performance to include ICT and 21st century monitoring. knowledge and skills MoEYS will integrate Procure and/or develop e-resources ICT as a teaching, for supporting teaching and learning learning, and (contents) knowledge sharing Train ICT teachers for all USSs and tool across the Lower Secondary Schools (LSS). education sector to equip students with Develop MoEYS capacity to produce ICT knowledge and high quality and relevant e- skills to transition to resources. the twenty-first century world of work. Cambodia’s Plan 2019 To prepare lifelong Develop a national plan to establish Education Not Not specified Education 2030 learners for an ICT infrastructure and connectivity Strategic Plan specified Roadmap interconnected and in all schools and development of 2019–2023 interdependent world, digital learning materials. who can act Improve pre-service teacher effectively and education curriculum, both in-service responsibly at local, and pre-service to include new and national, regional, and modern pedagogical approaches and global levels for a integrate ICTs, Global Citizenship more peaceful and Education (GCED) and Education sustainable world. for Sustainable Development (ESD). The overarching goal Promote the use of /ICT, mobile is to ensure inclusive technologies in literacy/numeracy and equitable quality programs. education and promote Develop a system that maximizes the lifelong learning use of ICT opportunities and data opportunities for all. connectivity for real-time Governance and information sharing, analysis, and management of reporting as well as feedback mechanism to ensure that the voices 104 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech education improves at and concerns of learners, parents, all levels. educators, and the public as a whole are heard, and they feed into planning, policy making, and services delivery. Education Strategic National 2019 Promotion of Digital Integrates ICT into teaching methods Cambodia’s Not This policy includes Plan 2019–2023 Education Education: Education 2030 specified a few EdTech- Improve pre-service teacher training Plan Roadmap related indicators, Integrate ICT into a by integrating new teaching methods including access to tool for teaching and and the use of ICT in teaching and electricity. learning and to share learning processes. knowledge across the According to the Use information technology (e-mail whole education latest Education and the internet) to promote new sector. Congress of 2021, learning approaches and  Number of Equip students with collaborative learning (for example, primary schools knowledge and skills through networking), to construct with a computer on ICT to transition and synthesize new information (for room for students into twenty-first example, through PowerPoint increased from 31 century employment. presentations); solve problems (for in 2019 to 76; example, through data processing Ensure all students programs that can find relationships,  The percentage of complete formal upper secondary sort data, and so on); and provide a education with schools using ICT means for dialogue, discussion, and knowledge and skills as a tool to debate; interactivity that leads to on ICT to support their support teaching social construction. further education and and learning professional work Encourage students to prepare increased from PowerPoint presentations in their 17% in 2019 to Increase the efficiency groups as a useful social channel for 24%; and and effectiveness of working together in teams to produce  The report further teaching and learning a creative product to share with the shows that 78% of in teacher training class. preschools, 85% centers, schools, and other educational Create a special room using digital of primary institutions by using tools such as LCD projectors, schools, and 96% computers, and other teaching of secondary schools had access 105 Budget Document EdTech-related Selected EdTech-related Alignment with allocated/ Implementation Policy/plan/strategy Year type goals/objectives proposals/strategies/activities other policies projected Progress/Result for EdTech ICT tools and e- materials to integrate quality to electricity, resources. citizenship into classrooms. against the targets of 37%, 55%, and Use e-learning to 83%, respectively, support the delivery of in the academic education services to year 2020/21. all sub-sectors in education and develop institutional capacity for life-long learning. Create standards for infrastructure and network connections at the national and subnational levels by integrating systems into a single internal network. Note: Most of text in this table has been copied verbatim from the listed policies and plans. 106 Table A.4: Determinants of student Khmer test scores Model 1 Model 2 Model 3 Model 4 Model 5 Student ICT access Internet access at home 8.5856* 7.8174 8.9460 (5.0343) (5.5798) (6.4463) Student use of school resources Paper textbooks, workbooks, or 14.8714** 21.2665*** 17.8030** worksheets (6.9451) (7.3086) (8.4617) Self-study worksheets (prepared by 12.9034*** 12.0641*** 16.3406*** MoEYS team) (4.1346) (4.6436) (5.3743) Digital textbooks, workbooks, or 8.0139* 11.6578** 8.9423 worksheets (4.5914) (5.1408) (5.7724) Real-time lessons by a teacher from my 6.4422 9.9305* 8.1210 school on a video communication program (4.9820) (5.7639) (6.3026) Real-time lessons by your extra-class −16.5733*** −13.0969** −16.0595*** teacher from your school on a video communication program (4.8099) (5.2564) (6.1681) Recorded lessons or other digital material 8.8219* 5.1716 5.2253 from other sources (4.5825) (4.8984) (5.6702) Caregiver support Ask what student is learning 10.8046* 5.9088 5.8218 (6.3563) (6.6328) (7.1821) Explain new content −10.3800* −13.8611** −11.0060 (5.5275) (5.9498) (6.8217) Buy some books or story books or other −9.6794** −8.2415* −6.3286 learning materials (4.3659) (4.6545) (5.2068) Allow student to borrow digital device 11.6955*** 10.4759** 11.2271** (smartphone or computer or tablet or others) (4.3433) (4.7393) (5.2778) School ICT access School has electricity −25.2766** −25.8680** −10.9506 (10.3822) (10.6614) (13.3338) School has internet 9.6861* 10.2601* 7.9609 (5.2365) (5.8965) (6.9003) Working devices in school (per 300 −0.3272*** −0.3059** 2.0951** students) (0.1224) (0.1393) (0.9174) Working devices available for students 0.6953** 0.7148** −3.8033* (per 300 students) (0.3231) (0.3399) (1.9395) School teaching activities and support Learning materials for self-study −11.9522** −11.9469* −13.3146* (5.7479) (6.2275) (7.7457) School assignments 11.5735* 6.1626 6.4741 (6.7087) (7.4262) (8.7957) 107 Model 1 Model 2 Model 3 Model 4 Model 5 Live virtual classes on a video 1.7637 −8.9475 −15.3542** communication program (4.4874) (5.5618) (6.1201) Asked to submit completed assignments 28.5057*** 23.6011*** 27.6301*** (6.8243) (8.1110) (9.2165) Share of schoolwork completed 35.4466*** 35.1728** 14.7091 (11.7919) (14.0053) (17.3166) Teacher ICT access at home Internet 10.3556** 10.5562* 3.8054 (5.2009) (5.8375) (6.8335) Teacher digital competency Laptop 14.8075* 16.1860* 19.2640 (8.0211) (9.3513) (12.3951) Smartphone 11.6080* 4.3670 0.7682 (7.0126) (8.9788) (10.6272) Downloading and using app −11.4042 −16.4963* 1.5596 (8.0837) (9.6346) (12.2036) Math score 2016 (matched) 0.0207 (0.0260) Student/household characteristics Student is female 9.1185** 5.2631 8.1535** 5.1038 3.7334 (3.8900) (3.7592) (3.5615) (4.1600) (4.6629) Number of siblings −4.1689*** −3.0308*** -3.8274*** −3.4911*** −3.9687*** (1.1618) (1.1721) (1.0651) (1.2247) (1.3542) Wealth quintile (reference: bottom quintile) Second quintile 9.8791 9.2129 13.0755** 8.0608 7.2335 (6.2778) (6.1503) (5.6531) (6.8424) (7.2654) Third quintile −1.7194 1.9203 2.0782 −2.1518 −2.6260 (6.3558) (6.0935) (5.7595) (6.9316) (7.3856) Fourth quintile 8.4584 16.5993** 18.0972*** 8.0689 4.2133 (6.6191) (6.5818) (6.1524) (7.2216) (7.8360) Top quintile 25.7945*** 36.1464*** 35.2361*** 25.8960*** 26.3161*** (7.7883) (7.3216) (6.9599) (8.6614) (9.8521) Lives with both parents 14.4821*** 7.8064* 9.4649** 11.2403** 11.1839** (4.5373) (4.6360) (4.3707) (4.9855) (5.4281) Father has university education 6.5594 15.0829* 12.0166 10.7715 4.5311 (7.9281) (8.8633) (8.1092) (9.5128) (12.4513) Child worked during school closure (reference: did not work) Worked some days −9.6172* −9.3658* −9.9955** −8.3216 −8.9213 (5.5742) (5.4919) (5.0761) (6.0775) (6.7238) Worked every day −22.1926* −24.4232** −30.4739*** −17.9583 −21.8036* (12.0574) (12.3479) (10.7101) (12.6864) (12.9458) Student experience/engagement Annual absentee days −1.2981*** −2.5126*** −1.3769*** −2.1647*** −2.4059*** (0.4199) (0.5073) (0.4150) (0.5621) (0.6906) Has repeated grade −8.7211** −10.8029** −9.1162** −9.0686* −7.6976 (4.4354) (4.2389) (4.0025) (4.7450) (5.4681) School characteristics Number of students in grade 6 0.1964* 0.2030** 0.1192 0.1864 0.0256 (0.1046) (0.1022) (0.1036) (0.1151) (0.1503) Private school 19.7000** 24.2913*** 23.5233*** 13.1556 - (8.1551) (8.8390) (7.6468) (10.1822) (.) 108 Model 1 Model 2 Model 3 Model 4 Model 5 Constant 415.9172*** 353.5244*** 375.8834*** 422.1533*** 327.0726*** (29.7187) (35.2918) (27.2318) (38.2537) (48.5079) Observations 2,120 2,124 2,476 1,796 1,306 R-squared 0.2197 0.2057 0.1879 0.2516 0.2248 Source: Authors’ calculations u NLA 2021, the supplemental principal and teacher surveys 2021, and NLA 2016. Note: Asterisks denote statistical significance at the 10 percent (*), 5 percent (**), and 1 percent (***) levels, respectively. 109