Digital Pathways for Education: Enabling Greater Impact for All Subhashini Rajasekaran, Taskeen Adam, Klaus Tilmes © 2024 International Bank for Reconstruction and Development / Digital Pathways for Education: The World Bank 1818 H Street NW, Washington, DC 20433 Enabling Greater Impact for All Telephone: 202-473-1000 Internet: www.worldbank.org Some rights reserved. Subhashini Rajasekaran, Taskeen Adam, Klaus Tilmes This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and con- clusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. Nothing herein shall constitute or be considered to be a limitation on or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. This publication was produced with the financial support of the European Union. Its contents are the sole responsibility of World Bank staff and do not necessarily reflect the views of the European Union. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denomina- tions, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territo- ry or the endorsement or acceptance of such boundaries. Rights and Permissions This work is available under the Creative Commons Attribution 4.0 International license (CC BY 4.0) https://creativecommons.org/licenses/by/4.0/, with the following mandatory and binding addition: Any and all disputes arising under this License that cannot be settled amicably shall be submitted to mediation in accordance with the WIPO Mediation Rules in effect at the time the work was published. If the request for mediation is not resolved within forty-five (45) days of the request, either You or the Licensor may, pursuant to a notice of arbitration communicated by reasonable means to the other party refer the dispute to final and binding arbitration to be conducted in accordance with UNCITRAL Arbi- tration Rules as then in force. The arbitral tribunal shall consist of a sole arbitrator and the language of the proceedings shall be English unless otherwise agreed. The place of arbitration shall be where the Licensor has its headquarters. The arbitral proceed- ings shall be conducted remotely (e.g., via telephone conference or written submissions) whenever practicable, or held at the World Bank headquarters in Washington, DC. Attribution – Please cite the work as follows: Rajasekaran, S., Adams, T., Tilmes, K. 2024. “Digital Pathways for Education: Enabling Greater Impact for All”. World Bank, Washington, DC. License: Creative Commons Attribution CC BY 4.0 IGO. Translations – If you create a translation of this work, please add the following dis- claimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations – If you create an adaptation of this work, please add the following dis- claimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content: The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to reuse a component of the work, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Cover illustration and interior design: © Elizaveta Tarasova ET-digital, Imatra, Finland. Cover Photos (Further permissions required for reuse): © Wavebreakmedia / Depositphotos. Used with the permission of Wavebreakmedia / Depositphotos. © Suriya / Adobe Stock. Used with the permission of Suriya / Adobe Stock. © MartaKlos / Adobe Stock. Used with the permission of MartaKlos / Adobe Stock. © Gajendrra Bhati / Adobe Stock. Used with the permission of Gajendrra Bhati / Adobe Stock. © Patrick / Adobe Stock. Used with the permission of Patrick / Adobe Stock. © Wazzkii / Adobe Stock. Used with the permission of Wazzkii / Adobe Stock. © Kinara Art Design / Adobe Stock. Used with the permission of Kinara Art Design / Adobe Stock. © AI Generated / Stockcake.com. © Kelley Lynch / World Bank. © Dominic Chavez / World Bank. Table of Contents Acknowledgments viii Note on Non-Endorsement and Neutrality x Abbreviations xi Key Terms xiii Executive Summary 1 1. Introduction 10 2. Enabling the Digital Transition of Education Systems: 13 Conceptual Framework 2.1 Proposed conceptual framework 13 2.2 Deepen digital transitions in education: Shift to a systemic 17 approach 2.2.1 Education Leadership and Governance (Strategic Layer) 18 2.2.2 Digital Public Infrastructure, Foundations and Platforms 19 (Partnerships Layer) 2.2.3 Digitally Enabled Education System Architecture (Education ICT 22 and Data Layer) 2.2.4 Digitally Enabled Service Delivery (Implementation Layer) 25 2.3 Scaling digital transitions in education: Shift towards stronger 31 learning-centered innovation ecosystems and partnerships 2.4 Unpacking the Layers: Digital pathways in education and skills 34 development 3. Digital Transitions Create both Opportunities and Risks 38 3.1 Current evidence landscape 38 3.2 Diving into the potential opportunities 39 3.2.1 Equitable access to quality learning for all 39 3.2.2 Better learning outcomes and more holistic learning opportunities 43 3.2.3 Increased effectiveness of teachers and teaching methods 45 3.2.4 Cost-efficiency and cost-effectiveness of investments 47 iii 3.2.5 Quality and relevance of education ‘at scale’ 48 3.2.6 System strengthening and resilience 49 3.3 Understanding potential risks and possible mitigation strategies 50 3.3.1 Digital risks 50 3.3.2 Risk mitigation initiatives 54 3.4 Shaping the future of evidence 55 4. Contextual Conditions for Unlocking the Power of Digital 60 in Education Systems 4.1 Digital readiness of the education system: The five pillars 60 4.1.1 Strategic leadership and governance 60 4.1.2 Data and technology infrastructure and management 62 4.1.3 Digitally enabled education service delivery and analytics 62 4.1.4 Education workforce capacity and culture 63 4.1.5 EdTech market and business models 64 4.2 Digital readiness assessment frameworks, methods, and tools 64 5. Tailoring Digital Pathways 68 5.1 Trade-offs and issues to consider in design and implementation 69 5.2 Common pitfalls in shaping digital transitions 71 5.3 Effective practices for digital transitions in education 73 6. Conclusion 75 References 76 Annex 1. Glossary 103 Annex 2: Sample GPT-4 Output 106 iv List of Figures: Figure ES1: Policy actions to realize an equitable future of learning 2 Figure ES2: Recognize and Support the Changing Role of Teachers 2 Figure ES3: Potential Digital Pathways for Equitable, Relevant and 3 Resilient Education Systems Figure ES4: Embrace Mindset Shifts in Digital Transitions in Education 4 and Skills Development Figure ES5: Enabling Architecture for the Digital Transition of 5 Education Systems Figure ES6: Principles of Digital Development 6 Figure 1: Key shifts required in the digital transitions in education 12 and skills development Figure 2: Depth and scale of digital transitions in education and 14 skills development Figure 3: Digital pathways in education and skills development – A 17 conceptual framework Figure 4: Unpacking the Strategic Layer 18 Figure 5: Unpacking the Partnerships Layer 19 Figure 6: Unpacking the Education ICT and Data Layer 22 Figure 7: Types of digitally enabled education service delivery 25 Figure 8: Array of applications in digitally enabled service delivery 26 Figure 9: Unpacking the layers: Digital pathways in education and 35 skills development Figure 10: Education platformization poses important risks for 51 learners Figure 11: Technology and learning contexts interact with the 56 characteristics of the learner Figure 12: Understanding the ESSA tiers of evidence in education 57 Figure 13: EdTech Tulna Framework 57 Figure 14: Evaluating educational programs leveraging EdTech 59 Figure 15: Assessment frameworks and tools for diagnosing 65 education needs and related digital readiness v Figure 16: Education ecosystem stakeholders illustrative map 67 Figure 17: Example of an agile delivery approach 73 List of Boxes: Box 1: The case for techno-realism - A long history 18 Box 2: Technology classification based on nature of cost savings 20 Box 3: Enterprise architecture in (Estonia) 21 Box 4: Data storage and management in education (Estonia) 23 Box 5: Data collection, analysis, and dissemination for the 24 Improvement of Learning (RIMA) (Guanajuato district in Mexico) Box 6: Changing role of teachers: From provider to facilitator to 28 cultivator and coach Box 7: Support teachers to strengthen pedagogical and 28 administrative practice (UK) Box 8: Applying early warning mechanism (EWM) to address 30 early school leaving (ESL) in Romania Box 9: TUMO Armenia 32 Box 10: Building the EdTech ecosystem for technological 33 innovations in education in Turkiye Box 11: Role of Education and Skills to better manage the Digital- 36 Climate Nexus Box 12: Ukraine’s expansion of digitally enabled education in the 40 midst of war Box 13: University of the People (on HE), also covered under 42 the Yidan Prize 2023 for Shai Reshef on Education Development: very relevant for refugee contexts Box 14: Use of ICAP Learning Framework in STEM Education 44 Box 15: The evolution of the ‘digital divide’ 51 Box 16: Persuasive design or deceptive design 52 Box 17: Critical considerations for AI in education 54 Box 18: EU’s New Landmark AI Law adopted in March 2024 61 vi Box 19: EdTech Tulna - Navigating the complex EdTech landscape 63 (India) Box 20: Possible issues and trade-offs to consider for policy 69 decision-making List of Tables: Table ES1: Tailoring Digital Pathways: Effective Practices and 9 Common Pitfalls Table 1: Digital Readiness Assessment of the Education System 66 vii ◄ Table of Contents Acknowledgements This book has been prepared by a core team led by Subhashini Rajasekaran and comprising Taskeen Adams and Klaus Tilmes. It has been prepared under the direction of Rita Almeida (Practice Manager for Education Global Practice) and Jamele Rigolini (Lead Economist for Human Development, Europe and Central Asia region) and support from Ahmet Levent Yener (Practice Leader for Human Development, South Caucasus) and Jamexis Denise Christian (Senior Operations Officer, Human Development, Europe and Central Asia). The review of evidence is based on work done in collaboration with Taskeen Adams, Co-Director, Open Development Education, and Klaus Tilmes, Senior Policy Adviser on emerging technologies and development. The team received valuable inputs from Robert Hawkins, Senior Education Specialist and Global Lead for Education, Technology, and Innovation. Michael Trucano, Visiting Fellow, Brookings (previously, Global Lead for Technology & Innovation in Edu- cation, World Bank) played a vital role in encouraging, advising, and providing early comments. The book has immensely benefited from inputs and case examples from Husein Abdul-Hamid, Bjorn Hassler, Adrien Samuel Julien Olszak Olszewski, Kristel Rillo, and Ravi Gupta, as well as editorial support from the World Bank Chennai Office and Fiona Mackintosh. The team is grateful for research assistance from Kexin Wang, Education Consultant, and for program assistance from Renata Pereira De Mello, Tania Fragnaud, and Hala Bishara. The team would like to thank World Bank colleagues and peer reviewers Alex Twinomugisha, Cristobal Cobo, Gabriel Demombynes, Husein Abdul-Hamid, Noah Yarrow, Sachiko Kataoka, Simon Carl O’Meally and Shobhana Sosale for the advice and guidance provided that contributed to the development of this work. The team would furthermore like to thank Stéphan Vincent-Lancrin (OECD), Karl Rectanus (previously Instructure), Natalia Kucirkova (University of Stavan- ger, Norway), Velislava Hillman (London School of Economics), Gouri Gupta (Director EdTech, Central Square Foundation, India), Michael Forshaw (Founder, Edtech Impact, UK), Daniel Stanhope (Founding Researcher, LearnPlatform by Instructure) and many others who have shared their reflections related to evi- dence and risks and provided insights from implementation experience in a wide range of policy environments globally. The team benefited from valuable consultations with a variety of stakeholders including many senior policymakers from the Europe and Central Asia region, Africa region and East and South East Asia region between 2023 and 2024, including as part of the Cambridge HP Edtech Fellowship. This helped gather valuable insights on relevance and use of the framework to inform their strat- egy and implementation related to the digital area in education. The team is indebted to all counterparts for their valuable reflections, insights, questions and, for their passion to address the learning and education needs of their coun- tries and regions. These have significantly contributed to strengthen the discus- sions on the digital pathways in education framework and the digital readiness assessments. viii ◄ Table of Contents We are immensely thankful for the strong encouragement, support and overall guidance from Michal Rutkowski, Regional Director for Human Development, Europe and Central Asia region; Luis Benveniste, Global Director for Education Global Practice; and Jaime Saavedra, Regional Director for Human Development, Latin America and the Caribbean. This product is part of a task under the Human Development Assessments in Eastern Neighborhood Countries (P173530). This work could not have come to life without the generous funding from the Trust Fund Program EU2020 Knowl- edge & Advisory Services (EU2020) of the European Commission’s Directorate General for Neighborhood and Enlargement Negotiations (DG NEAR). ix ◄ Table of Contents Note on Non-Endorsement and Neutrality Throughout this book, all references made to specific commercial digital prod- ucts, services, enterprises, and entities are intended to provide relevant and real-life examples to demonstrate the available solutions and clarify the related messages. None of the references signify an endorsement of the said products, solutions, or enterprises. The authors, as well as the World Bank, act as neutral arbiters of knowledge to help identify both opportunities and risks for improv- ing education outcomes. x ◄ Table of Contents Abbreviations AGI Artificial General Intelligence AI Artificial Intelligence AIEd AI in Education AR Augmented Reality AWS Amazon Web Services BESA British Educational Suppliers Association CoP Communities of Practice CRIA Child Rights Impact Assessment CRRE Center for Research and Reform in Education CSO Civil Society Organization CxO Chief X Officer (C-Suite Designation) DBE Department of Basic Education DCC Digital Credentials Consortium DLP Digital Literacy Program DSP Digital Service Provider EDRA Education’s Digital Readiness Assessment EEF Equitable Education Fund EHIS Estonian Education Information System EiE Education in Emergency EMIS Educational Management Information System ESL Early School Leaving ESSA Every Student Succeeds Act EU European Union EWM Early Warning Mechanism EVER Evidence Evaluation Routine FCV Fragility, Conflict and Violence GDPR General Data Protection Regulation GIS Geographic Information System GRM Grievance Redress Mechanism GTCI Global Talent Competitiveness Index HEMIS Higher Education Management Information System HIC High-Income Country ICT Information and Communication Technology IFI International Financial Institution IFOW Institute for Future of Work IoT Internet of Things IP Intellectual Property IRI Interactive Radio Instruction ITS Intelligent Tutoring Systems IVR Interactive Voice Response LAYS Learning-Adjusted Years of Schooling LIC Low-Income Country xi ◄ Table of Contents LMIC Lower-Middle-Income Country MIC Middle-Income Country MINTIC Ministry of Information Technologies and Communications MOE Ministry of Education MOOC Massive Open Online Course MOST Multiphase Optimization Strategy MR Mixed Reality NDEAR National Digital Education Architecture NEET Not in Education, Employment or Training NEP National Education Plan NERN National Education and Research Network NIA National Information Society Agency NLP Natural Language Processing OCR Optical Character Recognition OECD Organization for Economic Co-operation and Development OER Open Educational Resources OOS Out-of-School PAL Personalized Adaptive Learning PLN Professional Learning Network PPP Public-Private Partnership RCT Randomized Control Trial RFP Request for Procurement RLH Regional Learning Hub SD Standard Deviation SMART Sequential Multiple Assigned Randomized Trials SMEs Small and Medium Enterprises SMMR Systematic Mixed Methods Research SND Special Needs and Disabilities SSAMS Sudan Schools Attendance Monitoring System STEM Science, Technology, Engineering, and Math TCO Total Cost of Ownership TVET Technical and Vocational Education and Training TPD Teacher Professional Development UNESCO United Nations Educational, Scientific and Cultural Organization UNICEF United Nations Children's Fund USAID United States Agency for International Development VPLN Virtual and Global Professional Learning Network VR Virtual Reality xii ◄ Table of Contents Key Terms Digitization of the education system refers to the process of converting physical objects into a digital format. Digitizing paper-based education data and document scanning are relevant examples. In these processes, text extracted from physical paper is converted to PDF or other digital formats, which are subsequently stored on the computer. Digitization organizes data into bits (units of information encoded into ones and zeros) that allow computers to process, store, and transmit analog data.1 Digitalization refers to converting analog processes to digital processes such as automating existing processes or retraining workers on using computers. Digitalizing education data in an educational management information system (EMIS) is an example. Digitalization goes further than digitization but does not yet use digital technology to introduce systemic or organizational changes.2 Digital transformation is the redesign of the education ecosystem, including its operational, organizational, and cultural aspects. It is complex, multicausal, and constantly evolving. It requires integrating digital technology and data into daily life and effectively managing the resulting changes that they bring about. Digital transformation requires the organizational system to evaluate, reengineer, and reimagine its functions and strategic outcomes. An example is ensuring digitally and data-enabled decision-making in education at all levels using feedback loops. Such agility will facilitate ongoing digitalization initiatives but should not be confused with them. There is no well-defined endpoint for a digital transformation, which involves continuous improvement by default. Digital transitions refer to the advancements along a continuum with charac- teristic distinctions between analog, digitized, digitalized, and digitally trans- formed systems, with certain inflection points. For example, for countries/ regions, this would mean considering who they are (context), what they want (ambition), why (rationale), where they are (readiness), and what they expect to achieve (results/outcomes) to clarify the digital transitions that may need to occur in the economy, in society and consequently in education, and what processes and pathways to follow and what related mechanisms are needed for governance. EdTech refers to the private market and innovation ecosystem including universities, research institutions, nonprofits, start-ups, and enterprises that develop commercial products, applications, and services for educational and skills development stakeholders. These can include B2C (business to consumer), B2G (business to government), and B2B (business to business) products and services. Education interventions using digital solutions refer to education policy, projects, and programmatic interventions that use digital solutions as part of the intervention design and/or business process redesign. These digital solutions may or may not use EdTech products and services. Making a distinc- tion between EdTech and education interventions using digital solutions places the education-related outcomes and related interactions and processes at the center rather than the technology tools, products, and services. xiii ◄ Table of Contents Executive Summary EXECUTIVE SUMMARY Executive Summary Background: Learning is in Crisis The world is being progressively propelled by technol- in education. However, these are often expensive, require ogy and innovation, with education and skills develop- adequate capacity, and consume a large share of public ment systems racing to keep up. The possibilities of using education budgets. Consequently, the need for robust digital solutions to enhance teaching and learning and to research evidence is being felt much more strongly. revolutionize educational processes are attractive, but the related investment needs are substantial in a fiscally con- Amidst these persistent and complex challenges, AI is rev- strained post-pandemic context, and evidence of effec- olutionizing education and skills development through tiveness is scarce.i At the same time, there are growing its applications in computer vision, speech, machine concerns about the risks of these developments, includ- learning, big data, and natural language processing. Its ing deepening digital inequalities, data privacy infringe- rapid growth is reshaping interactions, communication, ments, algorithmic discrimination, harmful surveillance, living, learning, and work. AI in education (AIEd) has the access to inappropriate content, online predators, health potential to power personalized learning, support teach- and well-being issues, fake news and misinformation, and ers’ understanding, and revolutionize educational prac- cyberbullying. tices by applying technologies such as intelligent tutoring systems (ITS), chatbots, and automated assessment to At the same time, the world is facing a longstanding enhance learning, teaching, and administration. However, global learning crisis with long-term adverse impacts challenges persist, and its impact on education remains on lifetime earnings, health and well-being. This silent uncertain.4 The pandemic and the global adoption of AI crisis is amplifying intergenerational poverty and ineq- have further fueled longstanding debates, raising critical uities. Young people are not gaining equitable, relevant, questions on issues of equity, ethics, and fairness. and resilient knowledge, experience, and skills to pre- pare them for jobs, while a highly dynamic labor market In these chaotic times of rapid technological advance- increasingly exhibits skills mismatches. The results of ments, huge learning gaps, high inequities, and fiscal PISA 2022 demonstrate extremely large deficits in foun- constraints, passive, reactive, and fragmented education dational skills among students, including digital skills policy responses can deepen the learning crisis, leaving and creative thinking skills. This is particularly alarming many young people behind. Strong, proactive educa- considering the macro trends of rapid advancements in tion leadership that leverages transformative ownership artificial intelligence (AI), rising life expectancy, and rising and a wide range of partnership engagements is needed share of the working age population globally, indicating to effectively adapt to a rapidly changing education and the need for quality learning and retraining opportunities skills landscape. It is important to recognize the inter- at all ages to enable productive employment and civic par- connectedness of learning and skills development with ticipation throughout life. broader technological shifts as well as the critical need to achieve high-quality public benefits for all at affordable With the advent of AI, learning and work environments costs. A collaborative and iterative approach, deep contex- are going digital at an unprecedented speed and scale, tual awareness, effective and adaptive leadership, and an but the demand for research evidence on technology use evidence-informed roadmap is needed for steering digital in education and skills development has just started to transitions in education. gain momentum. The availability, affordability, and use of digital innovations in education (across formal, infor- What Should Education Leaders, mal, and nonformal systems) have increased rapidly in Policymakers, and Practitioners Do? recent years. Worldwide investment in EdTech companies is on an upswing, with global investment growing from First, it is vital for education stakeholders to keep steady US$9.56 billion in 2017 to US$25 billion in 2022.3 Many focus on the visionii of providing equitable, relevant, and more governments are advancing their digital investments resilient learning for all.5 The science of learning tells us i While efficacy trials of Edtech products and services show significant promise, measuring effectiveness of digitally enabled education intervention is more challenging and 1 hence, scarce. ii Learners are engaged, Teachers facilitate learning, Learning Resources are adequate, Schools are safe and inclusive, and Systems are well managed; with cross-cutting areas of Technology promotes Leaning and Home Learning environment are supportive (World Bank, 2021). ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Figure ES1. Policy actions to realize an equitable future of learning 1. Pursue systemic 2. Focus on equity reform supported and inclusion, 3. Focus on 4. Ensure financial by political through a results and use commitment that 5. Invest wisely commitment to progressive path evidence to keep is commensurate in technology learning for all towards universal- improving policy with what is children ism needed Source: Saavedra, J. C., et al (2020). that learning needs to happen with joy, purpose, and rigor, history than now to remind ourselves that learning, at its for students to be engaged and motivated to learn. Such core, is about human connections—between students, learning can happen for everyone, everywhere. But this can teachers, parents, caregivers, managers, administrators, only be realized through a fundamental transformation of and broader communities—with teachers being the most the different pillars of a robust education system: Learn- significant mediators of learning.6 More than two decades ers are engaged, Teachers facilitate learning, Learning of research has shown that human learning occurs in the Resources are adequate, Schools are safe and inclusive, and interactions between learners and educators, mediated by Systems are well managed; with two crosscutting areas: the learning environment (physical and digital) among Technology promotes Learning and Home Learning envi- other contextual factors.7 Thus, it will be essential to safe- ronments are supportive (World Bank, 2021). This can be guard the “learning core” by ensuring that the role of edu- accomplished by pursuing systemic reforms with a focus cators is expanded and altered as cultivators, coaches, and on equity and results, with commensurate and wise invest- critical pedagogues. ments including in the use of technology (Figure ES1). Third, tailor digital pathways that can enable a systemic Second, as education races forward to keep pace with shift towards equitable, relevant and resilient education accelerating digital transformations, it is critical to rec- and skills development outcomes for all. Education sys- ognize and support the changing role of teachers as cul- tems can take on different digital pathways, from incre- tivators, coaches, and critical pedagogues (Figure ES2). mental, to progressive, to even a sea change, depending There has perhaps never been a more critical moment in on the starting point and the prevailing circumstances. Figure ES2. Recognize and Support the Changing Role of Teachers Provider, Facilitator Provider Provider & Facilitator & Cultivator and Coach Role of teachers as sole providers Role of teachers also cannot be In the digital era, the teacher's of knowledge is no longer valid, limited to facilitators, connectors role is also to cultivate learning given the widespread democrati- and providers of knowledge experiences, nurture the human zation of information through potential of each learner, public community spaces, print strengthen critical judgement and and digital media discernment, enable collaboration and conviviality, and kindle curiosity and creative thinking The teacher's role has become ever more critical and complex to help students build 21st century skills required to survive and thrive in a digital era. Source: Author’s illustration. 2 ◄ Table of Contents Executive Summary Policymakers are best placed to astutely assess the deter- sidered. The rationale for education transformation, such minant conditions and consider the policy tradeoffs, pit- as equity and inclusion gaols, strengthening the quality falls, and effective practices for tailoring digital pathways and relevance of education, and improving the efficiency for education. The potential digital pathways are often and resilience of the system are core drivers for coherent affected by global, regional and country contexts. Global investments and focus on results. The potential digital and regional megatrends of technological advancements, pathways can then be based on the current state of digi- climate change, demographic changes and geopolitics tal readiness of the economy, society and the education increasingly affect various facets of people’s lives. At the system. Finally, results expected to be achieved play a same time, a country’s sociopolitical and economic con- determinant role in the pathways that need to be adopted text plays a key role in determining suitable digital path- (Figure ES3). For some countries, the transformation in ways. This includes considerations for demographics, education may seem far off, maybe even unattainable in social norms including gender norms, income distribu- the near future, yet a shared vision and policy direction tion, national wealth, economic growth and productiv- can guide and inform systemic actions that policymakers ity, political situation, stability and trust in government, can take now. Chapter 4 and Chapter 5 offer a detailed dis- access to capital and markets etc. The level of ambition cussion on a range of issues pertinent across low-income, related to national goals and priorities, as well as related middle-income, and high-income country contexts. education and skills needs often tend to be carefully con- Figure ES3. Potential Digital Pathways for Equitable, Relevant and Resilient Education Systems Global & Regional Context: Potential Digital Pathways Technological advancements, climate change, demographic change, geopolitics Incremental Country Context*: Socioeconomic and Political context Ambition • National goals and priorities • Education and skills needs Develop a shared Progressive Rationale vision and direction • Equity and inclusion for a Digital Investment strategy & • Quality and relevance implementation • E ciency pathway in education • Resilience and skills development Readiness** Transformational • Digital maturity of the economy • Digital readiness of the education system Expected results in near- to medium-term • Education and skills development outcomes • Socioeconomic outcomes Source: World Bank. Notes: * Demographics, gender, social norms, income distribution, national wealth, economic growth and productivity, political situation, stability and trust in government, access to capital and markets etc. ** • Latent; •• Emerging; ••• Establishing; •••• Advanced 3 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All How Can This Be Done: Enabling the Digital Transition of Education Systems To achieve dramatically improved outcomes, it is impor- 2 Shift from investing in fragmented digital “frills” to investing in integrated digital “core” that supports teaching, learning, and education management. tant for policymakers and practitioners to embrace three mindset shifts to lead digital transitions proactively, Digital transitions in education require public policy strategically, and with an evidence-based approach and government implementation to shift from invest- (Figure ES4). ments that increase fragmentation to those that drive system-wide embedded digital interventions. Invest- ing in digital frills essentially means ad hoc investments 1 Shift from supporting short-term, ad hoc digital inter- ventions to leading with a systemic, fit-for-purpose approach centered on learning and skills development. in hardware, software, and content platforms with no cohesive results orientation. In contrast, investing in an integrated digital core implies coherence across strategic governance, partnerships, data and technology infrastruc- It is vital to take a coherent and collaborative systems ture, and implementation, which deepens the transition. approach when adopting digital pathways in education This matters from the perspective of ensuring inclusion, and skills development rather than a fragmented mar- security and privacy, and high-quality standards for all; ket-led reactive approach where digital solutions are enabling cost-efficiencies from economies of scale and mere add-ons. This can be done by promoting proactive scope; and generating and systematically embedding qual- learning-centered public policy measures and entering ity innovations with measurable effect sizes on learning. meaningful partnerships with industry and civil society. This includes deepening digital transitions in education by taking a systemic approach in the strategic governance, infrastructure, public policy design, and government implementation to improve education outcomes and scal- 3 Shift from adopting techno-centric approaches to adopting education, skills, and learning-centered approaches. ing digital transitions in education with strong and learn- ing-centered private markets and innovation ecosystems Achieving equitable education impacts from digi- (including universities, research institutions, nonprofits, tal transitions in education will require a shift from start-ups, and enterprises) to improve quality and equity technocentric approaches towards learning-centered and increase the efficiency, and relevance of education. approaches. This often requires strong and robust inno- Figure ES4. Embrace Mindset Shifts in Digital Transitions in Education and Skills Development FROM TO 1 Supporting short-term, ad hoc digital 1 Leading with a systemic, SHIFT MINDSETS interventions fit-for-purpose approach centered on learning and skills development & ACTIONS 2 Investing in fragmented digital 'frills' 2 Investing in integrated digital 'core' that supports teaching-learning, and education management 3 Adopting Techno-centric approaches 3 Adopting education, skills and learning centered approaches Inequity and rent-seeking Equity, fairness, accountability and transparency OUTCOMES SHIFT Irrelevant and outdated learning Relevant, up-to-date and forward-looking learning Fragile education systems Resilient education systems better prepared for emergency and crises Source: Author’s illustration. 4 ◄ Table of Contents Executive Summary vation ecosystems and private markets that are incentiv- sions by education leadership show urgency and foster ized and supported to deliver contextualized, high-quality, collaborative, result-oriented governance with a laser and cost-effective educational products. The greater the focus on learning. This layer drives close engagements extent to which education relies on private markets and between stakeholders under the Partnerships Layer for innovations in universities, research institutions, non- Digital Public Infrastructure, Foundations and Platforms profits, start-ups, and enterprises, the more potential for to ensure appropriate coupling of education strategic achieving the transition at scale. These engagements can plans with state digital programs. This is often enabled provide educational institutions with emerging Edtech by university-industry collaboration, intra-government and AI based tools, foster competition, and drive demand and inter-government partnerships, and other innovation at more affordable prices. At the same time, a comprehen- ecosystems wherever useful and relevant. This is followed sive regulatory framework is essential to ensure equita- by the Education ICT and Data Layer comprising the Edu- ble, high-quality education and skills development and cation System Architecture and related ICT/Digital Infra- encourage fair and healthy competition. Collaborative structure driving, at its core, the data agenda and acting engagement between public policy and private markets is as a critical integration layer that glues together the other essential to support traditional and emerging education layers. The Implementation Layer is at the heart of the goals. education endeavor—empowering learners, teachers, par- ents, and administrators toward the design and delivery of Enabling Architecture for the Digital digitally enabled education services at scale for all. Transition of Education Systems This enabling architecture is crucial to: (i) be more Overall, digital pathways in education and skills devel- responsive to equity issues ensuring fairness, accounta- opment can be seen as the interplay of two dimensions: bility, and transparency in the face of a deep global learn- public policy along the vertical axis and market forces ing crisis and very large numbers of people who are out- along the horizontal axis. The digital arena of this inter- of-school and out-of-work; (ii) be more relevant to ensure play comprises at least four layers of the education ecosys- that learning is up-to-date, and forward-looking, given the tem and an overarching transversal layer to scale digital scale and speed of technological change, including AI; and with help of market forces, including research, CSOs, uni- (iii) be more resilient so that education systems are better versities, and startups (Figure ES5). The topmost Strate- prepared to ensure learning continuity in times of emer- gic Layer of Education Leadership and Governance drives gency and crises and of increasing climate-related shocks. the digital agenda where high-priority bankable deci- Figure ES5. Enabling Architecture for the Digital Transition of Education Systems Collaboration and Feedback loops Strategic Layer Lead strategically, collaboratively with Education Leadership and Governance result-oriented governance on education, Depth of Digital transitions digital and data Partnerships Layer Engage the ecosystem, tapping into a Digital Public Infrastructure, Foundations and whole-of-government ecosystem and related Platforms overarching digital infrastructure Education ICT and Data Layer Integrate the education system architecture Education System Architecture & related and education ICT infrastructure along with ICT/Digital Infrastructure high caliber human capital Implementation Layer Ensure high quality and a ordable edtech Digitally enabled education service delivery solutions are incorporated into education service delivery for all Scale of Digital transitions Supported by Market forces Leverage and enable stronger private market engagement, research, development and innovation ecosystems supporting the digital transitions in education Source: Author’s illustration. 5 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Digital Transitions Create • Stronger policy commitments are needed to generate Opportunities but Also Risks and use evidence such as adopting a hierarchy of evi- dence methods and embracing newer methods of gen- Countries that leverage digital solutions can realize erating evidence for measuring additional affordances equitable, relevant, and resilient education systems with of digital solutions that improve education outcomes. a positive impact on learning outcomes if certain con- ditions are in place. One of these conditions is that the At the same time, the digital risk landscape for educa- solutions are designed with the principles of learning in tion systems is complex. Digital service providers (DSPs) mind as well as the principles of digital development— and platform companies, whose business models rely on such as inclusion, scale, and sustainability (Figure ES6).8 sustained user engagement, play a vital role in providing This report outlines and reviews evidence on six potential digital services in education and skills development. The outcomes of education that digital transitions can affect. terms of engagement in these services often increase dig- While a growing number of evidence reviews have stud- ital exclusion, in addition to potentially exposing users to ied the impacts of digital interventions in education, their data privacy violations, online safety violations, threats to results have been mixed.9 Some key findings are as follows: personal health and wellbeing, lax cybersecurity, AI-re- lated risks, and the adverse effects of commercialization • There are many practical challenges for generating of education. There are also newly emerging risks often robust evidence including contextualization, inter- related to advanced technologies that policymakers need vention complexity often including several confound- to keep abreast of and guard against. Some mitigation ing variables, the need for iterative evaluation design measures include building capacity at all levels of the edu- approaches, combined with the cost and time taken for cation system for the responsible use of AI, the implemen- rigorous evaluations. tation of a regulatory framework, and the monitoring of algorithms. AI solutions, platforms, and tools are revolu- • It is vital to embrace plurality of evidence on digi- tionizing education by offering diverse capabilities but are tally enabled education interventions. These must go also introducing novel risks and challenges for students, beyond “what works” to address questions of “how,” teachers, school leaders, and policymakers, particularly in “why,” “under what conditions,” and “for whom.” terms of exacerbating exclusion, bias, and manipulation. Figure ES6. Principles of Digital Development Understand the Share, reuse, and Design with existing ecosystem improve people Design for Build for Establish people-first inclusion sustainability data practices Create open and Anticipate and Use evidence to transparent practices mitigate harms improve outcomes Source: https://digitalprinciples.org/endorsers/ 6 ◄ Table of Contents Executive Summary Research: Shaping the Future of EdTechiv evidence is distinct from evidence on educa- Evidence on the Use of Technology in tion interventions using digital solutions: Definitions and terminologies matter in how evidence is generated, Education interpreted, and used. Evidence on ‘education interven- Research shows that technology can improve educational tions utilizing digital solutions’ is not the same as evidence outcomes if certain conditions are put in place informed on ‘EdTech’. It is critical to distinguish between the assess- by learning sciences and digital development principles. ment of an education intervention that involves the adop- However, literature has not kept pace with educational tion, uptake, and use of digital solutions and the assess- practice of technology use for teaching and learning and ment of EdTech products that support market research for pedagogical and administrative management. Some of and procurement decisions. the reasons for the significant lag between research and praxis include incoherence between speed and scale of Policymakers can establish a conducive regulatory envi- technological advancements with regulation and research, ronment for promoting and supporting the development challenge of generating timely, contextually relevant, and of high-quality learning-centered products and services, cost-effective research, and insufficient demand from poli- requiring clear evidence at various stages of the innova- cymakers for robust evidence on cost and quality.10 tion-adoption-scale-up cycle. Research should be speedy, cost-effective, and scalable to produce the best outcomes. But, with the fast-paced growth in the number and vari- We propose that policymakers proceed with well-informed ety of EdTech solutions, tools, and platforms, it is chal- and qualified optimism13 when allocating marginal invest- lenging to identify quality. Aspects of quality include the ment dollars to the use of digital solutions in education. It is application of the science of learning, the safety of these important to first assess their contextual needs, education tools, the ease with which these tools can connect with objectives and digital readiness levels, in addition to the existing systems, their compliance with regulations and opportunity cost over analog solutions for making quality policies, their cost and eligibility for funding, and their learning, upskilling, and reskilling accessible to everyone. impact on users. Policymakers and institutional manag- ers often have to make rapid decisions regarding digital Context Matters for Unlocking procurement and have to choose between a multitude of the Power of Digital for Education options. With no shared global standards of quality, their Transformation decisions may be influenced by advertising and business metrics than the potential learning impacts of the tech- The readiness of a country to implement digitally ena- nology. Mislabeling digital products as educational can bled education services depends on important precon- have serious negative ramifications for learning. ditions. These include the country’s digital foundations such as high-speed affordable connectivity, digital identi- Therefore, there is a need for a shared set of global qual- fication and digital payment systems for individuals, com- ity assurance standards for digitally enabled educational munities, and societies to benefit more broadly. These also products and services. The EdTech markets are facing a include digital enablers at the national and sectoral levels problem with ‘lemons’11 in which poor-quality products such as digital infrastructure, interoperable data infos- drive out better-quality ones because of information tructures, digital skills and change management capacity, asymmetry, high proliferation of products and because no and innovation ecosystem, backed by relevant policies and foolproof way exists to gauge quality before purchasing. regulations. Together, these enable systemic digitalization iii In 2022, there were 455,000 ‘B2C educational’ apps in a of education service delivery at scale. predominantly unregulated and untested market. Hirsch- Pasek et al (2015) found that only a handful of apps are Systemic assessment of a country’s readiness for using designed with an eye towards how children learn.12 digital solutions in education is crucial for effective change management and for guiding digital transfor- Embracing different methodologies is crucial for mation efforts. Such readiness assessment enables poli- research on digital solutions in education. Digital tran- cymakers to objectively consider different scenarios and sitions in education involve various stages, including international best practices as well as to identify and pri- design, discovery, purchase decision, implementation, and oritize actions that can ensure that policies are relevant for application at scale. Research is lagging at all these stages, today and remain resilient to future challenges. Moreover, and traditional measures of the efficacy and effectiveness there is a significant danger of continuing and worsen- of the solutions are insufficient. The quality, equity, ethics, ing inequities, because of less mature foundational capa- security, safety, and user experience for purported ‘edu- bilities and digital enablers in lower-income and lower cational’ digital solutions must be evaluated using inde- middle-income countries compared to the upper-middle pendent expertise at different stages. income and high-income countries. iii The problem with lemons is outlined in George Akerlof’s 1970 paper on the effect of quality uncertainty on the market mechanism. 7 iv EdTech refers to the private market and innovation ecosystem that develops commercial products, applications, and services for educational and skills development stake- holders. These can include, for example, B2C, B2G, B2B products and services. Whereas education interventions using digital solutions refer to education policy, projects, and programmatic interventions that employ digital solutions as part of the intervention design and/or business process redesign. These digital solutions may or may not employ commercial EdTech products and services. Clarifying this terminology helps to place the education-related outcomes and related interaction processes at the center rather than the technology product/service at the center. ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Policymakers must ensure that the digital transition in mapping and engagement at all levels enhance the suc- education has a focus on equity and rights. Different path- cess of digital transition initiatives (Chapter 4, Figure 16). ways can be taken depending on the needs and context of each country. The digital maturity of an economy is highly Tailoring Digital Pathways – correlated with per capita income: the more mature and Tradeoffs, Common Pitfalls, and diversified an economy or ecosystem is, the likelier it is to Effective Practices have the requisite infrastructure, institutions, regulations, and human capital for the transition.14 Thus, public invest- Each country’s vision for its economy, society, and educa- ments must focus on the needs of marginalized groups tion will drive the digital pathways that it adopts. There (such as children, young people, women, people with disa- are three ways in which digital transitions in education bilities, minority groups, and marginalized communities). can mature and scale: (i) the overall digital ecosystem sets the pace, with education gradually incorporating digital The readiness of education systems for digital transi- resources; (ii) disruptive events or initiatives pressure edu- tions relies on at least five essential pillars: (i) strategic cation improvements, such as the pandemic accelerating leadership and governance; (ii) data and technology infra- digitalization efforts; or (iii) the education sector takes the structure; (iii) digitally enabled education service delivery lead, supported by the digital ecosystem. The third path is and analytics; (iv) an education workforce with the nec- most sustainable, as has been seen in country experiences essary capacity and culture; and (v) the existence of an such as Singapore, Estonia, South Korea and many others. EdTech market and business models. Each of these pillars plays a crucial role in shaping a country’s digital pathways It is crucial to tailor digital pathways to the country con- and together provide a comprehensive framework for text in education and skills development, with a keen informed decision-making Understanding this readiness focus on equity considerations. This involves adjusting level helps inform a country’s preparedness for successful strategies and incorporating digital technologies to meet digital transition in its education system.15 education objectives with a firm focus on inclusion, rele- vance, and resilience. Educational policies must align and Assessing the digital readiness of education and skills potentially strengthen the broader policy environment, in development systems is crucial for informing govern- acknowledgment of their impact on education outcomes. ment strategies and action plans. It encourages ecosystem It is important to consider stakeholder perspectives care- actors to align their approaches with national education fully and create responsive policies for optimizing digi- and skills development goals and priorities. Evaluating a tal transformation benefits and minimizing challenges system‘s readiness provides insights into its infrastructure, during policy implementation. governance, capacity, core education services, and market structures. It also considers how a country’s overall digital Policymakers face formidable and multifaceted chal- approach is supporting high-quality education and skills lenges, including financial considerations, when deploy- development. Such an assessment should take a compre- ing digitally enabled education. In many countries where hensive approach to identifying critical systemic barriers economic constraints are huge and public investment is (such as a lack of capacity or coordination) that may not at a premium, large upfront investment needed for dig- be directly visible or under the remit of the Ministry of ital transformation is a formidable challenge to over- Education. come.17 Policymakers may need to strike a delicate bal- ance between innovations, strategic scaling plans, and the The experiences of Moldova, Georgia, and Kosovo in dynamic education environment. Finding and sustain- the ECA region demonstrate how a systemic assessment ing along a strategic digital pathway should involve both of digital readiness can glean prioritized recommen- short-term and long-term policies, as well as the capacity dations and nuanced actions for advancing education to adapt to social, technological, and economic shifts. objectives in a particular context.16 The assessment must consider first the country’s broader national objec- Moreover, policymakers often have to grapple with criti- tives and digital maturity and the education sector’s chal- cal trade-offs in shaping digital transitions in education lenges and priorities and then use these findings to assess and skills development systems. The journey to embed where and in what ways the country could leverage dig- risk-informed digital approaches throughout an educa- ital solutions to advance education outcomes. By trian- tion system requires careful consideration of trade-offs gulating various data sources and stakeholder mapping, related to the starting point, desired destination, and the the assessment can offer recommendations to enhance “collective” ability of stakeholders to successfully manage foundational, functional, and frontier digital capabilities, change processes. These can lead to incremental, progres- aiming to stimulate action and dialogue on the develop- sive, or transformative impacts, depending on the context. ment of a transition roadmap for education. Stakeholder For example, an SMS-based tutoring guide might be trans- 8 ◄ Table of Contents Executive Summary formative in an FCV country but might be only considered supported by concrete funding, governance, and imple- a digital frill in another. mentation arrangements.vi When decision-makers rely on political considerations or the reputation of EdTech It is pivotal to balance realism with aspirational strate- innovations supersedes evidence and good practices, this gies, ensure durability of policies, and prioritize educa- is often a recipe for failure. At the same time, when deci- tional needs through careful resource allocation. This sion-makers seek evidence, challenges still abound, par- will promote the effective design and implementation of ticularly the difficulty of finding relevant and applicable digitally enabled education policy. The report identifies research as these often maybe outdated, complex, and not and discusses key trade-offs related to the influence of independently reviewed. Bridging this gap with accessible governance structures, the implications of data sharing, and high-quality research is essential to navigate the expo- data privacy and cybersecurity challenges, procurement nential growth of EdTech innovations and ensure that evi- strategies, coordination costs, the role of champions on dence plays a central role in shaping education systems fit effective and systemic use of digital in education, and col- for the digital age. laboration with authorities from other sectors. Outline of the Book While all countries and regions must pave their own digital pathways in education, they can learn from the This book presents a conceptual framework followed by common pitfalls encountered by and effective prac- a discussion of global evidence on the opportunities and tices used in other countries. Learning from experiences risks of going digital, the contextual preconditions, and avoids reinventing the wheel. Yet caution is needed when how digital pathways can be tailored to produce greater aspiring to adopt and implement digital policies of coun- educational impact at scale for all. Chapter 1 provides tries that may have a more developed environment, and important background that sets the stage for the concep- may not share similar contexts, while also balancing ben- tual framework and clarifies the motivation and purpose efits against risks. of this work. Chapter 2 discusses the elements of the con- ceptual framework, expanding on the systemic role played Partnerships between governments, international by digital solutions in the delivery of education services financial institutions (IFIs), the private sector, philan- to enhance inclusion, resilience, and effectiveness. Chap- thropists, and civil society are crucial for ensuring an ter 3 identifies potential benefits and risks of digital tran- equitable, relevant, and resilient digital future in educa- sitions in education, mapping the mixed evidence base tion.18 Digital inclusion and social inclusion are intricately and highlighting opportunities and mitigation measures. and directly enmeshed and require adaptive, responsive, It also offers an elaborate discussion to shape the future and timely public policy interventions. Inaction or a reac- of evidence. Chapter 4 discusses the underlying contex- tive approach is not viable as technology, platforms, and tual conditions that underpin the conceptual framework underlying data industry are not value neutral. Data, and can enable the large-scale design, implementation, in particular, is a critical frontier for all digital transi- and use of digital solutions in education systems. Chap- tions, including education. Multisectoral investments ter 5 discusses how governments and policymakers can are needed in electricity, foundational and shared public think about tailoring pathways for digital transitions in digital infrastructure, and “meaningful” connectivity,v, 19 education and skills development to inform short-term, while for digitalizing the education system architecture, it medium-term, and long-term policy and implementation is vital to leverage people, data, and technology. measures, including trade-off considerations as well as common pitfalls and effective practices. Chapter 6 offers Each country needs to develop a strategic vision for a concluding statement. digitally enabled education and skills development, Table ES1. Tailoring Digital Pathways: Effective Practices and Common Pitfalls Effective practices Common pitfalls • Ensure a high degree of coordination and coherence • Device- or technology-centric strategies and between across policies procurement • Use agile approaches • Not considering opportunity costs • Begin with the problem(s) • Inflexible, long-term plans • Align education sector plans and ICT infrastructure • Inadequate government support • Design for scale • Lack of stakeholder buy-in • Establish collaborative systems • Coordination issues • Incorporate flexibility and adaptability • Disjointed legacy systems • Focus on the usability of data systems • Data privacy and security • Learn from other industries v The meaningful connectivity target is a tool to raise the bar for internet access and set more ambitious policy goals for digital development. It sets minimum thresholds 9 across the four dimensions of internet access that matter most to users. These are (a) regular internet use | minimum threshold: daily use; (b) an appropriate device | minimum threshold: access to a smartphone; (c) Enough data | minimum threshold: an unlimited broadband connection at home or a place of work or study; and (d) a fast connection | minimum threshold: 4G mobile connectivity (A4AI, 2021). vi OECD Directorate for Education and Skills (2023) Policies for the Digital Transformation of School Education: Proposed Analytical Framework and Methodology, at https:// one.oecd.org/document/EDU/EDPC/SR(2023)2/en/pdf 1 Digital Pathways for Education: Enabling Greater Impact for All INTRODUCTION Learning and work environments are going digital at an learning approaches to help schoolchildren recover their unprecedented speed and scale. The availability, afforda- education.” Over 244 million children and young people bility, and use of digital technologies in education, encom- are out of school,28 and 468 million children lived in a con- passing formal, informal, and nonformal settings, have flict zone in 2023.29 Education inequality is particularly grown rapidly in recent years in many countries. Recent shocking, with average education investment per student groundbreaking technological advancements,vii such as in the richest countries (US$11,655) being 13 times higher artificial intelligence-enabled chatbots, encapsulate this than education spending in middle-income countries growth and its pervasive effects on every aspect of edu- (MICs, US$890) and 200 times higher than the spending cation and work. Although the speed and scale of digital in low-income countries (LICs, US$56).30 Globally, two- adoption and use in education vary across income groups, thirds of the world’s school-age children or 1.3 billion chil- demographic groups, and contextual conditions, most dren aged 3 to 17 years old do not have access to the inter- education systems are increasingly leveraging digital solu- net at home, even though an overwhelming majority of tions. The pandemic accelerated demand for emergency digital innovations need devices with internet access and remote teaching modalities worldwide, exponentially have limited compatibility with feature phones.31 catalyzing a widespread yet inequitable adoption of often piecemeal and fragmented digital solutions in education Young people are not gaining equitable, relevant, and systems.20 Furthermore, there are growing concerns about resilient knowledge, experiences, and skills to prepare the risks accompanying digital solutions, such as data pri- them for jobs of the present and the future, while skills vacy infringements, algorithmic discrimination, harmful mismatches are increasingly evident in a highly dynamic surveillance, access to inappropriate content, online pred- labor market. The problem extends throughout formal, ators, health and well-being issues, fake news and misin- nonformal, and informal education and skills develop- formation, and cyberbullying. ment from basic education to postsecondary, includ- ing vocational education, higher education, and lifelong At the same time, the world is facing a long-standing learning. This is particularly concerning given the recent global learning crisis,viii which will likely have a long- advances in artificial intelligence (AI), which are leading to term adverse impact on the lifetime earnings, health, changes in the skills required by employers. Moreover, the and well-being of today’s students, amplifying intergen- average job tenure is declining, and the gig economy offers erational poverty and inequities. Many basic education new and flexible opportunities, especially in the informal systems are not fit for purpose,ix,21,22,23 often exacerbating labor market. These shifts require workers to constantly structural inequities,24 and are insufficiently prepared to upskill and reskill in an increasingly digitally driven world. manage crises. While evidence consistently shows that the Furthermore, the high cost of higher education degrees most effective education interventions rely on teachers and incongruent returns in the labor market, combined and supportive school principals, in the vast majority of with several alternative ways to acquire credentials includ- countries, they do not appear to have the skills needed to ing short-term online courses and highly marketable cer- be effective.25 Pandemic-driven school closures deepened tifications, are prompting many young people to sidestep the crisis, sharply increasing learning povertyx,26 to an formal postsecondary education. This is especially the estimated 70 percent globally and exacerbating inequali- case in industries that value skills over traditional formal ties in education while exposing the deep digital divide.xi,27 credentials.32 Finally, life expectancy continues to increase UNICEF‘s 2022 Pulse Check revealed that “one-third of globally,xii and there is a rising share of working-age pop- nationally developed platforms have entirely shut down, ulations and those above the age of 65, indicating a need are outdated, or are no longer fully functional, limiting for quality learning and retraining opportunities to be vii OpenAI’s Generative AI-based chatbot software ChatGPT surpassed 1 million users in just five days of launch on November 30, 2022, growing 1,000 times to 1 billion users within 3 months. viii The learning crisis has predated the COVID-19 pandemic, as new evidence has confirmed the alarmingly high learning poverty rates. According to the learning-adjusted data published in the joint report ‘The State of Global Learning Poverty: 2022 Update’, the average global learning poverty rate in lower-middle-income countries (LMICs) indicated that nearly 6 out of 10 children were not acquiring minimal literacy proficiency by the age of 10. Sub-Saharan Africa faced an even more critical situation, with 86 percent of children experiencing learning poverty in 2019. ix According to the World Bank’s State of Global Learning Poverty: 2022 Update, even before COVID-19, nearly 6 of every 10 children in LMICs could not read and understand a simple text by age 10. While the last few decades have seen a surge in school enrollment rates, 2017 statistics indicated that over 260 million young people across the world (ages 6 to 17 years) were out of school and at the peak of the COVID-19 pandemic, this reached 463 million (UNESCO 2017; UNICEF 2020). For those enrolled, the promise of education has not been realized: learning outcomes are low, schools are failing learners, and systems are failing schools (The World Bank 2018). 10 x Learning poverty means being unable to read and understand a simple text by age 10. The indicator combines the share of primary-age children out of school, who are defined as schooling deprived, and the share of pupils below a minimum proficiency in reading, who are defined as learning deprived. Thus, it is a combined measure of schooling and learning (World Bank 2022). xi An abrupt and extensive transition to online only education, accompanied by disparities in digital infrastructure and access, exacerbated preexisting socioeconomic and dig- ital divides. Remote learning failed to reach a minimum of 31 percent of students, equivalent to nearly half a billion students because of inadequate technological resources, unsuitable policies, or unsuitable technological infrastructure (UNESCO 2023). xii Global life expectancy at birth reached 72.8 years in 2019, an increase of almost nine years since 1990. ◄ Table of Contents Introduction available for people at all ages to enable them to engage in the quality of learning, and increase its relevance, and productive employment and civic participation through- strengthen the resilience and efficiency of education sys- out life.33 tems. Digital inclusion and social inclusion are intricately and directly enmeshed and require adaptive, responsive, Amidst these persistent challenges, education is expected and timely public policy interventions. While the digital to continue to experience digital transitions driven by promise comes with perils,xiii,35,36,37 neither non-action nor the transformative wave from recent breakthroughs in a reactive approach is an option—the technologies, plat- AI. AI encompasses computer vision, speech, machine forms, and underlying data industry are not value-neutral. learning, big data, and natural language processing (NLP) Data, in particular, is a critical frontier for all digital transi- in enabling digital machines to perform tasks tradition- tions, including education. Multisectoral investments are ally associated with intelligent beings. Its rapid growth is needed in electricity, foundational and shared public dig- reshaping interactions, communication, living, learning, ital infrastructure, and “meaningful” connectivityxiv,38 and and work. AI in education (AIEd) has the potential to offer in digitalizing the education system’s architecture, lever- personalized learning, enhance teachers’ understand- aging people, data, and technology. Hence, policymakers, ing of students, and revolutionize educational practices IFIs, and decision-makers need to take proactive measures through the application of technologies such as intel- to enable an inclusive, fit-for-purpose, and resilient digital ligent tutoring systems (ITS), chatbots, and automated future for education.39 assessment. Despite AIEd’s promise, challenges persist, and its impact on education remains uncertain.34 In the As education races forward to keep pace with accel- context of rapid changes and uncertainty, it is imperative erating digital transformations, what should educa- to recognize the interconnectedness of learning and skills tion policymakers and practitioners do? It is critical to development with broader systemic shifts. Yet education remind ourselves that learning, at its core, is about human systems cannot navigate these digital transitions in silos. connections—between students, teachers, parents, car- Countries require a highly collaborative and iterative egivers, managers, administrators, and broader commu- approach in manageable stages, a deep contextual aware- nities—and is largely mediated by the changing roles of ness, effective leadership with transformative ownership, teachers.40 More than two decades of research has shown and an evidence-informed roadmap. that learning occurs in the interactions between learners and educators, mediated by the learning environment Partnerships between governments, international finan- (physical and digital) and affected by several contextual cial institutions (IFIs), the private sector, philanthropists, factors.41 The capability of digital solutions (even AI-en- and civil society are crucial for ensuring that digital tran- abled humanlike interactions) can never fully replace the sitions reduce rather than reinforce inequity, improve authenticity of sentient human interactions. There is a Figure 1. Key Shifts Required in Digital Transitions in Education and Skills Development FROM TO 1 Supporting short-term, ad hoc digital 1 Leading with a systemic, SHIFT MINDSETS interventions fit-for-purpose approach centered on learning and skills development & ACTIONS 2 Investing in fragmented digital 'frills' 2 Investing in integrated digital 'core' that supports teaching-learning, and education management 3 Adopting Techno-centric approaches 3 Adopting education, skills and learning centered approaches Inequity and rent-seeking Equity, fairness, accountability and transparency OUTCOMES SHIFT Irrelevant and outdated learning Relevant, up-to-date and forward-looking learning Fragile education systems Resilient education systems better prepared for emergency and crises Source: Author’s illustration. xiii The role of technology use in education has been a subject of intense debate for a long time. Promises of the past in the use and effectiveness of technology in education 11 are still unrealized and expectations are muted. Research over the last 40 years in the use of technology in education shows that outcomes are mixed at best and cautionary at worst, including expensive failures such as the one-laptop-per-child programs (Facer and Selwyn 2021; Reich 2020; Ozler 2012). xiv The meaningful connectivity target is a tool to raise the bar for internet access and set more ambitious policy goals for digital development. It sets minimum thresholds across the four dimensions of internet access that matter most to users. These are (a) regular internet use | minimum threshold: daily use; (b) an appropriate device | minimum threshold: access to a smartphone; (c) Enough data | minimum threshold: an unlimited broadband connection at home or a place of work or study; and (d) a fast connection | minimum threshold: 4G mobile connectivity (A4AI, 2021). ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All real risk that the widespread proliferation of digitally ena- present ways in which countries can work towards being bled learning solutions could lead to a price premium for able to enact systemic approaches for education transfor- learning with both human and technological interactions, mation enabled by digital technologies. while the poor may only be able to learn using technology. Therefore, it will be essential to safeguard the “learning Building on the World Bank’s education vision set out core” by ensuring that the role of teachers and educators in Realizing the Future of Learning.xv The objective is to is expanded to encompass being cultivators, coaches, unpack the cross-cutting area “Invest wisely in technology,” and critical pedagogues. When technology is harnessed looking into how this can be done to maximize impact at in a spirit of well-informed and qualified optimism,42 it scale for all. Furthermore, emphasizing the human ele- may offer a potent opportunity to make quality learning, ment at the core of learning, the report also draws from upskilling, and reskilling accessible to everyone. the World Bank’s Edtech approach paper “Reimagining Human Connections”44, aligning the conceptual frame- To be relevant, equitable, and resilient, it is important work presented in this work with the strategy’s five prin- for the education and skills development policymakers ciples: (i) ask why; (ii) for all, at scale; (ii) empower teach- to lead digital transitions proactively and strategically ers; (iv) engage the ecosystem; and (v) be data driven. The using an evidence-based approach (Figure 1). Dramatic conceptual framework proposed in Chapter 2 holds edu- and urgent mindset shifts must be embraced and acted cation goals of equity, relevance (quality, effectiveness, upon to achieve dramatically improved outcomes. The and efficiency), and resilience as central and the learning key shifts are as follows. Shift 1 involves moving away from interaction and the teacher’s role as key determinants of short-term, ad hoc digital interventions to a fit-for-pur- outcomes. It focuses on how education systems can use pose, systemic approach centered on learning and skills digital solutions in a proactive, strategic, evidence-based development. Shift 2 involves moving away from investing manner to meet these goals at scale and with speed. It also in fragmented digital “frills” to investing in an integrated offers a mental model to consider interactions between digital “core’” that supports teaching-learning and edu- inward looking core education elements such as learners, cation management. Shift 3 involves moving away from teachers, resources, schools, home/parents, and systems, adopting technocentric approaches to adopting learning- and outward looking interactions that have economy-wide and skills-centered approaches. These shifts would lead impacts, including partnerships, ecosystems, and markets. to better outcomes by moving from irrelevant and out- dated learning to relevant, up-to-date and forward-look- This work offers a high-level strategic approach to ing learning approaches and materials, from inequity and undertaking the digital transition in education and rent seeking to equity, fairness, accountability, and trans- skills development systems with a focus on “how” to parency and by turning a fragile education system into a make this happen. We first present a conceptual frame- resilient system better prepared for emergency and crises. work followed by a discussion of global evidence on the These shifts require an unwavering focus on learning opportunities and risks of going digital, the contextual pre- impact, inclusion, and innovation combined with efforts conditions, and how potential digital pathways can be tai- to ensure cost-efficiency, privacy, and security. lored to lead to greater educational impact at scale for all. Chapter 2 unpacks the elements of the conceptual frame- As inequities in inputs and outcomes continue to exist work, expanding on the systemic role of digital solutions in and grow between LICs, MICs, and HICs, developing education service delivery to enhance inclusion, resilience, tailored digital pathways and pursuing systemic reform and effectiveness. Chapter 3 identifies the potential bene- efforts becomes even more important. While on one hand, fits and risks of digital transitions in education, mapping digitalization is driving innovation, economic growth, and the mixed evidence base and highlighting opportunities job creation, generating positive spillovers in the broader and mitigation measures. It also briefly examines the need economy of countries, the digital divide both within and for evidence methods and strategies that are responsive to between countries continues to widen, exacerbating pov- digitally enabled education services. Chapter 4 discusses erty and productivity differences.43 This affects LICs and the underlying contextual conditions that underpin the MICs much more than HICs. Given that every country conceptual framework and can enable the large-scale has its own unique, dynamic context and varied levels of design, implementation, and use of digital solutions in digital maturity, undertaking deep systemic reforms may education systems. Chapter 5 discusses how governments not be either easy or feasible in all contexts and situations. and policymakers can think about tailoring pathways for At the same time, digital technology is deeply embedded digital transitions in education and skills development to in learning, life, and work in HICs and many UMICs and inform short-term, medium-term, and long-term policy is likely to become increasingly so in LICs and LMICs as and implementation measures, including trade-off consid- digital foundations strengthen and technology advances erations as well as common pitfalls and effective practices. into new markets. With this recognition, this work aims to Chapter 6 offers a concluding statement. xv Learners are engaged, Teachers facilitate learning, Learning Resources are adequate, Schools are safe and inclusive, and Systems are well managed; with crosscutting 12 areas of Technology promotes Learning and Home Learning environment are supportive (World Bank, 2021). 2 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Chapter 2: ENABLING THE Enabling the DIGITAL Digital TRANSITION Transition OF EDUCATION of Education Systems: SYSTEMS: Framework FRAMEWORK CONCEPTUAL Conceptual Today, the transition to digital in education and skills enabling the digital transitions in education and skills development must be seen as part of the broader, development based on three key shifts: cross-sectoral technological transformation that is affecting the lives of all individuals and communities, Shift 1. From supporting short-term, ad hoc digital both directly and indirectly. For example, the digital interventions to leading with a systemic, fit-for-purpose skills intensityxvi and related wage premiums in labor mar- approach centered on learning and skills development. kets are increasing, and digital and social-emotional skills are a significant predictor of a person’s ability to find and There is an important distinction between digital path- keep a job, to benefit from professional development, and ways in education that take a coherent and collaborative to access civic engagement opportunities. This is because systemic approach and those that take a fragmented and digital technologies have transversal applications, offering siloed approach when adopting digital solutions. A sys- a growing array of affordances45,xvii such as transparency, tems approach would entail using proactive public policy customer centricity, adaptability, customizability, replica- measures combined with meaningful market and civil soci- bility, programmability, interoperability, and interactivity. ety partnerships to enable effective innovations for scale. In the context of this report, two terms hold specific meaning: This ‘anytime, anywhere’ versatility of digital technolo- gies enables them to be used in various ways to enhance • Deepening digital transitions in education or “depth” learning environments and support learning continuity refers to the extent to which digital transitions take a within and beyond institutions and across age groups. systemic approach in the strategic governance, infra- Beyond direct applications in teaching and learning, they structure, policy design, and government-level imple- are being used to strengthen pedagogical management, mentation to improve the quality and equity and administrative management, and governance. In some increase the efficiency, and relevance of education. This countries, this is enabled by information systems and data is represented along the vertical axis in Figure 2. analytics, and in a smaller group of countries, these are further interlinked with the country’s whole-of-govern- • Scaling digital transitions in education or “scale” signi- ment digital foundations, infrastructure, and platforms. fies the extent to which digital transitions are supported In some contexts, Ministries of Education are adopting by strong and learning-centered private markets and a more strategic and systemic approach to use of digital innovation ecosystems (including universities, research solutions in collaboration with other ministries and rele- institutions, nonprofits, start-ups, and enterprises) to vant entities, while also taking steps to address risks such improve the quality and equity and increase the effi- as deepening inequities and increasing security breaches. ciency and relevance of education. This is represented along the horizontal axis in Figure 2. 2.1 Proposed Conceptual Framework When the education sector leads proactively and strate- Education and skills development systems play a dual gically supported by an education-friendly digital inno- role: they enable digital transitions of economies across vation ecosystem and when digital solutions are embed- sectors by preparing and supplying skilled human ded and integrated as “core support” across the different resources in a rapidly changing labor market and, simul- levels, they are likely to strengthen and enhance the taneously, they themselves become digitally enabled to coherence and effectiveness of the core architecture of become more relevant, equitable, and resilient across the education system. This will likely increase inclusion, all education levels. This chapter argues that, while every efficiency, and relevance; improve quality; foster higher country will traverse its own unique digital trajectory in public and private benefits; ensure the sustainability of its attempt to get dramatically improved learning out- investments (for example, using national education data comes and build a strong human capital base, they must to identify and reengage vulnerable learners); and increase all embrace and act upon urgent and important mindset the resilience of the sector in the face of potential crises. shifts. The chapter presents a conceptual framework for At the same time, in the evolving educational landscape, xvi How digitalised are the EU’s enterprises? across all skills levels—basic, medium, and high skill jobs (Eurostat 2022). This is based on the Digital Intensity Index (DII), a 13 composite indicator, derived from the survey on information and communication technology (ICT) usage and e-commerce in enterprises, which counts how many out of 12 selected technologies are used by each enterprise. xvii Affordances are relationships between a physical or digital object and a person. They help determine how an object can be used but are not features. ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Figure 2. Depth and Scale of Digital Transitions in Education and Skills Development Digital Embedded systemically; Embedded Systemic Weak Innovation Ecosystems; Digital Embedded systemically; Public Policy Design and implementation Higher Inclusion; Strong Innovation Ecosystems; Drives Public and Private benefits at Higher Inclusion; potentially higher costs; Drives Public and Private Benefits at Depth of Digital Transitions less e cient; competitive prices; lower risk of privacy violations Lower risk of privacy violations; Fragmented Digital Add-ons; Fragmented Digital Add-ons; Weak Innovation Ecosystem; Strong Innovation Ecosystems; Low Inclusion; Low Inclusion; Likelihood Drives Private Benefits at high costs; Drives Private Benefits at competitive prices; Lowest social benefit; Poor are still priced out; Fragmented Higher risk of privacy violations; Higher risk of privacy violations; Private Market Penetration and Innovation Ecosystem Weak Scale of Digital Transitions Strong Technocentric Learning centered Source: Author’s illustration. private sector involvement is becoming a crucial input for Making a digital transition in the education sector governments who are increasingly dealing with disrup- requires public policy and government implementation tion in the provision of education. Collaboration between to shift away from fragmented investments to those that public and private interests, with their respective comple- are strategic and system wide. Investing in digital frills mentary strengths, can expedite the digital transition in means that investments in hardware, software, and con- education systems. These alliances would need to leverage tent platforms (such as an SMS service, a digital content investments and professional knowledge while attracting platform, or a data automation module) are made on an ad innovative concepts and problem-solving capacity. hoc basis in a piecemeal, uncoordinated fashion. Invest- ing in an integrated digital core implies that the invest- However, in some cases, digital solutions are treated ments are made in a coherent way across strategic govern- merely as add-ons and/or fragmented ways to enhance ance, data and technology infrastructure, policy design, service delivery. This tends to happen when education and implementation. This deepens the digital transition, systems play an overwhelmingly passive or reactive role in which is crucial to ensure inclusion, security, privacy, and response to technological advances (for example, increas- high-quality standards for all. It also enables cost-effi- ing connectivity without a broader plan to ensure access) ciency and scale and makes it easier to generate evidence or when disruptive events (for example, COVID-19) or and systematically implement quality innovations that breakthroughs (for example, generative AI) create momen- have measurable effects on learning. The proposed con- tum for changes. Adopting digital solutions in such a frag- ceptual framework offers four layers of depth in the digital mented way is less likely to improve learning outcomes transitions of education systems: and equity at scale, potentially causing learning outcomes to deteriorate and deepening inequity. Although there • Strategic Layer: Leading strategically with urgency and may be short-term benefits for limited user groups, these fostering collaborative, result-oriented governance often come at high costs, leading to high private benefits with a laser focus on learning and on upskilling/reskill- and short-run investment horizons (for example, a large ing outcomes. For the deepest possible systemic impacts, number of content platforms developed during the pan- strategic high priority decisions and actions by education demic became defunct).xviii,46 Figure 2 outlines the benefits leadership are fundamentally important. These include of increasing the depth and scale of digital transitions. contextualized and adaptive management responsive to the country’s specific needs and to the fast-changing dig- Shift 2. From investing in fragmented digital “frills” ital development landscape, policies and regulation, and to investing in an integrated digital “core” that sys- institutional and public sector capacity development. temically supports teaching, learning, and education Determining strategic priorities involves making impor- management. tant trade-offs (discussed in Chapter 5) and conducting an integrated digital risk assessment. xviii The UNICEF (2022) Pulse Check revealed that “one-third of nationally developed platforms have entirely shut down, are outdated, or no longer fully functional, limiting 14 learning approaches to help schoolchildren recover their education.” ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework • Partnerships Layer: Establishing a whole-of-govern- • Implementation Layer: Empowering learners, teachers, ment ecosystem with shared digital public infrastruc- parents, and administrators to use digitally enabled ture, foundations, and platforms. This is often done at education services at scale for all. This is at the user/ the national government level and must be closely in implementation unit level. It This may require digital sync with sector-level efforts. The whole-of-government interventions in learning and skills development with ecosystem might include a single window procurement multichannel, multimodal affordances that can support system, a secured intranet, and a data pipeline or service teaching and learning as well as strengthen pedagogic bus to disseminate and receive education-related infor- and administrative management. At the school level, a mation. When ICT systems specifically for the education wide array of tools for personalized learning, content sector are linked with these whole-of-government digital development, formative assessments, budget planning, foundations, platforms, and infrastructure, the systems and procurements are available. B2C EdTech products will be able to take better and data-informed decisions and services are often used in this layer. However, the in harmony with other sectoral and national priorities. implementation success depends heavily on the matu- This ensures countrywide regulations are implemented rity of private markets and innovation ecosystems in by default and that education sectoral needs are coordi- the country as well as its public governance and regula- nated and aligned with broader government initiatives tory oversight. In low-capacity contexts with semi-cen- and programs such as cloud infrastructure, AI laws, tralized or fully centralized governance, the access to privacy laws, and telecom regulation. At the local level, high-quality and cost-effective EdTech tools may vary citizens and stakeholders can be encouraged to engage widely among students, teachers, schools, and higher in upskilling and reskilling through participatory digital education institutions. Without a systematic approach, tools. To enable all these developments, there is a critical this fragmentation can compromise safety and prevent need for sufficient human resources with adequate ICT/ the benefits of economies of scale in procurement. On digital skills in government. The depth and scale of the the other hand, a high level of fragmentation may still systemic transformation of education would increase be effective in this layer for high-income contexts with with greater digitalization of the system’s architecture, highly decentralized governance, empowered schools information systems, and related human capabilities, and educators, and formal/informal learning collabora- especially when coupled with whole-of-government tives amongst educators. digital infrastructure and platforms. However, progress at the sectoral level will be hindered if the centralized Shift 3. From adopting techno-centric approaches and shared public digital infrastructure and governing to adopting education, skills, and learning-centered institutions are not fit-for-purpose. approaches. • Education ICT and Data Layer: Facilitating the devel- Policymakers working on digital transitions in educa- opment of a system architecture and ICT and data tion must shift from taking technocentric approaches infrastructure for the education system. Equipping to focusing on education, skills and learning-centered the system for digital transformation will require local, approaches, and to collaborating with strong and robust regional, and national levels of government to invest in private markets and innovation ecosystems. In the con- hardware equipment and devices, software and network- text of this report, the term ‘scale of digital transitions’ ing applications, digital content, information systems, refers to the extent to which the digital transitions in and data analytics. By ensuring greater coordination and education rely on private markets and innovations from consolidation in this area, governments have an oppor- universities, research institutions, nonprofits, start-ups, tunity to use digital technologies at scale to differentiate and enterprises. These engagements and partnerships instruction based on students’ individual learning needs, can provide the sector with new educational technologies to enable procurements with standards of quality and and tools, as the recent surge in AI-based learning tools competitive prices, and to enforce and monitor security has illustrated. Countries with weaker private markets are and safety regulations uniformly. They can also use tools likely to fall back or depend on imported and relatively for pedagogic management such as helping teachers to expensive digital solutions for education that may often acquire new workplace skills through virtual onboard- not meet the needs of the population (for example, when ing, online digital skills training and digital credentialing the national language is not widely used in digital for- programs. Finally, a robust ICT and data infrastructure mats). Such solutions may also fail to meet the country’s is essential for modern day administrative management regulatory norms (for example, outdated data offshoring functions, for example, increasing school efficiency and cybersecurity and safety policies) and infrastructure using network optimization and geospatial analysis and limits (for example, taxation on imports). enabling rigorous impact evaluations using learning ana- lytics and visualization data dashboards. 15 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Robust private markets, research investments, nonprofit range from weak and/or technocentric private markets and organizations, and a digital innovation ecosystem can innovation ecosystems to strong and learning-centered pri- foster competition and drive demand at more afforda- vate markets and innovation ecosystems. ble prices. Private EdTech markets are a large and growing business frontier.xix,47 Collaborative engagement between The digital arena for this interplay between public public policy and private markets is essential to support policy and market forces can be thought of as a multi- both traditional and emerging education goals and to plex—comprising at least four layers of the education experiment with novel approaches such as structured ped- ecosystem (Figure 3). agogy, teaching at the right level, communities of learning, digital personalized learning, algorithms, and tele-coaching • The topmost Strategic Layer (Education Leadership and models. The resulting technologies, processes, systems, and Governance) drives the digital agenda. In this layer, poli- programs can be developed to cater to all education sub- cymakers must start by critically reflecting on if and why sectors (K-12, tertiary, technical and vocational education digital solutions are bankable strategies in the country and training, corporate, and lifelong learning). This can be in question and how best they can be adapted and used. enabled through collaboration among various stakeholders, These decisions are what should drive the close collabo- including private entities, public institutions, communities, rations and engagements in the Partnerships Layer. and associations. The innovation and research ecosystem can provide governments with proof of concepts, guidelines • The Partnerships Layer (Digital Public Infrastructure, for effective designs for scaling with quality, evidence from Foundations, and Platforms) involves engaging the eco- implementation efficacy trials, iterative improvements for system of stakeholders across government, private ensuring greater fidelity and higher evaluation impacts and sector, higher education, research, civil society, and identify learning and innovation frontiers. importantly education service providers and beneficiar- ies (see also Chapter 4, Figure 16) to ensure appropriate While non-government participation brings evident coupling of education strategic plans with state digital benefits, a comprehensive regulatory framework is programs and approaches. essential to ensure equitable and high-quality educa- tion and skills development, to encourage investment, • The Education ICT and Data Layer (Education System and to foster healthy competition. Governments should Architecture and related ICT/Digital Infrastructure) drives actively involve education stakeholders, including the pri- the data agenda. This is a critical integration layer that vate sector, in formulating policies, standards, strategic glues together the other layers. It operationalizes tech- plans, and regulatory frameworks supported by robust nology through provision, support, maintenance, mon- regulatory mechanisms and quality assurance systems. itoring, and capacity building for the Implementation Bilateral and multilateral funding agencies can promote Layer based on strategic foundations, while ensuring public-private partnerships (PPPs), assist in the develop- interlinkages with the developments in the broader eco- ment of tools, and encourage responsible private sector system to benefit learning and skills development. investment in education, while emphasizing collaboration and partnerships.48 Given the wide and increasing pleth- • The Implementation Unit/User Layer drives the heart of ora of solutions, products, and services available, there is the education endeavor, which is learning, and skills a need for substantial regulation and governance, partner- development delivered by digitally enabled services. ships, and multidisciplinary human capacity to translate This involves designing for scale and ensuring that between technological aspects and educational needs to users—students, parents, teachers and guides, admin- navigate the risks and harness opportunities effectively istrators, and job seekers—are empowered to learn and within Ministries of Education and across government. realize their human potential. Tying the Three Shifts Together in the Conceptual A data protection, privacy, and interoperability layer Framework indicated between each of the above layers highlights the critical importance of safeguarding education data The pathways towards a digital transition in education and personal data in a digitally enabled system. Further- and skills development can be conceptualized as the inter- more, feedback loops established across and between these play of two dimensions: public policy along the vertical layers enhance the effectiveness, equity, transparency, and axes and market forces along the horizontal axis (Figure accountability of education processes. There is a clear ten- 2). The depth dimension along the vertical axis indicates sion between safeguarding data and sharing data, that is, the design and implementation of public policy, moving between protecting data from misuse and enabling data from digital fragmentation to full systemic embeddedness. interoperability and sharing for purposeful feedback, which The scale dimension along the horizontal axis indicates the must be managed and improved on a continuous basis. xix The global investment in EdTech was estimated at US$123.40 billion in 2022 (Grand View Research,, 2023). 16 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Figure 3. Digital Pathways in Education and Skills Development - A Conceptual Framework Strategic Layer – Ask Why? Education Leadership and Governance Public Policy Design and Implementation Data Protection, Data Privacy & Interoperability layer Deepening Digital Transitions Partnerships Layer – Engage the Ecosystem Feedback loops Digital Public Infrastructure, Foundations & Platforms Data Protection, Data Privacy & Interoperability layer Education ICT Layer – Be Data Driven Education System Architecture and ICT/Digital Infrastructure Data Protection, Data Privacy & Interoperability layer Implementation Layer – Design for Scale, Empower Education Workforce Digitally-enabled Education Service Delivery Scaling Digital Transitions Private Market and Innovation Ecosystem Maturity Source: Author’s analysis. Note: The four layers are aligned with the core principles outlined in the World Bank EdTech Approach paper; Adaptable across school and post-secondary education, NEETs, out-of-school youth and FCV contexts; Requires engagement across local, regional, national levels; Relevant across modalities, for example, tech-enabled face-to-face, blended/hybrid and fully remote modalities; Education ICT and Data Layer includes hardware, software, networking, digital content, information systems, data analytics, and related human capabilities. The digital pathways of the education system will evolve 2.2 Deepening Digital Transitions in by traversing all the four layers along both the depth Education: Shifting to a Systemic and scale dimensions. Each layer can exert influence other layers both below and above it, and all layers con- Approach tinually and dynamically interrelate. Some layers may be This section discusses the depth of digital transitions, stronger and more established than others or may exert as illustrated in Figure 3, for each of the four layers. The greater influence on decision-making. Hence, care must topmost layer (Section 2.2.1) refers to the need for coun- be taken to ensure that the learning core is always central tries’ strategic leadership to be collaborative and results to all endeavors and strategies and that the chosen digital oriented. This layer would need to be integrated into the pathways are a good fit for the country’s unique contex- cross-sectoral, whole-of-government ecosystem and the tual dynamics and processes. The Implementation Layer country’s critical digital public infrastructure (Section is highly susceptible to fragmentation and is relatedly sus- 2.2.2) while interconnecting with the architecture and ICT ceptible to the formidable digital risks discussed in Chap- and data systems of the education system (Section 2.2.3). ter 3 given the wide array of EdTech products and services The user/implementing unit layer (Section 2.2.4) can be that exist. Hence, it is important to consider ways to disconnected from the rest of the education system unless ensure a minimum level of digital competence across the it is embedded within the education system’s architecture population in proportion to their level of digital exposure. and within the country’s digital public infrastructure so that they exist and function efficiently together. When the The following subsections will discuss each of the layers implementation layer can fit into a coherent whole with and provide examples of how digital technologies can be the layers above, it will help to produce measurable out- integrated to enable education transformation. comes. 17 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Figure 4. Unpacking the Strategic Layer Strategic Layer – Ask Why? Education Leadership and Governance Demonstrated Political Contextualized Strategy Financing Policy and Legislation Will & Durable Vision Adaptive management Institutional Capacity Intersectoral Collabora- Data & ICT Governance Policy and Legislation Development tion and Regulation 2.2.1 Education Leadership and Governance on Digital Learning highlighted the importance of these (Strategic Layer) efforts, noting that countries with a strategy and plan were more resilient in their use of digital learning during the The principle governing this layer is: ask why (Figure 4). pandemic.50 Countries are increasingly recognizing that Policymakers need to understand their educational chal- their education strategies need to evolve in a changing lenges and determine what educational changes they wish digital landscape while being grounded in their own con- to achieve. They should then enquire whether, to what textual realities. These strategies must be cross-cutting extent, and in what ways technology can support achiev- as they traverse and penetrate all parts of the education ing that educational change and, on that basis, develop a system as well as overlap with other sectors (for example, digital pathway that will help to achieve that vision. health and employment). Digital strategies must enable the achievement of the end educational goals rather than Leadership with a realistic approach to technology, a being the end goal in and of itself. Finally, implementing demonstrated political will, a durable vision for edu- digital strategies often requires both significant invest- cation, and strong partnership coalitions is critical for ments and adaptive management as digital solutions going digital in education. See Box 1. A country’s socio- require more skilled support and maintenance to ensure economic context, its developmental goals as outlined in that systems are functioning and relevant.51 its national plans and vision, and its ambition and durable commitment to build its human capital are paramount for In terms of data regulation, most countries and regions setting achievable goals for benefiting from technological are taking some measures, but this can be expected advances. Policymakers and IFIs increasingly recognize to continue to be challenging, especially regarding the need to take proactive measures to govern and admin- cross-border data sharing. Data governance is becoming ister the use of technology in education while at the same more complex due to increased data flows across organi- time creating an enabling innovation ecosystem. zations, sectors, and borders, necessitating multinational solutions. Although the components of data governance There have been recent urgent global calls for countries can be integrated into a comprehensive digital transfor- to develop digital strategies and action plans to trans- mation strategy, fragmented legacy systems often impede form education at scale49. UNICEF’s 2022 Pulse Check the implementation of such strategies. The EU has played Box 1. The Case for Techno-realismxx: A Long History Techno-realism demands that we think critically about the role that tools and interfaces play in human evolution and everyday life, anticipate the mixed blessings of today’s emerging technologies, and recognize the need to be forever on guard for unforeseen consequences that must be addressed by the thoughtful design and appropriate use of digital tools.52 It also requires us to consider the social and political implications of technologies so that we might all have more control over the shape of our future. The heart of the techno-realist approach involves a continuous critical examination of how technologies—whether cutting-edge or mundane—might help or hinder us in the struggle to improve the quality of our personal lives, our communities, and our economic, social, and political structures. Source: Shapiro et al (1998). xx Conceptualized in the original Overview and Statement of Principles, introduced on March 12, 1998, it was a collaboration of 12 technology writers—David Bennahum, 18 Brooke Shelby Biggs, Paulina Borsook, Marisa Bowe, Simson Garfinkel, Steven Johnson, Douglas Rushkoff, Andrew Shapiro, David Shenk, Steve Silberman, Mark Stahlman, and Stefanie Syman. ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework a leading role in taking several regulatory policies and becomes pivotal to digital transformation endeavors. actions to ensure that countries design data governance Proactive measures are necessary to enhance the capac- policies and enforce compliance. One key pillar of the ity of public officials and proactively anticipate emerging European strategy for data, the Data Governance Act seeks digital needs.55 For example, with increasing digitalization to increase trust in data sharing, strengthen mechanisms and related datafication, it is imperative to develop the to increase data availability, and overcome technical obsta- capacity of public officials to deal with the size and com- cles to the reuse of data. The EU Data Governance Act plexity of the data for which they are responsible as well as aims to support the setup and development of common the capacity to manage related risks56 as data management European data spaces in strategic domains involving pri- increasingly shifts towards nongovernmental actors, par- vate and public players in sectors such as health, envi- ticularly EdTech vendors. This has traditionally been the ronment, energy, agriculture, mobility, finance, manu- remit of specific departments and ministries, but evolving facturing, public administration, and skills. Similarly, the business and citizen demands require public sector staff Canadian 2023–2026 Data Strategy for the Federal Public to acquire an innovative, versatile, and customer-focused Service outlines a comprehensive vision, strategic actions, mindset. and real-world case studies to enhance the use and value of data Canadians. It aims to align government-wide pri- 2.2.2 Digital Public Infrastructure, Foundations, orities, shape digital initiatives, and foster coordinated and Platforms (Partnerships Layer) efforts among federal organizations and senior leaders, including chief data officers, to implement data policies The principle governing this layer is: engage the eco- and programs effectively. system (Figure 5). Education systems should take a whole-of-government and multi-stakeholder approach to Digital transformation is a complex challenge for the identify and incorporate innovative and effective ideas to public sector that requires thoughtful coordination, support students, teachers, school leaders and education advanced digital and change management skills, and the administrators. Ministries of Education should be proac- ability to legislate on complex regulatory frameworks tive in finding ways to identify, incentivize, integrate, and pertaining to data and technology, including the rap- sustain the creators in their country. idly evolving AI landscape. In the face of such complexity, committed leadership, sustained funding, and strategic By establishing close coordination and links between foresight are vital to mobilize effective digital initiatives.53 the education sector’s specific ICT systems and Adopting an intersectoral approach is critical for govern- whole-of-government digital foundations, platforms, ment organizations, as it promotes collaboration among and infrastructure, education policymakers will be professionals from diverse domains such as finance, edu- empowered to make better and more data-informed cation, social assistance, and health care in addition to decisions. The systemic transformation of public educa- technology specialists. These elements enable govern- tion would be deeper and more scalable when the sector is ments to deploy state-of-the-art infrastructure, exploit connected with whole-of-government digital infrastruc- data insights, and use cutting-edge technologies such as ture and platforms. Some countries have further devel- AI and 5G54 to increase the effectiveness of social service oped their education system architecture to interoperate delivery while also increasing inclusion and responsive- and integrate relevant information systems for targeted ness to the needs of the people and the planet. and holistic service delivery (see Moldova’s single window procurement systems and secure vendor management sys- As governments grapple with intricate and rapid techno- tems, Estonia’s X-Road, and India’s National Digital Edu- logical advances, skills development in the public sector cation Architecture or NDEAR). Similarly, SOL (short for Figure 5. Unpacking the Partnerships Layer Partnerships Layer – Engage the Ecosystem Digital Public Infrastructure, Foundations & Platforms Transactional Technologies Information Technologies Operational Technologies Ex. Digital Platforms; Distributed Ledger Ex. ERP, CRM, Big data & Al, Cloud Ex. Robots, IoT, 3D printing... Technologies (Blockchain)... Technologies... Digital ID Systems Digital Payment Systems Electrification, Meaningful Connec- tivity 19 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Online Bidding Solution or Solução Online de Licitação) is tion, and education can move from legacy systems to more a Brazilian digital solution designed for community-level fully interoperable and integrated systems, including such procurements for community-driven development pro- innovations as unique IDs. Aadhaar, India’s unique iden- jects. In its successful pilot in Brazil, SOL demonstrated its tification system, is the cornerstone of the country’s data potential to enhance efficiency, transparency, and govern- infrastructure, providing a centralized means of uniquely ance in procurement processes. The app streamlines con- identifying residents. This system plays a pivotal role in nections between community associations and suppliers, various data systems, offering a reliable foundation for automating the entire procurement process and increas- streamlined services, efficient governance, and enhanced ing audit capacity by providing secure data storage.57 data management across diverse sectors. Cross-sectoral human development interventions can be particularly While shared digital public infrastructure and govern- relevant for out-of-school youths, those not in education, ance structures tend to increase long-run efficiency employment, or training (NEET), and those in the school- and reduce fragmentation, they need engagement from to-work transition if the interventions focus on the devel- and close coordination among a broader ecosystem of opment of specific skills. Digital payments across all sec- partners. Some clear areas of synergy can be found in the tors, including in conditional cash transfer programs, is human development sectors, especially areas that could another crucial way to deepen the transition systemically. benefit from economies of scale. This is particularly rele- Digital payments enable transparent and efficient incen- vant in underpinning platforms, infrastructure, and other tive design, increase transparency and resilience, and foundational elements such as unique IDs, data system reduce switching costs, which is especially important for architecture (education information systems, health infor- protecting the most vulnerable in times of crisis. See Box 3 mation systems, and social protection information sys- for Estonia’s experience.59 tems), remote services (telemedicine and remote tutoring), and integrated community-level delivery of essential ser- To reach all learners, empower teachers, and facil- vices. Private sector innovation and engagement is critical itate data-driven decisions in the education sector, for enabling greater uptake and use of shared digital public a whole-of-government approach and strategy that infrastructure, for example, through global, regional, and engages a broader ecosystem of actors in a country local innovation communities and hubs. There is a need to is essential. Education systems, equipped with their consider the role played by transactional, informational, and resources, skills, and interdependencies, cannot bear the operational technologies in producing efficiency gains and sole responsibility for steering a country’s digital transition. reducing costs and the frontier developments (see Box 2).58 The complexity of this transformative journey necessitates a broader national context encompassing various sectors There are opportunities to harmonize data systems and and stakeholders. Effective leadership is pivotal in coor- infrastructure related to the human development sec- dinating the efforts of governmental bodies, educational tors. For example, the architecture of and approaches to institutions, and industry partners. To guide this intricate management information systems in health, social protec- process, an evidence-informed roadmap is indispensable. Box 2. Classification of Technology based on the Nature of Cost Savings New digital technologies vary in their underlying source of efficiency gains, one of which is cost savings. Based on the nature of their cost savings, these technologies can be organized into three types: • Transactional technologies digitize business models. Examples include digital ecommerce platforms and blockchain. The fundamental driver is the falling cost of matching demand and supply. The main effect is to reduce information asymmetries and facilitate market transactions that otherwise might not happen. • Informational technologies exploit the exponential growth of data. Examples include business management software, cloud computing, big data analytics, and machine learning. The fundamental driver is the falling cost of computing. The main effect is to lower coordination costs. • Operational technologies combine data with automation. Examples include “smart” robots, 3D printing, and the Internet of Things (IoT). The fundamental driver is the falling cost of automating routine functions with “smart” machines. The main effect is to reduce production costs including labor, materials, and, in many cases, energy. Source: Hallward-Driemeier et al (2020). 20 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Box 3. Enterprise Architecture in Estonia Estonia uses an interoperability framework that plays a role similar to an enterprise architecture. The Esto- nian IT Interoperability Framework is a set of standards and guidelines to ensure communication between the information systems of central and local government agencies. The IT Interoperability Framework employs three perspectives: organizational, technical, and semantic interoperability. This enables decentralized public and private databases to be interoperable at the national level. For example, when the birth of a child is regis- tered on the Estonian State Portal “eesti.ee,” the child is automatically registered for benefits and enrolled in school. This is possible only because of the interoperability of the Population Registry, the IT system of the Edu- cation Department, and the IT system of the Social Insurance Board. This Interoperability Framework is not written into law. However, its obligatory nature is regulated by the legal agreements among government agen- cies, the private sector, third sector organizations, and private citizens who can submit proposals to access data. The aim of the IT Interoperability Framework is to ensure that state information systems are citizen-focused and service-based. Estonia applies open standards and open-source-based solutions in information systems to ensure their interoperability. Over 150 public sector institutions are connected to X-Road, which provides unified and secure data exchange between the public and private sector bodies and is used daily by more than 1,000 organizations. It is calculated to save more than 820 years of working time for the state and citizens annually. However, these savings are the result of only 5 percent of queries done via the X-Road, as they do not take into account all the savings that occur because of the automatic machine-to-machine data exchange, which amounts to 95 percent of the queries. The Estonian government estimates that its digital infrastructure has led to annual savings of about 2 percent of GDP. Source: e-Estonia (n.d.) Liimatainen et al. (2011), Mayakul at al (2019), and Ministry of Economic Affairs and Communications (2011). This roadmap must be concise yet comprehensive, outlin- of the educational benefits of electrification initiatives ing strategic goals and milestones in manageable stages include reduced truancy rates in Argentina after electric while being flexible enough to adapt to evolving techno- lights were installed in schools and significantly increased logical landscapes and societal needs. See for example the completion rates for primary and secondary schools in National AI Policy for Rwanda that clearly identifies several Sudan and Tanzania because of the availability of solar lead institutional actors across the government. A method- energy.61 Without adequate energy supply, deep digital ical, purposeful, and staged approach helps to manage divides will continue to exclude poor rural communities. risks and allows for course corrections and continuous improvement to enable the equitable distribution of ben- The expansion of broadband networks that offer mean- efits. However, if the whole-of-government strategy does ingful connectivity is imperative for fostering educa- not consider the local country context, its readiness for the tional equity.62 A dependable, affordable, and good-qual- implementation of digital technologies, and the physical ity broadband connection is indispensable for students infrastructure needed to do so, it will not bear fruit. who need to access various online educational resources, including interactive learning platforms, virtual tutoring, The electrification of learning environments, both at and e-books. These resources enrich their learning expe- home and at school, is a nonnegotiable prerequisite riences and complement traditional classroom teaching, for pursuing digital transitions in education. It is foun- empowering students to delve into subjects beyond their dational for providing safe and conducive learning envi- textbooks. Particularly during crises, as exemplified by ronments in all seasons, even during extreme weather the recent COVID-19 pandemic, the necessity of remote events, while enabling the use of devices and connectivity learning becomes apparent. Expanding broadband net- and realizing the full potential of digital and ICT imple- works facilitates students’ engagement in virtual class- mentation. Furthermore, it extends study time by ensur- rooms, online lectures, and collaboration with their peers ing the consistent availability of lighting and heating in and educators through videoconferencing platforms. This the many schools that operate in multiple shifts or offer ensures the uninterrupted continuity of education irre- flexible remediation options. Adequate lighting is cru- spective of any physical barriers. In the contemporary cial for teachers’ activities such as lesson preparation and digital era, proficiency in technology and digital literacy planning. A study that examined the broader relationship is imperative for preparing students for the future work- between electricity consumption and education across 210 force. The expansion of broadband networks serves as a countries revealed a strong correlation (above 66 percent) catalyst for fostering these skills, providing students with between electricity consumption per capita and higher the essential tools for online research, communication, scores on the education index.60 Some tangible examples critical thinking, and problem-solving. 21 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All National Cloud Computing and broadband infrastruc- ogies that enable the issue, storage, and verification of ture are critical digital infrastructure that should be digital credentials.65 Another example of university-led shared among public agencies, including the public edu- shared infrastructure is the National Research and Edu- cation sector. This critical digital infrastructure should be cation Networks (NRENs) in which universities pool their shared among public agencies, mirroring the cost-saving connectivity needs and engage in collective bargaining to aggregated demand approach of higher education research reduce costs. NERNs also share computer infrastructure networks. A cost-effective way of minimizing expendi- and virtual labs and conduct joint research. tures for government cloud and internet infrastructure is to take a strategic approach instead of a fragmented 2.2.3 Digitally Enabled Education System one. The government can consolidate procurement and Architecture (Education ICT and Data standardize service delivery by distributing the recurring Layer) expenses related to cloud and connectivity services among various ministries. Within the domain of public services, The principle governing this layer is: be data driven Canada’s My Canada Sign-On optimizes efficiency and (Figure 6). Transparent standards and interoperable data convenience by consolidating access to various portals (for architecture are key to making evidence-based decisions example, healthcare, grants, driver’s licenses, and taxation) and building a culture of learning and experimentation. under a unified ‘single sign-on’ login system. Additionally, Countries should have flexible, scalable data systems that Ministries of ICT/digital and Ministries of Education are avoid data silos and vendor lock-in. A culture of gathering pivotal in driving digital transformation and educational rigorous data about the impacts of technology use is an initiatives. The focus on alignment between government emerging priority. bodies highlights the importance of cohesive strategies, while joint decision-making on budget allocations can be Education leaders must design, develop, agree on, and used to reduce costs and optimize resources. promote the architecture and shared digital public infrastructure of the education system to ensure that Integrating more complex digital data systems into it has a system-wide impact. The ICT architecture of education requires robust infrastructure support and the education sector includes hardware equipment and increased capacity of public institutions and public devices, software and networking applications, stand- sector personnel at all levels.63 Real-time data systems can ards and protocols, tech stacks, registers, microservices, range across the spectrum of digital technologies—from plug-ins, cloud services, data storage and protection pro- those that depend on face-to-face surveys and observa- cedures, information systems etc. Choosing appropriate tions to sensor-based automated computational systems technology and devices has a positive effect on learning that rely on web- or application-based tools for comput- outcomes.66 Education policymakers, like those in all ers, tablets, and phones (with smart or basic features) for industries, are recognizing that technology is not a luxury collecting, sharing, managing, and reporting data.64 For but is as fundamental as other resources (schools, electric- example, the Digital Credentials Consortium (DCC) is an ity, and learning materials) to promote learning. Devices international network of 12 universities that are working should be selected based on their educational use, such as to develop shared digital infrastructure for issuing digital collecting data, teacher professional development (TPD), credentials, conducting user research within higher edu- teaching reading and literacy, or use in remote or hybrid cation institutions, and producing open-source technol- learning. The education system’s ICT architecture should Figure 6. Unpacking the Education ICT and Data Layer Education ICT Layer – Be Data Driven Education System Architecture and Digital Infrastructure Dashboards & Data Analytics (Real-time data collection) Education Registry Asset Management Student Information Communication, Collabo- Platform System System ration & Security Tools Unique IDs FM & Payments System HRM System Procurement System Georeferenced data Learning Management Grievance Redressal Vendor Management System System System 22 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Box 4. Data Storage and Management in Education in Estonia The Estonian Education Information System (EHIS) is a state-run, web-based database that stores live data on education and is accessible to different public groups to a varied extent. All schools and institutions following a curriculum are obligated to enter up-to-date data into the EHIS by law, including kindergarten, primary schools, secondary schools, vocational training courses, universities, and adult education. The EHIS performs automatic logical consistency checks to ensure that the uploaded data are reliable and accurate. Each school has online access to the EHIS through the user interface and can enter data in three ways. First, schools’ digital tools and digital management software are connected to the EHIS so the data can be automatically transferred from the school management system to the EHIS without the need for any additional actions. Second, schools can generate Excel sheets and upload XML files to the EHIS. Third, the school data can be manually entered into the EHIS user inter- face. The E-Service Department of the Ministry of Education and Research is responsible for solving any data entry issues that schools might encounter. Schools can monitor students’ and teachers’ data through the EHIS user interface. In addition, an online platform in the EHIS named Educational Eye makes school statistical data avail- able to the public. In addition, about 50 specific service providers can access a broader range of contract-specified data in the EHIS under strict legal conditions when they need to certify students’ identities to provide them with a public service. Furthermore, over 20 other state-run information systems are connected to the EHIS for data exchange with other state-run information systems. Various digital online tools supporting education delivery and progression are connected to the EHIS via X-Road (see Box 3) to complement the data in the EHIS. Source: Maio et al (2020). combine technology, data, and, most crucially, the human student needs and to improve student learning. Learn- capabilities needed to leverage the benefits of these invest- ing management systems that use real-time data col- ments at scale. These investments may include a variety of lection and analytic tools to enhance classroom-level hardware devices, equipment, and software applications decision-making include blended, in-person, and offline that are linked to information systems/exchanges and modalities such as those in Kosovo (Shkollat.org), Guja- have real-time analytics capabilities. See Box 4 for Esto- rat, India (Command and Control Center), and the Gua- nian example.67 najuato district in Mexico (RIMA) (see Box 568). Real-time data also enables closed-loop experimentation for the iter- The education system architecture supported by digital ative impact evaluation of interventions.69 It can improve infrastructure enables learners, teachers, parents, and the accuracy of predictions and automate repetitive tasks administrators to be data driven, which improves the for teachers and school administrators. This data can be delivery of education services. Countries such as Estonia, used to create feature-rich lesson plans using feedback Uruguay, and the Republic of Korea have actively invested from teachers at local and regional levels. Data-driven and built these capabilities throughout their education sys- lesson plans can help the teacher plan and provide dif- tems at the national, regional, and local levels. As a result, ferentiated instruction, guiding students as they explore these systems are more resilient, adaptive, and responsive their curated learning journeys. Used well, these data can to the needs of students, teachers, and administrators in also help students to complete their homework, receive normal times and in crises. Critically, the architecture feedback in real time, and receive clarifications of any mis- of the sector needs to be aligned with the government’s conceptions using an inquiry-based Socratic process while structure (for example, centralized versus decentralized) teachers still receive an analysis of the student’s learning to ensure that all government officers, from local govern- data. ment authorities to national departments, can use and benefit from it. This includes infrastructure that allows for Digital technologies offer the potential for efficient col- the easy storage and charging of devices in schools (such as lection and analysis of real-time data, though the chosen charging cards), software and network protocols for inter- tool needs to be appropriate to the context and purpose. active and secure intranet capabilities, and user-specific Paper-based data collection systems are slow, adminis- dashboards to extract, collate, analyze, and potentially tratively burdensome, and prone to human error. Fac- forecast data in real time when systems are interoperable tors to consider when choosing a digital tool include its across education information system modules. hardware and software requirements, supported modal- ities, connectivity, offline capabilities, interoperability, The digitalization of real-time administrative and stu- backward compatibility, adaptive capabilities, real-time/ dent assessment data makes it possible for educators, dynamic capabilities, costs, cost-effectiveness, licens- policymakers and parents to respond dynamically to ing, functionalities, decision-making features, reliability, 23 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 5. Data Collection, Analysis, and Dissemination for the Improvement of Learning (RIMA) (Guanajuato District in Mexico) The Secretary of Education of the State of Guanajuato in Mexico has designed and implemented a diagnos- tic tool called Information Collection for the Improvement of Learning (RIMA) to assess student learning. This instrument uses a census-based measurement approach to evaluate students’ performance in mathematics and Spanish, socioemotional competencies, and school environment. Specifically, it targets primary students in grades 4 to 6 and secondary students in grades 1 to 3 in both public and private schools. RIMA is based on the formative evaluation approach, which provides reliable information to inform the continuous development of the teaching-learning process, of teacher training, and of family involvement in education. The state coordinates and implements the entire RIMA process, supervising the evaluation process and disseminating customized evaluation results based on user-type, be it teachers, students, parents or schools. Different stakeholders can access the RIMA analysis results and recommendations online, giving them the information needed to identify good practices and improve the education system. The analysis results can be viewed on the indicators portal of the website of the Guanajuato State Secretary of Education by parents or guardi- ans by typing in the student’s Unique Population Registry Code (CURP) in the RIMA portal here. Relevant supervi- sory, managerial, and teaching personnel can also view the results using their institutional credentials. The results are presented visually and show the percentage of correct and incorrect answers, which can be filtered by subject, school, grade, shift, and group. The site also gives recommendations to reinforce the learning and guidelines for parents or guardians. The result report can be downloaded in MS Excel. The RIMA data in 2021 were used to assess the learning losses of over 600,000 students in the State of Guanajuato during the school closures due to the pandemic and revealed that students achieved only 40 percent of the curriculum goals on average, with the most significant learning losses having been experienced by indigenous populations, girls, and students in the afternoon shifts at public schools and in rural areas. Source: Secretary of Education of Guanajuato (2021a and 2021b). target environment, target population, user-friendliness, the engagement and performance of their students. The design accessibility, digital literacy requirements, human platform enables them to provide timely interventions and resource requirements, language offerings, geographic support students who might be falling behind, demon- compatibility, subject/sector alignment, and data protec- strating how real-time data management can enhance tion and privacy provisions.70 EdDataTools offers a map- learning outcomes.71 Students, families, and communities ping of technologies throughout the data lifecycle in the can use real-time data to track their progress and use this education-in-emergency (EiE) sector, which has been information to advocate for their needs.72 compiled through research and crowdsourcing. While not comprehensive, this mapping can complement other Digital analytical platforms are being used to enable sources when exploring what tools are available. It can and support outcomes-based financing and adaptive serve as a resource to assist in determining which technol- management. A rise in interest in outcomes- and results- ogies are most effective for the specific context in ques- based financing (including social and development impact tion. bonds and outcomes funds) among policymakers, funders, and practitioners has led to increased use of real-time data Real-time data can include data for tracking ongoing and adaptive management. For effective results-based school activities and data for observing the progress of financing to work, real-time performance data from insti- instructors and learners, which together can be used tutions can provide feedback on necessary and timely to inform adaptations to educational inputs, activities, improvements required to qualify for the next financing and outputs. Administrators (such as school or govern- condition.73 In Argentina, for instance, the national Min- ment leaders or program facilitators) can use performance istry of Education used a digital platform to provide real- information to adaptively manage the resources of their time tracking of schools and measurement of their edu- institutions and tailor education programs to the needs of cational outcomes to facilitate the implementation of an students and instructors. Similarly, educators can use stu- outcomes-based financing model.74 Moreover, a variety of dent data to ensure that they are teaching at the right level digital payment systems exist that enable direct account and to customize their instruction based on individuals’ transfers, mobile money transfers, e-vouchers, and digital learning patterns and needs. The Ministry of Education in wallets to pay salaries and make other payments quickly Singapore has implemented the Student Learning Space and to distribute conditional cash transfers to schools, platform, which provides teachers with real-time data on teachers, and families. 24 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Digital analytical platforms using real time data and 2.2.4 Digitally Enabled Service Delivery adaptive management have the potential to revolution- (Implementation Layer) ize education systems by enabling schools and teachers to continuously adjust their strategies based on data- The principles governing this layer are design for scale driven insights.75 For instance, an adaptive management and empower teachers (Figure 7). Starting the design pro- system included in the Educate Girls Development Impact cess by considering how technology can be used to benefit Bond initiative in India allowed for timely course correc- everyone, leveraging ‘anytime, anywhere’ affordances of tions and has had positive impacts on student outcomes.76 digital solutions can ensure that such initiatives are both In Kenya, the Tusome Early Grade Reading Activity, sup- equitable and adaptable to specific contexts and thereby ported by United States Agency for International Develop- sustainable at scale. Technology should enhance teachers’ ment (USAID), used digital tablets to collect, analyze, and engagement with their students by giving them access to monitor data on students’ reading performance to inform content, data, and networks that allow them to provide teachers of the need for any necessary adjustments to personalized student learning. Digital solutions cannot their teaching strategies to improve learning outcomes.77 fully replace teachers; they can best be used to augment Establishing and refining adaptive management systems teaching and learning. Evidence from around the world is increasingly essential for service providers engaged in shows that, over time, the role of teachers becomes more outcomes-based financing. This requires specialized skills central, not less, for the effective use of technology. for digitally enabled real time data management capac- ity.78 This approach can be valuable at all stages of a pro- Digitalization efforts have the potential to significantly gram’s lifecycle, from design to implementation, and it can improve the delivery of services at various levels, making inform immediate operational adaptations at the front a tangible impact on learners, teachers, caregivers, and line of educational instruction as well as strategic adapta- administrative bodies. For learners, digital applications tions at the systemic level.79 Education data plays a crucial offer versatile learning and skills development oppor- role in enabling decision-making by government adminis- tunities through various channels and modes. Teach- trators at various levels, aiding in policy design, strategic ers, caregivers, and communities can use digital tools to planning, resource allocation, and personnel decisions, help them to efficiently manage the teaching and learn- including the employment and training of administrators, ing processes, enhance their pedagogy, and tailor their teachers, and staff.80 approaches to train and coach adult learners. Adminis- trators and deans can use digital technologies to improve Information systems and the intelligence that they their administrative management. This section highlights: contain must be carefully governed and safeguarded to (i) how technology can support each of these user groups ensure the depth of the digital transition. A coherent and through the adaptation and integration of low-tech and effective information system provides the right insights at high-tech tools and (ii) how digitally enabled service deliv- the right time, with the right frequency to the right people ery during implementation can better integrate with the in a clear, understandable and actionable manner. For this, previous layers (Figure 3) to deepen the transition and it is essential to have a well-thought-through education have a more sustainable impact. A broad overview of tech- enterprise architecture and implementation plan. While nologies, products, programs, and emerging trends is pre- various tools have different functionalities and features, sented in this section, while evidence of their effectiveness the core system architecture must be based on ‘core’ objec- is presented in Chapter 3. tives for the education and skills development sector. It must strike a balance between ensuring interoperability A wide array of hardware and software options con- between systems and tools and protecting users’ data. tribute to the multifaceted nature of the digital world Figure 7. Types of Digitally Enabled Education Service Delivery Implementation Layer – Design for Scale, Empower Education Workforce Digitally-enabled Education Service Delivery For Learners For Teachers & Parents For Administrators For Job seekers Learning and Skills Management of Leadership & Skills for Employability Development Interventions Teaching & Learning Operational Management and Entrepreneurship 25 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Figure 8. Array of Applications in Digitally Enabled Service Delivery For Learners For Teachers & Parents Learning and Skills Development Management of Teaching & Learning Interventions • Curriculum & Content Development • Remediation & Recovery (ex. virtual tutoring) • Teacher Professional Development & Coaching • Digital Personalized (Adaptive) Learning • Teacher Certifications & Micro-credentials • Intelligent Tutoring Systems • Lesson Preparation and Delivery • Serious Games • Homework Preparation and Follow-up • Experiential learning (Robots, AR/VR) • Parent-Teacher Communication • Self-paced practice & collaboration tools • Virtual Professional Learning Networks & Collabora- • Tech-enabled assessments tion • Peer learning & real-time feedback tools • Formative assessments (Just-in-time, online) ... • Automated Grading ... For Administrators For Job seekers Leadership & Operational Management Skills for Employability and Entrepreneurship • Policy evaluation, evidence & analytics • Career counselling • Targeted Interventions (equity, quality, e ciency) • Career support services • Large-scale assessments • Skills qualification • Accreditation • Digital Credentials • Digital Certifications • e-Portfolio • Budget & expense management • Digital Education Records • Promotions, Incentives & Job Placement • Job aggregator platforms • Procurement • Upskilling/reskilling • Monitoring and Evaluation • Fundraising ... • Networking platforms ... Source: Authors' illustration. (Figure 8). Hardware includes radio, TV, e-books, tablets, chronous modes. EdTech serves multiple purposes, acting smartphones, interactive whiteboards, and computers, as a primary, complementary, additional, or supplemen- while algorithms and software include QR code scanning, tal resource in various learning environments, supporting optical character recognition (OCR), word processors, vid- self-led learning, mentor connections, and teacher/parent eoconferencing tools, anti-plagiarism software, content/ interactions. learning management systems, chatbots, instant messag- ing, and AI. Moreover, as technology advances, hardware Regarding self-led learning, technology enables student and software are increasingly enmeshed such as in the doubt resolution, self-paced learning, and gamified or case of virtual reality/augmented reality (VR/AR) and sim- immersive approaches. Doubt resolution tools such as ulation games. These tools can be deployed remotely, vir- Doubtnut and Wolfram Alpha leverage OCR and AI tech- tually, in face-to-face settings, or in blended modes, offer- nologies, while pre-set self-learning content can be unse- ing practical solutions to enhance educational processes quenced (for example, on YouTube) or sequenced (for and outcomes in real-world scenarios. example, via online courses). Massive Open Online Course (MOOC) platforms provide wide-ranging accessibility Learning and Skills Development Interventions for by combining interactive self-learning and peer-to-peer Learners learning, including game-based approaches such as Eneza Education and gamified learning paths such as Kitkit The term “learners” encompasses individuals of all ages School and Read Along. Using gamified augmented and engaged in various kinds of learning, including formal virtual reality applications is becoming increasingly prev- education, professional or vocational training, and non- alent in corporate training, higher education, vocational formal education. Technology plays a versatile role in edu- programs, and academic institutions. This trend increases cation, facilitating engagement through remote, virtual, the effectiveness, cost-efficiency, adaptability, consistency, face-to-face, blended/hybrid, or synchronous or asyn- and security of apprenticeship opportunities. 26 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Despite a myriad of applications, most tools operate and open-access AI tools. In terms of OERs, the African in silos, which hinders data-informed service delivery. Storybook Project has created a digital library of openly Aggregated data often benefits product owners rather licensed African storybooks that constitute appropriate than public education providers and users. Integrating reading materials for young African children in home lan- synchronization features, as in Kolibri, is crucial for lever- guages that reflect local content and have been piloted aging aggregated data for learning analytics. While learn- in several African countries. Open or partially open-ac- ing platforms at various levels provide feedback to teach- cess AI tools such as Dalle, Stable Diffusion, Lexica, Chat ers (such as Tangerine: Teach, which also helps teachers GPT, Claude, Microsoft Bing, Meta Llama 2, and Google to interpret the data), there is a missed opportunity for Bard, among many others, are democratizing the creation greater regional and national aggregation and use of feed- of educational content by empowering students, teach- back-based learning analytics and formative assessment ers, and anyone with interest, access, and know-how to data. Developing interoperable products and programs create their own content. Moreover, there are various that adhere to the architecture and protocols of a nation’s digital tools aimed at enhancing reading literacy, such as education system is crucial. This is necessary to enable the digital storytelling and storyboarding, as well as tools that collection and analysis of real-time data while maintain- enhance creative writing. ing and safeguarding user privacy. Similarly, in lesson delivery and homework design, tech- Management of Teaching and Learning for Teachers nologies exist that help teachers to provide increasingly and Parents personalized instruction to students, as highlighted earlier in this report. For instance, the teacher assumes Digitally enabled education interventions can empower the role of an instructor in the TV-based Sesame Street teachers and parents. These interventions include new Workshop, while Matific and Mysticraft employ games and enhanced forms of lesson and content creation, and augmented and virtual reality simulations to facilitate teaching and lesson delivery aids, professional develop- learning. Similarly, there are homework preparation tools ment, coaching, professional learning networks (PLNs), that automate corrections and feedback,81 assist instruc- accreditation, formative assessment, and grading. There is tors in generating, assigning, and ensuring that students a huge variety of digital applications to support the plan- comply with and complete assignments, and save teach- ning, delivery, and management of teaching and learning, ers’ time and effort. Each of these tools fundamentally focused on strengthening the interactions between teach- alters the instructor’s function in the educational process ers and students, among teachers, between parents and and requires them to acquire fresh proficiencies. Simi- teachers, and between teachers and the education system larly, there are popular 24/7 homework help tools such as (see Box 6). Cheggxxi for high school and college students and for those going through the school-to-work transition. In lesson preparation, technologies are reducing the need for teachers to create engaging and tailored lesson For teachers, digital solutions offer a variety of profes- plans from scratch. EdTech Hub’s guidelines for effective sional development and coaching opportunities that lesson planning tools recommend tools that encourage would not otherwise be available. Technology can enable effective pedagogical practices (for example, student inter- many-to-many connected networks, scalability, adapta- action), align with national curricula, provide subject-spe- tion, and audiovisual features, among other affordances to cific examples and pedagogies, encourage collaboration help teachers with their professional development. From and reflection among teachers, are relevant to the teach- the use of apps such as WhatsApp, Skype, and Telegram ers’ culture, language, and context, and are adaptable to by teachers and school communities to videoconferencing the teacher’s prior knowledge and expertise levels. Notable and real-time audiovisual solutions, TPD is being revolu- products of this sort include LearnZillion with high-qual- tionized by digital tools that can simulate students giving ity text-based lesson plans, Diksha and Funda Wande real-time and post-teaching. Teachers have access to dig- with multimedia, Kolibri with offline open educational ital content that helps them to improve and achieve mas- resources (OER) and curriculum alignment, Edraak with tery over core academic concepts, improve their skills and question banks, Oppia with community-built resources classroom practices for effective delivery of instruction, that are adaptable, Teacherly with content that is editable and build their capacity for using digital tools in teaching. and strengthened through peer feedback, Khanmigo using See Box 7 on a UK experience.82 generative AI and has curriculum alignment functionali- ties, and Nearpod using AI, visualization, and interactivity. The anytime, anywhere availability of a wide range of professional learning networks (PLNs) has the potential Moreover, content development is changing rapidly with to support teachers’ professional development. Numer- the availability of open educational resources (OERs) ous virtual and global professional learning networks xxi Chegg generated the most revenue of any educational app in 2022 at US$766 million, out of 455,000 ‘educational’ apps in Google Play and Apple App Store. 27 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 6. Changing Role of Teachers: From Provider to Facilitator to Cultivator and Coach It is now widely acknowledged that the role of teachers as the sole providers of knowledge is no longer valid given the widespread democratization of information through public community centers and print and digital media. Yet teachers’ roles cannot be merely reduced to facilitators, connectors, and coordinators of knowledge. The teacher’s role is to cultivate learning experiences, nurturing the human potential of each learner, strengthen- ing their critical judgement and discernment, enabling collaboration and conviviality, and kindling curiosity and creative thinking. Their pedagogical role remains essential in the learning process and is ever more critical in the digital age, even though they may not be the sole purveyors of knowledge. Source: Authors. (VPLNs)83 cover educators’ diverse needs and interests. allow educators to earn points by providing evidence of They are enabling teachers to modify their mindsets and continuous professional development and lifelong learn- practices through collaborative reflection and by shar- ing and prior learning. Accumulating professional credits ing expertise across contexts, for example, Edmodoxxii can make teachers more competitive for certain roles or and Times Educational Supplementxxiii. Virtual one-on- promotions. For example, in the United States, many dis- one human coaching for teachers can be provided in real tricts have a “step and lane” pay scale where teachers can time, although evidence of its long-term sustained impact move over a lane (that is, increase their pay) by accumulat- on teacher effectiveness is mixed.84 This can be through ing credits, including micro-credentials. text messages, voice calls, or video calls. Teachers can also upload their instruction videos to platforms, and coaches Teachers can use technology to develop just-in-time then provide them with timestamped comments and sug- formative assessments to offer tailored support to learn- gestions. ers, while also including collaborative engagement with parents in their children’s learning. Using technolo- Similarly, albeit slowly, modularized learning offered gy-enabled formative assessments allows teachers to gain through micro-credential certification programs is gain- comprehensive insights into students’ learning, to identify ing traction in education and training in more advanced their strengths and any areas for development, and coun- economies. In many EU countries, micro-credentials are ter any individual or group misconceptions. An illustrative becoming more prominent within qualification systems example is ChalkTalk, an AI-enabled tool that enables the on the basis that they offer students greater flexibility and teacher to leverage student performance data to provide are more suitable for building sector- or occupation-spe- students with differentiated support through data dash- cific skills (reskilling/upskilling) in response to the chang- boards. In this context, teachers can assess lexile levels of ing needs of industry.85 Their links to formal educational texts,xxiv thereby customizing reading content to better qualification frameworks are growing with the adoption suit individual needs. Additionally, tools are being imple- of related regulations in many EU countries such as Esto- mented to empower parents with guidance and activities nia, Denmark, and Ireland. Notably, these frameworks to support their child’s learning and development at home. Box 7. Supporting Teachers to Strengthen Their Pedagogical and Administrative Practice in the UK In the UK, skilled teaching staff are vital for determining the quality of digitally enabled education. To strengthen the pedagogical and administrative practice in education using technology, the UK partnered with the Chartered College of Teaching to launch a free online training course available to all teachers and education leaders. This training course strives to improve how technology is used in teaching alongside other training opportunities offered by the ICT industry. The UK has also been supporting the British Educational Suppliers Association’s LearnED program, which brings together teachers and industry to showcase EdTech products and best teaching practices that use technology through eight accredited continuing professional development roadshow events that are free of charge. Teachers and education leaders are invited to observe a range of technologies in action and share practical technology-based teaching practices with each other. The UK also has plans to establish a network of “demonstrator” schools to showcase best teaching practices that use technology with the support of the EdTech Leadership Group. The network will draw on industry expertise and learning from various school leaders. The “demonstrator” schools are expected to offer visiting teachers with peer-to-peer support and training programs. Source: Hinds (2019). xxii Edmodo is an educational technology platform designed for K-12 schools and teachers to facilitate communication, collaboration, and learning management. It provides 28 tools for educators to create classes, share educational resources, assign tasks, and engage students in interactive learning activities. Edmodo aims to enhance classroom efficiency and connectivity through its online platform. xxiii The Times Educational Supplement (TES) is a UK-based publication and online resource widely recognized for its educational content and resources. Originally a news- paper, TES has evolved into a comprehensive online platform offering articles, teaching resources, lesson plans, and educational jobs. It serves as a valuable resource for teachers, educators, and school administrators worldwide, providing insights, news, and tools to support teaching and learning practices. xxiv While the lexile levels enable tech-based targeting, developing student’s reading skills is a complex task that cannot be swiftly codified and hence the role of teacher as a reflexive coach and cultivator is critical. See https://creativeteachersclassroom.com/2016/06/27/why-its-time-to-leave-lexiles-behind/ ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Through proactive communication, teachers and parents combined with decentralized digital initiatives adapted can seamlessly integrate each other into the student’s to local needs and enforcement of policies and stand- learning journey, fostering a cohesive connection between ards. A well-balanced approach to digital governance in home and school. Just as tools help teachers to provide education will foster accountability among teachers and feedback to learners (for example, Tangerine: Teach), sim- principals enabling them to make informed decisions that ilar tools can also assist coaches in offering constructive enhance school operations, teaching quality and learning feedback to teachers (for example, Tangerine: Coach). outcomes. In countries with high levels of inequity and decentralized/semi-centralized governance structures As with the plethora of products and programs to sup- where capacity varies widely across its territory, the aware- port learning, those currently created for teachers, ness, adoption, and use of high-quality and cost-effective coaches, parents, and communities are largely used in a educational technology tools may vary significantly. This disjointed way, often being implemented in a group of can lead to high fragmentation, compromising safety and schools by a particular network or donor. In some cases, hindering the realization of economies of scale in pro- sets of schools are implementing multiple (often una- curement. Conversely, high-capacity contexts with high ligned) TPD programs, which can overburden teachers, digital maturity are often characterized by centralized while other schools are implementing no programs. While governance of information and quality assurance, decen- learning analytics data is useful at the school level, if done tralized governance in service delivery and perhaps most nationwide, it can inform education decision-making. importantly, by empowered educators who, having been Where multiple programs are being implemented to sup- granted a high degree of autonomy, embrace innovation port teachers, they need to be regulated by the govern- and adaptability and actively participate in formal and ment and aligned with the national education plan and informal learning collaboratives. the country’s protocols, and curricula (such as the national MEWAKA program in Tanzania or the National Numer- Digitally aware, empowered, committed leaders and edu- acy Program in Malawi). cators are key. Enhancing and rewarding digital capacity development and promoting collaboration across the edu- Administrative and Operations Management for cation and broader public sector ecosystem—at all levels— Education Leaders, Administrators, and Multilevel are essential for ensuring, over time, that digital is not the Education Officers business of only one department or entity but becomes embedded across the ecosystem as the natural state of play. Governments—at the local, regional, and national In many LICs and LMICs that have adopted a semi-central- levels—have an opportunity to use digital technologies ized approach, if technology is strategically deployed, it can to support teaching and learning, pedagogic manage- enable the link between centralized policymaking, stand- ment, and administrative management functions. They ards development, and budgetary controls, and decentral- can do this by implementing national learning manage- ized management, which enforces policies and standards, ment systems for learners and teachers, improving teacher streamlines administrative processes, enhances communi- allocation and placement using geospatial analysis, cation, and facilitates data-driven decision-making at the improving teaching and learning through digital personal- local level. Embracing governance models that are appro- ized instruction, targeting resources and academic recov- priate for the digital readiness of the education systems ery programs based on data from the EMIS, and enabling (See Chapter 4.1) can harness the benefits of technological innovative policy design and rigorous impact evaluation advances while catering to diverse requirements, ultimately using learning analytics. contributing to a more responsive and agile system. To ensure for effective education administrative and Technology enables decision-makers to access to real- operational management, it is crucial to carefully bal- time data at the right time. This data-driven approach ance centralized and decentralized digital governance makes the education system more equitable, improves its policies in accordance with the capacity, context, and quality, and increases its efficiency. It enables the use of digital maturity of the country. Governments grappling data visualization and feedback tools, enhances procure- with fiscal constraints may be drawn to decentralization’s ment options, and improves feedback mechanisms to sup- potential to enhance spending efficiency or to centraliza- port early warning systems and grievance redress. Indeed, tion’s potential to leverage scale and exert stronger budg- digital solutions are becoming increasingly important in etary control. However, the administrative governance of educational institutions’ day-to-day administrative and digital investments in education must be well balanced to operational management. They are using applications that ensure efficiencies of scale. This might mean establish- help them with policy evaluation, evidence, and analytics; ing a shared nationwide digital infrastructure and cen- targeting interventions with a focus on equity, quality, and tralized data capabilities and monitoring and evaluation efficiency; conducting large-scale assessments; accredit- 29 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All ing and issuing digital certificates; managing budget and tual effectiveness of different learning, pedagogic, and expense management; informing career progression and administrative tools and platforms. Responding to this job placements; financial and resource management; pro- need, service providers such as EdTech Tulna have started curement decisions; and monitoring and evaluations to to provide support to enable policymakers to make evi- ensure continuous improvements. Digital technologies can dence-informed decisions.86 support the entire data value chain—from collection, clean- ing, analysis, and visualization to use, reuse, and feedback. Early warning systems with vulnerability and risk indica- tors can be enabled by real-time data. These systems can Given the extensive proliferation of digital tools, there is use interoperable data systems to assess the vulnerabilities a clear need to improve procurement processes for digi- of teachers and administrators during training and capaci- tal infrastructure and capabilities. This includes prepar- ty-building and to enable integrated case management and ing bid documents, evaluating bids, appraising, utilizing, referrals, especially for the vulnerable who may face cumu- and tracking the performance of the selected product, and lative barriers to accessing education. They can also use monitoring and evaluating the impact of its outcomes. predictive analytics to identify and actively support at-risk This requires relevant knowledge and independent exper- populations. Similarly, technology can be used in accred- tise for auditing, comparing, and measuring the contex- itation and certification processes to produce electronic Box 8. Applying Early Warning Mechanisms to Reduce Early School Leaving in Romania An early warning system developed and implemented in Romania is helping reduce school dropouts and early school leaving. The early warning system provides a unitary framework for identifying at-risk students based on real-time data collection, guiding the Ministry of Education, inspectorates, and schools to implement prevention, intervention, and compensation measures to mitigate the identified risks. Following the implementation of a pilot, the early warning system mechanism has been scaled up through a national program, the National Program for Reducing School Dropout (PNRAS), funded by the European Commission (EUR 0.5 billion) under the National Recovery and Resilience Plan. Along with the development of the early warning system, a web-based data module was embedded in the EMIS-Romanian Education Integrated Information System (SIIIR). The module collects data through observa- tion, evaluation sheets and self-reported screening tools to identify at-risk students and track targeted activities and progress through students’ case files, where teachers can access their student’s academic progress, attendance, behavior, and other factors that may indicate a risk of dropping out. This module facilitates the timely detection of risks through real-time monitoring, while increasing cost-effectiveness, increasing accessibility for users, and improving communication between different stakeholders. A training program has also been developed to equip users of the early warning system with the skills needed to use it. The training target group consists of teachers, principals, school administrative staff, county inspectors, and Ministry of Education staff. The training program covers: (i) the early warning system in education, including its components, implementation steps, approach, and tools; (ii) the data module functionalities, user roles, types of data collected, analyses, and reports that can be generated; and (iii) comprehensive methodological suggestions for school and classroom-based interventions for students at risk. It offers a range of flexible learning options, allowing participants to tailor their training to their own pace and schedule and access supplementary resources at their convenience. Since its launch, the online training course has been completed by over 40,000 participants, with an additional 8,000 currently enrolled and progressing through the course. The self-paced training course, developed as a virtual learning environment (https://mate.edu.ro/) and con- nected with a learning management system, plays a crucial role in effectively scaling up the early warning system in Romania. The customizable design of the learning management system leads to easy accessibility, seamless enrollment, progress tracking, automated assessment, and feedback processes, with the ability to engage a broad range of participants in a short timeframe. Moreover, the platform gathers relevant data to inform the continuous improvement of the early warning system, such as tracking of learners’ progress, enrollment on the website, completion rates, and the number of schools attended by the trainees, while remaining cost-effective and affordable, upholding the quality, consistency, and flexibility of the learning experience. Source: World Bank in partnership and with funding from the European Commission (EC). 30 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework portfolios, enable time-bound digital signatures for valida- Facebook, Dell, IBM, and Tencent). Amazon Web Services tion, and enable the use of blockchain-based smart con- (AWS) now offers cloud computing programs worldwide, tracts for tracking funds and for market signaling through while IBM created the Pathways in Technology Early Col- verifiable credentials (see Box 8 for Romania example). lege High Schools (P-TECH) initiatives to close skills gaps while opening career paths to workers who do not have a Grievance redress mechanisms (GRMs) have been imple- bachelor’s degree. Microsoft, Google, and many others offer mented in numerous countries to improve public services, free digital skills development courses. The pandemic led including education. These mechanisms for soliciting and some companies to quickly pivot their product offerings redressing citizen complaints have been widely used in to fit demand, such as Microsoft for Education and Google domestic and international development initiatives. GRMs Classrooms, competing with open-source solutions, such as are a way to enhance governance as a whole by promoting Moodle, in both developed and emerging markets. Second, accountability and transparency in public services with a major education technology companies, both private and specific focus on the welfare of marginalized communities. publicly traded, are now offering a wide range of B2B and Digital communication channels are increasingly used in B2C applications and tools such as enterprise software, net- GRMs from lodging complaints and queries to the resolu- working tools, learning management systems, auto-grading tion stage. Also, community engagement and information systems, ITS, skills assessments, and virtual training. New availability on GRM are increasingly enabled by technol- venture capital investments in education have grown by 13 ogy through, for example, large-scale push notifications percent.88 Third, the market also includes telecommunica- via subscriber applications, large-scale SMS, and other tions and internet service providers, impact market intel- online messenger and email-based systems, which can be ligence companies powered by AI tools such as HolonIQ particularly useful in areas facing crises or emergencies. and International Data Corporation (IDC), and data brokers Despite these achievements, GRM endeavors frequently and intermediaries essential for running the infostructures encounter constraints, predominantly due to their local or required for operating the digital ecosystem for education. subnational government focus and their limited ability to feedback into high-level policymaking. However, there is a vast array of unproven products and a lack of contextualization (such as availability in local 2.3 Scaling Up Digital Transitions or regional languages), indicating that ensuring inclu- in Education: Shifting towards sion and democratization of such products and services Stronger Learning-centered will require strong governance capacity and sustained Innovation Ecosystems and efforts. The prevailing market trends are focused on grades and credentials and tend to target population seg- Partnerships ments that are already connected, digitally savvy, and can The education sector is increasingly considered an enor- afford to pay the high costs of technology solutions. They mous but so far underinvested opportunity for digital- also carry worrying digital risks due to the large-scale ization. The overall valuation of the private (that is, not collection and exchange of education and personal data, publicly listed) EdTech startup enterprises from 1990 to including through cross-border transactions and flows, 2022 was US$332 billion (as a benchmark, the equivalent especially those targeting highly vulnerable groups—such in health was US$1.7 trillion), most of which came from as children and young people—for recommendations and four economic regions: the US, China, India, and Europe online advertising.89 (as of September 2022). Extensive market consolidation since 2020 are making some companies likely future goli- Interministerial, intersectoral, and public-private col- aths with extensive market power.87 Moreover, Open AI’s laboration is essential to mobilize financing, strengthen ChatGPT, launched on November 30, 2022, has become capacity development and high standards of quality, the fastest adopted tool in history, garnering over 1 million and ensure that policies, legislation, and regulation sup- users within five days of its launch and 1 billion visitors port innovation ecosystems while securing systemic and within only three months. By March 2023, it was esti- equitable impacts at scale. For example, the proposed mated to have 25 million daily users. This is expected to EVERxxv framework is a way to conduct the quality assur- have significant implications for education systems. ance of digitally enabled education products in which key stakeholders such as governments, postsecondary educa- The digital education economy is a business frontier grow- tion providers, and employers must make sure that their ing in size and importance. First, there has been significant underlying systems and technologies advance ethical and growth in the licensing of education hardware and software equitable access to education and lifelong learning.90,91 and the marketing of education content programming glob- Another example of collaboration comes from the Neth- ally since the pandemic by all of the major technology com- erlands. In response to the growing need for more STEM panies (such as Acer, Alphabet, Amazon, Apple, Metaverse/ graduates and higher-quality training practices, several xxv In developing such an evaluation routine, it is important to embrace methodological plurality that recognizes the value of all types of research, without positioning RCT evi- 31 dence as the best evidence for all EdTech. EVER can be used for EdTech of any type, including those that are designed to promote foundational skills in literacy and math, those that aim to change learners’ behavior, as well as those that combine assessment and intervention (Kucirkova et al, 2023). ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All government ministries and leading companies in the umbrella and leadership of the Federal Ministry for Digital industry are forging closer relationships with public edu- and Economic Affairs and a member of the Digital Skills cation through public-private partnership (PPP) centers. and Jobs Coalition . It has successfully facilitated (and ena- The PPP centers are established in schools that offer bled) the confident use of digital technologies by citizens upper-secondary TVET and higher professional education of all ages, through the full cycle development of the EU in collaboration with various private sector companies. DigComp framework (Standardizing—Evaluating—Qual- The PPP centers are supported by state grants and co-fi- ifying—Certifying—Profiling). The association has con- nanced with private capital with the aim of enhancing the tributed substantially to the Austrian National Coalition product market and educational innovation through a for Digital Skills and Jobs. It brings together a variety of structural exchange relationship between schools and the stakeholders from the private and public sectors (compa- labor market. The PPP centers are autonomous to create nies, institutions, and organizations) and provides them their own market niche and value and provide paid ser- with a platform to launch actions to upskill and raise the vices to the private sector. There are more than 160 oper- level of digital competence in the country. Similarly, the ational PPP centers across the Netherlands involving over Institute for Future of Work (IFOW) in South Africa is 9,800 companies, 5,000 teachers, and 84,000 students. An committed to addressing the challenges presented by the average PPP center engages with 35 companies and other rapidly changing nature of work. IFOW serves as a hub for organizations.92 See Box 9 for Armenia example. multidisciplinary research, policy development, and prac- tical solutions. With a focus on inclusivity and sustaina- With the aim of developing the digital skills of citi- bility, the institute collaborates with experts, researchers, zens, a whole-of-society approach has been adopted policymakers, and industry leaders to contribute valuable and implemented in some contexts. In Austria, econo- insights and strategies for navigating the digital future of my-wide efforts to develop citizens’ digital skills have been work. It actively engages in education and capacity-build- led by fit4internet, a civil society organization under the ing initiatives, fostering a culture of continuous learning Box 9. TUMO Armenia and TUMO Global The Tumo Center for Creative Technologies is an after-school learning environment for teens aged 12 to 18 years old that was set up first in Yerevan, Armenia in 2011. Currently, over 30,000 teens have attended TUMO centers globally. In Armenia, there are 5 TUMO Hubs and over 40 TUMO Boxes for rural outreach and access. Globally, there are 9 operational TUMO Centers while several more are planned (13 new centers are due to launch in 2024- 25). The Tumo program is free of charge and is open to all students. The program does not have any prerequisites, nor does it offer grades or diplomas. Instead, it helps the students to build a live portfolio of work to showcase. The self-paced learning process is based on students’ individual preferences combined with very specific learn- ing targets organized around 14 digital skills area, combining technology and design. These include animation, game development, web development, digital media among others. Guided by skilled educators and mentored by industry specialists, students’ complete activities in the form of tutorials, exercises, projects, and workshops in their personal learning plans via the Tumo World, a game-like virtual learning environment. For its international expansion, TUMO has partnered with local entities through a franchise structure, pro- viding a cloud-based education management system (TUMO Path). Through this cloud-based system it delivers hundreds of hours of educational content along with accompanying in-person workshops. In addition, TUMO provides ongoing support for the project, including regular capacity-building, quality assurance, and oversight. Partners are responsible for local projects and for the management of the centers, providing the locations, operat- ing staff, and funding for the project. Since 2018, TUMO has partnered with a diverse group of organizations. In Paris, TUMO partnered with the city government to run the center through an existing cultural institution Forum Des Images, which serves as the local operating partner. In Berlin, the project is funded by the Kf W, a federal bank and operated by Accenture, which won a public tender. In Tirana, the Albanian-American Development Foundation and the city of Tirana fund the project and have created a dedicated entity to operate the project. In Kyiv, Zurich, and Beirut, private, civic-minded organizations fund and operate the centers. TUMO is actively working with future partners in both the public and private sectors across Europe, the US, Africa, Latin America, and Asia. Source: TUMO. 32 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework and adaptability, thus playing a pivotal role in creating a enabled education services. The UK government has resilient, innovative, and inclusive future.93 been actively supporting the EdTech business sector with a clear vision, guidance, and financing support mecha- Such ecosystem collaboration can also help to ensure nisms. It is working closely with EdTech businesses and more adoption and use of STEM programs by girls and key sector organizations to ensure that EdTech products women. For example, in Poland,xxvi for over a decade, four match the education sector’s needs. The Department of ministries (Ministry of Science and Higher Education, Education has partnered with the British Educational Ministry of National Education, Ministry of Administra- Suppliers’ Association (BESA) in designing LearnED—an tion and Digitization, and Ministry of Labor and Social online platform—to match teachers with relevant qual- Policy, Government Plenipotentiary for Equal Treatment) ity EdTech products and help them to become more and private institutions have been encouraging female informed buyers of EdTech products. On LearnED, free high schoolers to pursue high education in IT and STEM trials of EdTech products are available with case studies fields through the Girls as Engineers! program (‘Dziew- detailing the impact of each product on school educa- czyny na politechniki!) and the Girls go Science! program tion. In collaboration with the Department for Business, (Dziewczyny do ścisłych!). Over 150,000 girls have partici- Energy, and the Industrial Strategy (BEIS), F6S, and Nesta, pated in these two programs, and talented female students UCL Educate—a leading research accelerator program are offered scholarships. for EdTech—has helped EdTech companies to work with schools, colleges, and higher education providers to design PPP coalitions are taking shape to complement each EdTech products aligned with the needs of educators and partner’s strengths in financing and delivering digitally students.94 See Box 10 for Turkiye example. Box 10. Building the EdTech Ecosystem for Technological Innovations in Education in Turkiye Under the World Bank’s Safe Schooling and Distance Education Project in Turkiye, the EdTech Hub, Eğitim Teknolojileri Kuluçka ve İnovasyon Merkezi (ETKİM), was created as an incubator center in the first national Technopark as a collaboration between the Ministry of National Education, the Ministry of Technol- ogy and Industry, SMEs, and Middle East Technical University. ETKIM aims to establish an education tech- nology ecosystem to support teaching, learning, and skills development. It will also promote the commercial- ization of the EdTech sector by creating public-private partnerships between entrepreneurs and industry, by engaging students and teachers with innovations, and by mobilizing researchers from tertiary education insti- tutions. ETKIM is implemented by the Ministry of National Education’s General Directorate of Innovation and Educational Technologies, with three areas of focus: (i) the preparation of technology-supported student-cen- tered, active learning-based teaching scenarios in cooperation with teachers and academics: (ii) R&D activities, research, matching up and collaboration between ecosystem actors for the development, testing, and integra- tion of educational technologies and materials into the educational processes; and (iii) incubation and mentoring support for entrepreneurs at the start-up level in line with the National Technology Entrepreneurship Strategy. The professional learning lab at the EdTech Hub established under the project is structured to support the teaching and learning of 21st century competencies. Modeled after Europe’s future classroom labs, it is sup- porting teachers in developing their capacity to deliver active learning and testing innovative technology-enabled pedagogical approaches. The learning lab hosts weekly workshops and training events for teachers focusing on different subjects in which teachers are trained to prepare teaching scenarios and then practice them in their own classrooms. Trainers follow up with teachers and assess them on their effectiveness. Under the project, 500 such professional labs are also being planned across the country. The EdTech Hub has attracted over 40 firms working on EdTech products including multimedia, gaming, interactive e-content, simulations, and virtual labs. The interest in the Hub and demand to join from the private sector is high. There are calls to expand the Hub with new modalities to encourage investments in the develop- ment of new commercial ideas in the EdTech sector. All of these efforts are supported by an EdTech Strategy for Turkiye, prepared under the project, that is focused on identifying EdTech innovations, promoting technology use in education, strengthening ecosystem interactions, supporting EdTech entrepreneurs, empowering the skills and innovation of teachers and pedagogical staff, and mobilizing EdTech researchers from schools and universities. Source: World Bank. xxvi On special nationwide open days, girls visit technical universities and interact with female students majoring in technical studies and female scientists. Complementary activ- 33 ities, such as running experiments, are also carried out during the campaign. In addition, many more initiatives have been run to encourage girls to pursue STEM, including ‘Lean in STEM’, ‘Girls Code Fun’, and ‘IT for She’ (the biggest IT camp for women in Europe). ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All There is potential for other useful partnerships to be forged in this area. These include partnerships The unique and dynamic interactions among the between universities and industry and through intra- and inter-government partnerships and cooperation various layers, and elements of the conceptual between IFIs. In many developing country contexts framework suggest that countries that carefully where markets and innovation ecosystems are not well tailor their digital pathways and related strate- supported, there will be a need for priming and much gies and actions will likely be able to change their stronger support in terms of guidance, analytics and course effectively as technology evolves. financing mechanisms to get these partnerships off the ground. Also, while PPP arrangements may be becoming common in some HICs and MICs, they may be harder to develop in many LIC and LMIC contexts with insufficient coincidence that many countries that have adopted digital fair market competition, limited skills supply and demand, strategies and solutions in their education systems have and low governance capacity, thereby requiring considera- also embedded and institutionalized them in their poli- ble external support. cies, practices, workflows, and daily lives as well as have deepened their links with industry and their engagement 2.4 Unpacking the Layers: Digital with the private sector in service to people and learning. Pathways in Education and Skills There are also significant steps many countries are taking Development to address the challenges of the digital-climate nexus through education and skills development (See Box 11.95 Based on the conceptual framework and the elabora- Role of Education and Skills to better manage the Digi- tion of each of the four layers into potential component tal-Climate Nexus). parts, we can put the various elements together. Figure 9 presents the four layers along the depth and scale dimen- In the following chapter, we synthesize the available evi- sions complete with their various component parts. This dence on the opportunities and risks involved in choos- is by no means a fully representative view and may include ing the right digital pathways in education and skills or exclude some elements based on their relevance in a development sectors from a global perspective, high- given context. The purpose here is not to delve into each lighting some mitigation initiatives and reflecting on of the component parts but to highlight the character- newer approaches that have become available to shape istics and potential of digital solutions and how they the future of evidence in service of learning and learners. might enable and strengthen education service delivery to improve outcomes and impacts at scale for all. The unique and dynamic interactions among the various dimensions, layers, and elements suggest that countries that carefully tailor their digital pathways and related strategies and actions will likely be able to change their course effectively as technology evolves. Overall, some countries are leading the way in learning and skills development by deepening the proactive gov- ernance of their digital transitions in education (from strategic layer all the way to the implementation layer) while collaboratively leveraging the breadth of offerings from the private sector. They have consistently updated their education vision, policies, investment strategies, and practices over several decades, embedding digital devel- opments and keeping pace with technological changes to enhance the impact of education. Singapore, Korea, China, and several high-income EU countries such as Aus- tria, Denmark, Estonia, and Finland not only perform well in international school-level PISAxxvii assessments (for 15 year olds) but also in adult learning assessments, such as the Programme for the International Assessment of Adult Competencies (PIACC) and in broader assessments, such as the Global Talent Competitiveness Index (GTCI). It is no xxvii Programme for International Student Assessment. 34 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework Figure 9. Unpacking the Layers: Digital Pathways in Education and Skills Development Strategic Layer – Ask Why? Education Leadership and Governance Demonstrated Political Contextualized Strategy Financing Policy and Legislation Will & Durable Vision Adaptive management Institutional Capacity Intersectoral Data & ICT Governance Policy and Legislation Development Collaboration and Regulation Data Protection, Data Privacy & Interoperability layer Partnerships Layer – Engage the Ecosystem Digital Public Infrastructure, Foundations & Platforms Public Policy Design and Implementation Transactional Technologies Information Technologies Operational Technologies Feedback loops Deepening Digital Transitions Ex. Digital Platforms; Distributed Ex. ERP, CRM, Big data & Al, Cloud Ex. Robots, IoT, 3D printing... Ledger Technologies (Blockchain).. Technologies... Digital ID Systems Digital Payment Systems Electrification, Meaningful Connectivity Data Protection, Data Privacy & Interoperability layer Education ICT Layer – Be Data Driven Education System Architecture and Digital Infrastructure Dashboards & Data Analytics (Real-time data collection) Education Registry Asset Management Student Information Communication, Collabora- Platform System System tion & Security Tools Unique IDs FM & Payments System HRM System Procurement System Georeferenced data Learning Management Grievance Redressal Vendor Management System System System Data Protection, Data Privacy & Interoperability layer Implementation Layer – Design for Scale, Empower Education Workforce Digitally-enabled Education Service Delivery For Learners For Teachers & Parents For Administrators For Job seekers Learning and Skills Management of Leadership & Skills for Employability Development and Entrepreneurship Interventions Teaching & Learning Operational Management Scaling Digital Transitions Private Market and Innovation Ecosystem Maturity Source: Author’s analysis. Note: The four layers are aligned with the core principles outlined in the World Bank EdTech Approach paper; Adaptable across school and post-secondary education, NEETs, out-of-school youth and FCV contexts; Requires engagement across local, regional, national levels; Relevant across modalities, for example, tech-enabled face-to-face, blended/hybrid and fully remote modalities; Education ICT and Data Layer includes hardware, software, networking, digital content, information systems, data analytics, and related human capabilities. 35 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 11. Role of Education and Skills to better manage the Digital-Climate Nexus As education systems worldwide increasingly integrate digital solutions, their environmental impact poses significant challenges. Digital pollution in education manifests in several forms. Firstly, there is the direct energy consumption of programs and devices. Secondly, the production and disposal of physical devices like tablets and smartphones significantly contribute to environmental degradation, as these devices quickly become obsolete and often end up in landfills without proper recycling. Thirdly, the continuous use of digital tools and cloud-based solutions generates substantial data traffic, consuming considerable energy resources (Racsko & Troll, 2020). At the same time, the potential of digital technologies can be effectively harnessed to catalyze the green transi- tion across various other sectors using relevant skills and competences delivered by the education sector. Stud- ies underscore the importance of occupation-specific cognitive and technical skills, alongside socio-emotional and digital competencies, and sustainability practices, in meeting the demands of green jobs (OECD, 2023). Countries like India, Indonesia, and Viet Nam are already emphasizing sustainability awareness and STEM-related skills in their educational frameworks to prepare their populations for a greening economy. Thus, situated at the intersection of digital technologies and sustainability efforts, Green Digital Practicesxxviii in education and skills development are emerging as a therapeutic approach. This approach encompasses using digital solutions in the education sector to promote environmental sustainability and efforts to make digital tech- nologies themselves more environmentally friendly. Digital solutions have an important role to play in monitor- ing, predicting, planning for, and responding to climate change and extreme events, and in protecting critical education infrastructure and vulnerable student populations. The application of digital technology in education spans both climate change mitigation and adaptation efforts. Mitigation efforts are aimed at reducing greenhouse gas (GHG) emissions to limit the increase in the aver- age global temperature to 1.5 degrees Celsius above preindustrial levels. Adaptation efforts are aimed at reducing vulnerability and exposure to climate variability, building adaptive capacity, and lowering the costs and damage from climate-related impacts and natural disasters. The following efforts collectively can be helpful to foster a more sustainable and environmentally conscious education system. 1. Cultivating green-digital competences: Ensure school managers, teachers, and students understand how to leverage green digital technologies, online platforms, and digital materials to enhance teaching, learning, and management. 2. Promoting Efficient Use of Digital Tools and use of Digital Learning Materials: Minimize energy consump- tion by optimizing the use of digital devices and applications, and reducing need for traditional print materi- als. 3. Building sustainable digital infrastructure: Building energy-efficient data centers, networks, devices and management systems. 4. Implementing Eco-Friendly E-Waste Management: Responsibly dispose of and recycle obsolete digital devices to reduce electronic waste. 5. Leveraging Digital Tools for Sustainable Practices: Use digital technologies to support and enhance sustain- able practices within educational institutions. 6. Integrating Environmental Sustainability into Curricula: Embed principles of environmental sustainability into educational content to foster awareness and responsibility among students. Many education systems have developed professional standards integrating research, digital, and green com- petencies to guide educators in their roles and professional development. These standards emphasize teacher agency, inquiry-based learning, action research, and collaboration as key competencies. For example, the Austral- xxviii Green digital practices encompass knowledges and practices to make environmentally responsible decisions in digital activities. 36 ◄ Table of Contents Enabling the Digital Transition of Education Systems: Conceptual Framework ian Professional Standards for Teachers, developed by the Australian Institute for Teaching and School Leadership (AITSL), provide guidelines for teacher preparation and development on green digital. In Scotland and Ireland, professional standards explicitly include digital and green competencies, embedding Learning for Sustainability and Digital Literacy across all educational levels (Staring & Schleicher, 2023). The integration of green digital practices into the education sector is increasingly evident through various global frameworks and initiatives. For instance, the Education 2030 Learning Compass, EU DigComp, and EU GreenComp frameworks highlight the importance of a strong foundation in science, digital skills, and positive environmental attitudes (Broberg, 2023). EU DigCompEdu framework addresses teachers’ ability to develop learn- ers’ digital competence in communication, content creation, responsible technology use, and problem-solving (Redecker, 2017). Environmental and sustainability education is now embedded in curricula worldwide, with 77% of secondary schools in 85 countries including these topics in their science and social science curricula (McKen- zie & Benavot, 2024). UNESCO guidelines for designing green curricula cover key areas such as climate science, ecosystems, and sustainable lifestyles (UNESCO, 2024a) while the UNESCO Green School Quality Standard (UNESCO, 2024b) and the EU Digital Education Action Plan (2021-2027) emphasize establishing green schools with energy-efficient lighting and renewable energy-powered data centers. Source: World Bank. 37 3 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All DIGITAL TRANSITIONS CREATE BOTH OPPORTUNITIES AND RISKS Countries that leverage digital solutions can realize eral factors: (i) the incoherence between speed and scale equitable, fit-for-purpose, and resilient education sys- of technological advancements on the one hand and reg- tems with a positive impact on learning outcomes if cer- ulation and research on the other; (ii) the challenge of tain conditions are in place. Solutions must be designed generating timely, contextually relevant, and cost-effec- with the principles of learning and of digital develop- tive research that can help education decision-makers to ment—such as inclusion, scale, and sustainability—in determine the possibilities and risks of specific solutions mind.96 In this chapter, we present the current evidence in their context; (iii) the fact that generalized evidence is landscape and discuss the limited supply of evidence on often less useful on the ground, whereas locally contextu- digitally enabled education interventions, which have alized studies are only relevant in their specific settings; often employed EdTech products and services. We outline and (iv) policymakers frequently make decisions about the potential outcomes of the ongoing digital transitions digital adoption without demanding the necessary evi- in education systems globally, highlighting the evidence dence, instead relying on political considerations, the rep- of potential opportunities, risks, and mitigation measures. utation of specific innovations, and the recommendations The review illustrates the evidence available from low-in- of influential individuals.99 come, middle-income, and high-income countries. We also highlight the increasing tendency of researchers and Digital interventions centered on self-led learning and analysts to use several different methodologies to review improvements to instruction were found to be the most evidence on the efficacy and effectiveness of technology effective forms of EdTech at raising learning outcomes use in education and share examples of policy responses in in a recent study. Rodriguez-Segura (2020) analyzed 67 select high-income countries (HICs). We also bring forth (quasi) experimental studies across 29 LMICs since 2002. newer approaches that should be considered for shaping The studies were classified into four groups, with 21 being the future of evidence and emphasize the key issue of a focused on access to technology, 9 on technology-enabled lack of globally shared standards of evidence for digitally behavioral interventions, 17 on improvements to instruc- enabled education interventions. tion, and 20 on self-led learning. Technology-enabled behavioral interventions were found to be less promising 3.1 Current Evidence Landscape for generating significant effects but highly cost-effec- tive. Although expanding access to technology alone was Past research has consistently shown that technology shown to be insufficient to improve learning, it is a nec- is no magic bullet; it can help to improve education if essary first step for other types of interventions. A simi- and only if certain conditions are in place informed by lar meta-review done by Escueta et al (2017) that focused the science of learning and principles of digital develop- on HICs, found that: (i) providing access alone is insuffi- ment.xxix,97,98 It is important for education decision-mak- cient; (ii) computer-assisted learning with personalization ers who are contemplating investing in digitally ena- applied in math is promising; (iii) behavioral interventions bled interventions in education at scale to be clear and such as large-scale text message campaigns are promising; thoughtful about why they are adopting a particular tech- and (iv) research on online learning is inadequate. nology and to what end—and give due consideration to its realistic potential to achieve this intended goal in their A study that applied the SAMRxxx model to 1,640 tech- particular context. This contextualization of digital solu- nology-related interventions found that most of them tions for teaching and learning is often essential to ensure used technology to augment (46 percent) or modify (34 that they are adopted by their primary users such as teach- percent) the educational experience, with relatively ers, students, and administrators. fewer aiming for technology substitution (15 percent) or redefinition (5 percent). Vegas et al (2019) found that, So far, the research literature has not kept pace with out of all of the interventions, only 11 percent were evalu- educational practice on using technology in the teach- ated externally, while 18 percent were evaluated internally, ing and learning process and in pedagogical and admin- highlighting that research only captures a small snapshot istrative management. This lag can be attributed to sev- of the EdTech sector. Only 26 EdTech innovations were xxix Released in 2014, the Educational Technology Topic Guide may be considered foundational, covering 83 resources (45 research documents, 20 literature reviews, and 18 gray literature 38 reports) on EdTech use in schools in LMICs (Power et al. 2014). The guide reviewed a range of EdTech uses and initiatives, including IRI, audio or video resources, student tablets and e-readers, computer-assisted learning, computer suites, and one laptop per child. Since then, the Educational Technology Map was developed, covering 401 resources in low-resource settings, including 279 observational studies, 81 quasi-experimental studies, 23 experimental studies, and 6 secondary studies (Hollow, Muyoya, and Brugha, 2016). The study shares a useful interactive educational technology database that assists in visualizing data. xxx The SAMR model developed by Ruben Puentedura in 2006 can help educators think about the role of technology in supporting learning. Substitution can be thought of as substituting an analog method, such as quizzes on paper, with a digital version. Augmentation implies an improvement of a function, such as automatically graded worksheets, freeing up teachers’ time. Modification allows for significant task redesign, such as long-distance peer learning or dual teacher broadcast model. Redefinition means that the use of technology creates an experience that was previously inconceivable. An example is dividing students into groups and having them collaborate on a video tutorial or content creation. ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks evaluated using randomized control trials (RCTs). Regard- tech for education in emergencies (EiEs), tech-supported ing goals, 84 percent focused on improving students’ skills; teacher professional development (TPD), tech for learners 24 percent on increasing access, attendance, completion, with disabilities, massive open online courses (MOOCs), and enrollment; 23 percent on improving teaching; and 7 blended learning, and voice response (VR)for education. percent on increasing administrative efficiency. In terms Another good resource for EdTech research is a 2023 Tech- of implementors, 46 percent were implemented by NGOs nology in Education database, which has over 117 system- and 40 percent by the private sector, while governments atic reviews and meta-analyses on a variety of subtopics made up only 11 percent, and 3 percent were collabora- (https://tech.eved.io/lib/). Notably, 34 were done between tions. Notably, students from low-income households and 2020 and 2022, highlighting the increase in number of those from rural areas were the largest groups served by studies and the impact of distance learning during the EdTech solutions. An overwhelming majority of EdTech COVID-19 pandemic on EdTech research. innovations (84 percent) focused on improving students’ skills, whether cognitive or socioemotional, with the top 3.2 Diving into the Potential three skills areas being literacy, 21st century skills, and Opportunities numeracy. This section outlines six potential outcomes of digi- Another study found that the presence of technology tal transitions in education. It maps the evidence on can benefit student achievement and improve various the extent to which these outcomes have been or can be pedagogical aspects. Bernard et al (2018) identified 20 realized and on what has hindered their realization and meta-analyses of sufficient methodological quality and discusses the recommendations suggested in the liter- found: ature. While a growing number of evidence reviews (for example, rapid evidence reviews, meta-analyses, and sys- • The presence of technology can benefit student tematic reviews) study the impacts of digital interventions achievement, even if the magnitude of its effect is not in education, there are relatively few rigorous evaluations staggeringly high. This benefit extends across the grade using experimental studies that offer robust evidence.107 levels from elementary education to graduate school Several observational and quasi-experimental studies education. have been undertaken on a wide range and variety of edu- cation interventions to better understand the impact of • Various pedagogical aspects of technology use are very the use of technology on education.108 At the same time, effective. These include cognitive support tools and newer methodological approaches to EdTech evaluations applications (that is, tools predominantly in the hands are emerging. A selection is presented in this section. The of students intended to intensify and structure their evidence reviewed in this section was gathered through interaction with the content to be learned100), the role searches of existing evidence libraries (https://education- that the quality and fidelity of the treatment implemen- evidence.io/) and resources shared by experts. tation process plays in affecting learning outcomes,101 the importance of teacher training and professional 3.2.1 Equitable Access to Quality Learning for development,102 the added value of embedded assess- All ments,103 the importance of matching interventions to the needs of the target population (for example, mul- Government-led technology-enabled education pro- timedia for children from less stimulating family envi- grams need to reach and help the most vulnerable, eco- ronments104), and the effectiveness of using technology nomically poor, and socially and politically marginal- in combination with other meaningful learning activi- ized people in the most difficult contexts.109 The Incheon ties.105 Declaration for Education 2030, which was adopted in 2015 at the World Education Forum in Incheon, South • The moderate use of technology has been shown to Korea, commits to addressing all forms of exclusion and outweigh technology applications that feature all of the marginalization in education, emphasizing that “no educa- bells and whistles.106 tion target should be considered met unless met by all.”110 This includes young people who are not in school, learners As the evidence base grows, systematic reviews and with disabilities, girls and women who live in oppressive meta-analyses—particularly in high-income contexts— environments, refugees and internally displaced persons, have shifted from general summaries to examinations of persons in emergency or fragile contexts and experienc- specific digital interventions. These include educational ing natural disasters, disadvantaged ethnic and religious radio, interactive radio instruction (IRI), interactive voice minorities and indigenous peoples, and people who live in response (IVR), educational television, digital personal- resource-constrained rural and remote areas and low-in- ized learning, gamified learning, accelerated learning, come households (See Box 12 on Ukraine example).111 39 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 12. Ukraine’s expansion of digitally enabled education in the midst of war Since Russia’s invasion of Ukraine began, online learning remains a defining feature of the Ukrainian educa- tion system. The FCV context in Ukraine is uniquely shaping the organization of online learning, with digital devices being the only bridge to education for many children. In directly affected regions, especially in the east and south of the country, most schools are providing only online classes for security reasons. For instance, as of February 2024, only 5 percent of the 50,000 students in Kharkiv can study in person, primarily from classrooms set up in metro stations. Meanwhile, educational institutions farther away from the frontlines are only permitted to resume regular operations if they have access to bomb shelters capable of accommodating all students and staff in the event of an air alarm. As a result, a quarter of all school students remain dependent on online learning – with many studying remotely since the beginning of the COVID-19 pandemic in 2020. Partnerships for digitalization and remote learning are essential for implementing the broader education strategy of ensuring access to learning. The key priority of the Ministry of Education and Science of Ukraine (MoES) has been to ensure access to education for all students. In addition to supporting the construction of thou- sands of bomb shelters in schools, the MoES has fostered international and local partnerships to provide access to online education, especially in areas closer to the frontlines. It has coordinated the delivery of more than 130,000 devices to teachers and students from different partners and donors. In addition, authorities are collaborating with partners on the ground to establish digital learning centers (DLCs), which are serving as safe spaces for stu- dents to access distance learning content. The MoES is developing DLCs in some areas, those that have suffered considerable levels of destruction, and communities hosting many internally displaced children. Close to 300 DLCs have been established or are under development. As part of its strategy to build the education system back better after the war, Ukraine is using digitalization to complement and support safe and quality face-to-face education with tools such as the All-Ukrainian Online School (AUSO). The AUSO is the core learning management system in Ukraine, providing access to more than 4,000 lessons, tests, and materials covering 18 subjects for students in grades 5 to 11. Each AUSO lesson is thor- oughly assessed for compliance with the curriculum. While this platform remains primarily an online content repository for self-study, it has evolved since the beginning of the war to offer new functionalities that comple- ment teachers’ instruction for both in-person and online classrooms. For instance, the AUSO provides methodical support to teachers, including courses and other professional development opportunities certified by the MoES. The platform also features a Teacher’s Cabinet, which enables teachers to design online courses as well as carry out diagnostic pre-tests in mathematics and Ukrainian language. The overall aim of the MoES is to build the resilience of the education system to ensure the continued relevance of digitalization in the future. Thus, while mobilizing the rapid deployment of digital devices and digital learning resources, the MoES is prioritizing the use of digitali- zation to enable the long-term recovery of the education system. Source: World Bank. To promote equitable benefits and meaningful access, it hard to find useful tools that are not English-only. technology-enabled education interventions must pro- However, advances in natural language processing (NLP) vide adequate support for marginalized learners.112 models hold promise for substantially improving machine Electricity, physical ICT infrastructure, connectivity, and translation capabilities. This is exemplified by tools such devices are all required for meaningful access. Meaning- as DeepL, which can produce translations that closely ful connectivity means having access to speeds similar to resemble those made by humans, albeit in a limited set of 4G, an appropriate device, an unlimited broadband con- languages. nection, and the opportunity for daily use.113 However, having access to technology is not enough. Sociopolitical Education data is predictive in addition to being adap- factors such as language, culture, literacy, community, tive. Early warning systems can track student attendance, and institutional factors need to be considered to ensure behavior, and performance, and detect or predict students adequate access.114 To promote equitable access for all likely to drop out or fall behind better than other systems, learners, learning, tutoring, and assessment systems must such as those based on regression analysis. Some plat- be designed in their own languages. More than half of the forms can track student behavior (for example, time taken material on the web is written in English, which makes to answer questions), compare them to other users’ per- 40 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks formance, determine the probability of a student passing One of the greatest sources of untapped potential in a quiz, and determine what can be done to increase the technology is providing learners with special needs and odds of success. These predictive systems also have real- disabilities (SND) opportunities for remote learning, time capability to analyze data on student performance flexible learning times, and accessibility tools such as and generate actionable recommendations for person- captioning, screen reading, speech-to-text tools, and alized learning. For example, a study of Knewton online braille displays.122 A systematic literature review cover- courses conducted by the Center for Research and Reform ing 187 studies on EdTech for learners with disabilities in Education (CRRE) at Johns Hopkins University found a in LMICs found that only few studies had addressed the positive correlation between student use of these courses impact of EdTech on learning outcomes or other key func- and online assessment of learning performance, irrespec- tional skills but instead focused on engagement.123 The tive of learning ability.115 technologies used included social media for communica- tion, software programs, and apps intentionally designed Similarly, effective use of technology in education goes for learners with disabilities, sign language apps to teach beyond basic digital competence to data literacy, critical mathematics, and low-vision devices.124 The existing digital literacy, and the ability to overcome socio-tech- studies focus on sensory impairments rather than learn- nological barriers. Some socio-technological obstacles to ing disabilities, are limited to a few dominant languages the use of technology by students that were identified in (particularly English), and are mainly conducted in special research conducted in Bangladesh, Pakistan, Kenya, and schools (in HICs125 and LMICs126/127). Hersh and Mourout- Ghana comprise restricted time to use devices, a lack of sou (2019) raised concerns that a “bring your own device” confidence in technology, inadequate digital literacy model— although providing students with device conti- among parents and learners, inadequate support, fears nuity—may further exclude less well-off SND learners. about privacy, insufficient awareness and promotion of digital education offerings, and preconceived notions Technology provides crucial opportunities to deliver about social media and entertainment.116 education in emergency and refugee contexts that may not otherwise be possible. While rigorous evidence is To ensure an equitable distribution of benefits, it is nec- limited, certain characteristics of effective technology essary to consider various personal and socioeconomic use in education have been identified. These include ena- factors, including household income, geographic loca- bling mobile learning for students on the move; provid- tion, occupation, gender, and age.117 Different people ing multimodal access on SMS, radio, mobile phones, and have different learning experiences depending on their (preloaded) tablets; supporting continuous TPD; provid- level of digital knowledge, language skills, and ability to ing psychosocial support; streamlining education data learn independently.118 To meet the specific needs of each collection; and using community-based practices when learner fairly, it is important to offer ongoing support, to learning is likely to be happening outside of classroom integrate (tech-supported) teaching at the right level, and settings.128 Recommendations from Dahya (2016) for to help students to develop self-directed learning abili- education in emergency (EiE) contexts include focusing ties. Offline capabilities, captions, transcripts, responsive on procuring and maintaining technology locally, clearly design that adapts to different screen sizes, short video defining the purpose of technology, aiming to strengthen lengths, and integration with familiar technologies (for the education system, ensuring privacy and preventing example, WhatsApp) whenever feasible are technological harm, and looking for partners in other sectors (for exam- recommendations that promote equitable benefit (as seen ple, in the health sector). Ensuring higher education con- in South Africa119). Offering data bundles and zero-rating tinuity opportunities can be particularly rewarding for ref- by telecommunication providers and the government are ugees and internally displaced populations. (See Box 13129 also good ways to lower data costs, particularly in LMICs.120 University of the People, a tuition free Online University). A significant proportion of studies on the use of digitally Academic recovery programs can be enabled by dig- enabled education to assist marginalized communities ital technology but results so far have been mixed (see has concentrated on technology-supported girls’ educa- examples in Indonesia130 and Türkiye131). In LMICs, tion. According to a rapid evidence review of 39 papers by such programs support over-age or out-of-school chil- Webb et al (2020, p. 13-14), “girls benefit more than boys dren who have fallen behind because of having to work from the use of technology when provided with access to to support their families, the cost of schooling, being it.” It was also discovered that “girls are likely to respond refugees, or being too old to enroll in basic education.132 with a high level of engagement when given access.” How- These programs aim to improve socioemotional well-be- ever, girls still do not have as much access to EdTech as ing, self-confidence, and self-regulated learning.133 In the boys because of cultural, religious, and gender-based Eastern Caribbean, a promising academic recovery pro- norms about work distribution and first preference.121 gram uses EdTech in integrated ways, such as open educa- 41 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 13. University of the People (on HE), also covered under the Yidan Prize 2023 for Shai Reshef on Education Development: very relevant for refugee contexts University of the People (UoPeople), founded in 2009 and accredited in 2014, is a pioneering non-profit, tui- tion-free online university. It offers undergraduate and graduate programs in Business Administration, Computer Science, Health Science, Information Technology, and Education to students worldwide. UoPeople’s most notable feature is its tuition-free model, where students only pay minimal exam fees rather than traditional tuition, makes higher education globally accessible, especially for vulnerable and disadvantaged. The innovative online delivery model has attracted students from over 200 countries, reaching underserved populations in the developing world. Additionally, UoPeople’s partnerships with institutions such as the University of Edinburgh and New York Uni- versity, along with UN organizations like UNESCO, UNICEF, and UNHCR, have facilitated academic and profes- sional development opportunities and numerous scholarships for at-risk populations. Founder and President Shai Reshef was honored with the 2023 Yidan Prize for Education Development for UoPeople’s role in breaking down barriers to higher education globally. UoPeople prioritizes educational access for refugees, serving over 137,000 students, including 16,500 refugees, and focuses on equipping students with technical skills for the high-tech industry. Through partnerships with UNHCR and the German and U.S. governments, UoPeople offers scholarships to Syrian refugees and students in Burkina Faso, providing higher education opportunities to those otherwise without access. In 2020, UoPeo- ple launched an Arabic program offering Business Administration degrees to support Arabic-speaking refugees, including English courses and employing refugees in administrative and academic roles within the program. Addi- tionally, UoPeople is introducing the world’s first tuition-free, 100% online Master of Science in Information Tech- nology (MSIT) degree program to equip students with essential skills for the tech industry, aiming to bridge the gap between technology and management. Despite its innovative approach to improving access for all, UoPeople also faces challenges. The reliance on peer-to-peer technologies, rather than traditional teaching faculty, is effective only if interactions are meaning- ful leading to some skepticism about education quality compared to that in traditional institutions. The lack of a physical campus and face-to-face interaction may raise concerns about the depth of student learning. Limited program offerings compared to established universities potentially restrict students’ choices for specialized or advanced degrees. Furthermore, debates continue about the recognition of UoPeople degrees by employers and academic institutions, potentially affecting graduates’ job prospects. Lastly, while UoPeople’s low-cost model aims to enhance accessibility, its long-term sustainability is uncertain due to heavy reliance on donations, partnerships, and exams. However, the pandemic experience has helped temper many of these concerns and validate the rele- vance of this approach. Source: Yidan Prize 2023; Oded, Y. 2019; Kalman, Y.M. 2010. tional resources (OER), to support learning and WhatsApp bination of radio, phone, and SMS to provide interactive to provide psychosocial support, but the program has not education during the Ebola and COVID-19 pandemics.138 yet been externally evaluated. Evidence on IVR is less conclusive. An study of an IVR program in Bangladesh reported large learning gains (0.65 Distance learning through educational radio, inter- SD or 0.64 LAYS139,xxxi,),140 but a study in Ghana where active radio instruction (IRI), and interactive voice the Rising on Air program was adapted to IVR, found no response (IVR) can provide access to learning opportuni- significant effects on learning.141 While IVR enables more ties for students in rural and remote areas. Educational interactivity than is usually possible in LMICs, its costs radio supports language learning more effectively than vary depending on telecommunications service providers mathematics.134 Radio is also cost-effective and, when within the country. used interactively in groups, can provide a better pedagog- ical experience for undertrained teachers (Sierra Leone, A rapid evidence review of educational television found Liberia, Zambia, and South Sudan135/136). The Somali IRI that it has the potential to improve learning outcomes, Program (SIRIP) was found to increase enrollment and socioemotional outcomes, and social reasoning in improve students’ test scores.137 More recently, the Rising low-income contexts and hard-to-reach areas.142 Regard- on Air program in Sierra Leone and Liberia used a com- ing social reasoning, educational television can help stu- xxxi Learning-adjusted years of schooling. 42 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks dents to deal with different emotions and situations as A systemic review of 91 causal studies on the impact of well as learn about different religions and cultures, dif- technology use on learning outcomes found positive ferent disabilities, and stigmatized groups (for example, effects across a range of instruction types. The types of people with albinism or HIV).143 If designed to include technology studied included: (i) video-based instruction; sign language and culturally diverse content and lan- (ii) online courses; (iii) computer-assisted instruction; (iv) guages, it can also support hard-of-hearing learners and mobile devices; (v) simulations; (vi) animations; (vii) games; minority groups.144 and (viii) flipped classrooms.157 These positive effects were found when technology leveraged effective learning prin- Text, instant messaging apps, and chatbots can support ciples.158 They were also observed when technology pro- learning, interaction, and collaboration of students and vided unique learning experiences that were not available teachers in low-tech ways. Messaging-based programs through traditional teaching (in line with the findings of can support learning through peer tutoring, collabora- Power et al, 2014), particularly when it saved time and tive learning, delivery of content, teaching, and assess- facilitated recall, revision, or testing through practice ment.145 SMS-based content produced positive learning questions, quizzes, or problem-solving activities. Con- gains and increased parental engagement in the MobiLit- versely, technology hindered learning when there were no eracy Uganda Program and a rapid trial in Botswana.146 opportunities to discuss or consult about what was being The Dr. Math function on the Mxit messaging and social learned, when unrestricted access meant students only media app in South Africa provided users with interac- looked at materials before exams, and when interventions tive text-based learning opportunities and connections prioritized access to technology over its effective use. to peer tutors.147 Despite the positive experience at the time, the drop in the popularity of Mxit has led experts Since the COVID-19 pandemic, many educational to recommend that content and program design should institutions have incorporated blended learning into not be tied to any particular platform.148 WhatsApp, which their teaching methods to build their resilience in is currently more popular, has been used in a variety of case of further crises. A meta-analysis of blended learn- formal learning programs as well as hosting spontaneous ing in higher education found that students could have peer support groups for collaborative learning (in South achieved 13 percent more if they had access to it.159 The Africa, Türkiye, Brazil,149 and Lebanese refugee camps150). meta-analysis also found that the providing technological While WhatsApp allows for sharing text messages, images, support (as opposed to content support) and increasing videos, and voice notes, its drawbacks include message student-teacher-content interactions further increased flooding, a lack of separation from general WhatsApp learning gains. A similar meta-analysis by Means et al use, and eye strain.151 While the use of instant messaging (2013) of 50 independent effect sizes found 11 effect sizes to platforms in LMICs deepens the personal nature of inter- be significantly positive toward online or blended learning actions, collaboration, and socioemotional support (as in models and three effect sizes to be significantly negative in the Zataari refugee camp in Jordan152), institutions and favor of face-to-face learning. The overarching finding was individuals in HICs typically prefer to maintain a clear that online and/or blended learning can produce stronger separation between their personal and professional life, as learning outcomes. However, the authors emphasized that evidenced by strict data protection policies in businesses non-technological pedagogical elements of blended learn- that limit employees’ personal instant messaging use.153 ing models such as additional learning time, instructional resources, and course elements (or confounding variables) 3.2.2 Better Learning Outcomes and More may play a larger role in achieving positive outcomes.160 Holistic Learning Opportunities Blended teaching and learning approaches are particularly relevant for STEM education (See Box 14: Use of ICAP Increases in school enrollment rates over the past few Learning Framework in STEM Education).161 decades have shown that schooling cannot be automat- ically equated with learning.154 The global learning crisis Consistent with GEEAP “smart buys” recommenda- has been exacerbated by the COVID-19 pandemic, result- tion,162 a meta-analysis of the effectiveness of technol- ing in further learning losses. While there is a global need ogy-supported personalized learning in LMICs covering to improve learning outcomes, particularly in foundational 16 randomized control trials (RCTs) found a statistically literacy and numeracy, the broader “citizen-focused” out- significant—if moderate—positive effect size of 0.18 on comes of learning—namely its social, cultural, and polit- learning (p = 0.001).163 The study found that mathematics ical awareness and knowledge—should not be forgotten and literacy interventions have similar effectiveness, that or be limited to a select, privileged few.155 Learning should the presence of teachers played a key role in the personal- encompass lifelong learning, global citizenship, socioemo- ization of learning, and that different lengths and inten- tional development, and the development of curious, crea- sities of learning content appear to have had similar pos- tive, critical, and socially conscious populations.156 itive effects. A broader meta-analysis reviewing 34 studies 43 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 14. Use of ICAP Learning Framework in STEM Education ICAP (Interactive, Constructive, Active, and Passive) learning framework is a well-recognized model to cat- egorize and scaffold student engagement based on cognitive processes involved in learning. Michelene T. H. Chi received the prestigious Yidan Prize for Education Research 2023 for the practical and easy applicability of ICAP in teaching and learning. ICAP framework is built around four levels of learner engagement—passive, active, constructive, and interactive—each linked to specific knowledge-change processes that lead to varying cognitive outcomes. In the passive mode, students merely store information, leading to limited cognitive outcomes. The active mode involves integrating new information with existing knowledge through activities like note-taking, which improves recall and application. In the constructive mode, students create new outputs, enhancing their understanding through inference and leading to a well-organized construction of new knowledge.The interactive mode fosters collaborative learning through dialogue, where students co-infer, resulting in the most dynamic and robust understanding (Chi & Wylie, 2014). The use of the ICAP learning framework is relevant for educators in curriculum development and course design. It is especially useful for designing blended programs in STEM (Science, Technology, Engineering, and Mathematics) education. Given the experimental, interdisciplinary and inquiry-based learning required in STEM subjects, ICAP supports educators in designing activities that advance students from passive receivers to active participants to creators and collaborators, thereby deepening their problem-solving abilities. At the same time, use of ICAP can support the creation of adaptive learning environments that address diverse learner needs, help enhance the flexibility and scalability offered by digital tools, improve access to high-quality resources and pro- mote deeper understanding of complex STEM concepts through practical application and collaborative learning (Eugenijus, 2023). ICAP provides a clear method for observing and assessing student interactions with learning materials and peers, allowing educators to tailor instructional strategies that emphasize practical applications. In blended learning environments, ICAP can help ensure thoughtful integration of online and face-to-face learning by guiding the design of activities that combine interactive and problem-solving tasks in the classroom with online content (Chi & Wylie, 2014; Wu et al., 2023). In practice, new advanced systems using ICAP framework and AI-driven computer vision technology have been developed to closely monitor and categorize learners’ behaviors, and respond with adaptive learning trajecto- ries for STEM blended learning and teaching. Specifically, the Real-time Automated STEM Engagement Detec- tion System (RASEDS) and Precision Education-based Timely Intervention System (PETIS) have been designed to analyze students’ learning processes captured by cameras. These two systems utilize object detection to identify images of learners’ hands and the learning materials in use. The interactions between hands and learming mate- rials are measured as indicators of learners’ immediate behaviors, which are mapped to the ICAP framework to assess engagement levels during STEM activities. • RASEDS, used in STEM workshops and Internet of Things (IoT) projects, detects learners’ hands and materials every 5 seconds, summarizing engagement indicators each minute. The predominant engagement level within that minute is recorded to guide adaptive learning recommendations. This real-time, automated approach has been shown to significantly enhance engagement and self-efficacy in STEM activities by tailoring learning materials to individual needs. • PETIS, applied in computer programming courses, tracks student engagement every 5 seconds over 5-minute inter- vals to identify when students are struggling. It records video and statistical data, enabling instructors to make informed decisions if immediate feedback and assistance are needed. PETIS not only helps address students’ learning difficulties but also provides valuable insights for curricular improvements, leading to significant gains in learners’ programming skills in K-12 education. Source: Chi, M. T., & Wylie, R., 2014; Eugenijus, L., 2023; Lee, H. Y. et al. 2023; Wu, T. T. et al. 2023. found medium effect sizes on learning achievements (ES methods and personalized learning software significantly = 0.673) and learning perceptions (ES = 0.259), implying moderated the effect sizes. Thus, focusing on these mod- that technology-supported personalized learning is more erating factors can lead to greater learning gains. Another effective than conventional learning.164 However, learning meta-analysis of 33 research studies suggested that ALEKS 44 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks (a personalized learning software in Latin America) was ITS and virtual tutoring show promising results, espe- especially effective when supplementing traditional cially in terms of cost-effectiveness. Three meta-analyses instruction in mathematics across various levels of edu- show varied effects of ITS over time (g = 0.01 to g = 0.09;173 cation.165 A longitudinal study examined the influence of 0.66 SD;174 0.6 SD175). Steenbergen-Hu and Cooper (2013) MATHia, Carnegie Learning’s AI-based tool, on the perfor- found that interventions of less than a year-long were mance of 1 million students in algebra, particularly Black, more effective and that the general population benefited Latino, English learner-designated, poverty-stricken, and more than low-performing learners. When ITS was com- female students. The study found that completing more pared with traditional instruction alone (that is, no human workspaces in MATHia leads to better performance in tutors), large effect sizes of 0.86 SD were found. Two vir- algebra, particularly by students with low test scores in tual tutoring programs implemented in Italy and Spain middle school.166 during the COVID-19 health emergency when children were in lockdown have been evaluated and found to have Digital game-based learning and serious games can large positive effects.176 Similarly, SMS-based and phone- encourage playful, curious, and creative learning experi- based tutoring in low-resource settings has been found to ences. These are among the most rigorously researched be effective.177 Virtual tutoring for language learning,178 subtopics in EdTech, with a minimum of 20 nuanced peer tutoring through mobile apps that match tutors to meta-analyses of game-based learning available. A students,179 and group online tutoring with instructors meta-analysis using 154 effect sizes found that scaffolding and peers180 have also had positive outcomes. could improve learning and is more effective in adventure, puzzle, and simulation games than in role playing and Virtual reality (VR), augmented reality (AR), and mixed strategy games.167 Four reviews found mixed effects on reality (MR) have the potential to create immersive serious games (that is, combining serious aspects and ped- teaching and learning experiences. Research on virtual agogical value into fun gamified learning) and reviewed reality in K-12 dates back to the 1990s,181 but since then, perceptual, cognitive, behavioral, affective, and motiva- the affordability of VR devices has increased, particularly tional impacts/outcomes.168 Four meta-analyses on math- due to mobile phone compatibility. In parallel, the field of ematics games found improved learning outcomes, with VR instructional design is maturing. Pellas et al (2021) cat- significant heterogeneity across studies.169 Regarding the egorized emerging instructional design strategies, namely, impact of video games on perceptual, attentional, and cog- activity-based, discovery, experiential, and collaborative, nitive skills, a meta-analysis found positive effect sizes (0.55 as well as instructional design techniques, namely, edu- SD), but these were reduced in the long term (0.34 SD).170 cational games, field trips, observations, role play, and simulation. Increased motivation and engagement from Phone-based, virtual, and intelligent tutoring systems students as well as improved learning outcomes were are another area in which technology provides learners observed in various studies. Drawbacks included simu- with greater access at scale. There are several pedagog- lator sickness, overloading students, distractedness, and ical benefits of high-dosage group tutoring, which tech- complexity of functionality. nology can enhance through careful design and by lever- aging the unique attributes of connectivity, interactivity, With the growing role being played by digital technology multimedia, and data processing. LMICs used SMS and in education, digitally enabled pedagogies that support WhatsApp during COVID-19 to facilitate learning recov- creativity, curiosity, critical thinking, and self-directed ery.171 Virtual coaching can be scaled up cost-effectively learning have emerged. Examples include connectivism182 and support learning engagement, rapid feedback, teach- and rhizomatic learning,183 both of which involve real-time ing at the right level, and more time on task. Intelligent constructed and negotiated learning through networks. tutoring systems (ITS) can support real-time dynamic Open educational practices and critical praxis promote responses to specific learning gaps and increase students’ epistemic diversity and encourage reflection on personal anonymity, along with automated real-time data pro- positions and subjectivity.184 Virtual exchange programs cessing, enabling nimble adjustments as well as mentor- have been found to cultivate global citizenship among ing relationships and human connection opportunities. learners by connecting teachers and learners to diverse Virtual AI-driven tutors, such as Knewton, understand people, cultures, and experiences. Critical digital literacy students’ strengths and weaknesses and help them to skills are essential in preparing students for the digital age. learn new concepts and master old ones independently. They offer customized learning content and adap- 3.2.3 Increased Effectiveness of Teachers and tive assessment tools to increase students’ engagement Teaching methods and improve their learning outcomes. Khanmigo is an AI-powered assistant that functions both as a virtual tutor Effective teacher professional development (TPD) lev- for students and a classroom assistant for teachers.172 erages the knowledge and skills of educators, integrates 45 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All pedagogies and subject-specific information, caters veyed in over 165 countries engaged with a CoP weekly.201 to teachers’ unique and situational requirements, and While informally organized CoPs are driven by teachers’ helps them to meet the varied needs of their students.185 needs and tend to be safe spaces, the lack of experts or TPD should be modeling and practice-based, provide moderators can result in the circulation of misinforma- secure learning environments, encourage discussion and tion or encourage negative teaching practices.202 Virtual critical reflection, and be teacher-led.186 TPD should be CoPs occur in a range of formats, such as instant mes- sustainably implemented through regular sessions, incor- saging platforms (Indonesia203), Facebook (Kenya204), porate time for professional development, and align with wiki forums (Mexico205), Twitter,206 and Instagram (Tür- national policies, objectives, and assessment frameworks. kiye207). CoPs have also been created as a forum for virtual exchanges between teacher groups from different coun- Digital content packs and software applications can sig- tries, allowing for inter-country sharing of practices, for nificantly enhance educators’ subject knowledge and challenging norms and beliefs, and for self-reflection.208 pedagogical expertise. They are especially helpful for Using technology in CoPs can facilitate post-training developing knowledge of mathematics and science and activities (South Africa209), teacher attendance monitor- increasing language proficiency among teachers.187 In con- ing (India210), and extended support (Indonesia211). texts such as rural areas and during crises when it may be difficult to provide TPD activities, digitalized text-based Virtual coaching is a cost-effective solution for TPD but and audiovisual teaching and learning materials, (scripted) is less sustainable than hybrid coaching as its effective- lesson plans, structured guides, teaching and coaching ness can be diluted over time. This was the case in the applications, teacher learning management systems, and virtual (tele-) coaching programs in South Africa212 and preloaded devices have been found to be useful. For exam- Senegal.213 A meta-analysis of 60 coaching studies in HICs ple, English in Action and other nationally scaled mobile by Kraft et al (2018) found large positive effects on instruc- learning TPD programs in Bangladesh have combined tion (0.49 SD) and smaller positive effects on achievement school-based support with audiovisual content already (0.18 SD), with no significant effect sizes between in-per- loaded on memory cards or Apple iPods.188 In a similar son and virtual models. Virtual coaching on Skype was vein, the degree to which subject-specific software such also found to be cost-effective (US$2.40 per person) to as Geogebra189 and MATLAB190 have aided in the develop- support pedagogical leaders in Brazil214. Virtual coaching ment of teachers’ abilities has varied when implemented via text has been used to reinforce new practices and send with support structures. reminders and nudges in Kenya215 and Malawi.216 While useful, it lacks key coaching elements such as observation Self-study and virtual learning communities are being and feedback, which are difficult to do via text.217 More implemented as online and blended learning opportu- recently, coaching software has been developed, such as nities to assist with TPD.191 MOOCs have provided new Tangerine: Coach and Tusome’s National Tablets that avenues and topics for teachers to upgrade their knowl- provide built-in observation protocols, automated feed- edge and skills (for example, in China,192 India,193 and back, and post-observation discussion prompts as well Uganda194), particularly in challenging-to-access envi- as support real-time data collection (as in Kenya218 and ronments.195 However, to use MOOCs to provide TPD, Uganda219). Research in Rwanda has examined the use the teachers must be self-motivated, have the capacity to of semiautomated chatbots on WhatsApp or Telegram to contextualize general content, and have adequate inter- provide teachers with micro-content, activities, support, net access. This is easier for teachers from higher socio- reminders, and motivation.220 economic groups than for those who are less well-off.196 Blended or wrapped courses designed to cater to a par- Technology provides novel models for teachers to gain ticular group of teachers have proven to be more effective feedback. The use of videos for modeling or microteach- because of the provision of contextualization and support ing was observed in 43 percent of the 170 studies reviewed (as in Lebanon197 and Kenya198). In 33 studies, most cover- by Hennessy et al (2022) and found to be highly effective. ing HICs, flipped learning in teacher education has been For example, Funda Wande in South Africa has a range of demonstrated to be more experiential and interactive videos for microteaching in a variety of local languages. than traditional methods.199 Video-based reflection and opportunities to practice newly learned pedagogies improved teaching in the OER4Schools Virtual or hybrid communities of practices (CoPs) foster program in Zambia.221 Not only do teachers see videos of collaborative learning among teachers and are particu- exemplary teachers but they can also record themselves larly useful to provide those in fragile settings with or their peers and self-evaluate (Türkiye222 and Cambo- socioemotional support, communication, and resource dia223). Caution should be exercised with the recording sharing.200 CoPs can be organized formally or organically. approach to ensure that teachers are comfortable and that During COVID-19, 54 percent of 20,679 teachers sur- the approach is aligned with cultural norms and practices. 46 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks Technology has helped teachers to increase their soci- the Indian adaptation of Sesame Street was around 16 oemotional learning, but it can also be a cause of anxiety. cents per student,233 and for Ubongo Kids in Tanzania was The Dream-a-Dream program in India has worked with 1 cent per student (production only234). Program costs vary over 12,000 teachers from low-cost schools to increase depending on whether the model includes group viewing teachers’ authenticity and empathy through monthly with teachers, additional resources, and the quality of the online workshops.224 The teachers gained more self-effi- content.235 cacy, creativity, communication, empathy, and emotional regulation, which improved their students’ all-round out- Reusing and adapting open educational resources (OER) comes.225 However, a review of teacher stress related to the to develop educational materials increases cost-effi- use of EdTech showed that teachers are experiencing high ciency, but aligning them with the curriculum and get- stress and anxiety because of being overloaded, unprepared ting government approval still take time and effort. The for using technology, feelings of incompetence, faulty soft- total cost savings from OER are difficult to quantify given ware, burnout, and striving for perfection.226 they are designed to be reused and adapted, but cost-ef- fectiveness is a key reason for using OER.236 It provides 3.2.4 Cost-efficiency and Cost-effectiveness educational materials to educators in basic education (for of Investments example, Open Up Resources: USA) and higher education, bringing contextualization, personalization, and creativity Evidence on the cost-effectiveness of EdTech is weak. into their teaching at little cost.237 OERs have also been This is mainly because: (i) many evaluations lack cost data, reused in national, curriculum-aligned learning manage- and when they do exist, it is difficult to determine the ment systems such as Uganda’s.238 However, in Kenya, true total cost of implementation and (ii) the frameworks local commercial publishers have resisted the govern- to measure cost-effectiveness are varied and complex, ment’s role as regulators and competitors in content pro- making it difficult to compare different cost-effectiveness duction.239 Adopting OER is not simple as governments calculations when they do exist.227 However, there are have stringent vetting approval processes and curriculum some studies from which lessons can be drawn. and cultural alignment needs.240 OER is being reused for TPD in different country contexts, for example, in Zam- Using text messages or videos to provide information bia’s OER4Schools project, the Leh Wi Lan project in (otherwise known as nudges) about educational benefits Sierra Leone, and Ghana’s Transforming Teacher Educa- can be low-cost228 (Chile229 and Peru230), but if resources tion and Learning (T-TEL) project. EdTech products reuse to act on those changed beliefs are not provided, it may OER. For example, Kolibri curates OER in its content cause further anxiety. Aurino et al (2022, p. 9) studied a management system, and Gooru reuses different OER trial in Ghana that had produced mixed results. The study in its learning pathways. OER has also reduced costs for found that behavioral nudges on average “decreased car- students in higher education (for example, via open text- egiver engagement, decreased self-reported school enroll- books) and in basic education. For example, using Siyavu- ment and attendance, decreased caregiver mental health, la’s open textbooks saved the South African government and decreased children’s academic skills,” indicating that as much as ZAR 1,200 million between 2011 and 2013.241 providing people with information without giving them the necessary means to act on it increases their stress. Technology is used in structured pedagogy and school- Similarly, Orozco-Olvera and Rascon-Ramirez (2022) on based national TPD programs as a cost-effective solution Nigeria found that videos to increase parental aspirations to provide continuous, quality TPD for all teachers. Build- had no impact on learning outcomes alone, but when cou- ing on evidence that infrequent teacher training workshops pled with preloaded smartphones, it resulted in increases and cascade models have not been very effective or cost-ef- in literacy and numeracy scores of 0.57 and 0.79 LAYS fective,242 LMICs are shifting to using continuous learning per US$100 respectively. Beam et al (2022) on Bangladesh models for their national TPD programs through school- showed coupling information with one-on-one phone based communities of learning.243 OER, digitalized mate- calls was more costly at US$4.48 per household, yet led to rials, audiovisual resources, learning management system, increased numeracy scores of 0.35 to 0.37 SD, whereas pro- and instant messaging platforms are key enablers of these viding information alone cost US$2.77 per household but models and help to make the model cost-effective.244 While only increased scores by 0.11 SD. the impact of virtual coaching as been found to decrease over time,245 early findings in Senegal show that tele-coach- Cost data on educational television broadcasting col- ing is more cost-effective than in-person coaching (costing lected over the last three decades illustrate the cost-ef- almost 83 percent less to achieve 0.1 standard deviation of fectiveness of using this method to reach learners in improvement in how teachers guided individual student rural areas.231 The total implementation costs for the reading practice). However, in-person coaching was pre- Brazil Telecurso program was US$26 per student,232 for ferred and has been found to be more effective.246 47 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All 3.2.5 Quality and Relevance of Education at by high-quality teachers to more than 100 million rural Scale students using satellite internet and/or CDs.251 Students watched the videos together and then, led by a local novice In reviewing 234 causal education studies in LMICs, teacher, completed workbooks and activities. A differ- Evans and Yuan (2022) found that effect sizes were halved ence-in-difference study showed that this had long-term when education projects scaled up from under 500 par- positive impacts, including students staying in school ticipants to over 5,000 participants.247 This indicates that longer and improved learning and labor market outcomes. efforts to scale up face many challenges including a lack One factor contributing to its success was that compli- of evidence-based designs, limited end user involvement, ance with government policy tends to be high in China.252 limited innovation funding, a focus on the product rather A similar approach is used in the 50-year-old Mexican than the underlying problem, and a lack of a strategic Telesecundaria model, where positive learning gains have approach from governments.248 Scaling is a challenge in also been observed.253 HICs, as Kraft et al (2018) demonstrated in their analysis of 60 studies where scaling reduced achievement effects Growing out of the distance learning and OER move- by a third. Technology can either aggravate this by adding ment, MOOCs have become the dominant way of pro- more complexity or alleviate it through smoother delivery viding massively scaled content. Over 20,000 open online depending on the nature of the intervention. Drawing on courses from 950 universities are currently being offered experiences of scaling up digital initiatives in Chile, China, to over 220 million students worldwide (excluding data Indonesia, and the US, the Omidyar Network (2019) devel- from China254). Many platforms have been modified to oped a useful model for equitable scaling using demand- freemium models that offer a basic version of the product led EdTech supply through sustainable models, enabling for free while charging for micro degrees. MOOCs were infrastructure, education policy and strategy, and human initially critiqued for being unidirectional, but many now capacity and multi-stakeholder collaboration to bring the incorporate interactive components through discussion vision to fruition. boards, quizzes, and peer assignment marking. MOOCs are notorious for their high attrition rates, with only 10 Evidence is growing on how to scale up education inter- percent of enrolled students completing their courses.255 ventions effectively, but the evidence mostly draws on However, they are now considered to be a drop-in model implementations done by teams outside of government. to which learners only come to acquire specific knowl- Crawfurd, Hares, and Sandefur, (2022) reviewed eight edge and then leave.256 Critics of MOOCs designed and highly effective Learning at Scale programs that predom- offered in the Global North claim that they promote only inantly focused on “structured pedagogy” or “teaching at Western knowledge systems and colonial languages, lack the right level”: Tanzania, Ghana Learning, Lecture Pour cultural diversity, and serve wealthier, degree-bearing Tous in Senegal, NEI Plus in Nigeria, the Pakistan Read- students.257 In recent years, there has been an increase in ing Project in Pakistan, Read India, SERI in India, and MOOC providers from other regions with more inclusive Tusome in Kenya.249 These studies were implemented by demographic profiles as indicated by the large user bases nongovernmental organizations (NGOs) or contractors so of Swayam in India (over 30 million users) and XuetangX they do not capture the full complexity of government-led in China (with over 80 million users). interventions. The features contributing to effective scal- ing include: (i) cultivating political will; (ii) focusing on More recently, digital personalized learning is reaching multilevel program implementation; (iii) setting out clear an increasingly large scale but without the necessary expectations and processes; (iv) ensuring buy-in from assessment tools and processes to precisely diagnose an (government) education stakeholders; (v) co-designing the individual’s learning needs. As a result, the personalized scale up with users; (vi) building on existing programs and learning environment is often more customized than per- technologies, (vii) integrating monitoring and accounta- sonalized258. Teaching at the Right Level is an approach bility; (viii) providing teacher support; (ix) adapting to the to help children acquire basic skills devised by Pratham, different needs of teachers and learners; (x) budgeting for an Indian NGO. It has been rigorously evaluated by over all of this.250 five randomized control trial (RCTs) involving 5 million children.259 Lessons learned from these trials that could Distance learning and the distribution of prerecorded be applied to scaling up a similar program implemented and/or preloaded content are the oldest and most by the government included the need to pilot and stress- cost-effective ways to scale up education delivery and, test the intervention in different contexts, the need to use when coupled with in-person support and interactive evidence to foster its acceptance by the government, and group activities, can yield significant learning gains. The ensuring that government staff are making long-term plans largest EdTech initiative to date is the government-led for the program’s large-scale implementation.260 Another 2004 reform in China that disseminated video recordings RCT study of a personalized technology-aided instruction 48 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks program for middle school students called Mindspark found that, despite half of the world being offline, 70 per- showed promising results for all students but particularly cent of platforms do not offer offline capabilities. These academically weaker students.261 In Uruguay, PAM, an findings illustrate the many challenges involved in scaling adaptive math platform, was rolled out as one of the gov- up digital learning, particularly the lack of incremental ernment’s recommended learning tools (via Plan Ceibal), development due to delivery pressures, the limited consid- hosting 100,000 curriculum-aligned exercises.262 Almost eration given to targeting the needs of specific groups, and 50 percent of learners in grades 3 to 6 produced learning significant financing and capacity constraints in delivering improvements of 0.2 SD, with larger gains for those with high-quality, scaled-up digital solutions in education. lower socioeconomic status.263 Malawi is among the first nations to scale up a government-owned digital personal- 3.2.6 System Strengthening and Resilience ized learning program countrywide to 3.5 million users in 5,770 public schools, which cost US$6 to US$8 per child One of the primary obstacles encountered in educational per year.264 The scaling approach incorporated teacher systems is the absence of transparency and accountabil- training, direct community involvement to reach out-of- ity with respect to the distribution of resources, teach- school learners, implementation toolkits, and localized ers, metrics for measuring progress in learning, quality content.265 However, these opportunities can also present assurance, and accountability measures.272 It can be hard significant risk to children and families both in terms of to get the right information to the right people at the right safety and privacy of their data and age appropriateness time, even when the data is available. The COVID-19 pan- of digital personalized learning in the foundational years, demic demonstrated that having robust and up-to-date that should be evaluated and mitigated against. education data is crucial for developing effective strategies to mitigate learning losses. While multi-country digital learning platforms can be both cost-effective and scalable, governments prefer Real-time data collection, cloud storage, learning ana- to be responsible for the platform’s content. Since the lytics, and dashboards can help adaptive management of COVID-19 pandemic, many multi-country learning hubs education systems. These can provide teachers, school have emerged that provide digital resources with a shared managers, and government officials appropriate infor- backbone of infrastructure, such as the Eastern and mation they require to make better decisions. With the Southern Africa regional learning hub (RLH).266 The RLH increasing trend towards decentralizing governance in plugs into the Learning Passport initiative, which is live many countries, the available technologies can help to in 27 countries and is in development in many more.267 improve how data is collected, analyzed, used, and stored. Learning Passport has been adopted by countries within Various tools for real-time data collection have emerged in the Europe and Central Asia region including Poland, recent years that can track teaching and learning activities Kazakhstan, the Kyrgyz Republic, Montenegro, Serbia, as well as educators’ and learners’ progress to inform future Tajikistan, Uzbekistan, Kosovo, and Ukraine.268 It is a improvements of these tools.273 If appropriately designed component of UNICEF’s regional implementation plan, dashboards can be useful for sharing relevant data with which supports platforms, content, teacher support, inter- different stakeholder groups (instead of overwhelming net access, devices, and youth engagement.269 A case study teachers with questions) and for providing them with the in Kosovo shows how Learning Passport was integrated resources and support required to make improvements274. into the preexisting national blended learning digital plat- There is significant opportunity for using real time robust form on Microsoft (shkollat.org), to provide a one-stop data systems for generating “evidence as a service.”275 as shop for digitally enabled education.270 Challenges in a cost-effective way of data collection, aggregation and uptake remain due to a lack of connectivity and limited analysis that contributes to causal evidence, such as exper- availability of devices as well as the absence of a long-term imental and quasi experimental evidence generation. political and financial commitment from the government, which has changed three times since shkollat.org was first Digital tools and processes are particularly useful for implemented.271 enhancing the resilience and transparency of EMIS in emergency contexts. Some factors that can make these While a range of digital learning platforms emerged EMIS work better include improving education data pol- during the pandemic aiming to reach learners at scale, icies, frameworks, and processes as well as lowering staff a significant number of them have not proved to be sus- turnover, overcoming poor infrastructure and intermit- tainable and/or effective. Findings from UNICEF (2022) tent connectivity, coordinating stakeholders, reducing revealed that “one-third of nationally developed platforms data fragmentation, duplication, and gaps, and increasing have entirely shut down, are outdated, or no longer fully accountability, trust, and buy-in from data collectors.276 functional, limiting learning approaches to help school- These can all be done using digital tools, as has happened children recover their education.” The report further in several EMIS projects in Africa and the Middle East.277 49 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All In South Sudan, for example, teachers used an SMS-based digital initiatives fosters innovation and supports the system to record daily attendance, which was then sent to development of a digitally enabled education ecosys- the South Sudan Schools Attendance Monitoring System tem. Some countries such as India and Uruguay are taking (SSAMS), an online cloud-based platform. SSAMS can national-level approaches to lay the foundations for digi- make progress reports and create dashboards for different talizing education ecosystems. The NDEAR in India was people.278 In Ethiopia, early efforts to use technology to formed through senior-level stakeholder engagement. improve the EMIS did not work because the computers Uruguay followed a similar top-down approach, forming and generators provided were not functional and capacity PLAN Ceibal and the Ceibal Center to expand the use of was adversely affected by staff turnovers.279 The country ICT and improve the quality of and increase access to edu- then developed an EMIS Action Plan to upgrade to faster, cation. However, the rapid expansion of digitalized edu- more automated technologies that were less dependent cation service delivery also poses potential risks that need on electricity and constant connectivity such as tablets to be addressed proactively. These will be reviewed in the and SMS.280 A geographic information system (GIS) was next section. also added to the EMIS to map schools and clusters to inform the allocation of resources and teachers.281 3.3. Understanding Potential Risks and Possible Mitigation Strategies Similarly, big data, learning analytics, and digital forma- tive assessment tools can provide personalized, dynamic The digital environment has become one of the most feedback on students’ progress and predict when they frequented spaces of daily life, blurring long-stand- will need early interventions.282 Although investing in ing distinctions between school, work, and home and learning analytics is becoming common in educational posing new risks for society. A critical factor in the institutions, two systematic reviews have found their evolving digital risk landscape involves increasingly using effectiveness to be inconclusive and have raised concerns digital devices and technologies to enable education, about usefulness beyond the institution given that the knowledge acquisition, communication, recreation, social analytics are specific to discipline and context.283 Noting connection, and civic discourse. Digital service providers that aggregated data is necessary to make learning analyt- (DSPs) and platform companies, whose business models ics more accurate, some countries such as Uruguay and rely on sustained user engagement, play a vital role in this Estonia have begun using learning analytics nationally.284 transformation. The digital risk landscape for education However, simpler school-level formative assessment tools systems, applications, and data is particularly complex, have been shown to improve learning outcomes285 such multifaceted, and rapidly evolving. as Tangerine’s Teach and Coach offline tools, which are being used in the Tusome project in Kenya.286 To address This section will offer perspectives on the major digital the siloed development of learning analytics and person- risks affecting access, privacy, safety, and security of edu- alization algorithms, EIDU has been introduced, which cation services. It will also discuss important policy initi- is a content-agnostic, open-source, anonymized learn- atives intended to mitigate digital risks and spotlight the ing dataset that different products can use to train and relevance of including an integrated risk management develop personalization algorithms.287 approach in the digital transition of any education system. Using building blocks in the development of digitally 3.3.1 Digital Risks enabled education platforms can be cost-effective and support interoperability and scalability, even across sec- One of the foremost risks of digital transitions is tors.288 Digital building blocks are open-source, modular, increasing exclusion and worsening the digital divide. interoperable pieces of code or software that can be (re) The pandemic highlighted and heightened concerns that used to build or tailor platforms.289 Building blocks pro- the rapid expansion of digitalized education service deliv- vide a middle ground between developing bespoke plat- ery has not been experienced equally, revealing a deep forms and buying off-the-shelf platforms. A system-wide digital divide that has created dramatic inequalities in building blocks approach decentralizes the development learning opportunities. The digital divide is not merely a of solutions, allowing different ecosystem actors to plug risk related to digital access alone but is also affected by in and integrate.290 For example, DHIS2, developed for access, autonomy, digital skills, motivation, purpose of the health sector, has also been created for the education use, quality of use, and social and cultural resources. Van sector, connecting the health and education MISs on the Dijk (2005) grouped these factors into three levels: (i) the same backbone architecture. material divide; (ii) the skills and usage divide; and (iii) the divide based on intended tangible outcomes, all driven by Developing a National Digital Education Architecture the users’ attitudes and motivation. This nuanced view of (NDEAR, India) that is linked to whole-of-government the digital divide is important because, even in contexts 50 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks Box 15. The evolution of the digital divide • The first digital divide: The rich have technology, while the poor do not. • The second digital divide: The rich have technology and the skills to use it effectively, while the poor have technology but lack the skills to use it effectively. • The third digital divide: The rich have access to both technology and people to help them to use it, while the poor have access to only to technology. Source: Trucano (2023). where there is widespread access to, for example, the Box 16 on Pursuasive Design or Deceptive Design) arise internet and devices, some population groups may face not only from students’ personal information that they, significant barriers in terms of skills, usage, and motiva- their parents, or friends knowingly share (“data given”) but tion. For example, countries and regions with large aging also from information gleaned from their online activi- populations, a high number of NEETs, significant dis- ties by data tracking technologies (“data traces”) as well as placed populations due to conflict or increasing climate from data derived from algorithms (“inferred data”). Yet, shocks, and/or considerable deficits in foundational skills the capacity of parents and teachers to effectively under- among the population. stand and supervise the digital activities of students is lim- ited, with many lacking the critical digital literacy skills to In the age of AI, from an equity perspective, the rich protect against these risks. may eventually have access to technology, digital skills, and human teachers, while the poor only have access to Merging public education with for-profit learning plat- technology with limited skills to use it (see Box 15). The forms, videoconferencing, and online proctoring ser- digital divide has also heightened concerns about the vices may pose severe risks to students’ privacy (Figure “datafication” of personal student data (i.e. commercial 10). During COVID-19, most EdTech products were made use of personal data), the algorithmic reinforcement of available for no direct financial cost, allowing govern- discrimination against structurally disadvantaged groups, ments to offload the actual costs of online education onto and the role of algorithmic processes in disseminating students, who were unknowingly forced to pay for their misinformation and disinformation, reinforcing the grow- access to learning with their rights to privacy, access to ing importance of media literacy. The pandemic made it information, and potentially their freedom of thought.291 clear that digital transformations are interconnected with Current consent models in school-based platforms often broader systemic shifts such as geopolitical competition, present a false choice, as opting out may exclude students political economy dynamics, and related inequalities and from accessing learning opportunities. Laws should be will continue to exert significant influence on human cap- enforced appropriately and regularly updated to ensure ital formation and a country’s growth trajectory. EdTech companies’ compliance with proper data privacy rights.292 Parents, teachers, and school administrators The risk of violations of data privacy and protection should learn more about how EdTech companies use stu- is increasing. These privacy and surveillance risks (See dent data.293 Figure 10. Education platformization poses important risks for learners Platformised Education Data capture / assymetrical Society Advertisers Data exchange data exchange Students, parents, Authorities Apps teachers = USERS Industry Source: Hillman (2024, forthcoming)294. 51 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 16. Persuasive design or Deceptive design Persuasive design is a form of user experience (UX) design that uses human psychology to design effective, fun, and engaging interactions. It is responsive to how we are as humans: how we think, react, and make decisions, leveraging common cognitive biases such as reciprocity, framing, and scarcity to spur decisions and actions, often in the digital world. For example, people like to have options when making a decision so persuasive design is used to frame those options in a way that guides them toward a “just right” choice or that indicates the scarcity of the product as people tend to be more decisive if they know something will not last much longer or that quantity is limited. However, these persuasive designs can be deceptive. Digitals apps, tools, and websites can employ tricks that make a person take actions that they did not mean to take, such as buying or signing up for something. This is a rapidly growing area of research, particularly in the fields of human-computer interaction (HCI) and law. Many types of deceptive patterns are already illegal in the EU and US, depending on the type and context of their use. In the EU, the Unfair Commercial Practices Directive (UCPD), the General Data Protection Regulation (GDPR), the Digital Services Act (DSA), and the Digital Markets Act (DMA) help to ensure a fairer internet for users. In the US, the Federal Trade Commission Act (FTC), the Restore Online Shoppers’ Confidence Act (ROSCA), and CAN-SPAM are major federal laws that pack a punch. Despite these regulatory steps, Deceptive.design has collected over 400 examples of use of deceptive patterns in these and other regions in a hall of shame. Sources: Authors’ analysis. Students are also facing commercial risks due to fraud- School-age children may also actively participate in or ulent or misleading claims. A recent review of 164 edu- be exposed to conduct and contact risks through hate- cational apps and websites across 49 countries found ful or harmful encounters in the digital environment. that nearly 90 percent of these tools were designed to Cyberbullying can lead to negative consequences for the pass information on to ad technology companies to victim’s personal development and safety. The exchange target students’ interests and influence their buying of sexual messages and images (known as “sexting”) is a preferences.295 Malware downloads and social network- growing practice among teens, with adverse implications ing apps open a backdoor to students’ personal infor- for their privacy, health, and well-being and with potential mation being used for profiling, violating their funda- criminal consequences.299 This risk category also includes mental legal rights and freedoms.296 The maturity and sexual exploitation and cyber grooming via email, chat age of the child may prevent them from comprehend- rooms, and social media platforms.300 ing the motivation behind this type of data collection or the longer-term privacy consequences. A few countries The exposure to the digital environment for school-age in the Organization for Economic Co-operation and children has raised widespread public concerns about the Development (OECD) have adopted laws that specifi- risks to health and well-being faced by youngsters. There cally address consumer risks for school-age children.297 is growing interest in better understanding the effects of digital media use on brain function and structure as well Students also face risks to their online safety, health, and as physical and mental health, education, social interac- well-being. In the digital environment, learners risk being tion, and politics.301 Intensive digital media use has been exposed to digital content that can be: (i) hateful and moti- implicated in reducing working memory capacity, caus- vated by the victim’s religion, race, gender, disability, or ing psychological problems (from depression to anxiety sexual orientation; (ii) harmful such as online scams, por- and sleep disorders), and reducing the level of text com- nography, or violent material; (iii) illegal to distribute under prehension while reading on screens. Higher use levels a country’s laws and sociocultural norms; or (iv) meant are associated with lower happiness levels, especially for to misrepresent news and information. Increased social girls. Heavy digital media multitaskers (MMT) are found media use contributes to poorer reading and shorter atten- to have poorer memory function, increased impulsivity, tion spans, so students are more vulnerable to believing less empathy, and a higher amount of anxiety.302 Early fake news and may lack the knowledge to identify mislead- extensive screen use in preschoolers can have a dramatic ing information. Finland, ranked first in Europe on resil- negative influence on their language development neural ience against misinformation, is making a concerted effort networks, leading to speech delays. Cyberbullying can to teach students about fake news. Starting in preschool, adversely affect the child’s mental health, including raising media literacy is part of the national core curriculum.298 their levels of depression, anxiety, and social exclusion.303 52 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks The escalation of cyber threats and cybercrime puts crit- state without their awareness and consent, prompting ical infrastructure, social cohesion, and mental health concerns about misclassification and stigmatization.313 A at risk.304 Education institutions are prime targets for second concern is that many AIEd tools tend to drive the cyberattacks as they host a wealth of data, including per- homogenization of students.314 The risk is that the abil- sonal information and sensitive research, and lack strong ity of teachers to personalize their teaching in response to cybersecurity measures. Attacks on educational institu- each student may be replaced by so-called individual path- tions are growing faster than in any other sector. A recent ways that are based mostly on averages of prior learners report found that 60 percent of all education institutions while collaboration and other social interaction aspects of globally were targeted by ransomware attacks in 2021.305 teaching and learning may be ignored, thereby replicating The attacks were largely successful, reducing their ability existing biases.315 The OECD recommends that, unless to operate. The report indicated that educational insti- appropriate measures are in place to protect children from tutions suffered from the highest rate of data encryp- harmful effects, digital platforms and applications should tion (73 percent) compared to other sectors, paid the not allow the profiling of children.316 highest ransomware amounts, and took the longest time to recover their data. Digital service providers have also Another important risk is the increasing commercializa- experienced damaging public relations fallout from cyber- tion of education, reflecting the interests of technology attacks (for example, Zoom bombing). Since it is almost providers without addressing deeper social and cultural impossible to protect fully against a cyberattack, having factors as these are difficult to change without broad robust and defensible processes in place should help to stakeholder participation and policy change. EdTech protect against cyberattacks on critical infrastructure and corporations increasingly sell their tools globally, creat- data systems and trigger disaster recovery and business ing asymmetries in power317 by extracting local data from continuity plans in the event of a major incident. Legis- host countries, by drawing entire state education systems lative initiatives related to cyberattacks are beginning to into adopting a narrow set of products across all institu- take shape, with California becoming the first state in the tions, and by imposing particular pedagogical approaches United States to require K-12 schools to report any cyber- through digitalized tutoring systems. The widespread attack affecting more than 500 students or personnel.306 practice of unbundling (the process of disaggregating edu- cational provision) and rebundling (the re-aggregation of AI technologies promise benefits but can create ine- those parts into new components and models) of educa- qualities, exclusion, discrimination, bias, and manipula- tion services may elevate the private good at the expense tion.307 Opening a backdoor to a “banality of surveillance” of the public good, creating an incoherent student experi- enabled by AI can inflict real harm. Students can be harmed ence, and lead to a monopolization of the education sector by data breaches, privacy invasions, heightened test anxi- by a few companies.318 Universities, EdTech companies, ety, unwarranted cheating flags, facial recognition bias, and investors have used market-led approaches to mone- and expensive hardware requirements. Institutions can be tize different aspects of the education experience, includ- harmed by a lack of compatibility with adaptive technol- ing student support, credentials, networks, curriculum, ogies, high costs of AI tools that are not fail-proof, viola- learning pathways, resources, and academic expertise. A tions of privacy laws, and liability exposure. And academic cautionary example is the replacement of free MOOCs freedoms can be damaged by the erosion of student trust with a freemium model, in which the lack of professional and the normalization of surveillance on students and fac- academic support for learners raises equity concerns, and ulty.308 Teachers may also face increased stress and anx- casts doubt on education-for-all programs. New forms of iety when using technology in teaching and learning.309 credentials, such as micro-credentials and digital badges, See, for example, Trucano (2023).310 which are central to the unbundling and rebundling of educational opportunities, have yet to demonstrate their While the purported benefits of AIEd suggest a future of value in the labor market. Another example of risk associ- virtual teachers and personalized learning, significant ated with the commercialization of EdTech is illustrated risks to both learning and learners have not yet been by the case of the multinational private company Byju’s, addressed and many risks also remain unknown.311 See which has reached over 100 million students in 120 coun- Box 17 on Critical considerations for use of AIEd. There is tries. It is alleged to have used coercive, unethical prac- a risk that AIEd systems will be deployed before the crea- tices targeting the poor and disadvantaged to increase tion of sufficient ethical frameworks that use learning and enrollments. Moreover, no rigorous evaluation was done human development as a starting point and explicitly con- to prove its efficacy,319 and the company is now mired sider issues such as fairness, accountability, transparency, in extensive allegations of financial fraud, regulatory bias, autonomy, agency, and inclusion.312 New emotion rec- non-compliance and an indictment from the National ognition capabilities, for instance, can give an idea of who Commission for the Protection of Child Rights in India you are and provide information about students’ mental due to its unethical practices. 53 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Box 17. Critical considerations for using AI in education AI solutions, platforms, and tools are revolutionizing education, offering diverse capabilities from prob- lem-solving to language understanding and image analysis. The integration of AI in education is evolving, and policymakers have yet to understand its impacts, especially in terms of ethics, equity, and data safety. The fast emergence of a wide range of free GenAI tools such as EduChat and MathGPT is fostering creativity by generating content across various domains, although critical evaluation is necessary to address potential biases. Alongside GenAI, AI’s conversational abilities enable adaptive tutoring, feedback systems for writing, and customized learn- ing experiences, particularly benefiting students with disabilities and English language learners. Furthermore, AI streamlines administrative tasks, such as drafting lesson plans and providing personalized recommendations for educators to meet diverse student needs. Adaptive platforms from educational technology leaders such as Carn- egie Learning and Knewton offer real-time customization and assessment, while university chatbots and features such as Microsoft’s Reading Coach and Assignments leverage AI to support learners and teachers by enhancing their reading skills and automating their grading processes respectively. Ethical considerations and collabora- tion between educational institutions and experts will be critical for maximizing AI’s benefits while ensuring its responsible integration into education and addressing concerns about ethics, equity, and data safety in the edu- cational landscape. The rapid technological developments in AI bring multiple risks and challenges. The list of AI-infused specific technologies being used in education grows longer every day, introducing novel risks and challenges for students, teachers, and school leaders. Teachers may struggle with the technical expertise needed to integrate AI into their teaching practices. Schools face the obstacle of the costs of AI tools and applications and must grapple with eth- ical concerns around privacy and security. In higher education institutions, integrating AI raises additional risks, including data privacy issues regarding student information and the necessity for transparent data usage policies. Furthermore, equity and accessibility concerns arise due to potential digital divides and the varied needs of students, particularly those with disabilities. Moreover, there is a risk of over-reliance on large language models, which allow students to complete assignments with little effort, leaving no room to interpret and understand, thus diminishing critical human skills such as problem-solving and interpersonal communication if not appropriately balanced. Edu- cational institutions must address these challenges to ensure responsible AI integration and maximize its benefits while safeguarding against potential risks in the evolving educational landscape. AI presents policymakers with complex challenges, including exacerbating digital inequality, biases, and obscure corporate practices. Concerns raised in the education sector include plagiarism, reduced human con- nection, unknown psychological impacts, and stunted intellectual development. Considering these obstacles, UNESCO (2023) advocates a human-centric approach that entails the implementation of specific regulations on AI ethics and copyright laws, international or regional data protection regulations, and whole-of-government approaches. It will also be critical to build capacity for responsible AI use, implement regulatory frameworks, and monitor algorithms in order to promote innovation while also taking account of ethical considerations, human rights, and equity in education and research. Reflecting on the long-term implications of AI for education and research requires multisector stakeholders to engage in comprehensive discussions to ensure that a forward-think- ing and ethically sound approach is taken to the integration of AI in education. Source: Authors‘ analysis. 3.3.2 Risk Mitigation Initiatives bly.322 The EU has been imposing significant regulatory fines on mishandling children’s data and improper data The digital environment presents both opportunities transfers in breach of the GDPR, reaching US$1.7 bil- and risks for students globally, which underscores the lion for Meta in 2022/2023 alone. OECD recently issued need for robust governance coalitions in each region recommendations to find a balance between protect- and worldwide. Clear guidelines and firm policy and ing children from risk and promoting the opportunities compliance frameworks are needed to ensure equitable and benefits that the digital environment can provide.323 access for all students. Several guidance documents have been released to protect students’ data and fundamental Child rights impact assessments (CRIAs) have been rights in educational settings, including by the Coun- adopted globally by countries in from Europe,xxxii,324 cil of Europe,320 the UN’s International Telecommuni- Australia, and Canada, and, recently, in the Global cation Union (ITU),321 and the Global Privacy Assem- Southxxxiii,325 in partnership with UNICEF. The Coun- xxxii Belgium, Bulgaria, Finland, Scotland, Spain, and Sweden. 54 xxxiii South Africa, India, Malaysia, Rwanda, Tanzania, Bolivia, Colombia, El Salvador, and Costa Rica. ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks cil of Europe’s recent recommendation on the rights of the child in the digital environment326 calls on states to 3.4. Shaping the Future of Evidence “require business enterprises to perform regular child- rights risk assessments for digital technologies, products, The impact of technology use in education has been services, and policies and to demonstrate that they are the subject of intense debate for several decades. These taking reasonable and proportionate measures to manage debates raise a number of pertinent questions. Does it and mitigate such risks.” To date, few global internet and democratize learning for all or exacerbate inequalities? mobile companies’ practices meet minimum standards in Does it compromise human skills and abilities or enhance the areas of privacy, freedom of expression, and remedy them? Does it encourage excessive personalization/indi- mechanisms for reported harms. However, there are some vidualization at the cost of social interactions and conviv- exceptions. For example, LEGO has pledged its compli- iality? Should it be used to teach young children or is it a ance with the Children’s Rights and Business Principles.327 risk to their development? Are expensive short-run invest- ments truly justifiable for delivering longer-run improved In another important step with significant implica- outcomes? Can the public interest and the commercial tions for mental health, brain development, and social interest ever be compatible? These debates have been fur- behavior affecting children, youths, and families, the ther fueled by the pandemic and the exponential global European Parliament overwhelmingly called for action uptake of AI and large language model tools that offer against the addictive design of apps, websites, and unparalleled speed and increasing accuracy. In this con- smartphones. Addictive design features of online services, text, there is a critical paucity of evidence not only on the especially social media, such as endless scrolling, videos learning impacts of technology but also on critical issues on autoplay, and default push notifications capture users’ of equity, ethics, and fairness. attention so they spend as much time as possible on plat- forms. The arguments tabled in this European Parliamen- Definitions and terminologies matter in how evidence is tary report are significant and offer important lessons on generated, interpreted, and used: Evidence on EdTechxxxiv potential risks and regulatory mitigation measures for is different from evidence on education interventions LICs, LMICs, and MICs as connectivity, device access, and using digital solutions. Evidence that an EdTech product usage is becoming increasingly ubiquitous. The argument or service works is different from evidence on education in favor of legislating in this area is that, while there are interventions that use EdTech or other digital solutions. regulations for addictions such as drugs, alcohol, tobacco, The latter needs to establish whether the education pro- or gambling, there have been none for addiction to digital cess using the EdTech/digital product or service improves platforms in general and social media in particular. outcomes. Essentially, a distinction must be made between the evaluation of EdTech products that can inform market While national and international support for digital risk discovery and procurement decisions and the evaluation mitigation initiatives is important and signals an emerg- of an education intervention that uses digital solutions, ing consensus, the interconnected nature of digital risks including EdTech products and services. The latter should cuts across the entire education delivery system. Manag- include impact and process evaluations as well as evalu- ing and measuring the system’s overall performance is the ations of implementation fidelity at scale. While it may primary responsibility of educational leadership in gov- be challenging to evaluate education interventions both ernment (at the federal, state, and local levels). How well is with and without digital solutions as digital solutions the entire system performing on strategy and operations become more embedded and ubiquitous in processes and in areas such as digital transition, user adoption, ecosys- practices, it is imperative to at least distinguish between tem dynamics, third-party networks? What risks are being efficacy (in a controlled environment) and effectiveness (at disclosed? This calls for an integrated risk management scale) of a given product or solution. approach that addresses four principal risk domains: (i) performance risks; (ii) resilience and continuity risks; (iii) With the advent of AI and the current sense of exuber- compliance risks including those linked to violations of ance about technology use in education, the demand is data privacy and cyberattacks; and (iv) risk assurance and starting to build for research evidence on technology use monitoring. Central to this endeavor of risk mitigation is in education interventions. The availability, affordability, the role of evidence. Evaluations of EdTech products and and use of digital innovations in education (in formal, evidence on the effectiveness of digitally enabled inter- informal, and nonformal systems) have all grown rapidly ventions in education are essential to determine where in recent years. According to some estimates, the global to invest the marginal dollar to improve outcomes. In the education market is projected to reach US$8 trillion in next section, we will delve into the broader question of value by 2030, up from US$6 trillion in 2022.328 Worldwide where current research stands and what are the important investment in EdTech companies is on an upswing, with considerations for the way forward. total investment growing from US$9.56 billion in 2017 to xxxiv EdTech refers to the private market and innovation ecosystem that develops commercial products, applications, and services for educational and skills development stake- 55 holders. These can include, for example, B2C, B2G, B2B products and services. Whereas Education interventions using digital solutions refer to education policy, projects, and programmatic interventions that employ digital solutions as part of the intervention design and/or business process redesign. These digital solutions may or may not employ commercial EdTech products and services. What is important to note is that this places the education-related outcomes and related interaction processes at the center rather than the technology product/service at the center. ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All US$25 billion in 2022.329 Governments around the world The digital transition in education includes a number accelerated their digital investments in education because of critical stages—from design to discovery to purchase of the pandemic and more recently because of advances in decision to implementation and diffusion at scale—and AI that promise to have long-reaching consequences on research is lagging at all stages. Moreover, traditional learning and labor markets. However, digital investments measures of efficacy and effectiveness alone are not suffi- are often expensive, require adequate capacity for opera- cient to judge whether a digitally enabled technology solu- tions and maintenance, and often need upgrades, which tion is appropriate for use in education as important ques- can consume a large share of public education expendi- tions related to quality, equity, ethics, security and safety, tures. Consequently, the need for robust research evidence and user experience that must also be considered. There is is being felt much more strongly.330 a need for independent evaluation expertise with relevant knowledge and understanding for each stage. With the fast-paced growth in the variety of EdTech solutions, tools, and platforms, it is challenging to iden- • Design of digitally enabled education products and tify quality. Assessing quality requires the ability to know, services: At this stage, research evidence based on the for example, which digital solutions are based on the sci- science of learning and user-centered design must ence of learning, their safety, the ease with which they can come together with the users. The technology and be connected with existing systems, their compliance with learning contexts interact with the characteristics of regulations and policies, their funding eligibility and costs, the learner (Figure 11). Together, these shape the learn- and their impact on users (students/teachers/others) in er’s experience and the outcomes associated with their their context. Too often, policymakers and institutional use of digital resources. This ecosystem is much more managers must make rapid decisions about procurement complex than the binary conceptions of technology use or the in-house development of such solutions, tools, and prevalent in the past.334 Hence there is a need for edu- platforms in an environment with a vast array of options. cators, researchers, and app developers to be involved Without shared global standards of quality and relevant in the evidence-based design and production of digi- evidence, these decisions may be driven more by market- tally enabled education products. ing and business metrics (such as user base, revenue, and margins) than by real impacts that technology can have on Figure 11. Technology and learning contexts interact learning. Mislabeling digital products as educational or with the characteristics of the learner mislabeling the extent or quality of their features, func- Technology Lerning tions, and capabilities can have serious negative ramifica- Outcomes e tions, including exacerbating learning inequities at scale. ctur tru A ective as ss ce Infr There is a need for a shared set of global standards of The Learner Ac quality assurance of digitally enabled educational prod- Digital Learning ucts and services. Due to the extensive information asym- Resources Behavioral metry between EdTech producers and education sector consumers, coupled with the extensive proliferation of Goodness of Fit products and services, there is no foolproof way to assess / Available Resources quality before purchase. This signifies a major risk that Skill poor-quality products would drive out better-quality Learning products (this is the classic “lemon problem” as outlined Activity Lea in George Akerlof’s 1970 paper on the effect of quality Lear in Cognitive rn uncertainty on the market mechanism).331 In the B2C seg- g G o als nin ment of the market, for example, there were 455,000 apps g Co advertised as “educational” in 2022, which was double the m m u nit y numbers that were available in 2014 (233,000 apps) in a Context largely unregulated and untested market332. Without any Source: Darling-Hammond et al (2014). Reused with permission. current standards for what qualifies as an educational app, there are only a handful of apps are designed with • Market discovery: At this stage, given the proliferation an eye toward how children actually learn. Hirsch-Pasek of solutions, research should not only evaluate the effi- et al (2015) found that only a small number of developers cacy of EdTech products but also their results. Then the in either small start-ups or bigger toy or media companies quality of these products from a technological, peda- have used research-based approaches with preliminary gogical, ethical, and financial standpoint should be cer- results of research.333 tified using robust quality assurance mechanisms and standards. Some good examples of such robust quality 56 ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks Figure 12. Understanding the ESSA tiers of evidence in education TIER TIER TIER TIER TIER Strong Moderate Promising Demonstrates 1 1 evidence 2 evidence 3 evidence 4 a Rationale Study design Well-designed and Well-designed and Well-designed and Well-defined logic model implemented experimental implemented quasi-experi- implemented correlational based on rigorous research study, meets WWC standards mental study meets WWC study, statisticall controls without reservation standard with reservations for selection bias Results of the Statistically significant Statistically significant Statistically significant An e ort to study the Study positive e ect on a positive e ect on a positive e ect on a e ects of the intervention relevant outcome relevant outcome relevant outcome is planned or currently under way Finding from No strong negative finding No strong negative finding No strong negative finding N/A related studies from experimental or from experimental or from experimental or quasi-experimental studies quasi-experimental studies quasi-experimental studies Sample size At least 350 participants, At least 350 participants, N/A N/A and setting conducted in more than conducted in more than one district or school one district or school Match N/A Similar population and N/A Similar population and N/A N/A setting to your setting setting to your setting ESSA = Every Student Succeeds Act, N/A = not applicable, WWC = What Works Clearinghouse. Source: REL Midwest. 2019. ESSA Tiers of Evidence What You Need to Know. Washington, DC: Regional Educational Laboratory at American Institutes for Research. https://ies.ed.gov/ncee/edlabs/regions/midwest/pdf/blogs/RELMW-ESSA-Tiers-Video- Handout-508.pdf. assurance processes are the frameworks used by EdTech fit their specific needs . These include aspects such as Impact, a UK company, and by Instructure’s Evidence- in-house development versus off the shelf purchase, as-a-service. Adopting tiered evidence-informed regula- cost-effectiveness, quality certifications, and procure- tion and enforcement has proven to be useful in the US ment feasibility within regulatory frameworks among with its ESSA Standards of Evidence. To understand the others. The Edtech Tulna Framework from India is a value of the different types of efficacy measurements, good example of such a framework that has been used the US Department of Education has defined four tiers for decision making (Figure 13). Furthermore, contex- of evidence as part of the Every Student Succeeds Act tual evidence is also needed when designing an inter- (ESSA) that encompass strong, moderate, promising, vention, including social (for example, cultural norms), and rationale-based evidence that schools should look environmental (for example, rurality), political (for at before procuring EdTech products (Figure 12).335 example, election cylces), and technological (for exam- Each facet of the ESSA tiers is designed to indicate the ple, mobile penetration) factors. level of confidence that school districts can have in the evidence. Chicago Public Schools include a question in Figure 13. EdTech Tulna Framework their standard request for information from EdTech P: Pedagogical T: Technology vendors asking them to show that their solution has an Alignment & Design evidence base aligned with the ESSA standards. Simi- Benchmarks for Benchmarks for ensuring use of ensuring design of larly, the Los Angeles Unified School District requests pedagogical meaningful user me al Te De its providers to show their evidence, along with data on strategies informed interaction gn gic nt ch sig & by theory and informed by and analysis of their solutions’ impact.336 no n Ali ago national sound design log educational d principles. Pe policies. y • Purchase decisions: At this stage, research evidence is Product required to evaluate a solution or product for public Design procurement. This often includes the potential pro- curer designing requests for procurement (RFPs) based Content Quality on the problem to be solved (“ask why” in Figure 13, Conceptual framework - strategic level) that ask for an in-depth evaluation of the product’s contextual fit and C: Content Quality Benchmarks for ensuring for any market discovery insights of potential products presence of high quality content or services. The procurer might also need a decision support framework to compare the quality and costs factors in the selection of EdTech solutions that best 57 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All • Early-stage implementation fidelity: At this stage, inter- • Process evaluations of education interventions: It is vention design must explicitly account for implemen- important to evaluate the effectiveness of the digitally tation factors (for example, logistical support, human enabled intervention from a technological, pedagogi- capacity, training, and maintenance). Meta-analyses cal, ethical, and financial standpoint to inform future attempting to assess EdTech in isolation consistently designs. The research evidence should help to answer raise concerns of confounding variables, where the “What additional affordances do EdTech components EdTech treatment, coupled with non-technological add to education interventions in their specific con- changes in practice (sometimes spurred by the EdTech text to improve outcomes? And is this value-add more intervention itself), led to positive outcomes.337 Imple- cost-effective than a non-technology enabled interven- mentation research evidence is essential for weeding tion?”356 A systemic approach is particularly useful to out teething issues often using iterative and incremen- measure the complexity of interventions that include tal user-feedback based design improvements in the real EdTech, especially in light of research that demon- world. Research on implementation can identify bottle- strates the sensitivity of outcomes to implementation necks and capacity constraints at various system levels choices.357 Examples of systemic frameworks include (national, subnational, school levels) that prevent policies the EdTech-specific 6Ps of the education system frame- from achieving their full potential. These include meth- work (People, Practices, Places, Provision, Products, and odologies such as action research,338 rapid evaluation Policies)358 or tech-supported structured pedagogy that cycles,339 design-based research (DBR),340 design-based maps how technology can support across the different implementation research (DBIR),341 problem driven iter- components of structured pedagogy (Leadership Devel- ative adaptation (PDIA),342 and the sandbox model.343 opment, Lesson Guides, Teaching Materials, Learning While these methodologies (aptly) recommend imple- Materials, Teacher Training, Ongoing Teacher Support, menters to co-design and co-research their products with Assessment and System Management).359 Sometimes implementation partners or the government, these part- more tailored methodologies are needed to achieve ners can bring their own biases and self-interest to the unique purposes. If trying to unpack system changes process so a concerted effort is needed to remain objec- to measure system effectiveness, outcome harvest- tive.344 Early stage implementation need not be in a small ing,360 network analysis,361 techniques associated with scale pilot. In fact it might be prudent to begin with an adaptive management,362 and political ethnography eventual scale up in mind as scaling up small pilot digital or political economy analysis363 may be more useful. interventions is fraught with challenges, can increase sys- Drawing on implementation science,364 a whole system temic fragmentation, and often prone to failure.xxxv,345,346 approach is useful to analyze complexity within inter- ventions that leverage EdTech, especially as research • Impact evaluation of education interventions at demonstrates the sensitivity of outcomes to implemen- scale: At this stage, robust experimental evidence based tation choices, particularly when scaled.365 on research methodsxxxvi including RCTs and differ- ence-in-difference methods are relevant.347 Recent Given the various reasons for seeking evidence, meth- approaches such as the Multiphase Optimization Strat- odological plurality must become the cornerstone of egy (MOST) and Sequential Multiple Assigned Rand- research on education interventions involving digital omized Trials (SMART)348 are important for determin- solutions. There is increasing recognition among inves- ing the effectiveness of scaled-up adaptive education tors, governments, and product developers of the inherent interventionsxxxvii on outcomes of interest. Other statis- difficulties involved in measuring the efficacy of technol- tical methods could be usedxxxviii,349 including computa- ogy use in education, which often requires resource-in- tionally intensive approaches (such as geoinformation tensive research projects that currently operate in the systems),350 SMART,351 and nimble randomized con- absence of any mutually agreed global metrics. As noted trol trials.352 Newer agile approaches to econometrics earlier, different research methodologies are applicable are using adaptive data collection for policy learning353 to different stages of the digital innovation, adoption, and determining how experimental design can be used and scale-up cycle to generate relevant evidence. The key in staggered rollouts (accounting for lack of statistical is to promote an innovation ecosystem that is driven by power).354 These methodologies can produce just-in- evidence-based quality standards as well as opportunities time findings in a cost-effective and timely manner. for fair selection, design improvements, and program- Finally, when considering scaling, holistic designs, such matic evaluations. Kucirkova et al (2023) propose a general as systematic mixed methods research (SMMR), can EdTech EVER framework to conduct the quality assurance meet the need for formative, fast feedback loops and of digitally enabled education products, potentially draw- larger-scale, longer-term rigor due to the fast-evolv- ing on the ESSA Standards of Evidence in the US or the ing nature of EdTech and its potential for large-scale Standards of Evidence in Australia, both of which embrace impact and reach.355 methodological plurality. xxxv The Principles of Digital Development (n.d.) tackle the issue of ‘piloititis’ and emphasize beginning with scale in mind through making initial cost-effective, contextualized, 58 consultative design choices that will support widespread uptake. Scaling limitations include a lack of evidence-based designs, limited end user involvement, limited innova- tion funding, a focus on the product or the tech as opposed to the problem, and a lack of a strategic approach to EdTech from governments (Simpson et al. 2021). xxxvi In this era of the near ubiquitous use of technology in teaching, the comparison of a ‘technology-present treatment group’ with a ‘no-technology control group’ as a research question is less relevant, and in many cases, not feasible. xxxvii In an adaptive intervention, the practitioner (or some other entity) modifies the dosage (duration, frequency, or amount), type, or delivery mode of an intervention to meet the unique and changing needs of individuals (or organizations). SMART involves multiple stages of randomizations, meaning that some or all individuals (or organizations) participating in a SMART are randomized more than once. xxxviii see Haßler et al. 2021 for more details on diverse methodologies. ◄ Table of Contents Digital Transitions Create Both Opportunities and Risks Figure 14. Evaluating educational programs leveraging EdTech rld education rld education al-wo sys al-wo sys re te re research te he The research he The m m s ion T T Co fac or t ct ta nt tor rs fa en ex s mentation facto Con em tu pl An EdTech al textual factors Im product, The educational service or goal programme ne n (E po po tio co s nt d) ne ple m ca m Te n co Edu Im ch ts No s n -t e tor c h n olo gic al fac Research on an EdTech intervention: "Does the Research on digitally-enabled education EdTech o ering improve learning outcomes"? interventions: "What additional a ordances does ? EdTech add to the education intervention?" • Is the research being conducted with or informed by long-term owners (e.g., the government) and stakeholders (e.g., teachers)? • Who is implementing the intervention? e.g., external experts, local NGOS, government, teachers, or volunteers • When is the intervention taking place? e.g., election years, census data collection, exam time in schools, religious festivals • How is it being implemented? e.g., dosage, blended, after school • What are the prerequisites for the intervention to work? e.g. electricity, connectivity, devices, support sta • What are technological components of the intervention? e.g., radio or augmented reality • What is the environment? e.g., country, rurality, rainy area • What is the educational purpose? e.g., literacy programme, TPD There is an urgent need for policymakers to create a the innovation-adoption-scale-up cycle to inform deci- more conducive regulatory environment for promot- sion-making, and publicly procure well proven products. ing and supporting the development of high-quality learning-centered products and services. Policymakers For evidence to be useful, a nuanced and deep under- are often required to make procurement and investment standing of various contextual factors is needed. This can decisions fairly quickly in a fiscally constrained environ- include the availability of suitable infrastructure and con- ment and with limited evidence. Yet these investments are ducive legal and policy frameworks for the deployment of often large and come with significant opportunity costs, promising EdTech products, adequate funding allocations especially in LICs and LMICs. Hence, governments can for their operation and maintenance, the alignment with play a more proactive role in demanding evidence that can national curriculum standards, the proficiency of educa- enhance decision-making and reduce investment risks. tors in using the technology effectively, and the presence Given the accelerating scale and speed of these invest- of supportive home and community environments condu- ments, this research must be fast, cost-effective, and scal- cive to learning. able. Policymakers can take an approach of well-informed and qualified optmism for allocating marginal investment The next chapter will explore the contextual conditions dollars in digital solutions in education. They can seek needed to enable education systems to make the digital evidence that answers the following questions: What transition. This will include a discussion of frameworks, purpose does this serve? How can we know that this is methods, and tools for assessing the current state of play working? How effective will this investment be in improv- of a country’s education system and technological readi- ing outcomes in my context? Why is this particular dig- ness and for determining appropriate courses of action.366 ital solution better than alternative analog or other dig- ital solutions? They can play an even more active role by requiring clear and concrete evidence at various stages of 59 4 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All CONTEXTUAL Chapter 4: CONDITIONS FOR UNLOCKING THE POWER OF CONTEXTUAL DIGITAL IN EDUCATION CONDITIONS SYSTEMS FOR UNLOCKING THE POWER OF DIGITAL IN EDUCATION SYSTEMS Various contextual conditions at the country and sector of any country and in providing a comprehensive frame- levels need to be in place for individuals, communities, work for informed decision-making. These contextual and societies to realize the potential benefits of incorpo- conditions collectively serve as a foundation for the digital rating digital solutions to improve education outcomes. transition of a country’s education system.369 The state of readiness of these contextual conditions has predictive power, especially during periods of heightened 4.1.1 Strategic Leadership and Governance uncertainty. A recent report by the Broadband Commis- sion on the potential for hybrid learning367 highlighted the This pillar aligns with the Strategic Layer of the conceptual importance of assessing a country’s digital maturity level to framework. help governments and organizations to determine its abil- Government policies, strategies, regulations, and com- ity to effectively manage and guide digital transformation mitment to digitally enabled education are significant efforts. This would also enable policymakers to consider determinants of digital readiness. These help to promote various possible scenarios and incorporate relevant inter- the safe, ethical, and equitable use of digital technologies national good practices and experiences into their plans to in education.370 Governments must come up with a clear ensure that policies in place today remain resilient to future vision and strategy for digitally enabled education systems. shocks, including climate change, conflict, and fragility. This should be supported by durable legislation and equi- table financing. However, it is crucial to recognize that the When a country is planning to undertake a digital trans- benefits of strategic leadership and governance depend on formation in support of an equitable, fit-for-purpose, meeting certain preconditions, such as adequate and sus- and resilient education system, it must prioritize equity tainable financing and procurement mechanisms. If these and rights. Going digital in education can be achieved via preconditions are not met, the advantages will not accrue different pathways depending on the country’s needs and automatically. Adequate and sustainable financing and context. The digital maturity of an economy (and, by exten- procurement mechanisms are necessary to incentivize sion, the maturity of its various sectors) is highly correlated schools and teachers to adopt and scale up technology-en- with per capita income as the more mature and diversified abled solutions to deliver education services. an economy or ecosystem is, the more likely it is to have the necessary infrastructure, institutions, regulations, and Many LICs, LMICs, and UMICs face challenges in trying human capital already in place.368 This means that public to digitally enable their education systems, particularly investments must be centered around the needs, readiness, in terms of collaborative leadership, the alignment of and potential benefits for those with the least power (such policies and action plans, and the mobilization of ade- as children, young people, women, people with disabilities, quate financing.371 Addressing these challenges requires minority groups, and marginalized communities). the existence of champions of digital transformation in the different levels of government, the strengthening of 4.1 Digital Readiness of the Educa- institutional capacity, and the setting of performance tion System: The Five Pillars standards for critical business processes to establish accountability. This requires that: The contextual conditions required for education sys- • Crucial areas for change management support are iden- tems to be ready to benefit from a digital transition tified. revolve around five essential pillars. These are: (i) stra- • Regulatory structures beyond the Ministry of Education tegic leadership and governance capable of delivering coalesce to ensure the safety and security of digital users. human-centered services; (ii) a robust data and technol- ogy infrastructure; (iii) digitally enabled delivery and ana- • Links are established between educational digital solu- lytics of education services and school digital maturity; (iv) tions and national digital infrastructure, such as the the capacity and culture of the education workforce; and Cloud, and digital government services. (v) the EdTech market and business models. Each of these • The education and skills development sector has the pillars plays a crucial role in shaping the digital pathways required political capital to enable change. 60 ◄ Table of Contents Contextual Conditions for Unlocking the Power of Digital Transitions For example, Norway has an integrated approach to digital learn and develop their competence is improved through education policy that is incorporated into its broader edu- digitalization; (ii) digital solutions constitute a high-qual- cation strategy. The Norwegian Directorate for Education ity, interoperable digital operating environment that sup- and Training oversees the digital competence of all stu- ports cooperation between actors; and (iii) digitalization dents, apprentices, and teachers.372 Also, the Finnish Min- supports knowledge-based development. These aspira- istry of Education and Culture recently published Policies tions are embedded in the Finnish vision to be the world’s for the digitalisation of education and training until 2027, leading developer and user of sustainable digitalization in which outlines the measures and responsibilities needed education, teaching, and training by 2027. to realize these three aspirations: (i) everyone’s ability to Box 18. EU’s New Landmark AI Law The EU’s new AI law that was adopted in March 2024 will have implications for human development objectives and outcomes in the region and beyond. While it remains to be seen how the new law will be enforced, imple- mented, and updated in a fast-changing digital and AI landscape, the law itself introduces important binding rules on AI applications and uses, enshrining the principles of “safety, fundamental rights and democracy, while enabling businesses to thrive and expand.” The law is not being applied in isolation and will be complementary to two related regulations, the Digital Services Act and the Digital Markets Act, that establish a complex transparency and data-access regime governing AI technologies in the EU. The EU AI Act is being recognized as the world’s first AI law with the necessary teeth to regulate AI based on its capacity to cause harm. It includes binding rules on transparency, accountability, and respect for fundamental human rights. Two and a half years after the bill was first introduced, EU lawmakers reached a political deal over what has become a global benchmark on regulating AI. The biggest challenge facing lawmakers was how deeply to regulate AI to safeguard the rights and freedoms of individuals without handicapping European start-ups and small and medium enterprises (SMEs). The law is a landmark in the sense that it is intended to be legally binding (going beyond the voluntary commitments made by tech companies in the US) and includes blanket bans, few exceptions, several obligations, guardrails, and monetary sanctions. It includes initial steps for the Act’s imple- mentation, compliance, and enforcement as well as steep fines for non-compliance (ranging from EUR 35 million or 7 percent of global turnover to EUR 7.5 million or 1.5 percent of turnover, depending on the infringement and size of the company). The landmark bill aims to mitigate harm in areas where using AI poses an “unacceptable” risk to fundamental rights, such as health care, education, border surveillance, and public services. It completely prohibits the use of AI for biometric categorization systems that use sensitive characteristics (for example, political, religious, philo- sophical beliefs, sexual orientation, or race), untargeted scraping of facial images from the internet or CCTV foot- age to create facial recognition databases such as Clearview AI, emotion recognition in the workplace and educa- tional institutions, social scoring based on social behavior or personal characteristics, AI systems that manipulate human behavior to circumvent their free will, and AI that is used to exploit people’s vulnerabilities (due to their age, disability, social, or economic situation). Predictive policing may only be used under strict rules such as clear human assessment and objective facts, not only algorithmic decision-making. The AI law applies more stringent compliance rules in terms of risk management and data governance for AI systems, models, and applications with greater computational power as they pose greater risks. “High risk” AI systems and foundational AI models will have to adhere to strict rules that require them to implement risk miti- gation systems, maintain high-quality data sets, comply with EU copyright law, keep better documentation, meet future-proof technical standards (including environmental standards) and ensure human oversight, for example. How best to assess the computational power of evolving AI models and who will do the assessments is unclear, as AI models are often not open source. Tech companies will have to notify people when they interact with a chatbot with a biometric categorization or emotion recognition systems. They are required to label deepfakes and AI-gen- erated content and design systems in such a way that AI-generated media can be detected. The vast majority of AI uses, such as recommender systems and spam filters, will broadly not be restricted. The law aims to help SMEs to develop innovative AI solutions without undue pressure from dominant industry giants by promoting regulatory “sandboxes” and real-world testing under the oversight of national authorities. Sources: European Parliament, Euractiv, MIT Tech Review, Atlantic Council, New York Times. 61 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All 4.1.2 Data and Technology Infrastructure and European Commission established CLARIN, an open and Management distributed digital infrastructure that offers an interoper- able language technology platform for multidisciplinary This pillar aligns with the Partnerships Layer and the Educa- research to advance digital linguistics. The EU’s General tion ICT and Data Layer of the conceptual framework. Data Protection Regulation (GDPR) provides a framework for data governance in digitally enabled education, balanc- Infrastructure is the backbone of any educational ing the need for data to improve learning with the need system, providing the essential foundation for effective to protect personal data.377 To address interoperability, teaching, learning, and administrative processes, par- the US Department of Education has developed the Ed-Fi ticularly in the context of the digital evolution of edu- Data Standard and Technology Suite, which provides a set cation. Effectively managed infrastructure enables access of rules for collecting, managing, and organizing educa- to knowledge and opportunities regardless of location or tion data to enable seamless exchange between systems. socioeconomic status. A reliable infrastructure, includ- More recently, there are significant efforts globally to reg- ing physical facilities, electricity, connectivity, and tech- ulate Artificial Intelligence applications including in edu- nological resources, is essential for fostering innovation, cation. See Box 18 for EU’s landmark AI Act. efficiency, and equitable access to education. A well-de- veloped physical and digital infrastructure empowers 4.1.3 Digitally Enabled Education Service the learning environment allowing educators to employ Delivery and School Digital Maturity diverse teaching methods, utilize digital tools, and make informed decisions through data analysis. This pillar aligns with the Education ICT and Data Layer and the Implementation Layer of the conceptual framework. The availability and adequacy of electricity, telecom- munications infrastructure including equipment and Service delivery is at the heart of the education process digital devices, and meaningful internet connectivity and is the key to unlocking the benefits of going digi- are essential part of the ICT backbone. Once these foun- tal. Depending on the extent to which digital solutions dational infrastructural barriers are overcome, it is neces- are embedded in education service delivery, they can sary to ensure access to high-speed internet in schools, to enable personalization of learning, dynamic responses to devices for use by teachers and students, and to appropri- the students’ and teachers’ learning and teaching needs, ate software and learning platforms.373 For example, the and better management and analytics to augment deci- national government in Korea has consistently invested in sion-making. The focus here is on teacher-learner inter- nationwide high-speed broadband infrastructure, leading actions, the instructional leadership of administrators to wider access to digital education resources. Similarly, and school managers, and the strengthening of the cur- after the Danish government invested in broadband infra- riculum, teaching and learning materials, assessments, structure, over 97 percent of households now have inter- analytics, and institutional management, including at net access.374 These significant investments have success- the school, local, and regional levels. Teachers need to fully allowed Danish schools to integrate digital learning be trained in the necessary competences (skills, knowl- into their curricula. In contrast, the internet penetration edge, and attitudes) to integrate technology into their rate in Chad is less than 5 percent,375 making the imple- teaching methods as well as into their curricular content mentation of digitally enabled learning solutions much to enhance learning outcomes. With reference to the more difficult. SAMR (Substitution, Augmentation, Modification and Redefinition) model xxvii, which describes how technol- Developing the architecture of a digitally enabled edu- ogy can be applied to teaching and learning, this goes cation system is essential, consisting of a coherent, beyond substituting traditional teaching methods with integrated blueprint that aligns people, data, and tech- digital ones into more advanced uses such as the aug- nology. This includes the variety of devices, operating sys- mentation, modification, and redefinition of learning. It tems, tech stacks, protocols, metadata, standards, work- will also be essential to implement robust and system- flows, applications, and networking equipment that can atic assessment of the digital competences of students. empower the service delivery and analytics space needed for learning. Effective data governance is crucial for main- Providing high-quality, localized digital content in the taining data privacy, security, and ethics in a digitally ena- languages spoken and understood by learners is crucial bled education system. In addition, interoperability—the for effective teaching and learning. Many LICs, MICs, ability of different information systems, devices, and appli- and also HICs lack digital content and devices that are cul- cations to access, exchange, integrate, and cooperatively turally and linguistically diverse. This makes it impossible use data in a coordinated manner—is vital for efficient for non-English-speaking learners to use digital content as and seamless digital learning experiences.376 In 2012, the effectively as their English-speaking peers. Moreover, AI 62 ◄ Table of Contents Contextual Conditions for Unlocking the Power of Digital Transitions and adaptive learning systems rely on codified, contextual- mation to help them make better informed procurement ized knowledge in the form of large training data sets, but decisions about digital products and services in education these remain underdeveloped in many languages across all with relevant quality standards that will serve the learner regions. The EU continues to invest in the agenda of lan- and learning (see Box 19). guage diversity in the digital age as recommended in the EU report Language equality in the digital age - Towards a 4.1.4 Education Workforce Capacity and Human Language Project commissioned by the European Culture Parliament in 2017. Lenovo recently launched a New Zea- land bilingual keyboard designed to seamlessly write in This pillar refers to the human resource capacity for designing the indigenous language, te reo Māori. This initiative was and implementing policies and practices to support the devel- driven by a student who had begun to learn te reo Māori opment of a digitally enabled education system. This applies in school and questioned why there was no hardware across the whole conceptual framework (the depth of digital available to support his learning.378 Canada has focused transitions). on creating digital content in both English and French, reflecting its bilingual policy and the diversity of cultural Enhancing the skills, capacity and collaboration across backgrounds among its population.379 a wide range of stakeholders is required to support the development of a digitally enabled education system. Multidisciplinary expertise is required to objectively There is a need for widespread knowledge of digital area assess the quality, cost-effectiveness, and risks of using and upskilling of the workforce with digital competences, digital products and services in educational settings. Too including for policymakers, bureaucrats, administra- often, educators, administrators, and institutional manag- tors, and teaching staff. Multidisciplinary teams with ers have to make complex decisions on the procurement specialization in ICT and educational fields are required and/or in-house development of digital tools, solutions to make well-informed decisions. There is also a need and platforms. Yet choosing between the growing plethora for technical support staff to facilitate the integration of digital solutions, tools, and platforms that are labelled as of digital technologies in education in schools. Policy- “educational” can be overwhelming. An understanding of makers need to foster a culture that focuses on the user different technological tools, educational needs, pedagog- needs, develops a positive attitude towards digitalization, ical knowhow, and contextual awareness are often needed encourages innovation, incentivizes skills development to weigh the merits and risks appropriately. Responding and collaboration, aims to increase trust in governance to this need, several entities—such as Education Alli- and data, and takes an evidence-based approach to man- ance Finland, LearnPlatform USA (Instructure), EdTech aging change and disruption. In Finland, for example, a Evidence Exchange, India’s EdTech Tulna, and Goldstar culture of trust in potential use of technology, a strong Education, among others— have been established to pro- belief in equity in education, and active parental involve- vide schools and governments with evidence-based infor- ment are integration of digital learning into education.380 Box 19. EdTech Tulna - Navigating the Complex EdTech landscape (India) EdTech Tulna provides an evaluation framework and standards to enable decision-makers and individual users in India to make an unbiased quality assessment of EdTech solutions. Tulna means comparative measurement in the Hindi language. EdTech Tulna offers three main services: (i) standards and quality frameworks; (ii) toolkits for making effective procurement decisions; and (iii) independent expert reviews of existing products. The company takes a research-based approach to creating frameworks that are sensitive to the local context and globally relevant. Based on global research literature, government policy documents, and stakeholder interviews, it emphasizes the mechanism of how and why a product is effective in the Indian context. EdTech Tulna’s standards have three dimen- sions: (i) Content Quality: the quality of educational content with regard to accuracy, coverage of national standards, appropriateness for targeted grade and learners, and language comprehensibility; (ii) Pedagogical Alignment: align- ment of EdTech solutions with national educational policy on pedagogy and assessments; and (iii) Technology and Design: how technological affordances integrate with pedagogy and content. The State governments of Haryana and Madhya Pradesh in India have used EdTech Tulna as the technical evaluation framework for the official process of comparing and selecting a proposal to procure Personalized Adaptive Learning (PAL) solutions for schools. EdTech Tulna has offered frameworks and evaluations catering to three key use cases for mathematics (in grades pre-K to 10): Personalized Adaptive Learning (PAL), Digital Class- rooms (DCRs), and Interactive Audio Visual (Interactive AVs). Source: Authors’ compilation. 63 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All The digital skills of the education workforce and public support educational processes, improve learning access, education administrators are crucial for effective digital and foster innovative teaching methods. Leadership and learning.381 Teachers need to be digitally literate to inte- governance play key roles in shaping a conducive envi- grate technology into their lessons, while public education ronment for EdTech innovation. Private sector participa- administrators at both the school and system levels need tion, supportive policies, and financing for research and digital skills to understand and make informed decisions implementation are essential components. Additionally, about digital learning technologies.382 In India, the gov- digital infrastructure and the digital competencies of staff ernment has implemented the Diksha platform, an ini- and students are vital for successful EdTech adoption and tiative to train teachers in digital pedagogies, and NGOs effective learning with technology. such as Pratham have recognized the gap in digital skills and are running digital literacy programs for students and Effective innovation ecosystems foster close collabora- teachers. Several OECD countries have professional devel- tion amongst universities, research institutions, start- opment programs to enhance teachers’ digital competen- ups, and private enterprises to enable the digital transi- cies such as Austria, Korea, France, Spain.383 However, tions in education. They contribute to the development there is still a pressing need for teacher training in digital of new digital learning tools, models, and strategies and skills in many LMICs and LICs. Socioeconomic inequali- foster a culture of innovation and entrepreneurship that ties, parental involvement, and cultural attitudes toward helps to drive the digital transformation of education.387 technology also influence the digital readiness of the For example, EdTech hubs in Kenya, such as iHub and education workforce.384 Educators’ working conditions, Nailab, foster digital education innovations by supporting career structures and professional support can facilitate start-ups that come up with solutions to local educational or constrain their ability to make effective use of digital challenges.388 Public-private partnerships (PPPs), where education technologies. Although the use of digital edu- feasible, can be useful for bridging resource gaps.389 In cation technologies may make some aspects of teachers’ Colombia, the Ministry of Education partnered with sev- work more efficient and given them access to an abun- eral tech companies to donate computers to implement dance of learning resources, navigating these resources, its Computers to Educate program (Computadores para preparing digitally enhanced lessons and learning how Educar), which increased the availability of devices in to do so effectively takes time385. Teachers’ working time schools and improved the acceptance rate of high school arrangements, incentives and career reward structures students in higher education.390 and opportunities for professional development must support digital pedagogy. In this, school leadership teams, High quality innovation and equitable distribution of in addition to education governance, play a central role in Edtech requires access to capital markets, supportive preparing teachers for the successful integration of dig- legislation, and viable business models that can lever- ital education technologies. They are critical to ensure age digital global value chains391. Innovative business that teachers remain motivated to engage in professional models could be sustained in a variety of ways, includ- growth, to appraise teachers and link their individual ing through direct consumer purchasing power, govern- learning with the school’s overall priorities and strategies ment procurement or grant programs, private invest- for the adoption of digital technologies. Decisions on the ment, or some combination thereof.392 By enabling management of staff time shape teachers’ opportunities financing options, fostering competition and setting to engage in collaborative practices and other forms of standards for quality and equity, policymakers can play peer learning that are critical to build school level capacity a proactive role in germinating an edtech innovation for the successful adoption of education technology386. ecosystem that supports education policy priorities and goals. At the same time, it is necessary to enforce regu- 4.1.5 EdTech Market and Business Models lations that strengthen the selection and procurement processes for high quality EdTech products and ser- This pillar refers to the maturity of the private market and vices that objectively match identified education needs. innovation ecosystem, which cuts across the whole conceptual framework (the scale of the digital transition). 4.2 Digital Readiness Assessment Frameworks, Methods, and Tools The EdTech market solutions comprise a network of technologies, platforms, and services aimed at enhanc- Conducting a thorough diagnosis of the existing context ing teaching and learning experiences. It involves stake- is essential for policymakers to understand the needs holders such as educators, students, EdTech innovation and challenges of a country’s education system and to ecosystem, policymakers, institutions, and investors. Col- develop a strategic roadmap for the digital transition in laboration among these parties is crucial for the ecosys- education (Figure 15). The diagnosis should be based on tem’s growth and sustainability. The ecosystem aims to three interconnected elements: (i) the needs of the coun- 64 ◄ Table of Contents Contextual Conditions for Unlocking the Power of Digital Transitions Figure 15. Assessment frameworks and tools for diagnosing education needs and related digital readiness • Digital Economy frameworks and related readiness assessment tools, guidelines, and methodologies Needs of • Several available by World Bank, USAID, ITU, UNDP, OECD, EU, and others a country • Tend to diagnose the state of digital connectivity, payments, ID systems, telecom regula- tion, whole-of-government infrastructure platforms, and data governance • Education sector analysis and assessments, guides, and methodologies • Aim to identify the systemic challenges within the education system Needs of education • A comprehensive methodological guideline co-developed by UNESCO, World Bank, system UNICEF, and GPE secretariat • Digital Readiness Assessment of Education systems and related guides, landscape reviews, and methodologies relating to ICT and digitalization in education Digital readiness • Available such as those by the World Bank, UNESCO, UNICEF, and others that o er a of education variety of assessment tools and methodologies and skills development • These diagnose the digital readiness within education systems/sub-systems Source: Author’s analysis. try; (ii) the needs of its education and skills development education services, and market structures in the coun- system; and (iii) the digital readiness of its education and try. It should also consider the extent to which a coun- skills development system. The diagnosis should first try’s overall approach in the digital area will support the explore the socioeconomic goals and education priorities achievement of high-quality education and skills devel- of the country along with the economy-wide state of the opment for all. A comprehensive view is needed to iden- digital foundations (including connectivity, payments, tify many critical systemic barriers to advance the digital ID systems, telecom regulation, whole-of-government transition, as these are not necessarily the most visible infrastructure platforms, and data governance). As for the (for example, capacity and coordination) and may not be needs and challenges of the education system, the diag- within the direct remit of the Ministries of Education. nosis should identify them from sectoral analyses, strat- egies, and action plans, define the problems that need to The Education Digital Readiness Assessments (EDRA) be solved, and consider the problems that digital technol- was conducted in Moldova, Georgia, and Kosovo between ogies might be able to solve. To establish the extent of the 2021 and 2024. These assessments helped evaluate the digital readiness of the education and skills development education system’s readiness to provide an objective diag- system, the diagnosis should determine the opportuni- nosis of the system across the five pillars outlined in Sec- ties, barriers, and risks for adapting, integrating, and scal- tion 4.1 Digital Readiness of the System. This helped iden- ing-up technology-enabled solutions in the sector. There tify and prioritize recommendations by mapping them to are readily available frameworks, assessment methods, strategic education objectives that could most benefit from and related tools to help determine the most appropriate digital advancements; and inform a near- to medium-term and contextually relevant strategies for incorporating dig- policy action and implementation roadmap.393 By triangu- ital solutions in education, which can be used to enable lating various data sources and stakeholder insights, the the digital transformation of any education system. diagnostic assessment can highlight the country’s digital readiness and offer detailed recommendations to enhance A comprehensive assessment of the digital readiness the system’s foundational, functional, and frontier digital of education and skills development systems is vital to capabilities, aiming to stimulate action and dialogue and inform government strategy and action plans and to help develop a transition roadmap. Systematic stakeholder ensure that the relevant actors align their approaches mapping and engagement of various system-level actors with the national education goals and priorities. The and elements at the national, subnational, and school/ diagnosis should provide essential insights into the matu- community levels is often key to enabling and implement- rity of relevant infrastructure, governance, capacity, core ing digital transition initiatives (Figure 16). 65 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All These diagnostic assessments need to pay special atten- tion to system-focused, equity- and inclusion-focused, A comprehensive assessment of the digital readi- and learning-focused indicators (Figure 15). While some tools include indicators that measure equity outcomes, ness of education and skills development systems they may not include other system-focused or learn- is vital to inform government strategy and action ing-focused indicators. It is important to evaluate assess- plans and to ensure that the relevant actors align ment tools based on whether they are people-centered, their approaches with the national education adequately draw interlinkages across various pillars, and goals and priorities. sufficiently consider values such as equity, solidarity, inclusion, and human rights. Furthermore, it is impor- tant to critically consider not only what is assessed but also whose views and experiences are taken into account In the next chapter, we discuss some of the key trade-offs to inform policymaking. Assessing the digital readiness in that policymakers might consider when developing a dig- any given country requires both a top-down and a bot- ital transition roadmap for education and skills develop- tom-up approach. Top-down perspectives include those ment. of national, regional, and local institutions, committees, and private sector CxOs, while bottom-up perspectives include those of children, young people, parents, teachers, and school administrators. This can result in the diagnosis reflecting a balance of practice and policy and a focus on equity and inclusion. Table 1. Digital Readiness Assessment of the Education System Areas covered under each assessment pillar Pillar 1 - LEADERSHIP AND GOVERNANCE Vision and strategy Institutional capacity Legislation, policy, and compliance Pillar 2 - ENABLING INFRASTRUCTURE Education enterprise architecture and data governance Connectivity Technology infrastructure Standards and services Pillar 3 - DIGITALLY-ENABLED EDUCATION DELIVERY AND SCHOOL DIGITAL MATURITY Content standards and instructional materials Digitally empowered learning environment Digital skills of students Teacher’s digital skills and pedagogical practices School management and administration Pillar 4 - HUMAN CAPACITY Policymaking School ownership Wider public ICT professionalism Pillar 5 - EDTECH MARKET and BUSINESS MODELS Capacity and culture for entrepreneurship Development, uptake, and management of Edtech products and services Quality assurance of EdTech Support for innovative business models and public- private partnerships 66 ◄ Table of Contents Legend Map Figure 16. Illustrative Map of Education Ecosystem Stakeholders • ? Contextual Conditions for Unlocking the Power of Digital Transitions Decision makers and regulators • Engaged stakeholders • Key influencers • Broader stakeholders Teacher Preparation and Education Service End-users - Clients Accreditation Providers Students Teacher Colleges Teachers Parents and Advocacy Pedagogical and groups Instructional Design School Leaders Experts Teachers Middle level Managers andAdvocacy groups Certifications and Accreditation Bodies Labor market Civil Society and NGOs Competition Regulators And Telecom Regulatory Local Universities and Authority Research institutions Local NGOs, Think Ministry of Data Privacy Tanks Health Commission Education System Administrators International NGOs MOER Digital Management Ministry of and Operations Ministry of Education and Education IT Leadership of Digital Department Economy Education Research Committee Platform Non-Governmental Ministry of Social A Ministry Funding Sources Protection, Labor & of Finance Family Start-ups Ministry of ICT/Telecom Donor and Partners Investors Enabling ICT Infrastructure Technology Companies & Business Community Local Aggregators PaaS and SaaS companies Telecom Operators Commercial O -the-Shelf Software Utilities and Power Solutions companies Internet Service Providers Custom Software Developers Informatics and Statistical Agencies Hardware companies Advocacy and Lobbying Groups Open-Source Solution Communities Media - International, national, local Source: Authors’ analysis. Note: This is an illustrative map that would need to be contextualized based on local and global trends, including emerging and new players. 67 5 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Chapter 5: DIGITAL PATHWAYS TAILORING TAILORING DIGITAL PATHWAYS Each country’s unique contextual conditions and vision While a country or region will have its own pathway to for its economy and society will drive which digital digitally enable education and skills development sys- pathway that it adopts for education and skills develop- tems, there may be overlaps with pathways that have ment. There are at least three different pathways: (i) the been followed by other countries. Therefore, having a set overall digital ecosystem sets the direction and pace, with of shared guidelines can be useful so as to not reinvent the the education sector taking a passive role (for example, if wheel, to be aware of common pitfalls and challenges, and the advent of connectivity and access to devices leads to to learn from the experiences of others. At the same time, teachers and schools incorporating digital resources in policymakers should be cautious about adopting the dig- their teaching and learning); (ii) disruptive events or initia- ital pathways taken by countries that do not share similar tives (from start-ups, private sector, or major international maturity levels and contexts as their own country. More- education initiatives) put external pressures on the sector over, the pervasive and interconnected consequences of to improve education services (for example, the pandemic digitalization on various aspects of education and skills catalyzed digitalization initiatives globally); or (iii) the development mean that careful thought must be given to education sector takes the lead (led by strategy at the top balancing benefits against risks by actively recognizing the and/or by local initiatives from the local level) supported inherent trade-offs in policy decisions. by an education-friendly digital ecosystem. While the first two pathways have been clear drivers in the digital trans- This chapter delineates some of the critical trade-offs formation of education so far, the third pathway—taking and common pitfalls in shaping digital transitions a strategic approach led by education and skills develop- within education and skills development systems and ment objectives—provides the most sustainable way for- underscores the need to take a strategic approach and ward as it ensures that relevant learning experiences and to identifying effective ways to balance conflicting fac- related goals stay at the core of any transformation efforts. tors. When considering adopting a risk-informed digital This can be seen in the experiences of countries such as strategy across the education system, policymakers need Singapore, Estonia, and South Korea. to carefully consider their key trade-offs, the desired out- come, and the country’s collective capability to manage It is important to be sensitive to the local context and the change process. While proactive public policy meas- to tailor digital pathways accordingly when developing ures and a coherent systems approach can yield increased education and skills development policies and strate- inclusivity and benefits, they may also incur higher costs gies. Tailoring consists of deliberately considering and and require strong leadership. On the other hand, adopt- fine-tuning strategies and policies to address the diverse ing short-run trajectories may offer immediate private and dynamic needs within the specific country context benefits but is likely to lead to fragmentation, greater when embedding digital technologies in education and social exclusion, and piecemeal results. For instance, an skills development. This involves meticulously aligning education policymaker with strategic or budgetary consid- educational policies with the broader and multifaceted erations that might choose to prioritize pressing concerns policy ecosystem. It implies the need to craft policies such as student safety, health and hygiene, and ensuring that cater to diverse actors both within and outside the adequate number of trained teachers, while acquiring low- education sector and acknowledge the potential impact tech digital tools and solutions, thus taking an incremen- of these policies on a broad spectrum of education out- tal pathway. Meanwhile, another country might choose comes. The concept of tailoring underscores the impor- a fully ICT-integrated curriculum while allowing teach- tance of responsiveness in policy design. The aim should ers to gradually assimilate digital pedagogic practices. It be to optimize the benefits of digital transformation while is pivotal to balance realism with aspiration, ensure the addressing the varied challenges and stakeholder interests durability of the adopted strategy, and acknowledge that involved in the educational landscape. the trade-offs made between different areas will affect the allocation of resources. 68 ◄ Table of Contents Tailoring Digital Pathways 5.1 Trade-offs and Issues to Consider ating an incoherent student experience and raising impor- in the Design and Implementa- tant questions about the role of the state. tion of Digital Pathways It is necessary to consider the effect of existing gov- Policymakers often have to grapple with a variety of ernance structures and mechanisms on the design and trade-offs when taking a decision on digital investments implementation of education policies and programs for education and skills development (see Box 20). It is imperative for policymakers and relevant ministries to Box 20. Possible issues and trade-offs to proactively lead the charge for the digital transforma- consider for policy decision-making tion of the education sector by playing a strategic and hands-on role in shaping policies that drive the integra- • Decisions based on total cost of ownership (TCO) tion of digital technologies into education. By taking this vs. short-term investment and budget cycles proactive stance, policymakers can guide the transforma- • Centralized or local-level equipment procurement tion process and ensure that policies are tailored to meet and ownership the specific needs and challenges of their country’s educa- tion system. They can also leverage the potential of digital • Bear coordination costs for whole-of-government innovations to enhance educational outcomes while effec- approach or business-as-usual siloed approach tively mitigating any potential disruptions, thus steering • Data sharing within and/or beyond borders the course of the digital transformation in a controlled and purposeful manner rather than being passively influenced • Data-driven efficiency vs. surveillance state by external forces. In doing so, policymakers will need to • International digital products vs. local innovation tackle several trade-offs, with their decision depending on their motivation and vision, appetite for risk, the availa- • Policies that drive private benefits and bility of fiscal space, and feasibility considerations such as fragmentation vs. Policies that drive public benefits whether workers with the relevant human capabilities are and integration at scale available. • Challenges with forecasting the competences Policymakers may need to examine the implications of (skills, knowledge, and attitudes) for the future unlocking private sector innovation and new business • Generalizability vs. contextualization and models and build in appropriate mitigation measures: adaptation of digital solutions Private market engagement and competition to increase the availability of context-relevant digital solutions for • Speed/cost-effectiveness vs. contextualization of education and skills development must be thoughtfully digital solutions managed. For example, the prevalent practice of unbun- • Unbundling and rebundling of all aspects of dling and rebundling digital componentsxxxix of education education provision products and services may elevate the private good at the expense of the public good and create incoherent student • Equity at scale using low tech vs. staggered experiences. This is particularly noticeable in higher edu- investments in high tech cation,394 which is digitalizing at a rapid pace. Market-led approaches supported by newer business models are able • Speed and visible results vs. reform continuity and to monetize all dimensions of the learning experience sustainable impact including curriculum and course development, teaching • Centralized vs. decentralized digital governance of and learning resources, credentials and certifications, education administration processes learning support and networks, flexible learning pathways and modalities, and even academic expertise.xl Moreover, • Personalized and individualized learning vs. social in other subsectors, new EdTech products and services learning through interaction and debate may initially be bundled for a lower price than the com- • Deciding whether to allow the use of AI tools in bined price of the individual components, which is simply schools economies of scale at work. However, market penetration enables firms to then unbundle and rebundle the offer- • Deciding whether to enable a ‘bring your own ings with the intention of elevating prices, including less device’ policy in schools useful products in the bundle (referred to as bloatware), or offering a complex mix of products at varying prices to • Offer options such as teachers plus tech or only apparently provide steep discounts. Such practices can put tech or only teachers. Source: Authors’ compilation. students and academic institutions at a disadvantage, cre- xxxix For example: e-readiness; program-level planning; course design and development; course delivery; course student support; course and program evaluation; and course 69 maintenance/updates. xl For example, freemium and subscription-based models. ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All with digital affordances. The centralized, semi-decentral- balance between unlocking the value of data for the public ized, and decentralized nature of governance mechanisms, benefit, fostering innovation through open-source princi- including fiscal, political, and administrative governance, ples, ensuring robust governance methods that empower in the country’s education systems have an outsized stakeholders, and addressing concerns over IP, particularly influence on how education policies and programs get with regard to private organizations.399 implemented. Centralized governance structures have more control over standards and greater economies of Inherent data privacy and cybersecurity challenges need scale, while decentralized governance structures allow to be carefully managed as any education system under- for greater customization of solutions and much greater goes a digital transition. While data-driven learning tools responsiveness to local needs. Decentralization is increas- have the potential to enhance student outcomes substan- ingly seen as the evolved response to the problems and tially, the effective use of student data in K-12 schools excesses of centralized governance. It is hailed for increas- remains in its early stages. The risks associated with data- ing the efficiency and effectiveness of organizations, facili- driven instructional methods and perceptions surround- tating quicker reaction to problems, and improving service ing these risks present significant obstacles to realizing delivery, while also increasing accountability, discourag- the benefits of large-scale adoption of data and analytics ing corruption, and increasing cost recovery. Deciding to in schools. Limited budgets, particularly in public schools, what extent and which aspects of digital and IT govern- make it difficult to prioritize investment in data-driven ance should be centralized or decentralized has signifi- tools and technologies. Using student data raises concerns cant implications. If policymakers value the benefits of about privacy, the risk of unauthorized access or sale of standardization, efficiencies of scale and scope, and lower personal information to third parties, and the broader coordination costs, they may choose to reduce on-prem- considerations of data collection and use within and out- ises systems and increase bandwidth capacity and cloud side the school system.400 Policymakers must support migration by centralizing IT digital platforms.395 However, education institutions in making informed investment concentrating governance power among platform owners decisions in digitally enabled education infrastructure by may disadvantage and alienate platform participants (for offering guidance on compliance with security and data example, third-party developers and end users) since plat- protection frameworks and promoting interoperability form owners can prioritize their own interests over those frameworks to ensure compatibility between different of stakeholders.396 Hence, decentralized digital platforms digital learning environments.401 based on distributed ledger technologies and Web3xli,397 are being explored in some countries to enhance greater Suitable institutional procurement strategies and budg- civic participation.398 etary practices are needed to ensure the judicious use of digital technologies within educational institutions. It is important to consider the implications of sharing This involves not only the provision of such necessary education data and personal data and adopt relevant infrastructure as broadband connectivity and access to safeguards within the evolving landscape of EdTech. digital equipment but also institutional funding and While EdTech holds transformative potential for enhanc- human resource policies to ensure their effective manage- ing students’ educational experiences and outcomes, con- ment and use at the local level.402 Often policymakers face cerns have surfaced about the invasion of learners’ privacy, trade-offs when making large-scale expensive procure- especially children, and about the lack of robust evidence ments with up-front cost estimates being much less than supporting the learning benefits that have been claimed estimates of the total cost of ownership (TCO), which for EdTech. Policymakers must grapple with the nascent could be three to seven times the initial investment. Find- sector’s complexities and need to consider restructuring ing a balance between up-front capital investments and the regulatory environment to safeguard students’ inter- the recurrent expenditures of using digital technologies ests. Balancing the roles of government and commercial is imperative for scale and sustainability. Furthermore, organizations in EdTech provision is crucial and requires meticulous planning of resource allocation and match- a governance framework that prioritizes the best interests ing is needed to handle logistical challenges, resource and of the student. The challenge lies in integrating informa- capacity constraints, and organizational change manage- tion that is held in different databases, such as students’ ment.403 The carefully tailored allocation of fiscal and grades and engagement data, to understand student needs budgetary resources will facilitate the acquisition and and preferences comprehensively. There may be a discrep- deployment of digital education technologies in accord- ancy in access to education records between ministries ance with the country’s overarching policy objectives. and private EdTech companies, often citing intellectual However, this undertaking is not without challenges, par- property (IP) or commercial reasons. This raises issues of ticularly in countries dealing with by severe financial con- reciprocity and the imperative to enable secure data shar- straints and/or hard-to-access rural settings.404 ing in public interest. Policymakers must strike a delicate xli Web3 is an ecosystem in which information is distributed and collectively managed across a network (Stackpole 2022). 70 ◄ Table of Contents Tailoring Digital Pathways There are coordination, financial, and political costs requisites; (iv) allocating physical and human resources; (v) involved in translating strategic intentions of a digitally fostering capacity development; and (vi) establishing robust enabled education system into effective policy imple- monitoring and evaluation systems. mentation. Strategic documents are most effective in stimulating and coordinating policy reform when comple- A strategic digital pathway for education must balance mented by time-bound action plans that identify specific short-term policies against medium- to long-term roles and responsibilities, establish shared governance policies as well as have the flexibility to accommodate structures, and specify funding arrangements. It is impor- changes and advances in response to social, techno- tant to recognize that proposed policies do not operate in logical, and economic shifts. The adaptability of educa- isolation but rather intersect with a spectrum of policies tion strategies becomes paramount as new technologies both within and outside the education sector, hence may emerge and needs from the field evolve. This flexibility only be realized by incurring significant coordination, and responsiveness are pivotal in a rapidly evolving field financial, and political costs. The willingness or unwilling- and can be ingrained at the design stage through pur- ness of policymakers to bear these costs will make or break poseful feedback loops. These loops identify emerging effective implementation. For example, policies related to needs, constraints, challenges, implementation issues, or regional development, tech sector regulations, and teacher undesirable effects, allowing policymakers to adjust their professional development are vital for implementing dig- strategic directions as the digitally enabled education eco- ital strategies in education. Thus, policymakers must aim system evolves and matures. The ability to juggle short- to align their strategic vision for digitally enabled educa- term, medium-term, and long-term considerations, coor- tion with the broader policy ecosystem to avoid a doubling dinate with relevant stakeholders, and make prudent fiscal of efforts and ensure synergy across various domains. trade-offs are all necessary digital education that aims for impactful and equitable outcomes. Fostering the active participation of a wide range of stakeholders and recognizing the role of champions is 5.2 Common Pitfalls in Shaping imperative for successfully implementing digitally ena- Digital Transitions bled education policies and action plans. The engage- ment of stakeholders fulfills a multitude of functions, Digitalization initiatives in many industries fail to reach such as promoting innovation (through the participation their desired outcomes, in both public and private sector. of private enterprises, academic institutions, and research Reports from a variety of strategy consultants, such as centers), assisting families in educating children about the BCG, McKinsey, and Bain, attest that only 15 to 25 percent educational potential and risks of digital technologies, of digital transformation initiatives end up meeting the and facilitating the procurement of digital technologies planned business objectives. The many failed experiences (through the involvement of local authorities). Policymak- of digital initiatives tend to share some common pitfalls ers need to investigate the protocols, systems and collab- observable across sectors and domains, several of which orative mechanisms that can be used to involve a diverse are also relevant in education. This section discusses some network of stakeholders. Effective and contextually rele- common challenges that have emerged from earlier digi- vant models of stakeholder engagement and public com- talization efforts in education in various countries. munication are needed to engage the stakeholders who influence how policies are implemented in the educational Policymakers should try to avoid strategies that are system, including but not limited to students, educators, device- or technology-centric and that view technology administrators, and subnational governing bodies. as an add-on. Strategies often emphasize rolling out hard- ware or building infrastructure without including a clear Substantial time and effort are often needed to collaborate pathway for how this will improve education delivery. with authorities from other sectors to ensure a coherent Success is often quantified by inputs, such as the rollout policy approach and effective implementation of digitally of devices or the construction of computer labs, without enabled education focused on learning. It is imperative acknowledging that these are only a means to achieve aca- to assess digitalization policies in education not in isola- demic outcomes. Additionally, digitalization is often seen tion but in the context of the governance environments in as a separate layer to be added onto an education compo- which they function while at the same time adjusting to the nent, as opposed to being built into it. For example, teach- demands and potentialities of emerging technologies. The ing and learning materials that may have been originally components of a strategic vision for digitally enabled edu- developed for traditional classroom instruction are simply cation include: (i) establishing clear objectives that are con- approved to be digitized, instead of being redesigned from nected to tangible policy actions on pedagogies, curricula, the outset using multimodal instructional design pro- and assessment; (ii) providing direction and regulation for cesses that emphasize interactive teaching and learning digitally enabled education; (iii) establishing essential pre- materials. 71 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All Large-scale purchases of technology should not be made proposed system might not meet their needs. They may in the absence of an assessment and analysis of oppor- have different expectations from digitalization initiatives tunity costs. Large-scale technology purchases are often than those of policymakers. This may lead to difficulties in made without either testing the proof of concept of the implementation and outright resistance at the school level technology in the intervention (especially considering because of a lack of motivation among teaching work- the high failure rate of digital transformation initiatives) force. Similarly, different government agencies may have or budgeting for the TCO, such as device maintenance, different interpretations of the country’s digitalization training, change management, or curriculum reform, strategy, resulting in a misalignment of efforts and poor which can be five to ten times higher than the initial cost investment decisions. These different approaches might of acquisition.405 Countries sometimes feel pressured to be due to tensions between government agencies, power embark on large-scale technology procurement because struggles, or political allegiances. Therefore, change man- of time-bound funding availability (for example, funds agement efforts might then be needed to overcome inter- from a donor need to be used in a specific time period) nal resistance. or political reasons (for example, the need to distribute devices before an upcoming election cycle). It is better to A lack of or poor-quality coordination, monitoring, make expensive digital technology investments in educa- evaluation, and accountability mechanisms can hamper tion after due consideration to the contextual needs and the implementation of digitalization efforts. In some related opportunity costs. countries, non-state actors such as private sector com- panies or NGOs are already implementing digitalization Inflexible, overly theoretical, long-term strategies often projects, but their activities are not always monitored or become outdated or limiting. High-level decision-makers integrated with national digitalization plans. For exam- or external consultants may take many months to develop ple, a development agency may be rolling out digitalized theoretical policies in boardrooms without engaging with teaching and learning materials that are incongruent with local-level stakeholders such as local government author- the national curriculum approval processes. In general, ities, teachers, students, parents, and communities to strategies often need to include sufficient monitoring and ensure their buy-in. These fixed, long-term plans cannot evaluation activities or fail to implement them at all or the adapt to the changing technology landscape due to their frameworks are too complex to be carried out frequently complexity and lack of contextual knowledge. Further- and effectively. Monitoring and evaluation are also often more, they are not resilient in terms of unforeseen events treated as an afterthought as opposed to playing their that might occur (for example, pandemics or natural dis- essential role in the feedback loop for adaptation of the asters). As technology is fast evolving, regulations (for strategy and its implementation. example, the need to store data locally) may lag or become outdated and restrictive. Disjointed legacy platforms and applications can be too expensive to fit into improved systems. While govern- Government officials often lack the capacity, resources, ments may acknowledge that systems can be improved, and time to fully support digital transition efforts. Gov- the cost, time, and approvals needed to update or merge ernment officials at the federal, state, district, and local them are high. For example, taking a building blocks levels often do not have the capacity, resources, and time approach to education data management makes sense in to develop the required skill sets for going digital, particu- theory. However, when governments already have many larly when it involves the use of new tools and processes. stand-alone incompatible systems in place, the politics These skills may include data management, data analysis, and costs of redesigning the underlying architecture can instructional design, and software-specific digital liter- make it an unfeasible prospect.406 This challenge is more acy skills. Governments may also lack access to appropri- acute in technologically advanced countries, while coun- ate software or approvals to use software or data storage tries just starting to enter the digital realm have the advan- mechanisms. Even when officials have the necessary skill tage of being able to start with a clean slate and adopt an sets and training, they are often overloaded when new dig- integrated system architecture from the outset. italization activities are added on top of their existing roles and responsibilities. Overloading is an issue of capacity Data privacy, data usability, and data security concerns management and resource allocation. may be too lenient or too restrictive. Governments may adopt data management processes that are either too Lack of stakeholders’ buy-in as sufficient time and effort strict or too lenient in terms of privacy and security. They are not put into understanding and responding to their may choose to restrict the hosting of education data to needs and pain points. Without concluding a needs assess- local data centers with the intention to maximize security. ment with government authorities (at the state, regional, However, by doing so, they then limit the availability and district levels) and with school managers and teachers, the usability of the data and increase the risk of entire plat- 72 ◄ Table of Contents Tailoring Digital Pathways forms going offline if local data centers malfunction. Con- The goals of the plan should be aligned with broader versely, a government may choose to transition to public education sector plans, existing and planned national cloud hosting for greater flexibility, but without necessary ICT infrastructure, and digitalization plans in other safeguards, this opens the data up to risks of exfiltration sectors.408 When policies related to ICT in education syn- or cyberattacks. Therefore, before aggregated data can be chronize with other sector’s plans, costs can be shared, shared openly to support better decision-making, it is vital and benefits can be optimized. For example, digital archi- to take proper measures for data anonymization, cleaning, tecture can be shared between primary and higher edu- and security. But this process may be considered cumber- cation or between education and health systems. Broader some and a bottleneck to making data open. digitalization plans, such as providing connectivity, over- lap strongly with connectivity availability in schools. The following section draws on these lessons to outline Installing undersea fiber optic cables or developing dig- some effective ways to overcome these challenges. ital learning platforms in shared languages may require greater regional cooperation. In addition to aligning with 5.3 Effective Practices for Digital other plans, ICT in education policies should align with Transitions in Education what aspects of governance are centralized and decen- tralized and other political economy considerations. Con- Due to the rapid evolution of technology, digitaliza- text-specific factors, such as socioeconomic or emergency tion strategies must be flexible, adaptive, and iterative variations between regions, should be taken into account to remain relevant. Traditionally, the development of a to ensure that policies can adapt to diverse needs. strategy or policy precedes the implementation of a dig- ital solution which can put the brakes on the rapid adop- It is useful to adopt agile delivery approaches where tion of the solution. Since most challenges discussed in possible and when applicable to develop responsive pol- the previous section are human-centered implementation icies and plans.409 These approaches start with a discov- challenges rather than technology-centered issues, the ery phase where actors seek to understand a problem (as guidelines outlined here are intended to address imple- discussed in Chapter 4), and then iteratively implement mentation challenges from the onset. alpha, beta, and trial phases to test, adapt, and improve as the policy is scaled up. In the alpha stage, multiple First and foremost, policymakers must begin with edu- prototypes of different approaches and ideas might be cation sector problems, identify those solvable using tested to determine what works most effectively and digital technologies and set clear milestones and goals. appropriately within a given context. Using this meth- As mentioned in Section 5.1, it is not possible to imple- odology can prevent the development of theoretical ment all interventions envisioned in a strategy at once. plans that might not work in practice (See Figure 17). Instead, the plan needs to focus on the specific, urgent challenges that the country is facing, which can be deter- Digital transition policies must take into account dif- mined through a comprehensive analysis of the education ferent dimensions and layers of the education system as sector. The goals that are set in the plan should balance well as their governance arrangements to achieve policy comprehensiveness and feasibility, while recognizing that coherence and alignment. To maintain adaptability and digitalization does not solve all problems and may not relevance, the policy ecosystem should incorporate feed- always be the best solution.407 back loop mechanisms to inform adjustments to the strat- Figure 17. Example of an agile delivery approach Alpha Beta National Trial Scale Testing multiple Testing the best idea Examining impact and Rolling out across the small prototypes and further refining it designing for national scale country, and continuing to /ideas in a number of schools iterate and improve 73 ◄ Table of Contents Digital Pathways for Education: Enabling Greater Impact for All egy in response to evolving trends and insights from stake- Decision-makers should focus on the usability of their holders. This dynamic and integrated approach ensures country’s national education management informa- that the digitally enabled education policy ecosystem tion system (EMIS) rather than on producing data remains responsive and effective in addressing the evolv- dashboards as an end in itself. It is crucial that deci- ing needs of education systems.410 sions regarding what an EMIS should measure and how it should operate should be influenced by the country’s Digital aspects should be considered and integrated into education policies and priorities. Policymakers should be intervention design upfront, with inclusion, scale and able to use data to evaluate what progress is being made sustainability for optimal impact. Scale cannot be an towards the achievement of those priorities. Therefore, afterthought. Many interventions fail because the pilot data architects must understand the relevant policies projects are not scalable.411 Designing for scale means and implement the necessary functionality accordingly. considering user costs, partners, funding models, and For instance, there is considerable demand for data per- technology choices upfront.412 Crucially, a major differ- taining to mandates regarding equity and equality to see ence between pilots and scaled projects is the amount of if children of all demographic backgrounds are receiving implementation and human support that is needed for the education to which they are entitled. Additionally, an the scale-up, which is often too costly. McLean et al (2020) EMIS could provide analysts with timely information on have argued that, instead of scaling up (for example, train- increases in school dropout rates and the factors that con- ing more teachers) or scaling out (for example, equipping tribute to them. more teacher resource centers), the focus should be on scaling impact. They outline four guiding principles for Providing guidance, standards, and formal regulations planning to scale for impact: (i) justify whether something is crucial to ensuring the safe and effective use of digital can and should be scaled; (ii) optimize scale by making technology in education systems. The growing cyberse- trade-offs, such as equity versus sustainability; (iii) coor- curity risks, data protection concerns, and potential algo- dinate different complex systems requiring multi-stake- rithmic bias make it necessary for policymakers to pay holder engagement, flexibility, and adaptability; and (iv) closer attention to complying with existing digital security dynamically evaluate the how taking a different action and data protection frameworks and to developing regu- affects impact, considering, why, for whom, and under lations in currently uncovered areas. Continuous regula- what conditions. tory efforts are essential to address privacy concernsxlii and limit inequitable practices as educational technologies Collaborations between different government stake- evolve. Lessons can be learned from other industries that holders can facilitate a systems way of thinking, encour- use data to improve the user experiences while embracing age trust and drive greater coherence and alignment413. transparent data use to improve results. As highlighted in UNICEF’s Pulse Check on Digital Learning414, strategies that overlook the human element have higher failure rates. Systems become more effective when they focus on learning instead of on technologies or devices.415 Learning applies not only to students and teachers but also to government officials at the federal, state, and local levels and all other stakeholders in the edu- cation ecosystem. Adopting a systems approach involves understanding the dynamics and complexity of interact- ing variables in a system, including stakeholders, policies, global events, regulations, and cross-sectoral influences.416 It is vital that decision makers periodically update the strategy in response to technological changes. It is impor- tant for a flexible, adaptable and collaborative approach in designing and implementing the strategy. This goes beyond soliciting stakeholder input to co-creating envi- ronments where various stakeholders can continuously and actively participate in sense-making.417,418 To ensure that the strategy remains responsive and resilient, it will need to be updated periodically, which is not usually the case with other strategies and policies. xlii The EdTech industry has been compared to regulation in the medical industry in regard to how it could be regulated: “If it is not allowed to sell ineffective medicine, why 74 should we allow the selling of ineffective learning tools?” 6 ◄ Table of Contents Tailoring Digital Pathways CONCLUSION Chapter 6: CONCLUSION A new collective understanding of digitally enabled edu- ducers and rapid consumers of the right skills, knowledge, cation systems is needed to achieve greater learning and and behaviors. At the same time, there is intrinsic value in well-being for all. Digital technologies are offering new education as part of the human endeavor to learn, grow, tools through which learning goals can be achieved. How- and participate in civil society as active and responsible cit- ever, this does not mean that achieving better outcomes in izens. Increasingly, civic participation is enabled through a digital world depends only on the rapid adoption of new access to digital services, financial transactions, tax pay- technologies in education systems. It crucially depends ments, voting, and many digital government services. In on decision-makers taking a value-based and purposeful the end, the human side of the human-machine interface approach to the discovery and design of digital solutions determines how technology can best serve education poli- and innovations in education, diffuse the benefits of dig- cymakers, administrators, service providers, teachers, and ital education technologies equitably and deploy them in the community of students and parents. an economically feasible way.419 The pandemic experience demonstrated the enormous cost to societies when education systems are not fit for purpose and are unprepared in the face of crisis.420 Dig- ital education technologies, while not a panacea, provide a minimum of services. They enable countries to be more resilient. During the pandemic, countries with a relatively mature digital backbone, an enabling ecosystem, and rel- evant digital capabilities in education were able to switch to and expand emergency remote education servicesxliii more rapidly and successfully than those with less digitally mature education systems. However, when those digital solutions were not well thought through, contextually tai- lored, and evidence based, they failed to meet their pur- pose, incurring significant opportunity costs. In the end, the human side of the human-ma- chine interface determines how technology can best serve education policymakers, administra- tors, service providers, teachers, and the commu- nity of students and parents. Today’s education policymakers must choose contextu- ally relevant digital pathways that help build a well-en- dowed and competitive labor force fit for dynamic and digitally intense labor markets. As every aspect of the digital economy is powered by people, education is both the key constraint and a key enabler of a successful digital transition. Hence, individuals need to be both avid pro- xliii For example, in Uruguay, a pioneer in Latin America and the Caribbean region with an exemplary educational technology initiative since 2007, early investments in edu- 75 cation technology are paying off. Every student had access to a personal computer and internet. Short videos and follow-up activities can be broadcast weekly using the national online learning platform. Teachers know how to work remotely using interactive lesson plans with students and set weekly goals for learners. Thus, Uruguay’s Ceibal at-home program was launched immediately after school closures because it could draw on the preexisting systematic deployment of Plan Ceibal’s digital capability. ◄ Table of Contents REFERENCES 1. Monton, A. L. (2022, March 22). Difference and Similari- com/downloadpdf/book/9789460919039/BP000006.pdf ties: Digitization, Digitalization, and Digital Transformation. (details) GlobalSign. https://www.globalsign.com/en-sg/blog/differ- ence-and-similarities-digitization-digitalization-and-digi- 7. Ganimian, Vegas, and Hess 2020; National Academy of tal-transformation (details) Science, Engineering, and Medicine 2000. 2. Bloomberg, J. (2018). Digitization, Digitalization, and Digital Ganimian, A. J., Vegas, E., & Hess, F. M. (2020, Septem- Transformation: Confuse Them At Your Peril. Forbes. https:// ber 10). Realizing the promise: How can education tech- www.forbes.com/sites/jasonbloomberg/2018/04/29/ nology improve learning for all? Brookings. https://www. digitization-digitalization-and-digital-transformation-con- brookings.edu/essay/realizing-the-promise-how-can-edu- fuse-them-at-your-peril/ (details) cation-technology-improve-learning-for-all/ (details) 3. Adkins, S. S. (2020). The 2019 Global Learning Technol- Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How ogy Investment Patterns: Another Record Shattering Year. people learn (Vol. 11). Washington, DC: National academy https://users.neo.registeredsite.com/9/8/1/17460189/ press. https://nap.nationalacademies.org/catalog/9853/ assets/Metaari-2019-Global-Learning-Technology-Invest- how-people-learn-brain-mind-experience-and-school-ex- ment-Patterns.pdf (details) panded-edition 4. Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. National Academies of Sciences, Division of Behavio- (2023). Systematic literature review on opportunities, chal- ral, Board on Science Education, Board on Behavioral, lenges, and future research recommendations of artifi- Sensory Sciences, Committee on How People Learn II, ... & cial intelligence in education. Computers and Education: Practice of Learning. (2018). How people learn II: Learners, Artificial Intelligence, 4, 100118. https://doi.org/10.1016/j. contexts, and cultures. National Academies Press. https:// caeai.2022.100118 (details) nap.nationalacademies.org/catalog/24783/how-people- learn-ii-learners-contexts-and-cultures 5. Saavedra, C., Jaime, A. I., Mario, C. A. D., & Omar, S. (2020). Realizing the future of learning: From learning Darling-Hammond, L., & Oakes, J. (2021). Preparing teach- poverty to learning for everyone, everywhere. World ers for deeper learning. Harvard Education Press. Bank, 154703. https://documents1.worldbank.org/curated/ en/250981606928190510/pdf/Realizing-the-Future-of- Darling-Hammond, L., Oakes, J., Wojcikiewicz, S., Hyler, M. Learning-From-Learning-Poverty-to-Learning-for-Every- E., Guha, R., Podolsky, A., Kini, T., Cook-Harvey, C., Mercer, one-Everywhere.pdf C., & Harrell A. (2019). Preparing Teachers for Deeper Learn- ing (research brief). Palo Alto, CA: Learning Policy Institute. 6. Goldin, C., & Katz, L. F. (2018). The race between edu- cation and technology. In Inequality in the 21st Century De Felice, S., Hamilton, A. F. D. C., Ponari, M., & Vigliocco, (pp. 49–54). Routledge. https://www.taylorfrancis.com/ G. (2023). Learning from others is good, with others is chapters/edit/10.4324/9780429499821-10/race-educa- better: the role of social interaction in human acquisition tion-technology-claudia-goldin-lawrence-katz (details) of new knowledge. Philosophical Transactions of the Royal Society B, 378(1870), 20210357. https://www.ncbi.nlm.nih. Tinbergen, J. (1974). Substitution of graduate by other gov/pmc/articles/PMC9791495/ labour. Kyklos: International Review for Social Sciences. https://repub.eur.nl/pub/8084/1974Kyklos.pdf (details) 8. Kucirkova, N., Brod, G., & Gaab, N. (2023). Applying the science of learning to EdTech evidence evaluations De Siqueira, A. C. (2012). The 2020 World Bank educa- using the EdTech Evidence Evaluation Routine (EVER). tion strategy: Nothing new, or the same old gospel. In The Npj Science of Learning, 8(1), 35. https://doi.org/10.1038/ World Bank and Education (pp. 67–81). Brill. https://brill. s41539-023-00186-7. (details) 76 ◄ Table of Contents Frau-Meigs, D., & Hibbard, L. (2016). Education 3.0 and Technologies, 27(7), 9953–10011. https://doi.org/10.1007/ Internet Governance: A new global alliance for children and s10639-022-11020-9 (details). young people’s sustainable digital development. https:// www.cigionline.org/sites/default/files/gcig_no27web_0. Bingimlas, K. A. (2009). Barriers to the successful inte- pdf (details) gration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, 9. Vegas, E., Ziegler, L., & Zerbino, N. (2019, November 20). Science and Technology Education, 5(3), 235–245. https:// How ed-tech can help leapfrog progress in education. doi.org/10.12973/ejmste/75275 (details) Center for Universal Education at The Brookings Institu- tion. https://eric.ed.gov/?id=ED602936 (details) Freeman, A., Becker, S. A., & Cummins, M. (2017). NMC/ CoSN horizon report: 2017 K. The New Media Consortium. 10. Olsen, B. (2023). Government Decision-Making on Educa- https://www.learntechlib.org/p/182003/ (details) tion in Low- and Middle-Income Countries: Understanding the Fit among Innovation, Scaling Strategy, and Broader Omidyar Network. (2019). Scaling Access & Impact Real- Environment. https://www.brookings.edu/wp-content/ izing the Power of EdTech. https://assets.imaginablefu- uploads/2023/08/Olsen-2023.pdf (details) tures.com/media/documents/ON_Scaling_Access__ Impact_2019_85x11_Online.pdf (details) 11. Labun, J. (2023). Fixing Ed-Tech’s Investment Model. https://ssir.org/articles/entry/fixing_ed_techs_investment_ 16. Rajasekaran, S., & Casap, L. (2022). Moldova – model. (details) Digital Education Readiness Assessment 2021-22. World Bank Group. http://documents.worldbank.org/ 12. Hirsh-Pasek, K., Zosh, J. M., Golinkoff, R. M., Gray, J. H., curated/en/099120006252220689/P17773104ea- Robb, M. B., & Kaufman, J. (2015). Putting education in 6f2040a88e02bdf9bbd04f6. (details) “educational” apps lessons from the science of learning. Psychological Science in the Public Interest, 16(1), 3-34. 17. Gkrimpizi, T., Peristeras, V., Magnisalis, I. (2023) Classifica- tion of Barriers to Digital Transformation in Higher Educa- 13. Autor, D. (2022). The labor market impacts of technological tion Institutions: Systematic Literature Review, https://www. change: From unbridled enthusiasm to qualified optimism mdpi.com/2227-7102/13/7/746 to vast uncertainty. National Bureau of Economic Research. https://www.nber.org/papers/w30074 (details) 18. International Commission on the Futures of Education. (2021). Reimagining our futures together: a new social Facer, K., & Selwyn, N. (2021). Digital technology and the contract for education. UNESCO. https://doi.org/10.54675/ futures of education: Towards ‘Non-Stupid’optimism. Paper ASRB4722 (details) Commissioned for the UNESCO Futures of Education Report. https://www.academia.edu/download/98619725/ United Nations. (2022). Assuring and improving quality public Unesco_Background_Ed_Tech.pdf (details) digital learning for all. United Nations. https://www.un.org/en/ transforming-education-summit/digital-learning-all (details) Shapiro, S. L., Schwartz, G. E., & Bonner, G. (1998). Effects of Mindfulness-Based Stress Reduction on Medical and 19. A4AI. (2022). Meaningful Connectivity — unlocking the full Premedical Students. Journal of Behavioral Medicine, 21(6), power of internet access. Alliance for Affordable Internet. 581–599. https://doi.org/10.1023/A:1018700829825 (details) https://a4ai.org/meaningful-connectivity/. (details) 14. Cullinan, J., Flannery, D., Harold, J., Lyons, S., & Palcic, 20. Hodges, C. B., Moore, S., Lockee, B. B., Trust, T., & Bond, D. (2021). The disconnected: COVID-19 and disparities M. A. (2020). The difference between emergency remote in access to quality broadband for higher education stu- teaching and online learning. https://vtechworks.lib.vt.edu/ dents. International Journal of Educational Technology in handle/10919/104648 (details) Higher Education, 18(1), 26. https://doi.org/10.1186/s41239- 021-00262-1 (details) 21. UNESCO. (2017). The world is off track in achieving the global education goal, SDG 4 - UNESCO Digital Library. https:// 15. Rajasekaran, S., & Casap, L. (2022). Moldova – Digital Edu- unesdoc.unesco.org/ark:/48223/pf0000368935 (details) cation Readiness Assessment 2021-22. World Bank Group. 22. UNICEF. (2020, August 26). COVID-19: Are children able to Godaert, E., Aesaert, K., Voogt, J., & Van Braak, J. (2022). continue learning during school closures? UNICEF DATA. Assessment of students’ digital competences in primary https://data.unicef.org/resources/remote-learning-reacha- school: a systematic review. Education and Information bility-factsheet/ (details) 77 ◄ Table of Contents 23. The World Bank. (2018). World Development Report 2018: decline of life on Earth points to the need for transform- Learning to realize education’s promise. (details) ative change. Science, 366(6471), eaax3100. https://doi. org/10.1126/science.aax3100 (details) 24. The Chronicle. (2020, June 30). How the Education System Exacerbates Inequality. The Chronicle of Higher 33. United Nations. (2022). World Population Prospects 2022. Education. https://www.chronicle.com/article/how-the-ed- United Nations, Department of Economic and Social ucation-system-exacerbates-inequality (details) Affairs, Population Division. https://www.un.org/develop- ment/desa/pd/sites/www.un.org.development.desa.pd/ 25. World Bank. (2023). Open Knowledge Repository. files/wpp2022_summary_of_results.pdf (details) https://openknowledge.worldbank.org/entities/publica- tion/211c4d69-9013-4942-831f-4dbae5ede977?deliv- 34. Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. eryName=DM209153. (details) Available under “World (2023). Systematic literature review on opportunities, chal- Bank. 2023. Making Teacher Policy Work. © Washing- lenges, and future research recommendations of artifi- ton, DC: World Bank. http://hdl.handle.net/10986/40579 cial intelligence in education. Computers and Education: License: CC BY 3.0 IGO.”. Artificial Intelligence, 4, 100118. https://doi.org/10.1016/j. caeai.2022.100118 (details) 26. The State of Global Learning Poverty: 2022 Update. (2022). https://thedocs.worldbank.org/en/doc/ 35. Facer, K., & Selwyn, N. (2021). Digital technology and the e52f55322528903b27f1b7e61238e416-0200022022/ futures of education: Towards ‘Non-Stupid’optimism. Paper o r i g i n a l / Le a r n i n g - p o v e r t y - r e p o r t - 2 0 2 2 - 0 6 - 2 1 - f i - Commissioned for the UNESCO Futures of Education nal-V7-0-conferenceEdition.pdf (details) Report. https://www.academia.edu/download/98619725/ Unesco_Background_Ed_Tech.pdf (details) 27. GEM Report UNESCO. (2023). Global Education Mon- itoring Report 2023: Technology in education: A tool on 36. Reich, J., Buttimer, C. J., Fang, A., Hillaire, G., Hirsch, K., whose terms? (1st ed.). GEM Report UNESCO. https://doi. Larke, L. R., Littenberg-Tobias, J., Moussapour, R. M., org/10.54676/UZQV8501 (details) Napier, A., & Thompson, M. (2020). Remote learning guid- ance from state education agencies during the COVID- 28. Archer, D. (2023). Transforming the financing of education 19 pandemic: A first look. https://doi.org/10.35542/osf. at the mid-point of the sustainable development goals. io/437e2 (details) International Journal of Educational Development, 103, 102931. https://doi.org/10.1016/j.ijedudev.2023.102931 37. Baird, S. J., Garfein, R. S., McIntosh, C. T., & Özler, B. (2012). (details) Effect of a cash transfer programme for schooling on preva- lence of HIV and herpes simplex type 2 in Malawi: a cluster 29. End the killing of children in armed conflict, UN com- randomised trial. The Lancet, 379(9823), 1320–1329. mittee urges. (2023). https://www.ohchr.org/en/state- https://doi.org/10.1016/S0140-6736(11)61709-1 (details) ments/2023/11/end-killing-children-armed-conflict-un- committee-urges (details) 38. A4AI. (2022). Meaningful Connectivity — unlocking the full power of internet access. Alliance for Affordable Internet. 30. Bend, M., Hu, Y., Pan, Y., Patrinos, H. A., Poulsen, T., Rive- https://a4ai.org/meaningful-connectivity/. (details) ra-Olvera, A., Tanaka, N., Antoninis, M., Murakami, Y., & Benveniste, L. (2023). Education Finance Watch 2023. 39. International Commission on the Futures of Education. https://policycommons.net/artifacts/6951221/education-fi- (2021). Reimagining our futures together: a new social nance-watch-2023-english/7861178/ (details) contract for education. UNESCO. https://doi.org/10.54675/ ASRB4722 (details) 31. UNICEF, I. T. U. (2020). Two Thirds of the World’s School- Age Children Have No Internet Access at Home, New United Nations. (2022). Assuring and improving quality UNICEF-ITU Report Says. https://www.unicef.org/press-re- public digital learning for all. United Nations. https://www. leases/two-thirds-worlds-school-age-children-have-no-in- un.org/en/transforming-education-summit/digital-learn- ternet-access-home-new-unicef-itu (details) ing-all (details) 32. Díaz, S., Settele, J., Brondízio, E. S., Ngo, H. T., Agard, J., 40. Goldin, C., & Katz, L. F. (2018). The race between edu- Arneth, A., Balvanera, P., Brauman, K. A., Butchart, S. H. M., cation and technology. In Inequality in the 21st Century Chan, K. M. A., Garibaldi, L. A., Ichii, K., Liu, J., Subrama- (pp. 49–54). Routledge. https://www.taylorfrancis.com/ nian, S. M., Midgley, G. F., Miloslavich, P., Molnár, Z., Obura, chapters/edit/10.4324/9780429499821-10/race-educa- D., Pfaff, A., … Zayas, C. N. (2019). Pervasive human-driven tion-technology-claudia-goldin-lawrence-katz (details) 78 ◄ Table of Contents Tinbergen, J. (1974). Substitution of graduate by other Facer, K., & Selwyn, N. (2021). Digital technology and the labour. Kyklos: International Review for Social Sciences. futures of education: Towards ‘Non-Stupid’optimism. Paper https://repub.eur.nl/pub/8084/1974Kyklos.pdf (details) Commissioned for the UNESCO Futures of Education Report. https://www.academia.edu/download/98619725/ The World Bank. (2018). World Development Report 2018: Unesco_Background_Ed_Tech.pdf (details) Learning to realize education’s promise. (details) Shapiro, S. L., Schwartz, G. E., & Bonner, G. (1998). Effects De Siqueira, A. C. (2012). The 2020 World Bank educa- of Mindfulness-Based Stress Reduction on Medical and tion strategy: Nothing new, or the same old gospel. In The Premedical Students. Journal of Behavioral Medicine, World Bank and Education (pp. 67–81). Brill. https://brill. 21(6), 581–599. https://doi.org/10.1023/A:1018700829825 com/downloadpdf/book/9789460919039/BP000006.pdf (details) (details) 43. World Bank. 2024. Digital Progress and Trends Report 41. Ganimian, Vegas, and Hess 2020; National Academy of 2023. © Washington, DC: World Bank. http://hdl.handle. Science, Engineering, and Medicine 2000. net/10986/40970 Ganimian, A. J., Vegas, E., & Hess, F. M. (2020, Septem- https://openknowledge.worldbank.org/entities/publica- ber 10). Realizing the promise: How can education tech- tion/7617f89d-2276-413d-b0a7-e31e7527d6af nology improve learning for all? Brookings. https://www. brookings.edu/essay/realizing-the-promise-how-can-edu- 44. Hawkins,Robert J.; Trucano,Michael; Cobo Romani,Juan cation-technology-improve-learning-for-all/ (details) Cristobal; Twinomugisha,Alex; Sanchez Ciarrusta,Inaki Alejandro. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn (Vol. 11). Washington, DC: National academy Reimagining Human Connections : Technology and Inno- press. https://nap.nationalacademies.org/catalog/9853/ vation in Education at the World Bank (English). Washing- how-people-learn-brain-mind-experience-and-school-ex- ton, D.C. : World Bank Group. http://documents.worldbank. panded-edition org/curated/en/829491606860379513/Reimagining-Hu- man-Connections-Technology-and-Innovation-in-Educa- National Academies of Sciences, Division of Behavio- tion-at-the-World-Bank ral, Board on Science Education, Board on Behavioral, Sensory Sciences, Committee on How People Learn II, ... & 45. Soegaard, M. (2015, July 5). Affordances. Interaction Practice of Learning. (2018). How people learn II: Learners, Design Foundation - IxDF. https://www.interaction-design. contexts, and cultures. National Academies Press. https:// org/literature/book/the-glossary-of-human-computer-in- nap.nationalacademies.org/catalog/24783/how-people- teraction/affordances learn-ii-learners-contexts-and-cultures 46. UNICEF. (2022). Pulse Check on Digital Learning. https:// Darling-Hammond, L., & Oakes, J. (2021). Preparing teach- www.unicef.org/reports/pulse-check (details) ers for deeper learning. Harvard Education Press. 47. Education Technology Market Size & Share Report, 2030. Darling-Hammond, L., Oakes, J., Wojcikiewicz, S., Hyler, M. (n.d.). Retrieved February 29, 2024, from https://www. E., Guha, R., Podolsky, A., Kini, T., Cook-Harvey, C., Mercer, grandviewresearch.com/industry-analysis/education-tech- C., & Harrell A. (2019). Preparing Teachers for Deeper Learn- nology-market (details) ing (research brief). Palo Alto, CA: Learning Policy Institute. 48. Engaging the private sector for stronger, more dynamic De Felice, S., Hamilton, A. F. D. C., Ponari, M., & Vigliocco, education - Creative. (2020, November 2). https://www. G. (2023). Learning from others is good, with others is creativeassociatesinternational.com/insights/engag- better: the role of social interaction in human acquisition ing-the-private-sector-for-stronger-more-dynamic-educ- of new knowledge. Philosophical Transactions of the Royal ation/ (details) Society B, 378(1870), 20210357. https://www.ncbi.nlm.nih. gov/pmc/articles/PMC9791495/ 49. Transforming Education Summit, 2022; UNESCO Commis- sion on Futures of Education, 2021 42. Autor, D. (2022). The labor market impacts of technological change: From unbridled enthusiasm to qualified optimism 50. UNICEF. (2022). Pulse Check on Digital Learning. UNICEF. to vast uncertainty. National Bureau of Economic Research. https://www.unicef.org/media/132096/file/Pulse%20 https://www.nber.org/papers/w30074 (details) Check.pdf (details) 79 ◄ Table of Contents 51. Coflan, C., Wyss, N., Thinley, S., & Roland, M. (2022). Devel- Ministry Economic Affairs and Communications. (2011). oping a national EdTech strategy. EdTech Hub. (details) Interoperability of the State Information System version 3.0. (details) 52. Shapiro, S. L., Schwartz, G. E., & Bonner, G. (1998). Effects of Mindfulness-Based Stress Reduction on Medical and 60. Welland, A. (2017). Education and the electrification of rural Premedical Students. Journal of Behavioral Medicine, schools. https://e4sv.org/wp-content/uploads/2017/05/ 21(6), 581–599. https://doi.org/10.1023/A:1018700829825 TR13-Education-and-the-Electrification-of-Rural-Schools_ (details) web-1.pdf (details) 53. Kelly, P. (2023). 5 Challenges of Digital Transformation in 61. Welland, A. (2017). Education and the electrification of rural the Public Sector. GovNet. https://blog.govnet.co.uk/tech- schools. https://e4sv.org/wp-content/uploads/2017/05/ nology/5-challenges-of-digital-transformation-in-the-pub- TR13-Education-and-the-Electrification-of-Rural-Schools_ lic-sector (details) web-1.pdf (details) 54. ITU. (2021, October 14). Ministerial Roundtable - “Digital- 62. Utilities One. (2023, November 24). The Impact of Broad- izing daily life: government services and content driving band Network Construction on Education. Utilities One. digital transformation” – Part 1 - ITU Telecom World. https:// https://utilitiesone.com/the-impact-of-broadband-net- digital-world.itu.int/ministerial-roundtable-digitalizing-dai- work-construction-on-education (details) ly-life-government-services-and-content-driving-digi- tal-transformation-part-1/ (details) 63. The Economist 2021. 55. ITU. (2021, October 14). Ministerial Roundtable - “Digital- 64. Deutsche Gesellschaft für Internationale Zusammenar- izing daily life: government services and content driving beit (GIZ) GmbH. (2017). Data for development: What’s digital transformation” – Part 1 - ITU Telecom World. https:// next? Concepts, trends and recommendations for German digital-world.itu.int/ministerial-roundtable-digitalizing-dai- development cooperation. https://webfoundation.org/ ly-life-government-services-and-content-driving-digi- docs/2018/01/Final_Data-for-development_Whats-next_ tal-transformation-part-1/ (details) Studie_EN.pdf (details) 56. Kamp-Hartong, S., Piattoeva, N., Saari, A., & Savage, G. Gustafsson-Wright, E., Osborne, S., & Aggarwal, M. (2022). Transformation of Education Policy and Gov- (2022). Digital tools for real-time data collection in edu- ernance in the Digital Era. https://academic.oup.com/ cation. https://policycommons.net/artifacts/4142237/ book/41546/chapter/353003648. (details) digital-tools-for-real-time-data-collection-in-educa- tion/4951284/ (details) 57. Brazil : The SOL-ution for Smart Community Procurement. (2022). World Bank. https://doi.org/10.1596/37687 (details) 65. Goger, A., Parco, A., Carter-Rau, R., Homma, K., Melikset- yan, A., & Milman, N. (2022). How learning and employ- 58. Hallward-Driemeier, Mary; Nayyar, Gaurav; Gill, Indermit; ment records shape access to quality education and jobs. Aridi, Anwar. (2020). Europe 4.0: Addressing the Digital Brookings. (details) Dilemma. World Bank, Washington DC. http://hdl.handle. net/10986/34746 66. Bethel, E. C. (2014). A Systematic Review of One-to-One Access to Laptop Computing in K-12 Classrooms: An Inves- 59. e-Estonia guide (n.d.). Retrieved March 19, 2024, from tigation of Factors That Influence Program Impact. (details) https://e-estonia.com/wp-content/uploads/eas-eestonia- vihik-a5-180404-view.pdf (details) McKinsey. (2022). Digital trust: Why it matters for busi- nesses | McKinsey. https://www.mckinsey.com/capabili- Heikkilä, J., & Penttinen, K. (2016). Overview of Enterprise ties/quantumblack/our-insights/why-digital-trust-truly-mat- Architecture work in 15 countries. (details) ters (details) Mayakul, T., Sa-Nga-Ngam, P., Srisawat, W., & Kiattisin, 67. Di Maio, G., Seitzl, L., & Unterweger, D. F. (2020). Strength- S. (2019). A Comparison of National Enterprise Architec- ening the Governance of Skills Systems: Lessons from ture and e-Government Perspectives. 2019 4th Technol- Six OECD Countries. https://www.alexandria.unisg.ch/ ogy Innovation Management and Engineering Science handle/20.500.14171/112292 (details) International Conference (TIMES-ICON), 1–6. https://doi. org/10.1109/TIMES-iCON47539.2019.9024591 (details) 68. Secretary of Education of Guanajuato. (2021). RIMA - Reco- pilación de información para la mejora de los aprendiza- 80 ◄ Table of Contents jes: Informe de resultados evaluación de Matemática y tivity, inclusivity, and adaptability. World Bank Publications. Español RIMA2020 y 2021 [Results report of the evalua- https://policycommons.net/artifacts/1286108/the-skills-bal- tion of Mathematics and Spanish RIMA 2020 and 2021]. ancing-act-in-sub-saharan-africa/1883972/ (details) (details) 75. Picciano, A. G. (2012). The evolution of big data and learn- Secretary of Education of Guanajuato. (2021). RIMA - Reco- ing analytics in American higher education. Journal of pilación de información para la mejora de los aprendizajes: Asynchronous Learning Networks, 16(3), 9–20. https://eric. Manual del proceso (aplicación y devolución de resultados). ed.gov/?id=EJ982669 (details) [RIMA - Collection of information for improving learning: Process manual (application and return of results]. (details) Penuel, W. R., & Gallagher, D. J. (2017). Creating Research Practice Partnerships in Education. ERIC. https://eric. 69. Kaufman, D. M., Geyer, H. L., Milstein, M. J., & Rosengard, ed.gov/?id=ED573803 (details) J. (2022). Kaufman’s Clinical Neurology for Psychiatrists - E-Book. Elsevier Health Sciences. (details) Ramalingam, B., Barnett, I., Levy, A., Oppenheimer, C., Valters, C., Whittle, D., & Wild, L. (2017). Bridging Real-Time 70. Koomar, S., & Blest, H. (2020). Using EdTech to Support Data and Adaptive Management. USAID. Effective Data Monitoring: A Curated Resource List. EdTech Hub. https://docs.edtechhub.org/lib/FS4CMYUB (details) 76. Gustafsson-Wright, E., Osborne, S., & Massey, M. (2020). Do impact bonds affect the ecosystem of social ser- Gustafsson-Wright, E., Hadani, H. S., Hirsh-Pasek, K., vices delivery and financing? https://policycommons. Jalbout, M., King, E. M., ODonoghue, J. L., Olsen, B., net/artifacts/4141410/do-impact-bonds-affect-the-ecosys- Shapiro, J., Vegas, E., & Winthrop, R. (2022). Global edu- tem-of-social-services-delivery-and-financing/4949753/ cation trends and research to follow in 2022. https:// (details) policycommons.net/artifacts/4142385/global-educa- tion-trends-and-research-to-follow-in-2022/4951073/ 77. Piper, B., Destefano, J., Kinyanjui, E. M., & Ong’ele, S. (2018). (details) Scaling up successfully: Lessons from Kenya’s Tusome national literacy program. Journal of Educational Change, Gustafsson-Wright, E., Osborne, S., & Shankar, A. (2022). 19(3), 293–321. https://doi.org/10.1007/s10833-018-9325-4 From Evidence to Scale: Lessons Learned from the Quality (details) Education India Development Impact Bond. Center for Uni- versal Education at The Brookings Institution. https://eric. 78. Group, W. B. (2014). The World Bank Group A to Z 2015. ed.gov/?id=ED626233 (details) World Bank Publications. (details) 71. Zhao, J., Yuping, W., Maideen, I., Moe, Z. K., & Nasirudeen, 79. USAID (2018). Discussion Note: Adaptive Management. A. M. A. (2018). The Relationship between Smartphone https://usaidlearninglab.org/library/discussion-note-adap- Use and Academic Performance in a Sample of Tertiary tive-management Students in Singapore: A Cross-Sectional Study. Journal of Educational Technology, 14(4), 28–35. https://eric.ed.gov 80. Gustafsson-Wright, E., Osborne, S., & Aggarwal, M. (2022, /?id=EJ1179530 (details) November 2). Digital tools for real-time data collection in education. Brookings. https://policycommons.net/ 72. Gustafsson-Wright, E., Osborne, S., & Aggarwal, M. artifacts/4142237/digital-tools-for-real-time-data-collec- (2022). Digital tools for real-time data collection in edu- tion-in-education/4951284/ (details) cation. https://policycommons.net/artifacts/4142237/ digital-tools-for-real-time-data-collection-in-educa- 81. Evgeniou, T., Hardoon, D. R., & Ovchinnikov, A. (2020, tion/4951284/ (details) August 13). What Happens When AI is Used to Set Grades? Harvard Business Review, 13. https://hbr.org/2020/08/ 73. Tomkinson, E. (2016). Does Outcomes-Based Reporting what-happens-when-ai-is-used-to-set-grades (details) Contribute to or Contradict the Realisation of Social Out- comes? In J. Butcher & D. Gilchrist, The Three Sector Solu- 82. Hinds, D. 2019. “Realizing the Potential of Technology tion: Delivering public policy in collaboration with not-for- in Education: A Strategy for Education Providers and profits and business (p. 185). ANU Press. https://press-files. the Technology Industry.” Accessed through https:// anu.edu.au/downloads/press/n1949/pdf/ch09.pdf (details) www.semanticscholar.org/paper/Realising-the-poten- tial-of-technology-in-education%3A-Hinds/d6c979294f- 74. Arias, O., Evans, D. K., & Santos, I. (2019). The skills balanc- 1cd41f294508abc694a91ba9732a60 ing act in Sub-Saharan Africa: Investing in skills for produc- 81 ◄ Table of Contents 83. Trust, T., Krutka, D. G., & Carpenter, J. P. (2016). “Together and jobs. Brookings. https://www.brookings.edu/research/ we are better”: Professional learning networks for teach- going-digital-how-learning-and-employment-records- ers. Computers & Education, 102, 15–34. https://doi. shape-access-to-quality-education-and-jobs/ (details) org/10.1016/j.compedu.2016.06.007 (details) 92. European Training Foundation. (2020). Public-private part- 84. Cilliers, J., Fleischz, B., Kotze, J., Mohohlwanex, N., Taylor, nerships for skills development: a governance perspec- S., & Thulare, T. (2020). Can Virtual Replace In-person tive. Volume II, Case studies. Publications Office. https:// Coaching? Experimental Evidence on Teacher Profes- data.europa.eu/doi/10.2816/371790 (details) sional Development and Student Learning in South Africa. https://riseprogramme.org/sites/default/files/inline-files/ 93. Institute for Future of Work. (n.d.). Home - Institute for Virtual%20vs%20InPerson%20Coaching%20Working%20 Future of Work. Retrieved March 1, 2024, from https://ifow. Paper.pdf (details) ac.za/ (details) 85. Hassinen, S. (2023). Guidelines, recommendations and 94. Hinds, D. 2019. “Realizing the Potential of Technology tool for non-formal learning methods. Quaderni Di Comu- in Education: A Strategy for Education Providers and nità: Persone, Educazione e Welfare Nella Società 5.0: 1, the Technology Industry.” Accessed through https:// 2023, 273–284. https://www.torrossa.com/gs/resource- www.semanticscholar.org/paper/Realising-the-poten- Proxy?an=5521735&publisher=F35884 (details) tial-of-technology-in-education%3A-Hinds/d6c979294f- 1cd41f294508abc694a91ba9732a60 86. Kucirkova, N. I., Campbell, J., & Cermakova, A. L. (2023). EdTech Impact Evaluation Frameworks: 95. Aithal, P. S., & Rao, P. (2016). Green education concepts & Summary 2023. WikIT. https://static1.squarespace. strategies in higher education model. International Journal com/static/62fc80ec4c86a26330d18835/t/651aafa- of Scientific Research and Modern Education (IJSRME), 362c10a6513e376b3/1696247723463/EdTech+Im- ISSN (Online), 2455-563. Retrieved from https://www. pact+Evaluation+Frameworks_WiKIT+2023.pdf (details) researchgate.net/publication/306031451 87. https://www.brighteyevc.com/post/the-evolution-of-ed- Broberg, N. (2023). Understanding the competencies tech-activity-in-private-and-public-markets needed for innovation in greening and digitalizing econ- omies: Insights from existing literature. OECD Edu- 88. https://www.brighteyevc.com/post/the-evolution-of-ed- cation Working Papers. OECD Publishing. https://doi. tech-activity-in-private-and-public-markets org/10.5267/j.uscm.2024.5.026 89. Jordan, K., David, R., Phillips, T., & Pellini, A. (2021). Edu- McKenzie, M., & Benavot, A. (2024). Climate change and cation during the COVID-19: crisis Opportunities and sustainability in science and social science secondary constraints of using EdTech in low-income countries. school curricula. UNESCO. https://unesdoc.unesco.org/ Revista de Educación a Distancia (RED), 21(65). https://doi. ark:/48223/pf0000390036 org/10.6018/red.453621 (details) Muench, S., Stoermer, E., Jensen, K., Asikainen, T., Salvi, 90. Kucirkova, N. I., Campbell, J., & Cermakova, A. L. M., & Scapolo, F. (2022). Towards a green and digital (2023). EdTech Impact Evaluation Frameworks: future, EUR 31075 EN, Publications Office of the European Summary 2023. WikIT. https://static1.squarespace. Union, Luxembourg. com/static/62fc80ec4c86a26330d18835/t/651aafa- 362c10a6513e376b3/1696247723463/EdTech+Im- OECD. (2023). OECD Skills Outlook 2023: Skills for a resil- pact+Evaluation+Frameworks_WiKIT+2023.pdf (details) ient green and digital transition. OECD Publishing. https:// doi.org/10.1787/27452f29-en Kucirkova, N., Brod, G., & Gaab, N. (2023). Applying the science of learning to EdTech evidence evaluations Redecker, C. (2017). European framework for the digital using the EdTech Evidence Evaluation Routine (EVER). competence of educators: DigCompEdu. Publications Npj Science of Learning, 8(1), 35. https://doi.org/10.1038/ Office of the European Union. https://op.europa.eu/en/ s41539-023-00186-7. (details) publication-detail/-/publication/fcc33b68-d581-11e7-a5b9- 01aa75ed71a1/language-en 91. Milman, A. G., Allyson Parco, Rohan Carter-Rau, Jessa Henderson, Kazumi Homma, Ani Meliksetyan, and Natalie. Sabarwal, S., Venegas Marin, S., Spivack, M., & Ambasz, (2022, December 19). Going digital: How learning and D. (2024). Learning to propel climate action. World Bank. employment records shape access to quality education Retrieved from http://creativecommons.org/licenses/by/3.0/igo 82 ◄ Table of Contents Staring, F., & Schleicher, A. (2023). Education and Innova- 26(1), 87–122. https://doi.org/10.1007/s12528-013-9077-3 tion for the Digital and Green Transitions: How Higher Edu- (details) cation Can Support Teachers and School Leaders. OECD Education Policy Perspectives. No. 82. OECD Publishing. 101. Cheung, A. C. K., & Slavin, R. E. (2013). Effects of Educa- tional Technology Applications on Reading Outcomes Sukumar, R. (2019). Green technology in education and for Struggling Readers: A Best-Evidence Synthesis. sustainable development. Journal of Emerging Technol- Reading Research Quarterly, 48(3), 277–299. https://doi. ogies and Innovative Research (JETIR). Retrieved from org/10.1002/rrq.50 (details) https://www.jetir.org/papers/JETIREW06022.pdf 102. Cheung, A. C. K., & Slavin, R. E. (2013). Effects of Educa- UNESCO. (2024a). Greening curriculum guidance: Teach- tional Technology Applications on Reading Outcomes ing and learning for climate action. UNESCO. https://doi. for Struggling Readers: A Best-Evidence Synthesis. org/10.54675/AOOZ1758 Reading Research Quarterly, 48(3), 277–299. https://doi. org/10.1002/rrq.50 (details) UNESCO. (2024b). Green school quality standard: Green- ing every learning environment. Retrieved from https:// 103. Sosa, G. W., Berger, D. E., Saw, A. T., & Mary, J. C. (2011). www.unesco.org/en/articles/green-school-quality-stand- Effectiveness of Computer-Assisted Instruction in Statis- ard-greening-every-learning-environment tics: A Meta-Analysis. Review of Educational Research, 81(1), 97–128. https://doi.org/10.3102/0034654310378174 World Bank (2023). Green digital transformation: How to (details); cited in: Bernard, R., Borokhovski, E., Schmid, R., & sustainably close the digital divide and harness digital Tamim, R. (2018). Gauging the Effectiveness of Educational tools for climate action. Climate Change and Development Technology Integration in Education: What the Best-Qual- Series. Washington, DC: World Bank. Retrieved from http:// ity Meta-Analyses Tell Us. In Learning, design, and tech- hdl.handle.net/10986/40653 nology (pp. 1–25). https://doi.org/10.1007/978-3-319-17727- 4_109-2 (details) 96. Kucirkova, N., Brod, G., & Gaab, N. (2023). Applying the science of learning to EdTech evidence evaluations 104. Takacs, Z. K., Swart, E. K., & Bus, A. G. (2015). Benefits and using the EdTech Evidence Evaluation Routine (EVER). Pitfalls of Multimedia and Interactive Features in Tech- Npj Science of Learning, 8(1), 35. https://doi.org/10.1038/ nology-Enhanced Storybooks: A Meta-Analysis. Review s41539-023-00186-7. (details) of Educational Research, 85(4), 698–739. https://doi. org/10.3102/0034654314566989 (details) Frau-Meigs, D., & Hibbard, L. (2016). Education 3.0 and Internet Governance: A new global alliance for children and 105. Cheung, A. C. K., & Slavin, R. E. (2013). Effects of Educa- young people’s sustainable digital development. https:// tional Technology Applications on Reading Outcomes www.cigionline.org/sites/default/files/gcig_no27web_0. for Struggling Readers: A Best-Evidence Synthesis. pdf (details) Reading Research Quarterly, 48(3), 277–299. https://doi. org/10.1002/rrq.50 (details) 97. Power, T., Gater, R., Grant, C., & Winters, N. (2014). Educa- tional Technology Topic Guide. Health and Education Advice 106. Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., Resource Team. http://oro.open.ac.uk/41070/ (details) & Abrami, P. C. (2014). A meta-analysis of blended learning and technology use in higher education: from the general to 98. Hollow, D., Muyoya, C., & Brugha, M. (2016). Education the applied. Journal of Computing in Higher Education, 26(1), Technology Map: Guidance Document. Jigsaw Consult. 87–122. https://doi.org/10.1007/s12528-013-9077-3 (details) (details) Cheung, A. C. K., & Slavin, R. E. (2013). Effects of Educa- 99. Olsen, B. (2023). Government Decision-Making on Educa- tional Technology Applications on Reading Outcomes tion in Low- and Middle-Income Countries: Understanding for Struggling Readers: A Best-Evidence Synthesis. the Fit among Innovation, Scaling Strategy, and Broader Reading Research Quarterly, 48(3), 277–299. https://doi. Environment. https://www.brookings.edu/wp-content/ org/10.1002/rrq.50 (details) uploads/2023/08/Olsen-2023.pdf (details) Takacs, Z. K., Swart, E. K., & Bus, A. G. (2015). Benefits and 100. Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., Pitfalls of Multimedia and Interactive Features in Tech- & Abrami, P. C. (2014). A meta-analysis of blended learning nology-Enhanced Storybooks: A Meta-Analysis. Review and technology use in higher education: from the general of Educational Research, 85(4), 698–739. https://doi. to the applied. Journal of Computing in Higher Education, org/10.3102/0034654314566989 (details) 83 ◄ Table of Contents 107. Vegas, E., Ziegler, L., & Zerbino, N. (2019, November 20). org/10.1177/0267658316637401 (details) How ed-tech can help leapfrog progress in education. Center for Universal Education at The Brookings Institu- Wolf, R., Cook, M., Reid, A., Neitzel, A., Ross, S., & Risman, tion. https://eric.ed.gov/?id=ED602936 (details) K. (2021). An Evaluation of the Literacy-Infused Science Using Technology Innovation Opportunity (LISTO) i3 Eval- 108. Power, T., Gater, R., Grant, C., & Winters, N. (2014). Edu- uation (Valid 45) Final Report. Grantee Submission. https:// cational Technology Topic Guide. Health and Education eric.ed.gov/?id=ED613901 (details) Advice Resource Team. http://oro.open.ac.uk/41070/ (details) 116. Adil, F., Nazir, R., & Akhtar, M. (2021). Investigating the Impact on Learning Outcomes Through the Use of EdTech Rodriguez-Segura, D. (2020). EdTech in Developing Coun- During Covid-19: Evidence from an RCT in the Punjab prov- tries: A Review of the Evidence. World Bank Research ince of Pakistan. EdTech Hub. (details) Observer, 37(2), 171–203. https://doi.org/10.1093/wbro/ lkab011 (details) Hodor, R., Owusu, E. A., Ofori-Davis, L., Afram, A., & Sefa- Nyarko, C. (2021). Voices and Evidence from End-Users Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. of the GLTV and GLRRP Remote Learning Programme (2017). Education Technology: An Evidence-Based Review in Ghana: Insights for inclusive policy and programming. (w23744; p. w23744). National Bureau of Economic EdTech Hub. (details) Research. https://doi.org/10.3386/w23744 (details) Islam, T., Hussain, M., Shimul, S. N., Rupok, R. I., & Orthy, S. 109. Unwin, T., Naseem, A., Pawluczuk, A., Shareef, M., Spies- R. K. (2021). Integration of Technology in Education for Mar- berger, P., West, P., & Yoo, C. (2020). Education for the most ginalised Children in an Urban Slum of Dhaka City During marginalised post-COVID-19: Guidance for governments the Covid-19 Pandemic. EdTech Hub. (details) on the use of digital technologies in education – Act one. UNESCO, UniTwin, EdTech Hub. (details) Tembey, L., Baier, J., Ogolla, C., & Mohan, P. (2021). Under- standing Barriers to Girls’ Access and Use of EdTech in 110. UNESCO. (2015). Education 2030: Incheon Declaration and Kenya During Covid-19. EdTech Hub. (details) Framework for Action Towards inclusive and equitable quality education and lifelong learning for all. UNESCO IITE. https:// 117. Rohs, M., & Ganz, M. (2015). MOOCs and the claim of edu- iite.unesco.org/publications/education-2030-incheon-dec- cation for all: A disillusion by empirical data. International laration-framework-action-towards-inclusive-equitable-qual- Review of Research in Open and Distance Learning, 16(6). ity-education-lifelong-learning/ (details) https://doi.org/10.19173/irrodl.v16i6.2033. (details) Available under Copyright International Review of Research in Open 111. Unwin, T., Naseem, A., Pawluczuk, A., Shareef, M., Spies- and Distance Learning Nov 2015. berger, P., West, P., & Yoo, C. (2020). Education for the most marginalised post-COVID-19: Guidance for governments 118. Adam, T. (2022). Digital Literacy Needs for Online Learn- on the use of digital technologies in education – Act one. ing Among Peri-Urban, Marginalised Youth in South Africa. UNESCO, UniTwin, EdTech Hub. (details) International Journal of Mobile and Blended Learning (IJMBL), 14(3), 1–19. https://doi.org/10.4018/IJMBL.310940. 112. Adam, T. (2022). Digital Literacy Needs for Online Learn- (details) Available under Access limited to members. ing Among Peri-Urban, Marginalised Youth in South Africa. International Journal of Mobile and Blended Learning 119. Adam, T. (2020). Addressing Injustices through MOOCs: A (IJMBL), 14(3), 1–19. https://doi.org/10.4018/IJMBL.310940. study among peri-urban, marginalised, South African youth (details) Available under Access limited to members. [Thesis, University of Cambridge]. https://doi.org/10.17863/ CAM.56608. (details) Available under Attribution-Non- 113. A4AI. (2022). Meaningful Connectivity — unlocking the full Commercial-ShareAlike 4.0 International. power of internet access. Alliance for Affordable Internet. https://a4ai.org/meaningful-connectivity/ (details) 120. McBurnie, C., Adam, T., Kaye, T., & Haßler, B. (2020). Zero-Rating Educational Content in Low- and Middle-In- 114. Warschauer, M. (2003). Technology and social inclusion: come Countries. EdTech Hub. (details) rethinking the digital divide. MIT Press. (details) 121. Adil, F., Nazir, R., & Akhtar, M. (2021). Investigating the 115. Conklin, K., & Pellicer-Sánchez, A. (2016). Using eye-track- Impact on Learning Outcomes Through the Use of EdTech ing in applied linguistics and second language research. During Covid-19: Evidence from an RCT in the Punjab prov- Second Language Research, 32(3), 453–467. https://doi. ince of Pakistan. EdTech Hub. (details) 84 ◄ Table of Contents Nicolai, S., Jefferies, K., & Lockhart, A. S. (2022). EdTech file/GIZ%20InDesign-Vorlage%20fu%CC%88r%20Pub- Evidence From Covid-19 Response: A thematic review of likationen%20%E2%80%93%20DIN%20A4%20hoch.pdf primary research from Bangladesh, Ghana, Kenya, Paki- (details) stan and Sierra Leone. EdTech Hub. (details) Mitchell, J., Ashlee, A., Clericetti, G., Gladwell, J., & Tor- 122. Coflan, C. M., & Kaye, T. (2020). Using education technol- rance, R. (2020). Education in Emergencies: A Rapid Evi- ogy to support learners with special educational needs and dence Review. EdTech Hub. (details) disabilities in low- and middle-income countries. Zenodo. (details) USAID. (2020). Delivering Distance Learning in Emergen- cies. https://www.edu-links.org/sites/default/files/media/ Lynch, P., Singal, N., & Francis, G. A. (2022). Educational file/DELIVERING%20DISTANCE%20LEARNING%20IN%20 technology for learners with disabilities in primary school EMERGENCIES.pdf (details) settings in low- and middle-income countries: a systematic literature review. Educational Review, 0(0), 1–27. https:// 129. Oded, Y. (2019) University of the People: changing the doi.org/10.1080/00131911.2022.2035685 (details) world, one student at a time. In Fair Planet.org https:// www.fairplanet.org/story/university-of-the-people-chang- 123. Lynch, P., Singal, N., & Francis, G. A. (2022). Educational ing-the-world-one-student-at-a-time/ technology for learners with disabilities in primary school settings in low- and middle-income countries: a systematic Kalman, Y. M. (2010). How Low Can You Go? The Tui- literature review. Educational Review, 0(0), 1–27. https:// tion-Free Business Model of the University of the People. In doi.org/10.1080/00131911.2022.2035685 (details) Chais conference on instructional technologies research, Ra’anana, Israel. 124. Lynch, P., Singal, N., & Francis, G. A. (2022). Educational technology for learners with disabilities in primary school 130. World Bank, Bill and Melinda Gates Foundation, FCDO, settings in low- and middle-income countries: a systematic UNESCO, UNICEF, & USAID. (2022). Guide for Learn- literature review. Educational Review, 0(0), 1–27. https:// ing Recovery and Acceleration: Using the RAPID Frame- doi.org/10.1080/00131911.2022.2035685 (details) work to Address COVID-19 Learning Losses and Build Forward Better. https://thedocs.worldbank.org/en/doc/ 125. Hersh, M., & Mouroutsou, S. (2019). Learning technology e52f55322528903b27f1b7e61238e416-0200022022/ and disability—Overcoming barriers to inclusion: Evidence related/Guide-for-Learning-Recovery-and-Accelera- from a multicountry study. British Journal of Educational tion-06-23.pdf (details) Technology, 50(6), 3329–3344. https://doi.org/10.1111/ bjet.12737 (details) Yaniawati, R. P., Kartasasmita, B. G., Kariadinata, R., & Sari, E. (2017). Accelerated learning method using edmodo to 126. Lynch, P., Singal, N., & Francis, G. A. (2022). Educational increase students’ mathematical connection and self-regu- technology for learners with disabilities in primary school lated learning. Proceedings of the 2017 International Con- settings in low- and middle-income countries: a systematic ference on Education and Multimedia Technology- ICEMT literature review. Educational Review, 0(0), 1–27. https:// ’17, 53–57. https://doi.org/10.1145/3124116.3124128 (details) doi.org/10.1080/00131911.2022.2035685 (details) 131. Akbıyık, C., & Şimsek, N. (2009). Accelerated Learning in 127. World Bank. (2021). A Landscape Review of ICT for Disabil- Classroom and Computer Environments. 21. (details) ity-Inclusive Education. https://openknowledge.worldbank. org/server/api/core/bitstreams/ef7fd554-8359-50f9-9718- 132. Damani, K. (2020). Accelerated Learning and EdTech: A 7161545dbc54/content (details) Rapid Evidence Review. EdTech Hub. (details) 128. Ashlee, A., Clericetti, G., & Mitchell, J. (2020). Refugee Edu- 133. Damani, K. (2020). Accelerated Learning and EdTech: A cation: A Rapid Evidence Review. EdTech Hub. (details) Rapid Evidence Review. EdTech Hub. (details) Islam, A., Wang, L. C., & Hassan, H. (2022). Delivering 134. Damani, K., & Mitchell, J. (2020). Radio: A Rapid Evidence Remote Learning Using a Low-Tech Solution: Evidence from Review. EdTech Hub. (details) an RCT during the Covid-19 Pandemic. EdTech Hub. (details) 135. McBurnie, C. (2020). Using Interactive Radio Instruction Dahya, N. (2016). Education in Conflict and Crisis: How to Mitigate the Educational Impact of Covid-19: A curated Can Technology Make a Difference? A Landscape Review. resource list. EdTech Hub. (details) GIZ. https://www.eccnetwork.net/sites/default/files/media/ 85 ◄ Table of Contents 136. Hodor, R., Owusu, E. A., Ofori-Davis, L., Afram, A., & Sefa- 145. Jordan, K., & Mitchell, J. (2020). Messaging Apps, SMS, Nyarko, C. (2021). Voices and Evidence from End-Users and Social Media: A Rapid Evidence Review. EdTech Hub. of the GLTV and GLRRP Remote Learning Programme (details) in Ghana: Insights for inclusive policy and programming. EdTech Hub. (details) 146. Angrist, N., Bergman, P., Brewster, C., & Matsheng, M. (2020). Stemming Learning Loss During the Pandemic: 137. Burde, D., Guven, O., Kelcey, J., Lahmann, H., & Al-Abbadi, A Rapid Randomized Trial of a Low-Tech Intervention in K. (2015). What Works to Promote Children’s Educational Botswana (SSRN Scholarly Paper 3663098). https://doi. Access, Quality of Learning, and Wellbeing in Crisis-Af- org/10.2139/ssrn.3663098 (details) fected Contexts. 93. (details) Jordan, K., & Mitchell, J. (2020). Messaging Apps, SMS, 138. Lamba, K., & Reimers, F. (2020). Sierra Leone and Liberia: and Social Media: A Rapid Evidence Review. EdTech Hub. Rising Academy Network on air (Education Continuity (details) during the Coronavirus Crisis). World Bank; OECD; Harvard Global Education Innovation Initiative; HundrED. (details) Pouezevara, S., & King, S. (2014). MobiLiteracy-Uganda program phase 1: Endline report. RTI International. https:// 139. Haßler, B., Megha-Bongnkar, G., Regis, C., & Blower, T. ierc-publicfiles.s3.amazonaws.com/public/resources/Mob- (2021). Final Report and Recommendations. Open Devel- iliteracy_Endline_Report_final_Rev_SUBMTITED_Jan%20 opment & Education. (details) 6%202014%281%29.pdf (details) 140. Islam, A., Wang, L. C., & Hassan, H. (2022). Delivering 147. Butgereit, L., Leonard, B., Le Roux, C., Rama, H., De Sousa, Remote Learning Using a Low-Tech Solution: Evidence M., & Naidoo, T. (2010). Dr Math gets MUDDY: the “dirt” on from an RCT during the Covid-19 Pandemic. EdTech Hub. how to attract teenagers to Mathematics and Science by (details) using multi-user dungeon games over Mxit on cell phones. The authors. https://researchspace.csir.co.za/dspace/ 141. Afoakwah, E., Carballo, F., Caro, A., D’Cunha, S., Dobrow- handle/10204/4085 (details) olski, S., & Fallon, A. (2021). Dialling up Learning: Testing the Impact of Delivering Educational Content via Interac- 148. Budree, A., & Hendriks, T. (2019). Instant Messaging Tutor- tive Voice Response to Students and Teachers in Ghana. ing: A Case of South Africa. 2019 9th International Con- EdTech Hub. (details) ference on Cloud Computing, Data Science & Engineering (Confluence), 615–619. https://doi.org/10.1109/CONFLU- 142. Watson, J., & McIntyre, N. (2020). Educational Television: A ENCE.2019.8776928 (details) Rapid Evidence Review. EdTech Hub. (details) 149. Jordan, K., & Mitchell, J. (2020). Messaging Apps, SMS, 143. Borzekowski, D. L. G., Singpurwalla, D., Mehrotra, D., and Social Media: A Rapid Evidence Review. EdTech Hub. & Howard, D. (2019). The impact of Galli Galli Sim Sim (details) on Indian preschoolers. Journal of Applied Develop- mental Psychology, 64, 101054. https://doi.org/10.1016/j. 150. Boujikian, M., Carter, A., & Jordan, K. (2022). The Sandbox appdev.2019.101054 (details) Model: A Novel Approach to Iterating while Implementing an Emergency Education Program in Lebanon during the Moland, N. A. (2019). Can Big Bird Fight Terrorism?: Chil- COVID-19 Pandemic. Journal on Education in Emergen- dren’s Television and Globalized Multicultural Education cies, 8(3), 215. https://doi.org/10.33682/rj45-k7z7 (details) (Illustrated edition). Oxford University Press. (details) 151. Gon, S., & Rawekar, A. (2017). Effectivity of E-Learning Watson, J. (2020, March 31). Learning through television through Whatsapp as a Teaching Learning Tool. MVP in low-income contexts: mitigating the impact of coro- Journal of Medical Sciences, 19–25. https://doi.org/10.18311/ navirus (COVID-19). EdTech Hub. https://edtechhub. mvpjms.v4i1.8454. (details) Available under Copyright (c) org/2020/03/31/learning-through-television-in-low-in- 2017 Array. come-contexts-mitigating-the-impact-of-covid-19/ (details) 152. Jordan, K., & Mitchell, J. (2020). Messaging Apps, SMS, 144. Hodor, R., Owusu, E. A., Ofori-Davis, L., Afram, A., & Sefa- and Social Media: A Rapid Evidence Review. EdTech Hub. Nyarko, C. (2021). Voices and Evidence from End-Users (details) of the GLTV and GLRRP Remote Learning Programme in Ghana: Insights for inclusive policy and programming. Motteram, G., Dawson, S., & Al-Masri, N. (2020). WhatsApp EdTech Hub. (details) supported language teacher development: A case study in 86 ◄ Table of Contents the Zataari refugee camp. Education and Information Tech- 161. Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking nologies, 25(6), 5731–5751. https://doi.org/10.1007/s10639- cognitive engagement to active learning outcomes. Edu- 020-10233-0 (details) cational psychologist, 49(4), 219-243. 153. Safeguarding In Schools. (2022). Why schools shouldn’t Eugenijus, L. (2023). Integrating blended learning and use Whatsapp - Safeguarding In Schools. https://support. STEM education: Innovative approaches to promote inter- safeguardinginschools.co.uk/article/36-why-schools- disciplinary learning. Research and Advances in Educa- shouldnt-use-whatsapp (details) tion, 2(9), 20-36. 154. The World Bank. (2018). World Development Report 2018: Lee, H. Y., Lin, C. J., Wang, W. S., Chang, W. C., & Huang, Learning to realize education’s promise. (details) Y. M. (2023). Precision education via timely intervention in K-12 computer programming course to enhance program- 155. Unwin, T., Naseem, A., Pawluczuk, A., Shareef, M., Spies- ming skill and affective-domain learning objectives. Inter- berger, P., West, P., & Yoo, C. (2020). Education for the most national Journal of STEM Education, 10(1), 52. marginalised post‑COVID-19: Guidance for governments on the use of digital technologies in education ACT TWO Wu, T. T., Lee, H. Y., Wang, W. S., Lin, C. J., & Huang, Y. M. (OF THREE): FULL REPORT (p. 118). EdTech Hub, UNESCO, (2023). Leveraging computer vision for adaptive learning UniTwin. (details) in STEM education: Effect of engagement and self-efficacy. International Journal of Educational Technology in Higher World Bank. (2021). A Landscape Review of ICT for Disabil- Education, 20(1), 53. ity-Inclusive Education. https://openknowledge.worldbank. org/server/api/core/bitstreams/ef7fd554-8359-50f9-9718- 162. GEEAP. (2020). Cost-Effective-Approaches-to-Im- 7161545dbc54/content (details) prove-Global-Learning-What-Does-Recent-Evidence- Tell-Us-Are-Smart-Buys-for-Improving-Learning-in-Low- 156. IADB. (2019). The Future is Now: Transversal Skills in Latin and-Middle-Income-Countries.pdf. Global Education America and the Caribbean in the 21st Century. Inter-Amer- Evidence Advisory Panel. https://documents1.worldbank. ican Development Bank. https://publications.iadb.org/ org/curated/en/719211603835247448/pdf/Cost-Effec- publications/english/viewer/The-Future-is-Now-Transver- tive-Approaches-to-Improve-Global-Learning-What-Does- sal-Skills-in-Latin-America-and-the-Caribbean-in-the-21st- Recent-Evidence-Tell-Us-Are-Smart-Buys-for-Improv- Century.pdf (details) ing-Learning-in-Low-and-Middle-Income-Countries.pdf (details) 157. Yeung, K. L., Carpenter, S. K., & Corral, D. (2021). A Com- prehensive Review of Educational Technology on Objec- 163. Major, L., Francis, G. A., & Tsapali, M. (2021). The effective- tive Learning Outcomes in Academic Contexts. Educa- ness of technology-supported personalised learning in tional Psychology Review, 33(4), 1583–1630. https://doi. low- and middle-income countries: A meta-analysis. British org/10.1007/s10648-020-09592-4 (details) Journal of Educational Technology, 52(5), 1935–1964. https://doi.org/10.1111/bjet.13116 (details) 158. Power, T., Gater, R., Grant, C., & Winters, N. (2014). Edu- cational Technology Topic Guide. Health and Education 164. Zheng, L., Long, M., Zhong, L., & Gyasi, J. F. (2022). The Advice Resource Team. http://oro.open.ac.uk/41070/ effectiveness of technology-facilitated personalized learn- (details) ing on learning achievements and learning perceptions: a meta-analysis. Education and Information Technologies, 159. Bernard, R. M., Borokhovski, E., Schmid, R. F., Tamim, R. M., 27(8), 11807–11830. https://doi.org/10.1007/s10639-022- & Abrami, P. C. (2014). A meta-analysis of blended learning 11092-7 (details) and technology use in higher education: from the general to the applied. Journal of Computing in Higher Education, 165. Sun, S., Else-Quest, N. M., Hodges, L. C., French, A. M., & 26(1), 87–122. https://doi.org/10.1007/s12528-013-9077-3 Dowling, R. (2021). The Effects of ALEKS on Mathematics (details) Learning in K-12 and Higher Education: A Meta-Analysis. Investigations in Mathematics Learning, 13(3), 182–196. 160. Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). https://doi.org/10.1080/19477503.2021.1926194 (details) The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature. Teachers College 166. Weeks, J., & Baron, P. (2021). Exploring Math Education Record: The Voice of Scholarship in Education, 115(3), 1–47. Relations by Analyzing Large Data Sets II. Research Mem- https://doi.org/10.1177/016146811311500307 (details) orandum. ETS RM-21-02. Educational Testing Service. https://eric.ed.gov/?id=ED617566 (details) 87 ◄ Table of Contents 167. Cai, Z., Mao, P., Wang, D., He, J., Chen, X., & Fan, X. (2022). ment: A meta-analysis. Journal of Computer Assisted Effects of Scaffolding in Digital Game-Based Learning on Learning, 35(3), 407–420. https://doi.org/10.1111/jcal.12347 Student’s Achievement: a Three-Level Meta-analysis. Edu- (details) cational Psychology Review, 34(2), 537–574. https://doi. org/10.1007/s10648-021-09655-0 (details) 170. Bediou, B., Adams, D. M., Mayer, R. E., Tipton, E., Green, C. S., & Bavelier, D. (2018). Meta-analysis of action video 168. Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & game impact on perceptual, attentional, and cognitive Boyle, J. M. (2012). A systematic literature review of empir- skills. Psychological Bulletin, 144(1), 77–110. https://doi. ical evidence on computer games and serious games. org/10.1037/bul0000130 (details) Computers & Education, 59(2), 661–686. https://doi. org/10.1016/j.compedu.2012.03.004 (details) 171. Jordan, K., & Mitchell, J. (2020). Messaging Apps, SMS, and Social Media: A Rapid Evidence Review. EdTech Hub. Girard, C., Ecalle, J., & Magnan, A. (2013). Serious games (details) as new educational tools: how effective are they? A meta-analysis of recent studies: Serious games as educa- 172. Abdullah, M., Madain, A., & Jararweh, Y. (2022). ChatGPT: tional tools. Journal of Computer Assisted Learning, 29(3), Fundamentals, applications and social impacts. 2022 207–219. https://doi.org/10.1111/j.1365-2729.2012.00489.x Ninth International Conference on Social Networks Anal- (details) ysis, Management and Security (SNAMS), 1–8. https://doi. org/10.1109/SNAMS58071.2022.10062688 (details) Riopel, M., Nenciovici, L., Potvin, P., Chastenay, P., Char- land, P., Sarrasin, J. B., & Masson, S. (2019). Impact of 173. Steenbergen-Hu, S., & Cooper, H. (2013). A Meta-Anal- serious games on science learning achievement com- ysis of the Effectiveness of Intelligent Tutoring Systems pared with more conventional instruction: an overview on K–12 Students’ Mathematical Learning. 18. https://doi. and a meta-analysis. Studies in Science Education, 55(2), org/10.1037/a0032447 (details) 169–214. https://doi.org/10.1080/03057267.2019.1722420 (details) 174. Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of Intelli- gent Tutoring Systems: A Meta-Analytic Review. 37. https:// Wouters, P., Nimwegen, C. van, Oostendorp, H. van, & doi.org/10.3102/0034654315581420 (details) Spek, E. D. van der. (2013). A meta-analysis of the cogni- tive and motivational effects of serious games. Journal 175. Xu, Z., Wijekumar, K. (Kay), Ramirez, G., Hu, X., & Irey, R. of Educational Psychology, 105(2), 249–265. https://doi. (2019). The effectiveness of intelligent tutoring systems org/10.1037/a0031311 (details) on K-12 students’ reading comprehension: A meta-analy- sis. British Journal of Educational Technology, 50(6), 3119– 169. Benavides-Varela, S., Callegher, C. Z., Fagiolini, B., 3137. https://doi.org/10.1111/bjet.12758 (details) Leo, I., Altoè, G., & Lucangeli, D. (2020). Effectiveness of digital-based interventions for children with mathe- 176. Carlana, M., & Ferrara, E. (2021). Apart but Connected: matical learning difficulties: A meta-analysis. Comput- Online Tutoring and Student Outcomes during the COVID- ers & Education, 157, 103953. https://doi.org/10.1016/j. 19 Pandemic (HKS Faculty Research Working Paper compedu.2020.103953 (details) Series RWP21-001). https://www.hks.harvard.edu/publica- tions/apart-connected-online-tutoring-and-student-out- Byun, J., & Joung, E. (2018). Digital game-based learning comes-during-covid-19-pandemic#citation (details) for K-12 mathematics education: A meta-analysis. School Science and Mathematics, 118(3–4), 113–126. https://doi. Arriola, M., Gortazar, L., Hupkau, C., Pillado, Z., & Roldán, org/10.1111/ssm.12271 (details) T. (n.d.). A high-impact, on-line tutoring programme for dis- advantaged pupils in response to Covid-19. https://www. Ran, H., Kasli, M., & Secada, W. G. (2021). A Meta-Analysis esade.edu/ecpol/wp-content/uploads/2021/09/AAFF_ on Computer Technology Intervention Effects on Mathe- Menttores_PolicyReport_ENG_2021_compressed.pdf matics Achievement for Low-Performing Students in K-12 (details) Classrooms. Journal of Educational Computing Research, 59(1), 119–153. https://doi.org/10.1177/0735633120952063 Gortazar, L., Hupkau, C., & Roldán-Monés, A. (2024). Online (details) tutoring works: Experimental evidence from a program with vulnerable children. Journal of Public Economics, 232, Tokac, U., Novak, E., & Thompson, C. G. (2019). Effects of 105082. https://doi.org/10.1016/j.jpubeco.2024.105082 game-based learning on students’ mathematics achieve- (details) 88 ◄ Table of Contents 177. Angrist et al. 2020; (Stemming Learning Loss During the 186. Hennessy, S., D’Angelo, S., McIntyre, N., Koomar, S., Pandemic: A Rapid Randomized Trial of a Low-Tech Inter- Kreimeia, A., Cao, L., Brugha, M., & Zubairi, A. (2022). vention in Botswana Angrist et al., 2020) Technology Use for Teacher Professional Development in Low- and Middle-Income Countries: A systematic review. Angrist, N., Bergman, P., & Matsheng, M. (2022). Experi- Computers and Education Open. https://doi.org/10.1016/j. mental evidence on learning using low-tech when school caeo.2022.100080 (details) is out. Nature Human Behaviour, 6(7), 941–950. https://doi. org/10.1038/s41562-022-01381-z. (details) 187. Hennessy, S., D’Angelo, S., Koomar, S., Adam, T., Cao, L., & Haßler, B. (2023). Technology use in teacher preparation 178. Herrera Bohórquez, L. I., Largo Rodríguez, J. D., Viáfara and professional development in low- and middle-income González, J. J., Herrera Bohórquez, L. I., Largo Rodríguez, J. countries. Global Education Monitoring Report. (details) D., & Viáfara González, J. J. (2019). Online Peer-Tutoring: A Renewed Impetus for Autonomous English Learning. How, 188. Power, T., McCormick, R., & Asbeek-Brusse, E. (2017). A 26(2), 13–31. https://doi.org/10.19183/how.26.2.503 (details) Quasi-Experimental Study of the Classroom Practices of English Language Teachers and the English Language 179. Tan, S. C., Chung, S. H., Cheung, Y. L., Looi, C. K., & Wong, Proficiency of Students, in Primary and Secondary Schools W. H. (2020). Holistic Design of a Mobile Peer Tutoring in Bangladesh. http://www.eiabd.com/publications/ Application based on Learning and User Needs Analysis. research-publications/qe-qs-studies.html (details) 673–684. https://www.learntechlib.org/primary/p/218864/ (details) Shohel, M. M. C., & Kirkwood, A. (2012). Using technology for enhancing teaching andlearning in Bangladesh: challenges 180. Wu, M., & Gao, Q. (2020). Using Live Video Streaming in and consequences. Learning, Media and Technology, 37(4). Online Tutoring: Exploring Factors Affecting Social Inter- https://doi.org/10.1080/17439884.2012.671177 (details) action. International Journal of Human–Computer Interac- tion, 36(10), 964–977. https://doi.org/10.1080/10447318.201 189. Golding, J., & Batiibwe, M. S. K. (2021). A Design Approach 9.1706288 (details) to Mathematics Teacher Educator Development in East Africa. Journal of Research and Advances in Mathemat- 181. Youngblut, C. (1998). Educational Uses of Virtual Reality ics Education, 6(1). https://doi.org/10.23917/jramathedu. Technology. INSTITUTE FOR DEFENSE ANALYSES ALEX- v6i1.11898 (details) ANDRIA VA. https://apps.dtic.mil/sti/citations/ADA339438 (details) 190. Amevor, G., Bayaga, A., & Bosse, M. (2021). Analysis of Rural-Based Pre-Service Teachers Spatial-Visualisation 182. Downes, S. (2008). Places to go: Connectivism & connec- Skills in Problem Solving in Vector Calculus Using MATLAB. tive knowledge. Innovate: Journal of Online Education, International Journal of Emerging Technologies in Learning 5(1), 6. (details) (IJET), 16. https://doi.org/10.3991/ijet.v16i10.19269 (details) 183. Cormier, D. (2008). Rhizomatic Education: Community 191. Hennessy, S., D’Angelo, S., Koomar, S., Adam, T., Cao, L., & as Curriculum. Innovate: Journal of Online Education, 4. Haßler, B. (2023). Technology use in teacher preparation (details) and professional development in low- and middle-income countries. Global Education Monitoring Report. (details) 184. Adam, T. (2020). Open educational practices of MOOC designers: embodiment and epistemic location. Distance 192. Wang, Q., Chen, B., Fan, Y., & Zhang, G. (2018). MOOCs Education, 41(2), 171–185. https://doi.org/10.1080/01587919. as an alternative for teacher professional development: 2020.1757405 (details) examining learner persistence in one Chinese MOOC. Peking University. (details) Farrow, R. (2017). Open education and critical pedagogy. Learning, Media and Technology,42(2), 130–146. https:// 193. Wolfenden, F., Cross, S., & Henry, F. (2017). MOOC adap- doi.org/10.1080/17439884.2016.1113991 (details) tation and translation to improve equity in participation. Journal of Learning for Development, 4(2), 127–142. https:// 185. Hennessy, S., D’Angelo, S., McIntyre, N., Koomar, S., doi.org/10.56059/jl4d.v4i2.209 (details) Kreimeia, A., Cao, L., Brugha, M., & Zubairi, A. (2022). Technology Use for Teacher Professional Development in 194. Oyo, B., Kalema, B. M., & Byabazaire, J. (2017). MOOCs for Low- and Middle-Income Countries: A systematic review. in-service teachers: the case of Uganda and lessons for Computers and Education Open. https://doi.org/10.1016/j. Africa. Revista Española de Pedagogía, 75(266). https:// caeo.2022.100080 (details) doi.org/10.22550/rep75-1-2017-07 (details) 89 ◄ Table of Contents 195. Kennedy, E., & Laurillard, D. (2019). The potential of MOOCs cation program. Turkish Online Journal of Distance Educa- for large-scale teacher professional development in con- tion, 19(1). https://doi.org/10.17718/tojde.382663 (details) texts of mass displacement. London Review of Education, 17(2), 141–158. https://doi.org/10.18546/lre.17.2.04 (details) 204. Bett, H., & Makewa, L. (2020). Can Facebook groups enhance continuing professional development of 196. Castillo, N. M., Lee, J., Zahra, F. T., & Wagner, D. A. (2015). teachers? Lessons from Kenya. Asia-Pacific Journal of MOOCs for development: trends, challenges, and oppor- Teacher Education, 48(2). https://doi.org/10.1080/13598 tunities. Information Technologies & International Develop- 66x.2018.1542662 (details) ment, 11(2). (details) 205. Stewart, B. L. (2015). Cultivating a Personal Learning Liyanagunawardena, T., Williams, S., & Adams, A. (2013). Network that Leads to Professional Change. (details) The impact and reach of MOOCs: A developing countries’ perspective. ELearning Papers, 33, 38–46. (details) 206. Sunday, A. A. (2021). Use of Twitter Technology in Educa- tional Learning. The Turkish Online Journal of Educational 197. Inter-Agency Network for Education in Emergencies. Technology, 20(2). (details) (2022). Promising Practices in Teacher Professional Devel- opment (Teachers in Crisis Context). (details) 207. Ramazanoğlu, M., & Toytok, E. H. (2021). An Investigation of pre-service teachers’ motives for using Instagram: Pre-ser- 198. Onguko, B. B. (2014). JiFUNzeni: a blended learning vice teachers’ motives for using Instagram. International approach for sustainable teachers’ professional develop- Journal of Curriculum and Instruction, 13(1). (details) ment. The Electronic Journal of E-Learning, 12(1). (details) 208. Spiro, J. (2011). Guided interaction as intercultural learn- 199. Han, H., & Røkenes, F. M. (2020). Flipped Classroom in ing: designing internationalisation into a mixed delivery Teacher Education: A Scoping Review. Frontiers in Educa- teacher education programme. Higher Education Research tion, 5. https://doi.org/10.3389/feduc.2020.601593 (details) & Development, 30(5). https://doi.org/10.1080/07294360.2 011.598453 (details) 200. El-Serafy, Y., Adam, T., & Haßler, B. (2022). The effective- ness of technology-supported teacher professional learn- Üzüm, B., Akayoglu, S., & Yazan, B. (2020). Using telecol- ing communities in emergency settings. Future-Proofing laboration to promote intercultural competence in teacher Teacher Education: Voices from South Africa and Beyond. training classrooms in Turkey and the USA. ReCALL, 32(2). In S. Gravett & N. Petersen (Eds.), Future-Proofing Teacher https://doi.org/10.1017/S0958344019000235 (details) Education: Voices from South Africa and Beyond (1st ed.). Routledge. https://doi.org/10.4324/9781003185499 209. Moodley, M. (2019). WhatsApp: Creating a virtual teacher (details) community for supporting and monitoring after a profes- sional development programme. South African Journal of Hennessy, S., D’Angelo, S., Koomar, S., Adam, T., Cao, L., & Education, 39(2). https://doi.org/10.15700/saje.v39n2a1323 Haßler, B. (2023). Technology use in teacher preparation (details) and professional development in low- and middle-income countries. Global Education Monitoring Report. (details) 210. Nedungadi, P., Mulki, K., & Raman, R. (2018). Improving educational outcomes & reducing absenteeism at remote 201. Pota, V., Hennessy, S., Koomar, S., Kreimeia, A., Zubairi, A., villages with mobile technology and WhatsApp: findings Aerts, C., & Gault, C. (2021). Turning to technology: A global from rural India. Education and Information Technologies, survey of teachers’ responses to the Covid-19 pandemic. 23(1), 113–127. https://doi.org/10.1007/s10639-017-9588-z T4 Education & EdTech Hub. (details) (details) 202. Manca, S., & Ranieri, M. (2013). Is it a tool suitable for learn- 211. Habibi, A., Mukinin, A., Riyanto, Y., Prasohjo, L. D., Sulistiyo, ing? A critical review of the literature on Facebook as a U., Sofwan, M., & Saudagar, F. (2018). Building an online technology-enhanced learning environment. Journal of community: student teachers’ perceptions on the advan- Computer Assisted Learning, 29(6). https://doi.org/10.1111/ tages of using social networking services in a teacher edu- jcal.12007 (details) cation program. Turkish Online Journal of Distance Educa- tion, 19(1). https://doi.org/10.17718/tojde.382663 (details) 203. Habibi, A., Mukinin, A., Riyanto, Y., Prasohjo, L. D., Sulistiyo, U., Sofwan, M., & Saudagar, F. (2018). Building an online 212. Cilliers, J., Fleischz, B., Kotze, J., Mohohlwanex, N., Taylor, community: student teachers’ perceptions on the advan- S., & Thulare, T. (2020). Can Virtual Replace In-person tages of using social networking services in a teacher edu- Coaching? Experimental Evidence on Teacher Profes- 90 ◄ Table of Contents sional Development and Student Learning in South Africa. B., Hennessy, S., & Hofmann, R. (2018). Sustaining 221. Haßler, https://riseprogramme.org/sites/default/files/inline-files/ and Scaling Pedagogic Innovation in Sub-Saharan Africa: Virtual%20vs%20InPerson%20Coaching%20Working%20 Grounded Insights For Teacher Professional Develop- Paper.pdf (details) ment. Journal of Learning for Development, 5(1). https:// doi.org/10.56059/jl4d.v5i1.264 (details) Kotze, J., Taylor, S., & Fleisch, B. (2018). Moving towards cost-effective delivery models of teacher coaching: evi- 222. Batman, D., & Saka, A. Z. (2021). The Effects of Micro-Re- dence from field experiments in South Africa. (details) flective Teaching Practices on the Professional Skill Devel- opment of Pre-Service Physics Teachers. The Turkish 213. Bagby, E., Swift-Morgan, J., Niang, A., & Upadhyay, A. (2022). Online Journal of Educational Technology, 20(4). (details) Achieving Cost-Effective Instructional Coaching at Scale: Evidence from Senegal. Chemonics International. (details) 223. Lok, L., Schellings, G., Brouwer, N., & Brok, P. D. (2018). Investigating effects of using digital video in teacher train- 214. Bruns, B., Costa, L., & Cunha, N. (2018). Through the ing in Cambodia. Journal of Technology and Teacher Edu- looking glass: can classroom observation and coaching cation, 26(2). (details) improve teacher performance in Brazil? Economics of Education Review, 64, 214–250. https://doi.org/10.1016/j. 224. Inter-Agency Network for Education in Emergencies. econedurev.2018.03.003 (details) (2022). Promising Practices in Teacher Professional Devel- opment (Teachers in Crisis Context). (details) 215. Jukes, M. C., Turner, E. L., Dubeck, M. M., Halliday, K. E., Inyega, H. N., Wolf, S., Zuilkowski, S. S., & Brooker, S. J. 225. Inter-Agency Network for Education in Emergencies. (2017). Improving literacy instruction in Kenya through (2022). Promising Practices in Teacher Professional Devel- teacher professional development and text messages opment (Teachers in Crisis Context). (details) support: a cluster randomized trial. Journal of Research on Educational Effectiveness, 10(3). https://doi.org/10.1080/19 226. Fernández-Batanero, J.-M., Román-Graván, P., Reyes-Re- 345747.2016.1221487 (details) bollo, M.-M., & Montenegro-Rueda, M. (2021). Impact of Educational Technology on Teacher Stress and Anxiety: 216. Slade, T. S., Kipp, S., Cummings, S., Nyirongo, K., & A Literature Review. International Journal of Environmen- Pouezevara, S. (2018). Short message service (SMS)– tal Research and Public Health, 18(2), 548. https://doi. based remote support and teacher retention of training org/10.3390/ijerph18020548. (details) Available under gains in Malawi. In Cultivating dynamic educators: case http://creativecommons.org/licenses/by/3.0/. studies in teacher behavior change in Africa and Asia (pp. 131–167). RTI Press. (details) 227. Mitchell,J., & D’Rozario, J. (2022). Cost-Effective EdTech Paper 1: A position piece on how the sector can make pro- 217. Hennessy, S., D’Angelo, S., Koomar, S., Adam, T., Cao, L., & gress. EdTech Hub. https://docs.edtechhub.org/lib/RPR- Haßler, B. (2023). Technology use in teacher preparation 47JXT (details) and professional development in low- and middle-income countries. Global Education Monitoring Report. (details) 228. GEEAP. (2020). Cost-Effective-Approaches-to-Improve- Global-Learning-What-Does-Recent-Evidence-Tell-Us- 218. Piper, B., Oyanga, A., Mejia, J., & Pouezevara, S. (2017). Are-Smart-Buys-for-Improving-Learning-in-Low-and-Mid- Implementing large-scale instructional technology in dle-Income-Countries.pdf. Global Education Evidence Kenya: Changing instructional practice and developing Advisory Panel. https://documents1.worldbank.org/curated/ accountability in a National Education System. Interna- en/719211603835247448/pdf/Cost-Effective-Approaches- tional Journal of Education and Development Using ICT, to-Improve-Global-Learning-What-Does-Recent-Evidence- 13(3). (details) Tell-Us-Are-Smart-Buys-for-Improving-Learning-in-Low- and-Middle-Income-Countries.pdf (details) 219. Pouezevara, S., Brunette, T., Jordan, R., & Nakyejwe, D. (2019). Uganda Impact Study Report. RTI International. 229. Allende, C., Gallego, F., & Nielson, C. (2019). Approximating (details) the Equilibrium Effects of Informed School Choice. (details) 220. VVOB. (2021). Teacher motivation for online learning: pilot- 230. IPA.(2016, May 4). The Impact of Information Provision ing a microlearning support system | VVOB in Rwanda. on Human Capital Accumulation and Child Labor in Peru. VVOB. https://rwanda.vvob.org/news/teacher-motiva- Innovations for Poverty Action. https://www.poverty-action. tion-online-learning-piloting-microlearning-support-sys- org/study/information-about-returns-post-primary-educa- tem (details) tion-peru (details) 91 ◄ Table of Contents 231. Watson, J., & McIntyre, N. (2020). Educational Television: A Grounded Insights For Teacher Professional Develop- Rapid Evidence Review. EdTech Hub. (details) ment. Journal of Learning for Development, 5(1). https:// doi.org/10.56059/jl4d.v5i1.264 (details) 232. Wolff, L., Castro, C. de M., Navarro, J. C., & García, N. (2002). EDUCATION: EXPERIENCE OF MEXICO AND 243. Hennessy, S., D’Angelo, S., Koomar, S., Adam, T., Cao, L., & BRAZIL. Technologies for Education: Potentials, Parame- Haßler, B. (2023). Technology use in teacher preparation ters and Prospects, 144–152. http://woulibrary.wou.edu. and professional development in low- and middle-income my/weko/eed502/Chapter_10_Technologies_for_educa- countries. Global Education Monitoring Report. (details) tion.pdf (details) 244. Adam, T., El-Sefary, Y., Haßler, B., Khalayleh, A., Kremeia, A., 233. Batada, A., Banarjee, S., & Subramanian, M. (2016). Opening Proctor, J., & Mtebe, J. (2021). The Use of Technology in the new doors: Community engagement in India. In C. F. Cole CPD Implementation Plan in Tanzania. EdTech Hub. (details) & J. H. Lee (Eds.), The Sesame Effect: The Global Impact of the Longest Street in the World. Routledge. (details) 245. Cilliers,J., Fleischz, B., Kotze, J., Mohohlwanex, N., Taylor, S., & Thulare, T. (2020). Can Virtual Replace In-person 234. Watson, J., Hennessy, S., & Vignoles, A. (2020). The rela- Coaching? Experimental Evidence on Teacher Profes- tionship between educational television and mathematics sional Development and Student Learning in South Africa. capability in Tanzania. British Journal of Educational Tech- https://riseprogramme.org/sites/default/files/inline-files/ nology, 52(2). https://doi.org/10.1111/bjet.13047 (details) Virtual%20vs%20InPerson%20Coaching%20Working%20 Paper.pdf (details) 235. Watson, J., & McIntyre, N. (2020). Educational Television: A Rapid Evidence Review. EdTech Hub. (details) 246. Bagby, E., Swift-Morgan, J., Niang, A., & Upadhyay, A. (2022). Achieving Cost-Effective Instructional Coaching at Scale: 236. McGreal, R. (2019). A survey of OER implementations in 13 Evidence from Senegal. Chemonics International. (details) higher education institutions. The International Review of Research in Open and Distributed Learning, 20(5), 141–145. 247. Artuc,E., Cull, R., Dasgupta, S., Fattal, R., Filmer, D., Gine, https://doi.org/10.19173/irrodl.v20i5.4577 (details) X., Jacoby, H., Jolliffe, D., Kee, H. L., Klapper, L., Kraay, A., Loayza, N., Mckenzie, D., Ozler, B., Rao, V., Rijkers, B., 237. Blomgren, C. (2018). OER Awareness and Use: The Affin- Schmukler, S. L., Toman, M., Wagstaff, A., & Woolcock, M. ity Between Higher Education and K-12. The International (2020). Toward Successful Development Policies: Insights Review of Research in Open and Distributed Learning, from Research in Development Economics [Working 19(2). https://doi.org/10.19173/irrodl.v19i2.3431 (details) Paper]. World Bank. https://doi.org/10.1596/1813-9450- 9133. (details) Available under http://creativecommons. 238. Kabugo, D. (2020). Utilizing Open Education Resources to org/licenses/by/3.0/igo. Enhance Students’ Learning Outcomes during the COVID- 19 Schools Lockdown: A Case of Using Kolibri by Selected Crawfurd, L., Hares, S., & Sandefur, J. (2022). What has Government Schools in Uganda. Journal of Learning for Worked at Scale? In Schooling for All Feasible Strate- Development, 7(3), 447–458. https://doi.org/10.56059/jl4d. gies to Achieve Universal Education. Centre for Global v7i3.465 (details) Development. https://www.cgdev.org/sites/default/files/ schooling-for-all-feasible-strategies-universal-eduction. 239. Groeneveld, C., Kimenyi, E., & Kaye, T. (2021). SWOT Anal- pdf (details). Schaffner, J., Glewwe, P., & Sharma, U. (Eds.). ysis of the Kenya Education Cloud. EdTech Hub. (details) (2021). Why Programs Fail: Lessons for Improving Public Service Quality from a Mixed-Methods Evaluation of an 240. Chandra, S. (2020). Why the World Needs a Curriculum Unsuccessful Teacher Training Program in Nepal. https:// Alignment Hub. Learning Equality. (details) doi.org/10.22004/ag.econ.316663 (details) 241. Lambert, S. (2019). The Siyavula Case: Digital, Collabora- 248. Simpson, L., Carter, A., Rahman, A., & Plaut, D. (2021). Eight tive Text-Book Authoring to Address Educational Disad- Reasons Why EdTech Doesn’t Scale: How sandboxes are vantage and Resource Shortage in South African Schools. designed to counter the issue. EdTech Hub. (details) International Electronic Journal of Elementary Education, 11(3), 279–290. https://doi.org/10.26822/iejee.2019349252 249. Stern, J., Jukes, M., Piper, B., DeStefano, J., Mejia, J., (details) Dubeck, P., Carrol, B., Punjabi, M., Harris-Van Keuren, C., & Tufail, F. (2021). Learning at scale: interim report | Unesco 242. Haßler, B., Hennessy, S., & Hofmann, R. (2018). Sustaining IIEP Learning Portal. RTI. https://learningportal.iiep.unesco. and Scaling Pedagogic Innovation in Sub-Saharan Africa: org/en/library/learning-at-scale-interim-report (details) 92 ◄ Table of Contents 250. Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, H., 255. Reich, J., & Ruipérez-Valiente, J. A. (2019). The MOOC Mukerji, S., Shotland, M., & Walton, M. (2017). From Proof pivot. Science, 363(6423), 130–131. https://doi.org/10.1126/ of Concept to Scalable Policies: Challenges and Solutions, science.aav7958 (details) with an Application. Journal of Economic Perspectives, 31(4), 73–102. https://doi.org/10.1257/jep.31.4.73 (details). 256. Murray, S. (2019, March 4). Moocs struggle to lift rock-bot- Crouch, L., & DeStefano, J. (2017). Doing Reform Differ- tom completion rates. Financial Times. https://www.ft.com/ ently: Combining Rigor and Practicality in Implementation content/60e90be2-1a77-11e9-b191-175523b59d1d (details) and Evaluation of System Reforms. https://www.rti.org/ publication/doing-reform-differently (details). 257. Emanuel, E. J. (2013). MOOCs taken by educated few. Nature, 503(7476), 342–342. https://doi.org/10.1038/503342a. Gove, A., Korda Poole, M., & Piper, B. (2017). Designing (details) Available under 2013 Nature Publishing Group, for Scale: Reflections on Rolling Out Reading Improve- a division of Macmillan Publishers Limited. All Rights ment in Kenya and Liberia. New Directions for Child and Reserved.., Czerniewicz, L., Deacon, A., Small, J., & Walji, S. Adolescent Development, 2017(155), 77–95. https://doi. (2014). Developing world MOOCs: A curriculum view of the org/10.1002/cad.20195 (details) . MOOC landscape. Journal of Global Literacies, Technol- ogies, and Emerging Pedagogies, 2(3), 122–136. (details), Nicolai, S., Jordan, K., Adam, T., Myers, C., Jefferies, K., & Lockley, P. (2018). Open Initiatives for Decolonising the Lockhart, A. S. (in review). Toward a holistic approach to Curriculum. In Decolonising the University (pp. 145–173). EdTech effectiveness: Lessons from Covid-19 research in Pluto Press. (details) Bangladesh, Ghana, Kenya, Pakistan and Sierra Leone. International Journal of Educational Development. (details). 258. France, P. E. (2017). Is Standardization the Answer to Per- sonalization? Educational Leadership,74(6), 40–44. (details), Schaffner, J., Glewwe, P., & Sharma, U. (Eds.). (2021). Fisher, J. F. (2018, May 17). Is standardization the enemy Why Programs Fail: Lessons for Improving Public Service of personalization? Not necessarily. Christensen Institute. Quality from a Mixed-Methods Evaluation of an Unsuc- https://www.christenseninstitute.org/blog/is-standardiza- cessful Teacher Training Program in Nepal. https://doi. tion-the-enemy-of-personalization-not-necessarily/ (details) org/10.22004/ag.econ.316663 (details). 259. Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, H., 251. Stern, J., Jukes, M., Piper, B., DeStefano, J., Mejia, J., Mukherji, S., Shotland, M., & Walton, M. (2016). From Proof Dubeck, P., Carrol, B., Punjabi, M., Harris-Van Keuren, C., & of Concept to Scalable Policies: Challenges and Solu- Tufail, F. (2021). Learning at scale: interim report | Unesco tions, with an Application (Working Paper 22931). National IIEP Learning Portal. RTI. https://learningportal.iiep.unesco. Bureau of Economic Research. https://doi.org/10.3386/ org/en/library/learning-at-scale-interim-report (details). w22931 (details) Yarrow, N., Abbey, C., Shen, S., & Alyono, K. (2023). Using 260. Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, H., Education Technology to Improve K-12 Student Learning in Mukherji, S., Shotland, M., & Walton, M. (2016). From Proof East Asia Pacific: Promises and Limitations. https://elibrary. of Concept to Scalable Policies: Challenges and Solu- worldbank.org/doi/abs/10.1596/40508 tions, with an Application (Working Paper 22931). National Bureau of Economic Research. https://doi.org/10.3386/ 252. Stern, J., Jukes, M., Piper, B., DeStefano, J., Mejia, J., w22931 (details) Dubeck, P., Carrol, B., Punjabi, M., Harris-Van Keuren, C., & Tufail, F. (2021). Learning at scale: interim report | Unesco 261. Muralidharan, K., Singh, A., & Ganimian, A. J. (2019). Disrupt- IIEP Learning Portal. RTI. https://learningportal.iiep.unesco. ing Education? Experimental Evidence on Technology-Aided org/en/library/learning-at-scale-interim-report (details) Instruction in India. American Economic Review, 109(4), 1426–1460. https://doi.org/10.1257/aer.20171112 (details) 253. Fabregas, R. (2019). Broadcasting education: The long- term effects of Mexico’s telesecundarias. Working Paper. 262. Perera, M., & Aboal, D. (2019). The impact of a mathematics (details). Borghesan, E., & Vasey, G. (2021). The marginal computer-assisted learning platform on students’ mathe- returns to distance education: Evidence from mexico’s matics test scores. MERIT Working Papers. https://ideas. telesecundarias. (details). repec.org//p/unm/unumer/2019007.html (details) 254. Shah, D. (2021, December 14). A Decade of MOOCs: A 263. Perera, M., & Aboal, D. (2019). The impact of a mathematics Review of MOOC Stats and Trends in 2021. The Report by computer-assisted learning platform on students’ mathe- Class Central. https://www.classcentral.com/report/moocs- matics test scores. MERIT Working Papers. https://ideas. stats-and-trends-2021/ (details) repec.org//p/unm/unumer/2019007.html (details) 93 ◄ Table of Contents 264. Imagine WorldWide, H. A. (2022). Building Educational 275. Platform, I. L. (2023). EdTech Evidence Mid-Year Report. Foundations through Innovation & Technology (BEFIT) https://www.instructure.com/resources/research-reports/ Malawi Scale-Up Program Overview. https://www.imagine- edtech-evidence-2023-mid-year-report worldwide.org/wp-content/uploads/Malawi-Project-Over- view-General-11-16-22.pdf (details) 276. McBurnie, C., & Beoku-Betts, I. (2021). Mapping the educa- tion data ecosystem in Sierra Leone. EdTech Hub. (details) 265. Pitchford, N. (2023). Customised E-Learning Platforms. In T. Madon, A. J. Gadgil, R. Anderson, L. Casaburi, K. Lee, & Selwaness, I., Adam, T., Lawson, L., & Heady, L. (2022). A. Rezaee (Eds.), Introduction to Development Engineer- Guidance Note on Education Data Mapping in Sub-Saha- ing: A Framework with Applications from the Field (pp. ran Africa: Moving from theory to practice. EdTech Hub. 269–292). Springer International Publishing. https://doi. (details) org/10.1007/978-3-030-86065-3_11 (details) UNESCO. (2021). Strengthening education management 266. Groeneveld, C., Michels, G., & Kaye, T. (2022). Develop- information systems for increased resilience to crises: a ing a Proof of Concept for a Regional Learning Hub for synthesis of case studies - UNESCO Digital Library. https:// Eastern and Southern Africa Part 5: Final report. EdTech unesdoc.unesco.org/ark:/48223/pf0000378150 (details). Hub. (details) 277. UNESCO. (2021). Strengthening education management 267. Learning Passport. (n.d.). Learning Passport: Where We information systems for increased resilience to crises: a Work. UNICEF. Retrieved December 26, 2022, from https:// synthesis of case studies - UNESCO Digital Library. https:// www.learningpassport.org/where-we-work (details) unesdoc.unesco.org/ark:/48223/pf0000378150 (details) 268. Learning Passport. (n.d.). Learning Passport: Where We 278. Mwaniki, D. (2021). Strengthening Education Manage- Work. UNICEF. Retrieved December 26, 2022, from https:// ment Information Systems (EMIS) and Data for Increased www.learningpassport.org/where-we-work (details) Resilience to Crisis: country case study: South Soudan. UNESCO. https://unesdoc.unesco.org/ark:/48223/ 269. UNCEF ECARO. (2021). LearnIn Implementation Plan pf0000375815?posInSet=3&queryId=e42c738a-ce84-4fa 2020 - 2025. https://www.unicef.org/eca/media/19846/file 7-ba4a-dea123e05842 (details) (details) 279. Szucs, E. (2021). Strengthening Education Management 270. UNICEF Kosovo Office. (2021). Reimagine Education with Information Systems (EMIS) and Data for Increased Resil- the Learning Passport: The case of Kosovo. UNICEF. ience to Crisis: country case study: Ethopia - UNESCO https://www.unicef.org/kosovoprogramme/media/2511/ Digital Library. UNESCO. https://unesdoc.unesco.org/ file/Case%20Study%20Learning%20Passport.pdf (details) ark:/48223/pf0000375813?posInSet=7&queryId=5e6b08 72-9326-4b6a-8e57-2634e50b6fe1 (details) 271. UNICEF Kosovo Office. (2021). Reimagine Education with the Learning Passport: The case of Kosovo. UNICEF. 280. Szucs, E. (2021). Strengthening Education Management https://www.unicef.org/kosovoprogramme/media/2511/ Information Systems (EMIS) and Data for Increased Resil- file/Case%20Study%20Learning%20Passport.pdf (details) ience to Crisis: country case study: Ethopia - UNESCO Digital Library. UNESCO. https://unesdoc.unesco.org/ 272. Pritchett, L. (2015). Creating Education Systems Coher- ark:/48223/pf0000375813?posInSet=7&queryId=5e6b08 ent for Learning Outcomes: Making the Transition from 72-9326-4b6a-8e57-2634e50b6fe1 (details) Schooling to Learning (Research on Improving Systems of Education, p. 47). Rise Programme. (details) 281. Szucs, E. (2021). Strengthening Education Management Information Systems (EMIS) and Data for Increased Resil- 273. Gustafsson-Wright, E., Osborne, S., & Aggarwal, M. (2022, ience to Crisis: country case study: Ethopia - UNESCO November 2). Digital tools for real-time data collection Digital Library. UNESCO. https://unesdoc.unesco. in education. Brookings. https://policycommons.net/ org/ark:/48223/pf0000375813?posInSet=7&query- artifacts/4142237/digital-tools-for-real-time-data-collec- Id=5e6b0872-9326-4b6a-8e57-2634e50b6fe1(details) tion-in-education/4951284/ (details) 282. Alexander, B., Ashford-Rowe, K., Barajas-Murphy, N., 274. Pritchett, L. (2018). The Risks of Dangerous Dashboards Dobbin, G., Knott, J., McCormack, M., Pomerantz, J., Seil- in Basic Education. Research on Improving Systems hamer, R., & Weber, N. (2019). EDUCAUSE Horizon Report: of Education (RISE). https://doi.org/10.35489/BSG- 2019 Higher Education Edition. EDUCAISE. (details) RISE-RI_2018/006 (details) 94 ◄ Table of Contents 283. Larrabee Sønderlund, A., Hughes, E., & Smith, J. (2019). 291. Han, H. J. (2022). “How Dare They Peep into My Private The efficacy of learning analytics interventions in higher Life?” Human Rights Watch. https://www.hrw.org/ education: A systematic review. British Journal of Edu- report/2022/05/25/how-dare-they-peep-my-private-life/ cational Technology, 50(5), 2594–2618. https://doi. childrens-rights-violations-governments (details) org/10.1111/bjet.12720 (details), Namoun, A., & Alshanqiti, A. (2021). Predicting Student Performance Using Data 292. West, T. (2022, November 3). Children’s Privacy: An Evalu- Mining and Learning Analytics Techniques: A Systematic ation of EdTech Privacy Policies. (details) Literature Review. Applied Sciences, 11(1), 237. https://doi. org/10.3390/app11010237. (details) Available under http:// 293. Marlow, C. (2023). Digital Dystopia The Danger in Buying creativecommons.org/licenses/by/3.0/. What the EdTech Surveillance Industry is Selling. American Civil Liberties Union (ACLU). (details) 284. Ruiz-Calleja, A., Garcıa, S., Tammets, K., Aguerrebere, C., & Ley, T. (2019). Scaling Learning Analytics up to the national 294. Hillman, V. (forthcoming). Taming (ed)Tech: Why Chil- level: the experience from Estonia and Uruguay. (details) dren Stand to Lose in an Unregulated Digital Education. Bloomsbury Academic. (details) 285. Olubendi, D. (2019). A Framework For Evaluating The Outcome Of Use Of ICTOn Early Grade Literacy Assess- 295. Han, H. J. (2022). “How Dare They Peep into My Private ment; A Case Study Of Tusome Literacy Programme In Life?” Human Rights Watch. https://www.hrw.org/ Kenya [Thesis, University of Nairobi]. http://erepository. report/2022/05/25/how-dare-they-peep-my-private-life/ uonbi.ac.ke/handle/11295/109880. (details) Available childrens-rights-violations-governments (details) under Attribution-NonCommercial-NoDerivs 3.0 United States. 296. ICO (UK). (2022, October 17). 12. Profiling. https://ico.org. uk/for-organisations/guide-to-data-protection/ico-codes- 286. Olubendi, D. (2019). A Framework For Evaluating The of-practice/age-appropriate-design-a-code-of-prac- Outcome Of Use Of ICTOn Early Grade Literacy Assess- tice-for-online-services/12-profiling/ (details) ment; A Case Study Of Tusome Literacy Programme In Kenya [Thesis, University of Nairobi]. http://erepository. 297. OECD. (2020, April 3). Learning remotely when schools uonbi.ac.ke/handle/11295/109880. (details) Available close: How well are students and schools prepared? under Attribution-NonCommercial-NoDerivs 3.0 United Insights from PISA. OECD. https://www.oecd.org/corona- States. virus/policy-responses/learning-remotely-when-schools- close-how-well-are-students-and-schools-prepared-in- 287. Friedberg, A. (2022). Introducing EIDU’s Solver Platform: sights-from-pisa-3bfda1f7/ (details) Facilitating Open Collaboration in AI to Help Solve the Global Learning Crisis. In M. M. Rodrigo, N. Matsuda, A. I. 298. Gross, J. (2023). How Finland Is Teaching a Generation Cristea, & V. Dimitrova (Eds.), Artificial Intelligencein Edu- to Spot Misinformation. The New York Times. https://doi. cation. Posters and Late Breaking Results, Workshops and org/10.2307/jj.1357298 (details) Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium (pp. 104–108). Springer International 299. Doyle, C. (2021). The outcomes of sexting for children Publishing. https://doi.org/10.1007/978-3-031-11647-6_18 and adolescents: A systematic review of the literature. (details) Journal of Adolescence, 92(October), 86–113. https://doi. org/10.1016/j.adolescence.2021.08.009 (details). 288. Dial. (2019). SDG Digital Investment Framework A Whole-of-Government Approach to Investing in Digital 300. Heikkilä, H., Maalouf, W., & Campello, G. (2021). The United Technologies to Achieve the SDGs. https://www.itu.int/ Nations Office on Drugs and Crime’s Efforts to Strengthen dms_pub/itu-d/opb/str/D-STR-DIGITAL.02-2019-PDF-E.pdf a Culture of Prevention in Low- and Middle-Income (details) Countries. Prevention Science, 22(1), 18–28. https://doi. org/10.1007/s11121-020-01088-5 (details) 289. Adam, T., El-Sefary, Y., Haßler, B., Khalayleh, A., Kremeia, A., Proctor, J., & Mtebe, J. (2021). The Use of Technology 301. Korte, M. (2020). The impact of the digital revolution on in the CPD Implementation Plan in Tanzania. EdTech Hub. human brain and behavior: where do we stand? Dia- (details) logues in Clinical Neuroscience, 22(2), 101–111. https://doi. org/10.31887/DCNS.2020.22.2/mkorte (details) 290. Choudary, S. P., & Maruwada, S. (2022). The building block thesis. (details) 302. Barry, E. (2023, January 3). Social Media Use Is Linked to Brain Changes in Teens, Research Finds. The New York 95 ◄ Table of Contents Times. https://www.nytimes.com/2023/01/03/health/ Framework. International Journal of Artificial Intelligence social-media-brain-adolescents.html (details) in Education, 32(3), 504–526. https://doi.org/10.1007/ s40593-021-00239-1 (details) 303. Zhu, C., Huang, S., Evans, R., & Zhang, W. (2021). Cyberbul- lying Among Adolescents and Children: A Comprehensive 313. Hernandez-de-Menendez, M., Morales-Menendez, R., Review of the Global Situation, Risk Factors, and Preven- Escobar, C. A., & Arinez, J. (2021). Biometric applications tive Measures. Frontiers in Public Health, 9. https://doi. in education. International Journal on Interactive Design org/10.3389/fpubh.2021.634909 (details) and Manufacturing (IJIDeM), 15(2), 365–380. https://doi. org/10.1007/s12008-021-00760-6 (details) 304. World Economic Forum. (2022). Global Risks Report 2022. World Economic Forum. https://www.weforum.org/reports/ 314. Sarwar, M. B. (2022, April 21). Reading Audrey Watters: A global-risks-report-2022/in-full/ (details) reflection on personalised learning via education technol- ogy through a decolonial lens. EdTech Hub. https://edtech- 305. Sophos. (2022). The State of Ransomware in Education hub.org/2022/04/21/personalised-learning/ (details) 2022 – Sophos Partner News. Sophos Partner News July, 12. https://partnernews.sophos.com/en-us/2022/07/ 315. Mills, M., & Whittaker, M. (2019). Disability, Bias, and AI. AI resources/the-state-of-ransomware-in-education-2022/ Now Institute Report. (details) (details) 316. OECD. (2021). Development Co-operation Report 2021: 306. Kuykendall, B. K. (2022). New California Law Requires Shaping a Just Digital Transformation. Organisation for Schools to Report All Cyberattacks Impacting 500 or More Economic Co-operation and Development. https://doi. -. THE Journal. https://thejournal.com/articles/2022/10/06/ org/10.1787/ce08832f-en (details) new-california-law-requires-schools-to-report-all-cyber-in- cidents-impacting-500-pupils-or-staff.aspx. (details) OECD. (2021). Children in the digital environment: Revised typology of risks. OECD. https://doi.org/10.1787/9b- 307. Hasse, A. (2019). Youth and Artificial Intelligence.“Berkman 8f222e-en (details) Klein Center. https://cyber.harvard.edu/publication/2019/ youth-and-artificial-intelligence/where-we-stand (details) OECD. (2021). OECD Legal Instruments. https://legalinstru- ments.oecd.org/en/instruments/OECD-LEGAL-0389%20 308. Caines, A., & Silverman, S. (2021, December 10). Back (details) Doors, Trap Doors, and Fourth-Party Deals: How You End up with Harmful Academic Surveillance Technology on 317. Adam, T. (2019). Digital neocolonialism and massive open Your Campus without Even Knowing. The Journal of Inter- online courses (MOOCs): colonial pasts and neoliberal active Technology and Pedagogy. https://jitp.commons. futures. Learning, Media and Technology, 44(3), 365–380. gc.cuny.edu/?p (details) https://doi.org/10.1080/17439884.2019.1640740 (details) 309. Fernández-Batanero, J.-M., Román-Graván, P., Reyes-Re- 318. Czerniewicz, L. (2018). Unbundling and Rebundling Higher bollo, M.-M., & Montenegro-Rueda, M. (2021). Impact of Education in an Age of Inequality. https://er.educause.edu/ Educational Technology on Teacher Stress and Anxiety: articles/2018/10/unbundling-and-rebundling-higher-edu- A Literature Review. International Journal of Environmen- cation-in-an-age-of-inequality (details) tal Research and Public Health, 18(2), 548. https://doi. org/10.3390/ijerph18020548. (details) Available under 319. Williamson, B. (2022). Big EdTech. Learning, Media and http://creativecommons.org/licenses/by/3.0/. Technology, 47(2), 157–162. https://doi.org/10.1080/174398 84.2022.2063888 (details) 310. Trucano, M. (2023). AI and the Next Digital Divide in Educa- tion. (details) 320. Council of Europe. (2021). Children’s data protection in an education setting - Guidelines. Council of Europe 311. Holmes, W., & Tuomi, I. (2022). State of the art and practice Publishing. https://edoc.coe.int/en/children-and-the-inter- in AI in education. European Journal of Education, 57(4), net/9620-childrens-data-protection-in-an-education-set- 542–570. https://doi.org/10.1111/ejed.12533 (details) ting-guidelines.html (details) 312. Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, 321. ITU. (2020). Child Online Protection | ITU COP Guidelines. E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., https://www.itu-cop-guidelines.com/ (details) Cukurova, M., Bittencourt, I. I., & Koedinger, K. R. (2022). Ethics of AI in Education: Towards a Community-Wide 322. GPA. (n.d.). Adopted Resolutions – Global Privacy Assem- 96 ◄ Table of Contents bly. Retrieved January 6, 2023, from https://globalpriva- Bozkurt, A. (2020). Educational Technology Research Pat- cyassembly.org/document-archive/adopted-resolutions/ terns in the Realm of the Digital Knowledge Age. Journal (details) of Interactive Media in Education, 2020(1), 1–17. https://doi. org/10.5334/jime.570 (details) 323. OECD. (2021). Development Co-operation Report 2021: Shaping a Just Digital Transformation. Organisation for Platform, I. L. (2023). EdTech Evidence Mid-Year Report. Economic Co-operation and Development. https://doi. https://www.instructure.com/resources/research-reports/ org/10.1787/ce08832f-en (details) edtech-evidence-2023-mid-year-report#:~:text=This%20 report%20looks%20at%20the,%2D%20December%20 OECD. (2021). OECD Legal Instruments. https://legalinstru- 31%2C%202022 (details) ments.oecd.org/en/instruments/OECD-LEGAL-0389%20 (details) 331. Labun, J. (2023). Fixing Ed-Tech’s Investment Model. https://ssir.org/articles/entry/fixing_ed_techs_investment_ OECD. (2021). Children in the digital environment: Revised model. (details) typology of risks. OECD. https://doi.org/10.1787/9b- 8f222e-en (details) 332. Wylie, Louise. (2024). Education App Revenue and Usage Statistics. Business of Apps, accessed Jan 28, 2024. 324. Child Rights Impact Assessment - A tool to realise chil- dren’s rights in the digital environment. (2021). http://fra. 333. Shing, S., & Yuan, B. (2016). Apps Developed by Academ- europa.eu/en/content/child-rights-impact-assessment ics. Journal of Education and Practice, 7(33), 1–9. (details) (details) Hirsh-Pasek, K., Zosh, J. M., Golinkoff, R. M., Gray, J. H., 325. Child rights impact assessment. (2014, November 24). Euro- Robb, M. B., & Kaufman, J. (2015). Putting education in pean Union Agency for Fundamental Rights. https://digital- “educational” apps lessons from the science of learning. futurescommission.org.uk/wp-content/uploads/2021/03/ Psychological Science in the Public Interest, 16(1), 3-34. CRIA-Report.pdf (details) 334. Darling-Hammond, L., Zielezinski, M., & goldman, shelley. 326. Council of Europe. (2021). Children’s data protection in (2014). Using Technology to Support At-Risk Students’ an education setting - Guidelines. Council of Europe Learning. Alliance for Excellent Education. (details) Publishing. https://edoc.coe.int/en/children-and-the-inter- net/9620-childrens-data-protection-in-an-education-set- 335. Asian Development Bank. (2023). Reimagine Tech-Inclu- ting-guidelines.html (details) sive Education: Evidence, Practices, and Road Map. Asian Development Bank. https://doi.org/10.22617/TCS230233 327. The LEGO Group. (2018). Responsibility Report 2018. (details) (details) 336. Asian Development Bank. (2023). Reimagine Tech-Inclu- 328. Global Education’s Tech Reboot. (n.d.). Morgan Stanley. sive Education: Evidence, Practices, and Road Map. Asian Retrieved March 19, 2024, from https://www.morgan- Development Bank. https://doi.org/10.22617/TCS230233 stanley.com/ideas/education-system-technology-reboot (details) (details) 337. Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). 329. Adkins,S. S. (2020). The 2019 Global Learning Technol- The Effectiveness of Online and Blended Learning: A ogy Investment Patterns: Another Record Shattering Year. Meta-Analysis of the Empirical Literature. Teachers College https://users.neo.registeredsite.com/9/8/1/17460189/ Record: The Voice of Scholarship in Education, 115(3), 1–47. assets/Metaari-2019-Global-Learning-Technology-Invest- https://doi.org/10.1177/016146811311500307 (details) ment-Patterns.pdf (details) 338. McNiff, J. (2017). Action Research: All You Need to Know Labun, J. (2023). Fixing Ed-Tech’s Investment Model. (First edition). SAGE Publications Ltd. (details) https://ssir.org/articles/entry/fixing_ed_techs_investment_ model. (details) 339. American Institute for Research. (2018). Sequential, Multi- ple Assignment, Randomized Trial. https://optimizingout- 330. Allman,B., Kimmons, R., Rosenberg, J., & Dash, M. (2023). comes.air.org/innovative-designs.php (details) Trends and Topics in Educational Technology, 2023 Edition. Techtrends, 67(3), 583–591. https://doi.org/10.1007/s11528- 340. Crowley, K. (2016). Design-Based Research and Design-Based 023-00840-2 (details) Implementation Research | InformalScience.org. https:// 97 ◄ Table of Contents www.informalscience.org/news-views/design-based-re- 351. Ewens, W. J., & Grant, G. R. (2001). Computationally Inten- search-and-design-based-implementation-research sive Methods. In W. J. Ewens & G. R. Grant (Eds.), Statistical (details) Methods in Bioinformatics: An Introduction (pp. 349–363). Springer. https://doi.org/10.1007/978-1-4757-3247-4_12 341. Ubaidullah, A. (2016, June 10). Design Based Implemen- (details) tation Research (DBIR) for Sustaining Innovation in Class- room Instruction. Changing Classrooms - Challenges For 352. EEF. (2021). National Tutoring Programme: Nimble RCTs. Teachers & Teacher Educators. (details) EEF. https://educationendowmentfoundation.org.uk/ projects-and-evaluation/projects/national-tutoring-pro- 342. Samji, S., Andrews, M., Pritchett, L., & Woolcock, M. (2018). gramme-nimble-rcts (details) PDIA toolkit: A DIY Approach to Solving Complex Prob- lems. Center for International Development at Harvard 353. Zhan, R., Zhimei, R., Athey, S., & Zhou, Z. (2021). Policy University. https://bsc.cid.harvard.edu/files/bsc/files/pdia- Learning with Adaptively Collected Data. https://www. toolkit_ver_1_oct_2018.pdf (details) gsb.stanford.edu/faculty-research/working-papers/poli- cy-learning-adaptively-collected-data (details) 343. EdTech Hub. (2022). Sandbox Handbook V.2.0. EdTech Hub. (details) 354. Xiong, R., Athey, S., Imben, G. W., & Bayati, M. (2019). Optimal Experimental Design for Staggered Rollouts. Stan- 344. Boujikian, M., Carter, A., & Jordan, K. (2022). The Sandbox ford Graduate School of Business. https://www.gsb.stan- Model: A Novel Approach to Iterating while Implementing ford.edu/faculty-research/working-papers/optimal-experi- an Emergency Education Program in Lebanon during the mental-design-staggered-rollouts (details) COVID-19 Pandemic. Journal on Education in Emergen- cies, 8(3), 215. https://doi.org/10.33682/rj45-k7z7 (details) 355. Haßler, B., El-Serafy, Y., Khalayleh, A., & McBurnie, C. (2021). Syṡtemic Mixed-Methods Research — a conceptual 345. Principles of Digital Development. (2022). Design for Scale. framework for scalable EdTech research. EdTech Hub. Principles for Digital Development. https://digitalprinciples. (details) org/principle/design-for-scale/ (details) 356. Yeung, K. L., Carpenter, S. K., & Corral, D. (2021). A Com- 346. Simpson, L., Carter, A., Rahman, A., & Plaut, D. (2021). Eight prehensive Review of Educational Technology on Objec- Reasons Why EdTech Doesn’t Scale: How sandboxes are tive Learning Outcomes in Academic Contexts. Educa- designed to counter the issue. EdTech Hub. (details) tional Psychology Review, 33(4), 1583–1630. https://doi. org/10.1007/s10648-020-09592-4 (details) 347. Simpson, L., Carter, A., Rahman, A., & Plaut, D. (2021). Eight Reasons Why EdTech Doesn’t Scale: How sandboxes are 357. Kerwin, J. T., & Thornton, R. L. (2020). Making the Grade: designed to counter the issue. EdTech Hub. (details) The Sensitivity of Education Program Effectiveness to Input Choices and Outcome Measures. The Review of Principles of Digital Development. (2022). Design for Scale. Economics and Statistics, 1–45. https://doi.org/10.1162/ Principles for Digital Development. https://digitalprinciples. rest_a_00911 (details) org/principle/design-for-scale/ (details) 358. Plaut, D., Carter, A., Dixon, M., & Salami, T. (2020). EdTech 348. Nahum-Shani, I., Almirall, D., & Buckley, J. (2020). An Intro- Innovation for COVID-19: Insights from our global call for duction to Adaptive Interventions and SMART Designs in ideas. https://doi.org/10.5281/zenodo.4768571 (details) Education. (details) 359. Adam, T., Chuang, R., & Haßler, B. (2021). Structured Peda- 349. Haßler, B., El-Serafy, Y., Khalayleh, A., & McBurnie, C. gogy and EdTech. EdTech Hub. (details) (2021). Syṡtemic Mixed-Methods Research — a conceptual framework for scalable EdTech research. EdTech Hub. 360. Wilson-Grau, R. (2018). Outcome Harvesting: Principles, (details) Steps, and Evaluation Applications. (details) 350. Ewens, W. J., & Grant, G. R. (2001). Computationally Inten- 361. Wasserman, S., & Faust, K. (1994). Social Network Analysis. sive Methods. In W. J. Ewens & G. R. Grant (Eds.), Statistical https://doi.org/10.1017/CBO9780511815478 (details) Methods in Bioinformatics: An Introduction (pp. 349–363). Springer. https://doi.org/10.1007/978-1-4757-3247-4_12 362. Pasanen, T., & Barnett, I. (2019). Supporting adaptive man- (details) agement. (details) 98 ◄ Table of Contents 363. Whaites, A. (2017). The_Beginner_s_Guide_to_PEA. review of the literature. Eurasia Journal of Mathematics, pdf. UKAID. https://assets.publishing.service.gov.uk/ Science and Technology Education, 5(3), 235–245. https:// government/uploads/system/uploads/attachment_data/ doi.org/10.12973/ejmste/75275 (details) file/766478/The_Beginner_s_Guide_to_PEA.pdf (details) Freeman, A., Becker, S. A., & Cummins, M. (2017). NMC/ 364. Forman, S. G., Shapiro, E. S., Codding, R. S., Gonzales, J. CoSN horizon report: 2017 K. The New Media Consortium. E., Reddy, L. A., Rosenfield, S. A., Sanetti, L. M., & Stoiber, https://www.learntechlib.org/p/182003/ (details) K. C. (2013). Implementation science and school psychol- ogy. School Psychology Quarterly, 28(2), 77. https://doi. Omidyar Network. (2019). Scaling Access & Impact Real- org/10.1037/spq0000019 (details) izing the Power of EdTech. https://assets.imaginablefu- tures.com/media/documents/ON_Scaling_Access__ 365. Kerwin,J. T., & Thornton, R. L. (2020). Making the Grade: Impact_2019_85x11_Online.pdf (details) The Sensitivity of Education Program Effectiveness to Input Choices and Outcome Measures. The Review of 370. Resta, P., & Laferrière, T. (2015). Digital equity and intercul- Economics and Statistics, 1–45. https://doi.org/10.1162/ tural education. Education and Information Technologies, rest_a_00911 (details) 20(4), 743–756. https://doi.org/10.1007/s10639-015-9419-z (details) 366. Rajasekaran, S., & Casap, L. (2022). MOLDOVA – DIGITAL EDUCATION READINESS ASSESSMENT 2021- 371. Unwin, T., Naseem, A., Pawluczuk, A., Shareef, M., Spies- 22. World Bank Group. http://documents.worldbank. berger, P., West, P., & Yoo, C. (2020). Education for the most org/curated/en/099120006252220689/P17773104ea- marginalised post‑COVID-19: Guidance for governments 6f2040a88e02bdf9bbd04f6. (details) on the use of digital technologies in education ACT TWO (OF THREE): FULL REPORT (p. 118). EdTech Hub, UNESCO, Adam, T., McBurnie, C., & Haßler, B. (2020). Rolling Out UniTwin. (details) a National Virtual Learning Environment. EdTech Hub. (details) Unwin, T., Naseem, A., Pawluczuk, A., Shareef, M., Spies- berger, P., West, P., & Yoo, C. (2020). Education for the most Ganimian, A. J., Vegas, E., & Hess, F. M. (2020, Septem- marginalised post-COVID-19: Guidance for governments ber 10). Realizing the promise: How can education tech- on the use of digital technologies in education – Act one. nology improve learning for all? Brookings. https://www. UNESCO, UniTwin, EdTech Hub. (details) brookings.edu/essay/realizing-the-promise-how-can-edu- cation-technology-improve-learning-for-all/ (details) International Forum on AI and the Futures of Education, developing competencies for the AI Era, 7-UNESCO. 367. Cullinan,J., Flannery, D., Harold, J., Lyons, S., & Palcic, (2020). December 2020: synthesis report - UNESCO D. (2021). The disconnected: COVID-19 and disparities Digital Library. (n.d.). Retrieved December 27, 2022, from in access to quality broadband for higher education stu- https://unesdoc.unesco.org/ark:/48223/pf0000377251 dents. International Journal of Educational Technology in (details) Higher Education, 18(1), 26. https://doi.org/10.1186/s41239- 021-00262-1 (details) 372. Krumsvik, R. J. (2014). Teacher educators’ digital compe- tence. Scandinavian Journal of Educational Research, 368. Cullinan, J., Flannery, D., Harold, J., Lyons, S., & Palcic, 58(3), 269–280. https://doi.org/10.1080/00313831.2012.72 D. (2021). The disconnected: COVID-19 and disparities 6273 (details) in access to quality broadband for higher education stu- dents. International Journal of Educational Technology in 373. OECD. (2015). Does having digital skills really pay off? Higher Education, 18(1), 26. https://doi.org/10.1186/s41239- OECD. https://doi.org/10.1787/5js023r0wj9v-en (details) 021-00262-1 (details) 374. OECD (2019), Measuring the Digital Transformation: A 369. Godaert, E., Aesaert, K., Voogt, J., & Van Braak, J. (2022). Roadmap for the Future, OECD Publishing, Paris, https:// Assessment of students’ digital competences in primary doi.org/10.1787/9789264311992-en school: a systematic review. Education and Information Technologies, 27(7), 9953–10011. https://doi.org/10.1007/ 375. UNICEF, I. T. U. (2020). Two Thirds of the World’s School- s10639-022-11020-9 (details). Age Children Have No Internet Access at Home, New UNICEF-ITU Report Says. https://www.unicef.org/press-re- Bingimlas, K. A. (2009). Barriers to the successful inte- leases/two-thirds-worlds-school-age-children-have-no-in- gration of ICT in teaching and learning environments: A ternet-access-home-new-unicef-itu (details) 99 ◄ Table of Contents 376. Freitas, E., Fonseca, F., Garcia, V., Ferreira, R., & Gašević, Thinking. The Journal of Korean Association of Computer D. (2020). Towards a Maturity Model for Learning Analyt- Education, 23(3), 1–8. https://koreascience.kr/article/ ics Adoption An Overview of its Levels and Areas. 2020 JAKO202023852335493.page (details) IEEE 20th International Conference on Advanced Learn- ing Technologies (ICALT), 122–126. https://doi.org/10.1109/ 384. Ritzhaupt, A. D., & Hohlfeld, T. N. (2018). An examination of ICALT49669.2020.00059 (details) the digital divide and its dividing factors in formal educa- tional settings. Crossing the Bridge of the Digital Divide: A 377. General Data Protection Regulation (GDPR) – Official Legal Walk with Global Leaders, 19–36. (details) Text. Retrieved March 19, 2024, from https://gdpr-info.eu/ (details) 385. Minea-Pic, A. (2020), “Innovating teachers’ professional learning through digital technologies”, OECD Education 378. Lenovo. (2023, March 21). Lenovo Focuses on Digital Inclu- Working Papers, No. 237, OECD Publishing, Paris, https:// sion with Launch of Bilingual Keyboard. Lenovo StoryHub. doi.org/10.1787/3329fae9-en. https://news.lenovo.com/pressroom/press-releases/digi- tal-inclusion-maori-bilingual-keyboard/ (details) 386. Boeskens, L. and D. Nusche (2021), “Not enough hours in the day: Policies that shape teachers’ use of time”, OECD 379. Lenovo. (2023, March 21). Lenovo Focuses on Digital Inclu- Education Working Papers, No. 245, OECD Publishing, sion with Launch of Bilingual Keyboard. Lenovo StoryHub. Paris, https://doi.org/10.1787/15990b42-en. https://news.lenovo.com/pressroom/press-releases/digi- tal-inclusion-maori-bilingual-keyboard/ (details) 387. Russell, R., Kim, L., Hamilton, L., Lazovich, T., Harer, J., Ozdemir, O., Ellingwood, P., & McConley, M. (2018). Auto- 380. Salmi, J. (2020). COVID’s Lessons for Global Higher Edu- mated vulnerability detection in source code using deep cation: Coping with the Present While Building a More representation learning. 2018 17th IEEE International Con- Equitable Future. Lumina Foundation. https://eric.ed.gov ference on Machine Learning and Applications (ICMLA), /?id=ED611329 (details) 757–762. https://doi.org/10.1109/ICMLA.2018.00120 (details) 381. Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Students’ Use of and Engagement with ICT at 388. Bhanot, N., Ahuja, J., Kidwai, H. I., Nayan, A., & Bhatti, R. Home and School. In J. Fraillon, J. Ainley, W. Schulz, T. S. (2023). A sustainable economic revival plan for post- Friedman, & E. Gebhardt, Preparing for Life in a Digital Age COVID-19 using machine learning approach–a case study (pp. 125–166). Springer International Publishing. https://doi. in developing economy context. Benchmarking: An Inter- org/10.1007/978-3-319-14222-7_6 (details) national Journal, 30(6), 1782–1805. https://doi.org/10.1108/ BIJ-09-2021-0564 (details) Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age: The IEA Inter- 389. VERGER, A., & MOSCHETTI, M. (2017). Partnering with national Computer and Information Literacy Study interna- the private sector in the post-2015 era? The main politi- tional report. Springer Nature. https://library.oapen.org/bit- cal and social implications in the educational arena. Pro- stream/handle/20.500.12657/28001/1001996.pdf (details) gressive Lab for Sustainable Development, 245. https:// www.academia.edu/download/53505790/Verger___ 382. Tondeur, J., Van Braak, J., Ertmer, P. A., & Ottenbreit-Left- Moschetti_2017.pdf (details) wich, A. (2017). Understanding the relationship between teachers’ pedagogical beliefs and technology use in 390. Barrera-Osorio, F., & Linden, L. L. (2009). The Use And education: a systematic review of qualitative evidence. Misuse Of Computers In Education: Evidence From A Ran- Educational Technology Research and Development, domized Experiment In Colombia. The World Bank. https:// 65(3), 555–575. https://doi.org/10.1007/s11423-016-9481-2 doi.org/10.1596/1813-9450-4836 (details) (details) C., R., F., S., & J, M. Z. (2011). Impacto del Programa “Com- 383. OECD (2023), OECD Digital Education Outlook 2023: putadores para Educar” en la deserción estudiantil, el Towards an Effective Digital Education Ecosystem, logro escolar y el ingreso a la educación superior [Impact Chapter 7, OECD Publishing, Paris, https://doi.org/10.1787/ of the Programme Computers to Educate on Student c74f03de-en. Dropout, School Achievement and Admission to Higher Education. Serie Documentos CEDe, 15. https://repositorio. Jeon, S., Son, Y., & Park, S. (2020). Analysis of the ICILS uniandes.edu.co/bitstream/handle/1992/8254/dcede2011- 2018 Results by Korean Students’ Educational Experience 15.pdf?sequence=1&isAllowed=y (details) in Computer and Information Literacy and Computational 100 ◄ Table of Contents 391. Loonam, J., & O’Regan, N. (2022). Global value chains 399. Commission, D. F. (2023). A Blueprint for Education Data: and digital platforms: Implications for strategy. Strategic Realising Children’s Best Interests in Digitised Educa- Change, 31(1), 161-177. tion. https://digitalfuturescommission.org.uk/wp-content/ uploads/2023/03/A-Blueprint-for-Education-Data-FI- 392. Omidyar Network. (2019). Scaling Access & Impact Real- NAL-Online.pdf (details) izing the Power of EdTech. https://assets.imaginablefu- tures.com/media/documents/ON_Scaling_Access__ 400. Chui, M., & Sarakatsannis, J. (2015). Protecting Student Data Impact_2019_85x11_Online.pdf (details) in a Digital World. https://www.mckinsey.com/industries/ education/our-insights/protecting-student-data-in-a-digi- 393. Rajasekaran, S., & Casap, L. (2022). MOLDOVA – tal-world. (details) DIGITAL EDUCATION READINESS ASSESSMENT 2021- 22. World Bank Group. http://documents.worldbank. 401. OECD Directorate for Education and Skills. (2023). https:// org/curated/en/099120006252220689/P17773104ea- one.oecd.org/document/EDU/EDPC/SR(2023)2/en/pdf 6f2040a88e02bdf9bbd04f6. (details) (details) 394. Czerniewicz, L. (2018). Unbundling and Rebundling Higher 402. OECD Directorate for Education and Skills. (2023). https:// Education in an Age of Inequality. https://er.educause.edu/ one.oecd.org/document/EDU/EDPC/SR(2023)2/en/pdf articles/2018/10/unbundling-and-rebundling-higher-edu- (details) cation-in-an-age-of-inequality (details) 403. OECD Directorate for Education and Skills. (2023). https:// 395. Denford, J. (2016). Idea to Retire: Decentralized IT Govern- one.oecd.org/document/EDU/EDPC/SR(2023)2/en/pdf ance. (details) (details) 396. Arrieta-Ibarra, I., Goff, L., Jiménez-Hernández, D., Lanier, Olsen, B. (2023). Government Decision-Making on Educa- J., & Weyl, E. G. (2018). Should We Treat Data as Labor? tion in Low- and Middle-Income Countries: Understanding Moving beyond “Free.” AEA Papers and Proceedings, 108, the Fit among Innovation, Scaling Strategy, and Broader 38–42. https://doi.org/10.1257/pandp.20181003 (details) Environment. https://www.brookings.edu/wp-content/ uploads/2023/08/Olsen-2023.pdf (details) Posner, E. A., & Weyl, E. G. (2018). Radical Markets: Uprooting Capitalism and Democracy for a Just Society. 404. Olsen, B. (2023). Government Decision-Making on Educa- In Radical Markets. Princeton University Press. https://doi. tion in Low- and Middle-Income Countries: Understanding org/10.23943/9781400889457. (details) Available under the Fit among Innovation, Scaling Strategy, and Broader De Gruyter expressly reserves the right to use all content Environment. https://www.brookings.edu/wp-content/ for commercial text and data mining within the meaning of uploads/2023/08/Olsen-2023.pdf (details) Section 44b of the German Copyright Act.. 405. Haßler, B., & Khalayleh, A. (2019). Process for iterative and Zhu, F. (2019). Friends or foes? Examining platform owners’ adaptive implementation of government policy in order to entry into complementors’ spaces. Journal of Economics & utilise educational technology. (details) Management Strategy, 28(1), 23–28. https://doi.org/10.1111/ jems.12303. (details) 406. Adam, T., El-Sefary, Y., Haßler, B., Khalayleh, A., Kremeia, A., Proctor, J., & Mtebe, J. (2021). The Use of Technology 397. Stackpole, T. (2022). What Is Web3? Harvard Business in the CPD Implementation Plan in Tanzania. EdTech Hub. Review. https://hbr.org/2022/05/what-is-web3. (details) (details) 398. Kud, A. (2023). Decentralized Information Platforms in 407. Coflan, C., Wyss, N., Thinley, S., & Roland, M. (2022). Devel- Public Governance: Reconstruction of the Modern Democ- oping a national EdTech strategy. EdTech Hub. (details) racy or Comfort Blinding? International Journal of Public Administration, 46(3), 195–221. https://doi.org/10.1080/019 408. Khalayleh, A. (2021). Determining the suitability of Whatsapp 00692.2021.1993905 (details) for Jusoor Azima Project [Presentation]. EdTech Hub. https://doi.org/10.5281/zenodo.4783101. Available under Yan Chen, Jack I. Richter, & Pankaj C. Patel. (2020, April Creative Commons Attribution 4.0 International. (details) 30). Decentralized Governance of Digital Platforms - Yan Chen, Jack I. Richter, Pankaj C. Patel, 2021. https://journals. 409. gov.uk. (2016). Core principles of agile. GOV.UK. https:// sagepub.com/doi/10.1177/0149206320916755 (details) www.gov.uk/service-manual/agile-delivery/core-princi- ples-agile (details) 101 ◄ Table of Contents 410. OECD Directorate for Education and Skills. (2023). https:// one.oecd.org/document/EDU/EDPC/SR(2023)2/en/pdf (details) 411. Wilson, K., Gertz, B., Arenth, B., & Salisbury, N. (2014). The Journey to Scale: Moving Together Past Digital Health Pilots (p. 48). PATH. (details) 412. Principles of Digital Development. (2022). Design for Scale. Principles for Digital Development. https://digitalprinciples. org/principle/design-for-scale/ (details) 413. World Bank. (2018). World development report 2018: Learning to realize education’s promise. The World Bank. 414. Tungatarova, A., & van Cappelle, F. (2022). Pulse Check on Digital Learning. UNICEF. (details) 415. Haßler, B., & Khalayleh, A. (2019). Process for iterative and adaptive implementation of government policy in order to utilise educational technology. (details) 416. Bapna, A., Nicolai, S., Myers, C., Pellini, A., Sharma, N., & Wilson, S. (2021). A Case for a Systems Approach to EdTech. EdTech Hub. (details) 417. Al-Hindawi, H., & Hashem, L. (2020). Jordan’s EdTech response to COVID-19 [Case Study]. EdTech Hub. https:// doi.org/10.5281/zenodo.4706057 (details) Bapna, A., Nicolai, S., Myers, C., Pellini, A., Sharma, N., & Wilson, S. (2021). A Case for a Systems Approach to EdTech. EdTech Hub. (details) 418. Haßler, B., & Khalayleh, A. (2019). Process for iterative and adaptive implementation of government policy in order to utilise educational technology. (details) 419. Hawkins, R., Trucano, M., Cobo, C., Twinomugisha, A., & Ciarrusta, I. S. (n.d.). Technology and Innovation in Educa- tion at the World Bank. https://www.worldbank.org/en/topic/ edutech/publication/reimagining-human-connections-tech- nology-and-innovation-in-education-at-world-bank (details) Kickbusch, I., Piselli, D., Agrawal, A., Balicer, R., Banner, O., Adelhardt, M., Capobianco, E., Fabian, C., Gill, A. S., Lupton, D., Medhora, R. P., Ndili, N., Ryś, A., Sambuli, N., Settle, D., Swaminathan, S., Morales, J. V., Wolpert, M., Wyckoff, A. W., … Wong, B. L. H. (2021). The Lancet and Financial Times Commission on governing health futures 2030: growing up in a digital world. The Lancet, 398(10312), 1727–1776. https://doi.org/10.1016/S0140-6736(21)01824-9 (details) 420. UNICEF. (2022). Pulse Check on Digital Learning. https:// www.unicef.org/reports/pulse-check (details) 102 ◄ Table of Contents Annex 1 GLOSSARY Artificial Intelligence (AI) is the science of teaching tive and efficient use of information in enabling an machines to learn humanlike capabilities. Artificial organization to achieve its goals. It establishes the pro- General Intelligence (AGI) is the point at which an AI cesses and responsibilities that ensure the quality and can perform all human cognitive skills better than the security of the data used across a business or organ- smartest humans. Artificial Capable Intelligence (ACI) ization. Data governance defines who can take what can achieve a wide range of complex tasks but is not as action, upon what data, in what situations, and using far along as AGI. what methods. Blended Learning refers to a pedagogic approach that Data integrity refers to the consistency of data on the same involves a mixed modality of face-to-face teaching and variables collected from different sources. learning as well as distance learning through online, mobile, or other modes for teachers, trainers, and stu- Data quality refers to data adequacy, accuracy, relevance, dents. and explanatory capacity to inform decisions. B2B, B2C, and B2G: B2B refers to business to business, EdTech refers to the private market and innovation eco- B2C refers to business to consumer, and B2G refers system that develops commercial products, applica- to business to government. These are three common tions, and services for educational and skills develop- types of marketing strategies. ment stakeholders. These can include B2C, B2G, B2B products and services. Common data standards, terminologies, and structures on education data refer to specific provisions about Education interventions using digital solutions refers managing information, design, data collection, data to education policy, projects, and programmatic inter- verification, and archiving. ventions that employ digital solutions as part of the intervention design and/or business process redesign. Computational thinking refers to a set of problem-solv- These digital solutions may or may not employ com- ing methods that involve expressing problems and mercial EdTech products and services. It is important their solutions in ways that a computer could also to note that this places the education-related outcomes execute. It involves automation of processes and use and related interaction processes at the center rather of computing to explore, analyze, and understand pro- than the technology product/service at the center. cesses (natural and artificial). Interoperability Standards across Applications, Devices, Data specialist refers to a person with specialized com- and Equipment refers to developing and using data petencies in developing, monitoring, and maintaining standards that enable consistent and accurate collec- databases; monitoring performance; and interpret- tion and exchange of information across systems. This ing raw data and turning it into usable feedback and includes mechanisms for information exchange across applications. These are often statistics and computer applications, devices, and equipment that support the science experts who can manage the data pipeline and country’s health, education, and social protection ser- find information signals in the noise of big data. vice delivery. Data exchange refers to the process of sending and receiv- Digital competencies refer to the confident and critical ing data in such a manner that the information content use of the full range of digital technologies for infor- or meaning assigned to the data is not altered during mation, communication, and basic problem-solving in the transmission. all aspects of life. Data governance refers to a collection of processes, roles, Digital intensity refers to the degree to which different policies, standards, and metrics that ensure the effec- industry sectors have adopted digital technologies. 103 ◄ Table of Contents For example, the EU Digital Intensity Index (DII) is a Education technology (EdTech) refers to the combination composite indicator derived from the survey on ICT of ICT products and services to facilitate and enhance use and e-commerce in enterprises. Given changes in learning. technology, DII composition variability limits tempo- ral comparability. Enterprise architecture refers to a coherent, integrated ‘blueprint’ to optimize the often-fragmented legacy Digital pedagogy refers to pedagogy supporting the devel- of processes (both manual and automated) into an opment of digital skills and targeted and methodically integrated environment that supports service deliv- meaningful use of digital solutions, learning resources, ery while being responsive to changes. It provides a and content in teaching and learning. common vocabulary to discuss implementation across entities. This includes the overall design of a comput- Digital literacy refers to the ability to access, manage, ing system and the logical and physical interrelation- understand, integrate, communicate, evaluate, and ships between its components in reference to comput- create information safely and appropriately through ers, software, or networks. The architecture specifies digital technologies for employment, decent jobs, and the hardware, software, access methods, and protocols entrepreneurship. It includes competencies that are used throughout the system. variously referred to as computer literacy, ICT literacy, information literacy, and media literacy. Electronic data infrastructure refers to the structure and interaction of the major data types and sources, Digital skills refer to “…skills and competencies needed including logical and physical data assets and related to make use of digital technology and benefit from its data management resources. Data quality (adequacy, growing power and functionality…” encompassing “…a accuracy, relevance, explanatory capacity), data flows, range of different abilities, many of which are not only storage, protection, standards, interoperability, foun- ‘skills’ per se, but a combination of behaviors, expertise, dational data such as IDs, birth registries, and so on are know-how, work habits, character traits, dispositions, often important enabling factors. Electronic education and critical understandings… best understood as exist- records or any other data assets capture information ing on a graduated continuum from basic functional regarding a child/teacher/parent’s engagement with skills to a higher level, specialist skills.” the education system. (Basic) Digital skills refer to skills required to operate E-governance refers to the use of emerging ICT to facili- devices; find and use information; communicate and tate the processes of government and public adminis- collaborate; create basic content; understand and tration. It is about providing citizens with the ability apply safety, ethics, and privacy issues; and simple to choose the manner in which they wish to interact problem-solving and basic computational thinking. with their governments and the choices governments Related cognitive domain skills focus on remembering, make about how ICT will be deployed to support citi- understanding, and applying. Relevant for all citizens. zen choices. (Intermediate) Digital skills refer to basic digital skills, Integration refers to the process of linking independently media creation, problem-solving, computational designed applications to work together as one system thinking and coding, fundamentals of data analysis, AI, so that the data contained in each become part of a robotics, and Makerspace activities. It may also include larger, more comprehensive system that quickly and digital entrepreneurship skills, including starting and easily shares data when needed. Integration enables running an online business and using digital tools to access to data and functionality from such independ- run any business. Related cognitive domain skills focus ent applications through a single interface or service. on analysis, evaluation, and creation. Relevant for sec- ondary schools and HEIs. Interoperability refers to the ability to share information and services. Defining the degree to which the infor- (Advanced) Digital skills refer to skills needed by special- mation and services are to be shared is a useful archi- ists in ICT professions, such as computer program- tectural requirement, especially in a complex organi- ming and network management, including advancing zation and/or extended enterprise. This enhances the the digitalization of education systems. These include ability of organizations to interact toward mutually applications of advanced computational thinking, cut- beneficial goals by means of the exchange of data with ting-edge technologies like AI and big data, coding, other systems using common standards. Interoperabil- cybersecurity, the IoT, mobile app development, and ity also includes the ability of systems to provide and related transversal skills. receive services from other systems and to use the ser- 104 ◄ Table of Contents vices so interchanged to enable them to operate effec- tively together. Interoperability framework refers to an agreed approach of interoperability for entities that wish to work together toward the joint delivery of public services (without integrating all of their subsystems into one large system). Microservices architecture refers to a type of application architecture where the application is developed as a collection of services. It provides the framework for independently developing, deploying, and maintain- ing microservices architecture diagrams and services. Each microservice is a single service built to accommo- date an application feature and handle discrete tasks. Each microservice communicates with other services through simple interfaces to solve business problems. This allows for greater adaptability and flexibility and improves fault isolation. Mobile apps. The term ‘app’ has evolved to specifically connote software designed to reside on a mobile plat- form such as a tablet or mobile phone. It encompasses a user interface that interoperates with web-based resources that provide access to a wide array of infor- mation relevant to the app and local processing capa- bilities that collect, analyze, and format information in a manner best suited to the mobile platform. Addi- tionally, a mobile app provides persistent storage capa- bilities within the platform. Mobile apps are generally downloaded from application distribution platforms operated by the owner of the mobile operating system. Procedures refer to the steps that define the specific use of each system element or the procedural context in which the system resides. Socioemotional skills refer to noncognitive, soft, or char- acter skills, which are foundational human capabil- ities involved in achieving life goals, interacting with others, and managing emotions. Open Educational Resources (OER). OER are teaching, learning, and research materials intentionally cre- ated and licensed to be free for the end user to own, share, and, in most cases, modify. OERs are designed to reduce accessibility barriers by implementing best practices in teaching and to be adapted for local unique contexts. 105 ◄ Table of Contents Annex 2 SAMPLE GPT-4 OUTPUT This response was generated using the latest GPT-4 model with web browsing enabled on May 22, 2023. The model generated, consolidated, and improvised the response within 2–4 minutes. The output is not fully representative of country programs and their particularities or fully accurate. This is purely to be treated as an early demonstration example of the capability of large language AI models. PROMPT 1: The contextual conditions for the readiness of education systems for digital transitions relate to five pillars. This includes (i) strategic leadership and governance for citizen-centered service delivery, (ii) better and relevant data and technology infrastructure and management, (iii) digitally enabled education service delivery and analytics to inform education sector institutions in going digital, including up-to-date education programs, curricula, and inputs, (iv) education workforce capacity and culture, and (v) financing and innovation ecosystem for enabling val- ue-based and rights-based digital innovations in education to test, iterate, and scale. Can you provide at least 5–8 real and recent examples of actions and readiness in LIC, LMIC, MICs, and HICs in each of the above? Provide this in the form of a comprehensive table. Include sources and citations. Thank you. PROMPT 2: Regenerate the above tables by combining the responses. Do not draw three columns for LIC, LMIC, and HIC; instead, suffix the example with the terms LIC, LMIC, MIC, and HIC in brackets. RESPONSE: Absolutely. Here is the combined table, with examples of readiness and actions related to digital transitions in the education systems. The examples are suffixed with the type of the country (LIC, LMIC, MIC, HIC): Area of Readiness Examples Rwanda's Smart Rwanda 2020 Master Plan advances ICT in education (REB, 2020) (LIC). Viet Nam's Ministry of Education and Training launched the "Digital Transformation Program," embedding ICT in all education activities by 2025 (Vietnamnet, 2020) (LMIC). Strategic Leadership South Africa's Department of Basic Education (DBE) has an e-Education Strategy integrating and Governance digital technology into education (DBE, 2020) (MIC). Uganda's Digital Uganda Vision 2040 framework enhances digital infrastructure in rural areas (ICTA, 2020) (LIC). Singapore's ICT Masterplan promotes digital learning and equips students with digital literacy skills (MOE, 2020) (HIC). Brazil's NEP includes provisions for improving ICT infrastructure in schools (INEP, 2020) Data & Technology (MIC). Korea's National Information Society Agency (NIA)provides broadband access Infrastructure to all schools, supporting digitally enabled education (NIA, 2020) (HIC, EAP). Sweden's Broadband Strategy ensures fast and reliable internet access nationwide, benefiting schools (PTS, 2020) (HIC, Europe). Kenya's DLP delivers digital devices to primary schools and trains teachers in digital literacy (DLP, 2020) (LIC). India's National Digital Library offers students digital educational Digitally Enabled resources (NDL, 2020) (LMIC). Indonesia's "Merdeka Belajar" initiative leverages analytics Education Service to personalize learning experiences (Tempo.co, 2020) (LMIC). China's "Internet Plus" Delivery & Analytics initiative uses big data analytics to inform policy decisions and improve education service delivery (CGTN, 2020) (MIC). Canada's e-Learning Strategy promotes the use of technology and data to improve education outcomes (GOC, 2020) (HIC, Americas). 106 ◄ Table of Contents Area of Readiness Examples Burkina Faso's Can't Wait to Learn project uses digital tools to deliver quality education in conflict-affected areas, training teachers on their use (War Child Holland, 2020) (LIC). Education Workforce Pakistan's DigiSkills Training Program equips teachers with digital skills (DigiSkills, 2020) Capacity & Culture (LMIC). Thailand's "Equitable Education Fund" (EEF) provides teacher training to improve digital literacy and pedagogical skills (EEF, 2020) (MIC). Australia's Digital Literacy School Grants program supports teacher training in digital technologies (DESE, 2020) (HIC). Ethiopia's Digital Ethiopia 2025 strategy has secured international financing to improve digital innovation in education (World Bank, 2020) (LIC). The Philippines' DOST-SEI offers scholarships for innovation and ICT programs (DOST-SEI, 2020) (LMIC). Colombia's Financing and Ministry of Information Technologies and Communications (MINTIC) provides grants to Innovation Ecosystem support digital innovation in education (MINTIC, 2020) (MIC). UK's Innovate UK provides funding for digital innovation in education (UKRI, 2020) (HIC). References: 1. Rwanda Education Board (REB). (2020). Smart Rwanda 2020 Master Plan. 2. Vietnamnet. (2020). Ministry of Education and Training launches digital transformation program. 3. Department of Basic Education (DBE), South Africa. (2020). e-Education Strategy. 4. Information and Communication Technology Authority (ICTA), Uganda. (2020). Digital Uganda Vision 2040. 5. National Digital Library of India (NDL). (2020). NDL services. 6. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (INEP), Brazil. (2020). National Education Plan (NEP). 7. Digital Literacy Program (DLP), Kenya. (2020). Digital Learning Program. 8. Tempo.co. (2020). Merdeka Belajar. 9. CGTN. (2020). China’s Internet Plus Education Strategy. 10. War Child Holland. (2020). Can’t Wait to Learn Project. 11. DigiSkills Training Program, Pakistan. (2020). Digital skills for teachers. 12. Equitable Education Fund (EEF), Thailand. (2020). Teacher training program. 13. World Bank. (2020). Digital Ethiopia 2025. 14. Department of Science and Technology - Science Education Institute (DOST-SEI), Philippines. (2020). Scholarships for innovation and ICT. 15. Ministry of Information Technologies and Communications (MINTIC), Colombia. (2020). Grants for digital innovation in education. 16. Ministry of Education (MOE), Singapore. (2020). ICT Masterplan. 17. National Information Society Agency (NIA), South Korea. (2020). Broadband for Schools. 18. Post and Telecom Authority (PTS), Sweden. (2020). Broadband Strategy. 19. Government of Canada (GOC). (2020). e-Learning Strategy. 20. Department of Education, Skills and Employment (DESE), Australia. (2020). Digital Literacy School Grants. 21. UK Research and Innovation (UKRI). (2020). Innovate UK funding. 107 This work aims to offer a strategic approach to policymakers when undertaking digital transformation reforms in education and skills development systems, with a focus on “how.” It builds on the World Bank Education Vision framework offered in Realizing the Future of Learning by unpacking the digital cross-cutting area of “Invest wisely in technology”, looking into how this may be done to maximize impact at scale for all. We promote cautious optimism and techno realism, clari- fying how the education and skills sector can use digital technologies to its advantage by being proactive, strategic, and evidence-based, consid- ering carefully why to use digital and in what conditions the existing and emerging technologies might be positively leveraged. It is widely recog- nized that one size does not fit all and that policymakers need to have a laser focus on learning, weighing in contextual needs, and purpose- fully using the next marginal investable dollar on digital solutions to fulfill education objectives equitably at scale for all. Along with policy makers in government who are the primary audience for this work, it is intended to enable dialogue and critical partnerships across industry, academia, researchers and multilaterals, and World Bank staff to support and deepen our country engagements as countries increasingly expand the digital reach of public education services. Contact us at: rsubhashini@worldbank.org EdTech_Core_Team@worldbank.org © 2024 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org 108