Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series (#08) Edmond Gaible and Anush Shahverdyan 2016 saber.worldbank.org To cite this publication: Gaible, E. & Shahverdyan, A. 2016. Building and sustaining national ICT education agencies: Lessons from Armenia (NaCET). World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series (#08). Washington, DC: The World Bank. Available at: http://saber.worldbank.org Rights and Permissions This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) http://creativecommons.org/licenses/by/3.0/igo. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: Gaible, E. & Shahverdyan, A. 2016. Building and sustaining national ICT education agencies: Lessons from Armenia (NaCET). World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series (#08). Washington, DC: The World Bank. License: Creative Commons Attribution CC BY 3.0 IGO Translations—If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations—If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content—The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third-party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to re-use a component of the work, it is your responsibility to determine whether permission is needed for that re-use and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. © 2016 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington DC 20433 USA Telephone: +1-202-473-1000; Internet: www.worldbank.org Some rights reserved Disclaimer: The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. saber.worldbank.org 2 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) Table of Contents 1. Overview of NaCET .............................................................................................................................. 1 Governance, Funding, and Personnel ......................................................................................... 1 2. Primary Activities .................................................................................................................................. 3 ASIN ............................................................................................................................................ 3 Content Development and Distribution………………………………………………………….……. 4 Information Management ............................................................................................................ 4 Training and Outreach …............................................................................................................. 4 Mobile Internet Computer Station ............................................................................................... 5 Other Activities ............................................................................................................................ 5 3. Origins and Influences ......................................................................................................................... 6 Project Harmony and the Founding of NaCET ............................................................................ 6 PIU and ICT in Armenian Schools .............................................................................................. 7 NaCET Today ............................................................................................................................. 8 4. Issues and Questions ......................................................................................................................... 10 Hardware Replacement and Organizational Change ................................................................ 10 Observations and Lessons Learned .......................................................................................... 11 saber.worldbank.org 3 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) Acknowledgements The World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series explores a variety of topics and issues related to the use of information and communication technologies (ICTs) in the education sector. The Systems Approach for Better Education Results (SABER) initiative seeks to improve the global knowledge base related to education systems analyses, assessments, diagnoses, and opportunities for dialogue. SABER-ICT aims to improve the availability of policy-related data, information, and knowledge on what matters most in using ICTs to improve the quality of education. This publication series is made possible through generous support from the Government of Korea, most notably through the Korea - World Bank Partnership Facility. saber.worldbank.org 4 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) Executive summary The National Center for Education Technology (NaCET) has for nearly ten years served as the primary support for education technology in Armenian schools. NaCET manages the Armenian Schools Internet Network (ASIN), plus school information management, content development and dissemination, training in information technology, and development and dissemination of education content. NaCET was founded to meet urgent needs for technical capacity and contract management with the Ministry of Education and Science (MoES). In meeting these needs, NaCET has overseen the extension of Internet connectivity to more than 99 percent of schools as of 2013. This milestone is accompanied by new pressures—the need to begin the process of upgrading hardware—and by a new opportunity for broader assessment of the role of education technology in schools. saber.worldbank.org 5 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) 1. Overview of NaCET NaCET was established by an act of government in 2004, as part of the World Bank-funded Education Quality and Relevance Project to manage access to and distribution of Information and Communication Technologies (ICT) to Armenian schools. NaCET administers programs in the following areas: • Armenian Schools Internet Network (ASIN) • Armenian Education Portal • Education Management Information System (EMIS) for schools and students • Training in computer literacy for teachers and other school staff • Management of software licensing and vendor partnerships • Development and dissemination of electronic content for education Unlike some similar organizations in other countries, NaCET is not involved in the specification or procurement of computer or other hardware for schools, although the organization does participate in procurement processes for connectivity and software. Governance, Funding and Personnel NaCET receives its mandate from the Ministry of Education and Science (MoES) of Armenia. MoES allocates funds based on budget planning and proposed activities, issuing a new contract to NaCET each year. Annual budgets over the past five years are as follows: Year Budget Budget Percent AMD USD change 2008 344.436.500 $1,091,503 - 2009 557.680.200 $1,822,484 +60.0% 2010 697.951.300 $1,846,432 +01.3% 2011 678.029.380 $1,793,739 -03.9% 2012 698.532.200 $1,809,669 +<01.0% The NaCET Executive Body, comprising the organization’s director and Board of Directors, approves the annual NaCET action plan and staffing, reviews and approves financial information, and monitors NaCET activities. (Monitoring information is typically generated by NaCET and shared with the Executive Body.) saber.worldbank.org 1 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) NaCET employs a staff of 45, including 18 staff in areas that are primarily technical and administrative; eight staff are engaged in content development. saber.worldbank.org 2 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) 2. Primary Activities NaCET’s primary activity is management of the Armenia School Internet Network (ASIN), which provides secure, government-funded Internet connectivity to schools. Students and teachers generally access computers in their schools’ Internet Computer Centers (ICCs). ICCs are school computer labs—classrooms with computers, Internet connections and related tools, such as printers. NaCET also provides education content, supports school-focused and collaborative communications, builds capacity among school staff, collects and analyzes information from schools, and manages software licensing. ASIN NaCET is responsible for network administration, maintenance, and conducting upgrades to the network equipment as needed. As of October, 2012, ASIN connects 1,394 schools plus 50 access points at a maximum of 2 MBPS; more than 99 percent of Armenian secondary schools have Internet connectivity. (Approximately 230 schools have subscribed directly to private- sector Internet service; some of these schools maintain their separate service-agreements as well as using ASIN.) NaCET also manages contractual, operational and financial relations with the Internet Service Providers (ISPs) that deliver connectivity to schools. NaCET develops technical specifications that serve as the basis for tenders and contracts, conducts an annual open-bidding process among ISPs, receives bids, and selects ISPs to provide ASIN service. ASIN architecture routes all Internet traffic for Armenian schools through the ASIN central server. This configuration ensures that viruses and objectionable content are filtered. The configuration also supports collection of data on educational Internet use, central network monitoring and maintenance and provision of communications services such as email, FTP, and hosting of approximately 350 school websites. The ASIN central server is connected directly to Armenia’s fibre-optic network at 100 MBPS; the Armenian fibre-optic network is in turn connected through Georgia to the Trans-Asia- Europe fibre-optic backbone. ASIN marries this nationwide fibre-optic network to various “last-mile” solutions that enable the extension of connectivity to schools. As no schools are connected directly to the fibre-optic network, connectivity is distributed to them via several different technologies, including a wireless gateway, with all of the last-mile networking hardware owned by NaCET. Content Development and Distribution A second major sphere of activity within NaCET centers on the development and provision of digital content. Over the past several years, NaCET has conducted a range of activities in this area: • Armenian education portal (www.armedu.am) The education portal provides information, supports collaboration and offers e-learning courses, primarily on technology topics. Information available on the portal includes education news, access to education competitions, information about the Armenian education system, critical legal and saber.worldbank.org 3 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) governmental documents pertaining to education, and useful hyperlinks. The portal also provides teachers and students with online collaboration spaces, discussion forums, and with the opportunity to subscribe to RSS feeds and to sign up for email accounts. • e-Learning Via the portal, NaCET administers e-learning courses on the e-learning system, informatics, beginning computer graphics, and school-network administration. A secondary-level physics course is in development. • e-Textbooks A series of 18 electronic resources has been provided to Armenian schools, in subjects that include Armenian language and literature, geometry, algebra, Russian and English. In addition, NaCET has published both print and digital versions of Handbook of a Network User (also available on the portal) to help guide users of school networks and the Internet. • Digital library NaCET’s digital library provides digitized books for primary and secondary students. Information Management In 2008, NaCET assumed management of the Armenian Education Management Information Systems (EMIS) program, which collects data from all Armenian schools. The EMIS program supports planning, management, policy development and other decision-making processes of the central government; EMIS data are also used by regional education departments. These data are published in the annual General Education in Armenia report, which is available online. 1 Training and Outreach NaCET provides training to teachers and other school personnel focusing on the development of technology skills. • ICT training for teachers About 14,000 (or 35 percent) of Armenia’s secondary teachers have participated in technology training. • Network training for school personnel 620 school personnel participated in Network Administration training, improving their abilities to support use of ASIN • Training for informatics teachers 620 informatics teachers participated in training in the use of online tools for education NaCET’s plans for 2013 call for providing training to network administrators and informatics teachers in an additional 450 schools. Mobile Internet Computer Station In 2008, NaCET launched and staffed Mobile Internet Computer Station (MICS) to provide computer and Internet access, plus informatics training, in schools that do not have ICC labs or 1 http://armedu.am/arm/official.php?sec=org&id=377 saber.worldbank.org 4 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) that were not included in ASIN. The mobile-lab vehicle is equipped with seven computer workstations, and connects via satellite to the Internet. Between 2009 and 2012, over 1,900 students and 847 teachers in 80 communities participated in MICS technology training. Other Activities Other activities conducted by NaCET include: • Electronic diaries Electronic diaries enable parents to view their children’s test performances and other education records online. • e-Twinning At least one Armenian school has been selected to participate an e-twinning project as part of the e-twinning initiative of the European Commission (EC). The EC e-twinning portal (www.etwinning.net) provides online tools that help teachers find partners, set up projects, share ideas and collaborate. • Software licensing Since 2009, NaCET has annually purchased a licensing package from Microsoft, providing all Armenian schools with licensed and up-to-date operating-system and office-productivity software. saber.worldbank.org 5 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) 3. Origins and Influences NaCET’s current activities are shaped by the conditions that led to the organization’s founding. The resulting organizational contours, including NaCET’s strong technical capacity and management capacities in support of Internet connectivity, via ASIN, reflect the priorities and perspectives in relation to education technology that have come to the fore in Armenia. Today, technology in Armenian schools is both a set of tools that supports learning and a subject to be learned. By way of the informatics curriculum (taught in middle schools and high schools) and the training of school personnel, technology use in schools focuses primarily on the development of computer skills. Mirroring this emphasis, the main thrust of NaCET’s activities is technical: While content development and learning-focused activities are supported, the use of technology for learning across the curriculum and particularly to support different forms of learning (such as project-based learning, collaborative learning, or game-based learning) is not explored intensively or at scale. NaCET’s activities and areas of focus have reflected the priorities and approaches of the MoES, and are consistent with those of organizations involved in the initial stages of wide-scale roll-outs of technology in education systems in many other countries. Project Harmony and the Founding of NaCET In 2004 NaCET was organized to continue activities of the Armenian Schools Connectivity Project (ASCP), a US $4 million initiative that was co-financed by the governments of Armenia and the United States. ASCP was originally implemented by Project Harmony, a U.S.-based NGO that has also worked in other CIS states, plus Macedonia and Turkey. Through ASCP, information technology was introduced to Armenian schools and the ICC labs as a means of supporting the integration of technology into education and ultimately as a means of providing “Armenians with the 21st century education that today’s global economy demands.” 2 The transfer of management and operation of ASCP from Project Harmony to MoES helped to shape NaCET’s mandate and organization, and to identify capacities that the organization would require. Planned for 2006 and 2007, the transfer required that MoES have the technical capacity to maintain the existing 350 ICC labs in schools and to start to expand the program across Armenia. Meeting this requirement was the responsibility of NaCET, which at the time was a very young organization. Project Harmony was designed to provide a range of services to the participating schools that was much wider than IT training and connectivity. These services included staging of seminars and other events focused on integration of technology into education, and organizing more than 55 tele-collaborative projects engaging students and teachers in Armenia and the United States. (Other services included procuring computer hardware for approximately 170 ICC labs and providing training to school staff.) Project Harmony thus established a technology model for schools and introduced elements of a framework for technology-supported learning. This ambitious combination was intended to spark long-term school improvement, information literacy, and increased civic engagement in Armenia. In contrast, NaCET’s founding targeted immediate and specific needs: to assume operation and support of the school network and its central server, and to manage private-sector contracts. At the time of ASCP’s transfer to MoES operation in 2006, however, NaCET did not have the technical capacity to assume management and operation of the school network and server. 2 From the draft Transfer Plan for the Future of the Armenia School Connectivity Program 2006- 2007, a document developed by MoES and Project Harmony (January 2006). For more information about Project Harmony, visit www.ph-int.org. saber.worldbank.org 6 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) NaCET staff required substantial related training. At the same time, Project Harmony technical staff, who had been responsible for management of the network to that date, were in the process of creating an Armenian NGO that would service technical needs in schools and the education system after Project Harmony’s role in ASCP ceased. To address NaCET’s need for increased capacity and to ensure reliable network administration, it was agreed that staff of this new organization, which was to be called Harmony Foundation, would be “seconded” to NaCET to manage network operations and to provide on-the-job coaching and training. NaCET’s focus at that time was both essential and ambitious: to ensure that schools participating in ASIN (the new name for ASCP) had functioning computers and connections, personnel with technical skills, and teachers who were proficient computers users. It is worth noting that at the time, moreover, the Armenian education system was addressing fundamental challenges in the areas of primary- and secondary-school completion, teachers’ skills and approaches, and inequalities among schools in rural and urban areas. It is arguable that the transformation of teaching and learning, as framed by Project Harmony, required conditions that did not exist on a system-wide basis in Armenia at the time of NaCET’s founding. NaCET’s focus on technical and contractual expertise was considered both necessary and appropriate at the time that NaCET was formed and as it built its core capacities. PIU and ICT in Armenian Schools While NaCET has highly developed technical and operational capacities, education initiatives involving technology in Armenia are generally conceived by others. NaCET has good relations with the primary initiator of technology programs, the Center for Education Projects, also known as the Project Implementation Unit (PIU). Established in 1998, PIU has implemented two large-scale projects, the Education Financing and Reform Project (1998-2002) and the Education Quality and Relevance Project (2003-2014), both of which have been funded by the World Bank. These projects include the use of information technologies to improve learning outcomes, enhance information management, deliver professional development to teachers and increase access to curriculum-related content. In the course of these projects, PIU has launched several important initiatives, including the Armenian Education Portal and the Armenian EMIS. NaCET assumed management of the portal in 2007 and of the Armenian EMIS in 2008. In addition, PIU has undertaken three expansions of ASIN under the Education Quality and Relevance Project, adding 320 schools in 2008, 300 schools in 2010 and 459 schools in 2012. In initiating the ASIN expansion, PIU has developed tenders and managed procurement of hardware and software for schools’ ICC labs. As these labs are installed and connected, they become part of ASIN, with NaCET providing secure connectivity and education content. PIU, however, has a portfolio that is much broader than NaCET’s, extending into higher education, teacher training, and K-12 education reforms. The resources allocated by PIU to information technology in schools, as a result, are not extensive: In the past two years, PIU’s ICT in Education department has been trimmed from two persons to one. (Total staff at PIU is less than 28 people.) saber.worldbank.org 7 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) NaCET Today Today, NaCET has developed strong technical and management capacity, plus good relations with partners, such as ISPs and PIU, with MoES as a whole, and with schools. NaCET leadership changed in 2011, with Mr. Artak Poghosyan assuming the directorship. The founding director of NaCET, Mr. Edward Hovsepyan, served from 2004 to 2006; Mr. Areg Gigoryan served as director from 2006 to 2011. Under Director Poghosyan, NaCET is implementing new activities. In 2012, these included the extension of Internet connectivity to include an additional 459 schools in ASIN, in coordination with PIU’s procurement of hardware, and the design and launch of an online repository for education documents. In 2013, NaCET will extend ASIN-based connectivity and support to 12 vocational education and training colleges. In addition, in response to the second education- sector improvement project, NaCET plans during 2014 to strengthen its technical capacity, to assess and strengthen the technical capacity of school staff, and to increase the efficiency of information management by improving the integration of technology in schools into processes of data collection, analysis, and monitoring and evaluation. saber.worldbank.org 8 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) saber.worldbank.org 9 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) 4. Issues and Questions For many education-technology agencies such as NaCET, the establishment of system-wide technology access in schools represents a significant milestone. This achievement can be accompanied by questions about next steps and the potential for new areas of activity. As of the start of 2013, NaCET has successfully extended access to technology by trained personnel to almost all schools in Armenia, and has built its capacities to deliver education content and resources and to support information management. The first questions confronting NaCET, at this juncture, are both pressing and practical: Eight years have passed since the launch of ASCP by Project Harmony, and the age of the first computers installed in schools and the heterogeneous mix of operating systems and software applications poses a maintenance challenge. This challenge will only increase as the computers age, as will the need to procure replacements. The immediate task for NaCET, one that is perhaps unfamiliar but that is directly linked to its support of ASIN and the ICC labs, is to convince decision-makers of the need to upgrade computer hardware in schools. However the prospective cost of replacing computers is accompanied by the opportunity to assess current approaches and consider alternatives based on new tools and developments, activities that lie much father beyond the boundaries of NaCET’s historic areas of endeavor. Hardware Replacement and Organizational Change The issue of hardware maintenance, upgrades and replacement underscores the need for technology-focused advocacy within MoES and frames an example of organizational change arising in response to this need. Over the past decade, with support from NaCET and PIU, Armenian schools have experienced a rapid influx of computers that has increased the difficulty of hardware and software maintenance. From 2004 to 2012, the ratio of students to computers improved from 400:1 to 23:1. Versions of the Windows operating system released in this eight-year period—during which NaCET maintained licensing agreements with Microsoft—included Windows XP, Windows ME, Windows Vista, and Windows 7, with each version increasing minimum hardware requirements and requiring updates to other software. Today, the nearly 18,000 computers owned by schools and MoES vary greatly in terms of their technical specifications and the software that they are running. The age of some computers poses a challenge in terms of maintenance, and variations among operating systems and other software increases the difficulty of meeting that challenge. Replacement of computer hardware is becoming a critical issue. Individual schools and the MoES are faced with the need to follow their initial investments in education technology with follow-on investments to upgrade hardware in the ICC labs. The need for investment in hardware upgrades opens an opportunity to consider new tools that have emerged in the decade since the ICC labs were first introduced. Potentially valuable new solutions include tablet computers, pico-projectors and literacy-assessment software, as well as other tools with direct relevance to teaching and learning in schools. Gauging the value and the feasibility of these tools and of new approaches is complementary to, and perhaps no less critical than, the replacement of hardware that can no longer be maintained. NaCET has begun, within the framework of its mandate to support and improve educational computing in Armenia, to highlight the need for financial planning and funding allocations for the replacement of out-dated equipment. At present, these advocacy activities have taken the form of presentations of the case for replacement by Director Poghosyan to the education mission of the World Bank and to other groups. NaCET’s efforts, and MoES decision-making processes saber.worldbank.org 10 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) addressing this area, might benefit from a more structured approach to advocacy, in which staff gather relevant information about computer use in schools, effectively marshal that information in relation to policy goals and objectives, and provide evidence that can undergird the director’s efforts to secure support. To the extent that such activities constitute a new area of emphasis for NaCET, they create within the organization the need for learning and for the development of new capacities. NaCET’s effort to initiate the replacement of computer hardware represents a significant step beyond its historical role as an implementing agency. NaCET is broadening its portfolio to position itself, at least potentially, as an advocate in its dealings with the policymakers and decision-makers who determine approaches to education technology. This process of organizational change is driven by pressing factors—aging hardware—that have emerged in direct relationship to NaCET’s current activities. However there is an opportunity and perhaps a need for further change on the part of NaCET, in response to factors that are less direct but no less pressing, into an organization that both accurately assesses the role of computers in Armenian schools today and that provides guidance about the new tools and new approaches that can enhance technology-supported learning tomorrow. Observations and Lessons Learned The eight-year history of NaCET’s development in relation to education computing in Armenia yields observations that are potentially relevant to the establishment and direction of other national education-technology agencies. These include: Collecting information about computer use in schools is an essential function. Although NaCET manages the Armenian EMIS program, there is a need for improved information about the education technologies that have been deployed. The need to replace aging hardware sharpens focus on such information, however informed (data-driven) decision- making in many areas, both within NaCET and at the levels of policy and planning, requires ongoing capture and analysis of the ways that students and teachers use computers in schools. While extremely valuable information can be generated through smaller-scale research, monitoring and evaluation activities, system-wide information can be most relevant for the crafting of policy and plans. And while ASIN serves as a platform for the cost-effective collection of information with regard to web use and communications, additional information such as students’ rates of usage, in-class activities and attitudes with regard to technology could provide valuable support for educational computing. Financial sustainability calls for an agency with the skills and mandate for advocacy. While almost all school-based technology implementations require ongoing funding, in some instances these costs are unanticipated or otherwise excluded from plans and budgets. Organizational capacity to support sustainability planning is a necessity. However, decision- making bodies should also institute regular consultations with implementing agencies to support planning for near-term and long-term sustainability. Initiation and implementation are interrelated and equally important capacities. Although NaCET was originally conceived as an agency with capacity to implement or assume management of existing initiatives—such as ASIN, the Armenia EMIS, the education portal— the effective continuation of those initiatives demands vision and initiative. In the successive expansions of ASIN, NaCET and PIU have envisioned, procured, implemented and maintained a series of solutions to the problem of “last-mile” connectivity, such as use of a wireless gateway, to extend Internet access to rural and hard-to-reach schools. These expansions, which would have been impossible to approach as a sequence of replications, required problem solving, technical capacity, collaboration with the private sector, and the ability and willingness to try new approaches. Support of the portal, of information gathering, and certainly of the distribution of saber.worldbank.org 11 Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) education resources and content would benefit from application of these same activities and skills. Mechanisms for change should be established in conjunction with establishment of the organization. An organization’s structure, capacities and areas of focus can be persistent, and that persistence can inhibit learning or change. In such instances, NaCET and similar organizations can become locked in on strengthening existing capacities and expanding current initiatives, although new capacities and new paths might be called for. In light of the tremendous levels of experimentation and rates of change associated with technologies in education, attention to organizational learning is important. However the processes of learning and change are likely to require regular prompting and support. Forecasting can, for example, be built into periodic reviews and annual reports. Such forecasts can identify relevant trends, innovations that have potential, upcoming technology-related challenges within the education system (such as the looming need to replace hardware), and appropriate responses that can be taken. These forecasts can be accompanied by identification of new capacities or capacities to be strengthened to support the agency’s proposed response. (This observation is interrelated with the preceding observations about support for sustainability planning and about the need for both initiation capacity and implementation capacity.) saber.worldbank.org 12 saber.worldbank.org 13 World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series [1] SABER-ICT Framework Paper for Policy Analysis: Documenting national educational technology policies around the world and their evolution over time (Michael Trucano) [2] Building and sustaining national ICT/education agencies: Lessons from international experiences (Michael Trucano & Gavin Dykes) [3] Building and sustaining national ICT/education agencies: Lessons from Korea (KERIS) (Youngsun Kwon & Sanghyun Jang) [4] Building and sustaining national ICT/education agencies: Lessons from Malaysia (Smart Schools) (Molly N.N. Lee & Soon Seng Thah) [5] The Role and Status of National Research and Education Networks (NRENs) in Africa (Michael Foley) [6] Building and sustaining national ICT/education agencies: Lessons from England (Becta) (Gavin Dykes) [7] Building and sustaining national ICT/education agencies: Lessons from Chile (Enlaces) (Eugenio Severin) [8] Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) (Edmond Gaible and Anush Shahverdyan) [9] Building and sustaining national ICT/education agencies: Lessons from Uruguay (Plan Ceibal) (Eugenio Severin) [10] Building and sustaining national ICT/education agencies: Lessons from Indonesia (PUSTEKKOM) (Neil Butcher & Petra Bodrogini) [11] Evoke -- Developing Skills in Youth to Solve the World’s Most Complex Problems: The Social Innovators’ Framework (Barbara Freeman & Robert Hawkins) [12] Technologies in education across the Americas: The promise and the peril – and some potential ways forward (Michael Trucano) [13] Building and sustaining national ICT/education agencies: Lessons from Costa Rica (The Omar Dengo Foundation) (Carla Jimenez) [14] Building and sustaining national ICT/education agencies: Lessons from Thailand (NECTEC/Schoolnet Thailand) (Saowaruj Rattanakhamfu) [15] Building and sustaining national ICT/education agencies: Lessons from the Philippines (Benjamin Vergel De Dios) [16] Building and sustaining national ICT/education agencies: Lessons from Australia (EdNA) (Gerald White & Lesley Parker) [17] ICT and the Education of Refugees: A Stocktaking of Innovative Approaches in the MENA Region. Lessons of Experience and Guiding Principles (Kent Lewis with Simon Thacker) [18] Digital teaching and learning materials: Opportunities, options and issues (Michael Trucano) [19] Developing Skills in Youth to Solve the World’s Most Complex Problems: Contextualization, Implementation, and Experimental Research. Lessons from Evoke (Barbara Freeman & Robert Hawkins) [20] Developing Skills in Youth to Solve the World’s Most Complex Problems: Applications and Sustainability. Lessons from Evoke (Barbara Freeman & Robert Hawkins) World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series @WBedutech saber.worldbank.org worldbank.org/education blogs.worldbank.org/edutech worldbank.org/en/topic/edutech series coordinator: Michael Trucano