Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series (#14) Saowaruj Rattanakhamfu 2016 saber.worldbank.org To cite this publication: Rattanakhamfu, S. 2016. Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand). World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series (#14). Washington, DC: The World Bank. Available at: http://saber.worldbank.org Rights and Permissions This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) http://creativecommons.org/licenses/by/3.0/igo. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: Rattanakhamfu, S. 2016. Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand). World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series (#14). Washington, DC: The World Bank. License: Creative Commons Attribution CC BY 3.0 IGO Translations—If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations—If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content—The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third-party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to re-use a component of the work, it is your responsibility to determine whether permission is needed for that re-use and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. © 2016 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington DC 20433 USA Telephone: +1-202-473-1000; Internet: www.worldbank.org Some rights reserved Disclaimer: The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. saber.worldbank.org Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) Table of Contents 1. Introduction: Background and context................................................................................................... 1 2. The development of SchoolNet Thailand ………………………….......................................................... 2 3. Challenges in the implementation of SchoolNet Thailand ........................................…………………… 5 4. Conclusion and lessons learned ............................................................................................................ 7 Bibliography............................................................................................................................................... 9 saber.worldbank.org Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) Acknowledgements The World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series explores a variety of topics and issues related to the use of information and communication technologies (ICTs) in the education sector. The Systems Approach for Better Education Results (SABER) initiative seeks to improve the global knowledge base related to education systems analyses, assessments, diagnoses, and opportunities for dialogue. SABER-ICT aims to improve the availability of policy-related data, information, and knowledge on what matters most in using ICTs to improve the quality of education. This publication series is made possible through generous support from the Government of Korea, most notably through the Korea - World Bank Partnership Facility. saber.worldbank.org Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) Executive summary Thailand’s national initiative on ICT usage in schools was launched in 1995 by the country’s National Electronics and Computer Technology Center (NECTEC) under the pilot project known as the Thailand School Communication Network (more commonly referred to as ‘SchoolNet’). The main objectives of SchoolNet were to network schools inside and outside Bangkok; to connect them to the Internet; and to promote the use of the Internet for teaching and learning in schools. After operating for eight years, the project was transferred from NECTEC to the Ministry of Education (MOE) and served as the foundation for the country’s national education network. Experience from SchoolNet Thailand provides useful lessons related to what can be achieved by small groups with vision and passion; how to operate without dedicated budget allocations from government in the start-up phase; the important role of support from widely respected and influential people outside the particular line ministries in starting something new; how to transition from a small and dynamic pilot initiative into a large government bureaucracy; and the importance of cooperation among related agencies across ministries. saber.worldbank.org saber.worldbank.org Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) 1. Introduction: Background and context Information and communication technologies (ICT), and especially the Internet, have recently played an important role in the development of education in many countries around the world. In the 1990s, there was a notable ‘digital divide’ in access to ICTs in schools across Thailand, especially between schools in Bangkok and those in rural areas. Students in rural areas did not have access to the Internet. As part of larger efforts to reduce various gaps across the education system, efforts to promote equitable access to the Internet gained increasing prominence. Thailand’s national information technology (IT) policy called for the promotion of ICT use in education and human resource development to increase the country’s national competitiveness. To be aligned with the policy, in 1995 the National Electronics and Computer Technology Center (NECTEC), under the National Science and Technology Development Agency (NSTDA) of Ministry of Science, Technology and Environment (MOSTE), launched a new pilot project: the Thailand School Communication Network. More commonly known as ‘Schoolnet’, this initiative sought to promote Internet connectivity in schools so that teachers and students could access and utilize the Internet in support of their teaching and learning practices. There were four main areas of activity under the SchoolNet project. First, Schoolnet connected schools to the Internet and promoted Internet usage through the development of free operating software known as Linux School Internet Server (Linux-SIS). Second, the development ICT skills among personnel in educational institutes was supported. During the early stages of Schoolnet, human resource development was considered to be of critical importance, and NECTEC supported various efforts to for staff to develop related skills, through efforts such as its cooperation with Rajabhat Institutes to provide training courses. Third, digital learning resources were developed. At that time, there were few such resources available in Thai. Teachers were encouraged and supported to develop and share educational content through the network. A new digital library was created to serve as a knowledge resource, providing material in ten academic subject areas. NECTEC also developed the Digital Library Toolkit to help teachers to create their own websites. Fourth, NECTEC developed the website of SchoolNet to be the focal point for schools to share information, data, news, opinions and to provide solutions for technical problems in order to facilitate knowledge exchange across the network. According to UNDP’s Human Development Report 2001 and APEC’s New Economy Report 2001, SchoolNet was considered as best practice in promoting ICT in education. In spite of the limited budget from the government, SchoolNet provided internet access to 5,000 schools nationwide during 1995-2003. In 2003, responsibility for SchoolNet was transferred to the Ministry of Education (MOE) as part of the effort to develop the national education network (known as EdNet), merging and expanding SchoolNet with the national university network (UniNet) and MOE education management network (MoENet). EdNet aimed to provide internet access to 34,000 educational institutes nationwide at all levels, including schools. saber.worldbank.org 1 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) 2. The development of SchoolNet Thailand In alignment with Thailand’s eighth national economic and social development plan (1997-2001) and the country’s first national ICT policy (IT 2000), SchoolNet was launched in 1995 by NECTEC to be a pilot project for promoting the ICT usage in education. This project aimed to upgrade the education level of Thai students through access to self-learning resources from various knowledge sources around the world and to become a communication tool enabling the exchange of knowledge among teachers and students. The development of SchoolNet can be classified into four general stages. First stage (1995-1997): Introduction of IT in schools In 1995, the Thai Social/Scientific Academic Research Network (ThaiSARN), a project under NECTEC, expanded the Internet connection it provided from higher education to secondary education in order to promote the use of the Internet in secondary schools. Initially, ten secondary schools were targeted. Later, and in alignment with the national IT policy (IT2000) approved in 1996, the scale of Internet use in secondary schools in Thailand was expanded. The name of the project was changed to SchoolNet, with the target of 50 participating schools in 1996, and 120 schools nationwide in 1997. Under this project, an Internet server was set up under the name of “k12.nectec.or.th” (or “k12”). Each school was allocated two Internet accounts and five MB data storage. In addition, training courses about Internet usage and web page creation were provided to teachers. Schools could connect to the Internet by using a dial-up modem, connecting to 39 phone lines (later increased to 120 phone lines). No budget was allocated by government to support these activities and, given that a number of target schools also had no computers, NECTEC asked for support from the private sector to donate hardware (such as computers and modems) and operating system software to schools, especially in remote areas. In this period, SchoolNet faced three major implementation challenges. First, provincial schools had to pay a high cost for the long distance call to connect them to the Internet server, which was located in Bangkok. Second, some schools did not have computers, modems, and phone lines. Third, most educational content on the Internet was in English, and the English language ability of most Thai teachers and students was quite limited. Second stage (1998-2000): SchoolNet@1509 In the second stage, the target number of participating schools in SchoolNet was increased to 1,500 nationwide. Importantly, the national IT committee (NITC) appointed the Sub-Committee Coordinating Working Team for Internet use in education as a mechanism to build cooperation among three major ministries: the Ministry of Transport (MOT) to provide internet service with reasonable cost to schools by Telephone Organization of Thailand (TOT) and Communication Authorization of Thailand (CAT); the Ministry of Education (MOE) to select qualified schools and to provide training on basic internet usage; and the Ministry of Science, Technology and Environment (MOSTE) to promote Internet use in education and to prepare guidelines for the use of the Internet in schools. Under MOSTE, CAT was responsible for providing Internet connectivity. SchoolNet got support from TOT for domestic Internet bandwidth and CAT for International internet bandwidth. Participating schools from across the country could connect to the Internet by dialing the phone number ‘1509’, paying only the very small cost of three baht per connection. Three main achievements marked this second phase of activity. First, schools nationwide could connect to the Internet network without paying for long distance calls. Consequently, students and teachers in schools outside Bangkok could access the Internet for the first time. Second, the Digital Library was developed to provide knowledge and useful learning materials in Thai. saber.worldbank.org 2 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) Third, Linux-SIS, a free packaged software application, was developed to serve as the Internet/Intranet server for schools. Third stage (2001-2002): Expansion In this period, Schoolnet grew significantly to reach 5,000 schools at the including primary, secondary and vocational levels. In addition, SchoolNet shifted its role from network development to content development, especially digital learning resources in Thai, as the increase of Internet service providers (ISPs) meant that connectivity was widely available at much lower costs. Fourth stage (2002-2003): Transition from NECTEC to MOE The SchoolNet pilot project was formally transferred to the Ministry of Education (MOE) during this stage. Under the MOE, the Bureau of Information Technology of the Office of Permanent Secretary was responsible for managing this project. There were four groups directly in charge of project operation and implementation: the planning and policy group, responsible for training; the network and technology group, responsible for network and hardware; the system development group, responsible for providing software such as Computer Assisted Instruction (CAI) and database management software.; and the information group, responsible for data about schools. During this period, network coverage expanded to 38,000 schools nationwide. Later, the project’s name was officially changed to the Education Network Service (EdNet). Key players in SchoolNet Thailand Cooperation among a number of related agencies was of critical importance to the development of SchoolNet. In particular, as the secretariat office of the national IT committee (NITC) under the umbrella of NITC, NECTEC cooperated with TOT to get support for domestic Internet bandwidth, with CAT for support of international internet bandwidth, and with the MOE to select qualified participating schools and provide training. Under NITC, there was Sub-Committee Coordinating Working Team for Internet usage in education. Its major tasks were classified into three dimensions: network, content and human resource development. For the network task, the working team members included NECTEC, CAT and TOT; content coordinated by MOSTE; and human resource development activities were handled by both MOE and MOSTE, with MOSTE responsible for public relations and identifying agencies to receive, while the MOE was responsible for cooperation with Rajabhat Institutes to provide trainers. Some problems with collaboration ensued as, in practice, MOSTE and MOE worked separately and uncoordinatedly in many cases. NECTEC therefore took on an unofficial role in searching for educational supervisors and teachers to be volunteers to support the core tasks, such as providing solutions about basic network and Internet usage problems. saber.worldbank.org 3 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) figure 1. Structure of joint working teams table 1. Expenses in the implementation of SchoolNet Fiscal year Private sector (Baht) TOT/CAT (Baht) NECTEC (Baht) 1996 2,880,000 - 1,195,450 1997 1,030,000 - 4,857,800 1998 - 32,200,000 26,380,000 1999 1,254,811 32,200,000 2,380,000 2000 - 82,517,950 2,380,000 2001 - 82,517,950 66,600,000 2002 - 82,517,950 4,585,321 2003 - 82,517,950 3,816,130 Total (1996-2003) 5,164,811 394,471,800 112,194,701 Total (1996-2003) 511,831,312 saber.worldbank.org 4 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) 3. Challenges in the implementation of SchoolNet Thailand SchoolNet faced six major implementation challenges: 1. The MOE did not formulate a clear policy on ICT in education. As a result, school administrators did not realize, and were not aware of, the importance of ICT use in education. Consequently, when teachers faced problems, such as having no modem or computer, high phone costs, and no direct phone line to connect the internet, they did not get help or support from principals to help solve such problems. NECTEC stepped in and tried to solve such problems by asking for assistance from various other groups, brokering hardware donations from the private sector and network support from CAT and TOT. With the passage of the National Education Act of 1999, things improved considerably, as Chapter 9 under this Act focused on technology for education. This helped to build the awareness of using ICT in education for executives in MOE and schools. 2. Network infrastructure was not accessible equitably in different areas of Thailand. In the beginning of the project (1995-1997), ICT usage was very costly. At that time, there were few Internet Service Providers (ISPs), most of which were located in major provinces. Those wishing to connect to the Internet from schools in small provinces or remote areas had to pay the high cost of long distance call. This led to the development of a ‘digital divide’ between schools in major provinces and rural areas. 3. Some schools had difficulty connecting to the Internet through phone lines. Some schools were in remote areas so they did not have phone lines. Others -- especially small schools -- had only 1-2 phone lines dedicated for use by the school administration. This meant that they did not have a direct phone line to connect to the Internet; they often experienced frequent disruptions in their Internet connections as well. 4. Content on the Internet was of limited practical use in education. During the first era of Internet use in Thailand, most content was in English, and thus it was difficult for teachers to use this content in their teaching. Later, more content was developed in Thai, but most of this was created for entertainment purposes, not for education. As a result, teaching and learning content meant to be used for educational purposes was very limited. 5. Teachers lacked the skills and experience to use ICT in education. Most teachers did not have the necessary skills to be able to use the Internet, let alone use ICt in support of their pedagogical practices. Some teachers had a very high workload, so they thought that they did not have time to learn how to use new technologies. Some teachers had difficulties in understanding English – the predominant language of the Internet at that time – which further complicated their efforts to use the Internet. Sometimes teachers simply were not aware of what was available on the Internet, and so were note motivated to use the new technologies provided to them. 6. Resource management in schools did not maximize the efficiency of computer and network usage. Network administration in schools was often not sophisticated enough to maximize available bandwidth. There was no integration between computer usage and learning in core academic subjects, and limited cooperation among teachers to manage computer labs or computers to be saber.worldbank.org 5 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) used for learning in computer classes and in other subjects. Some schools, it should be noted, did not have enough budget to purchase computers and other related equipment. SchoolNet tried to solve these sorts of problems in a number of ways. To solve the network problem related to the high cost of a long distance call, SchoolNet got support from Her Royal Highness Princess Sirindhorn to allow for the merging of the SchoolNet with the Kanchanpisek Network (an online mass-educational project) so that a large-scale nationwide IP network called SchoolNet@1509 could be established. This provided one telephone number (‘1509’) which users nationwide could use to access the network at low cost. To deal with the problem of limited content to be used for learning in schools, SchoolNet developed prototype digital learning content as part of a “Digital Library” and a Digital Library Toolkit, which helped teachers with basic ICT knowledge create and distribute their own contents on the network. NECTEC developed Linux-SIS as free software which could help to manage local school networks, and distributed this software by CD-ROM, together with related training manuals and courses. To help build awareness of the benefits of ICT use in education, SchoolNet organized a number of outreach activities, including regional seminars and exhibitions; training courses for teachers and student; rewards for outstanding schools, administrators and teachers who promoted ICT use in schools; and online initiatives to demonstrate various applications of ICT use in education. Timeline: Key events and milestones of Schoolnet Thailand 1995 Launched the project of internet in secondary schools; 10 schools participated. 1996 The national ICT policy (IT2000) approved. 1996 Extended the project of internet in secondary schools to reach 50 schools and named it “SchoolNet”. 1997 Extended the number of participating schools to 120. 1998 Support from the Royal Highness Princess Sirindhorn to use the Kanchanapisek Network established a large-scale nationwide IP network called SchoolNet@1509. 2001 Increased available phone lines to 1,650 and the number of participating schools to 4,000 schools. 2001 Recognized as Best Practice for ICT usage as a tool to reduce the education gap and digital divide in the UNDP Human Development Report 2001 and APEC New Economy Report 2001 2003 SchoolNet transferred to be the Ministry of Education (MOE). 2003-7 Extended the number of participating schools to 38,000. saber.worldbank.org 6 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) 4. Conclusion and lessons learned A number of key lessons can be learned from the development of SchoolNet Thailand. The vision of leaders of the project team at NECTEC was a key factor in the birth of SchoolNet. NECTEC executives were keenly interested in technology and realized the potential benefits of ICT use in education. In addition, passion and personal commitment among operating staff can be an important striver of action and success, especially in the early days. Most NECTEC executives were university professors and were passionate about education. Their interest in connecting schools was an extension of this passion. Consequently, they were eager to launch the pilot project to provide Internet connectivity to secondary schools across Thailand. As the secretariat of the national IT committee, NECTEC operated the project by seeking cooperation from related agencies, such as TOT, CAT and MOE. At that time, NECTEC also oversaw the existing project of Thai Social/Scientific Academic Research Network (ThaiSARN), which was Internet network for higher education. The experience of operating ThaiSARN project helped NECTEC to expand the Internet connection network from higher education to secondary education. In addition to executives, NECTEC staffs were capable and eager to operate the project successfully, in spite of very limited budget and personnel. The creation and implementation of SchoolNet without an allocated budget from the government during 1996-2000 demonstrates that a lack of dedicated funds need not imperil a project of this sort, provided that people are able to be creative. SchoolNet did not get supporting budget from the government until 2001, because the budget bureau considered that it was not the main responsibility of NECTEC to promote the ICT usage in schools. Accordingly, NECTEC had to find its own ways to support the projects. The organization structure of NECTEC and its internal budgeting practices were flexible enough to give space for staff to allocate and manage the internal budget and resource to operate SchoolNet. To minimize costs, the network resources for the project were shared with the Kanchanapisek network and the ThaiSARN network. In addition, NECTEC got financial support from other groups, such as the private sector for equipment and TOT and CAT for the network. High level support from a widely respected and influential people outside the particular responsible line ministries (MOE and MOT) was a key to success. The support of Princess Sirindhorn was particularly notable and important in attracting partners to the initiative (like TOT and CAT), as well as helping to instill a sense of pride among key staff working on the project. In the early stages, the absence of related government policies provided space for groups to pilot new initiatives and try new things. NECTEC had done a connectivity project in higher education, and then decided that it would do something similar for schools. Although it had no mandate to do this, there was nothing preventing it from doing this either. As a national agency for education development, the MOE was not involved in the beginning, largely because there was no governing government policy. In addition, when Schoolnet Thailand was conceived and rolled out, the MICT did not exist (it was established in 2002) or was only in its early stages of operation. This meant that the conception and early implementation of SchoolNet was largely a result of decisions by NECTEC itself. The transition from a small and dynamic pilot initiative housed within a small institution into a large government bureaucracy (like the MOE) can be difficult and needs to be managed carefully. Once a governing government policy was in place, the functions of Schoolnet were transferred from NECTEC to the MOE. The MOE did not set up a separate dedicated structure to mirror what had been established within NECTEC. Transferring of “soft lessons”, focusing on content and developing human capacity, might not occur naturally, especially where responsibilities were transferred, but staff were not. Having a program saber.worldbank.org 7 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) overseen and implemented by a small group like those working on the Schoolnet project at NECTEC meant that there was focus. This focus, however, can be difficult to maintain when responsibilities are transferred to a large governmental ministry, which has a multitude of other priorities, and whose staff may not have the same types of backgrounds and expertise and/or who may have other responsibilities as well. To expand and ‘scale up’ a project of this sort, cooperation among related agencies is necessary. To enable this in Thailand, the joint working committee for ICT in education was set up, consisting of representatives from MOT, MOE, and MOSTE. Timing can be of critical importance here: If this is established too late, it may be difficult to create the necessary level of partnership for an effort like this to be developed and sustained over time. saber.worldbank.org 8 Building and sustaining national ICT/education agencies: Lessons from Thailand (Schoolnet Thailand) Bibliography Juthai. (2013). Special interview with Charnsak Leungtrairat, the head of technology and network group of the Ministry of Education, Thailand. National Electronics and Computer Technology Center (NECTEC). (2005). Assessing Impact of SchoolNet. Paisal Kiattananan & Thaweesak Koanantakool. (1998). SchoolNet Thailand: An information infrastructure for the future of Thailand. Bangkok: Minustry of Science, Technology and Environment. Siriporn Pansawat. (undated). 5,000 pilot schools: SchoolNet from NECTEC to Ministry of Education. UNESCO. (2004). SchoolNetworking: lessons learned. Bangkok: UNESCO-Bangkok. saber.worldbank.org 9 saber.worldbank.org World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series [1] SABER-ICT Framework Paper for Policy Analysis: Documenting national educational technology policies around the world and their evolution over time (Michael Trucano) [2] Building and sustaining national ICT/education agencies: Lessons from international experiences (Michael Trucano & Gavin Dykes) [3] Building and sustaining national ICT/education agencies: Lessons from Korea (KERIS) (Youngsun Kwon & Sanghyun Jang) [4] Building and sustaining national ICT/education agencies: Lessons from Malaysia (Smart Schools) (Molly N.N. Lee & Soon Seng Thah) [5] The Role and Status of National Research and Education Networks (NRENs) in Africa (Michael Foley) [6] Building and sustaining national ICT/education agencies: Lessons from England (Becta) (Gavin Dykes) [7] Building and sustaining national ICT/education agencies: Lessons from Chile (Enlaces) (Eugenio Severin) [8] Building and sustaining national ICT/education agencies: Lessons from Armenia (NaCET) (Edmond Gaible and Anush Shahverdyan) [9] Building and sustaining national ICT/education agencies: Lessons from Uruguay (Plan Ceibal) (Eugenio Severin) [10] Building and sustaining national ICT/education agencies: Lessons from Indonesia (PUSTEKKOM) (Neil Butcher & Petra Bodrogini) [11] Evoke -- Developing Skills in Youth to Solve the World’s Most Complex Problems: The Social Innovators’ Framework (Barbara Freeman & Robert Hawkins) [12] Technologies in education across the Americas: The promise and the peril – and some potential ways forward (Michael Trucano) [13] Building and sustaining national ICT/education agencies: Lessons from Costa Rica (The Omar Dengo Foundation) (Carla Jimenez) [14] Building and sustaining national ICT/education agencies: Lessons from Thailand (NECTEC/Schoolnet Thailand) (Saowaruj Rattanakhamfu) [15] Building and sustaining national ICT/education agencies: Lessons from the Philippines (Benjamin Vergel De Dios) [16] Building and sustaining national ICT/education agencies: Lessons from Australia (EdNA) (Gerald White & Lesley Parker) [17] ICT and the Education of Refugees: A Stocktaking of Innovative Approaches in the MENA Region. Lessons of Experience and Guiding Principles (Kent Lewis with Simon Thacker) [18] Digital teaching and learning materials: Opportunities, options and issues (Michael Trucano) [19] Developing Skills in Youth to Solve the World’s Most Complex Problems: Contextualization, Implementation, and Experimental Research. Lessons from Evoke (Barbara Freeman & Robert Hawkins) [20] Developing Skills in Youth to Solve the World’s Most Complex Problems: Applications and Sustainability. Lessons from Evoke (Barbara Freeman & Robert Hawkins) World Bank Education, Technology & Innovation: SABER-ICT Technical Paper Series @WBedutech saber.worldbank.org worldbank.org/education blogs.worldbank.org/edutech worldbank.org/en/topic/edutech series coordinator: Michael Trucano