Column C- Country - Describes the country in which the intervention took place. Document overview: The document provides a database of existing phone-based assessment interventions using SMS, IVR and phone calls and their key features. There are two tabs in the workbook: (1) 'Focus on Student Assessment' : This tab contains the interventions focused on student assessments using SMS, IVR and phone calls. (2) 'Use SMS-IVR-Calls (not for SA)' : This tab contains the interventions, programs, initiatives that focus on using SMS, IVR and phone call technologies but not necessarily for student Column C- Country - Describes the country in which the intervention took place. Column D- Region - Describes the region where the country is located. For example, Afghanistan is in the South Asia region (SAR). Column E- Scale - Describes the scale at which the intervention was implemented, for example, whether in two villages or nation-wide. This column does not exist under the tab 'Use SMS-IVR-Calls (not for SA)'. Column F- Project/Platform Name - Describes the name of the project or the platform used to implement the technology solutions. Column G- Implementor/Donor - Describes the name(s) of the organistaion that implemented or sponsored the work on remote formative assessments. Column H - Year of Implementation - Describes the year or duration during which the intervention was implemented. Column I-K- Short Message Service (SMS), Interactive Voice Response (IVR), Phone Calls - Describes the technology solution used as a part of the intervention. For example, if an intervention used only SMS and IVR, the respective columns are marked 'yes'. Column L- Mode of Assessment (Synchronous/Asynchronous) - Describes whether the assessment was synchronous or asynchronous. This column does not exist under tab 'Use SMS-IVR- Calls (not Column for M- SA)'. Grade/Level - Describes the grade and/or the level of education for which the intervention took place. For example, Grade 5/primary level. Column N- Number of Participants - Describes the total number of participants who were a part of the intervention. Column O - Frequency of Communication - Describes how frequently the participants were reached out as a part of the intervention, i.e., daily, weekly, bi-weekly etc. Column P- Information on Content Provided/Assessed - Describes the context about the content that was provided to the participants as a part of interventon. For example, "curriculum aligned revision material in all subjects for primary and secondary learners on any device." Column Q - Subject/Skills Assessed - Describes the subject area or the skills that were assessed as a part of the intervention. For example, numeracy including place value and number operations tasks. This column does not exist in tab 'Use SMS-IVR-Calls (not for SA)'. Column R - Modality of Content Delivery - Describes how the content was delivered to the participants. For example, using SMS, radio, IVR etc. Column S - Assessment Tool Used - Describes the name or provides details of the assessment tools used as a part of the intervention. For example, ASER tool. This column does not exist under the tab 'Use SMS-IVR-Calls (not for SA)'. Column T- Assessment Procedure - Provides details on how the assessment was conducted using SMS, IVR and phone calls. This column does not exist under the tab 'Use SMS-IVR-Calls (not for SA)'. Column U - Language of Assessment - Describes the langauge that was used to assess the participants. This column does not exist in tab 'Use SMS-IVR-Calls (not for SA)'. Column V - Results/Impacts/Outcomes - Describes the impact/outcome of the intervention. This column does not exist under the tab 'Use SMS-IVR-Calls (not for SA)'. For countries where this column is blank, the intervention may still be ongoing or the sources did not provide this information. In such cases, the intervention can still be referred to, to see what was done. Column W - Implementation Challenges/Lessons Learned - Describes the challenges that were faced during the implementation of the intervention and lessons learned as a result of it. This column does not exist under the tab 'Use SMS-IVR-Calls (not for SA)'. Column X - Link/Source - Describes the link/source from where the information about the intervention was taken. Column Y- Additional Details - Describes the additional information such as links that may be useful to know about the intervention. Regi S.No Country Scale on 1 Afghanistan SAR two villages 2 Africa AFR No information (Rwanda, Kenya, Ghana, Ivory Coast) 3 Africa (Uganda, AFR No information Rwanda, and Kenya) 4 Bangladesh SAR No information g 5 Bangladesh SAR Nationwide 6 Botswana AFR nine out of ten regions 7 Côte d'Ivoire AFR A rural village in the Adzopé Department 8 Ghana AFR No information 9 India SAR Nationwide 10 India SAR New Delhi 11 India SAR 150 Villages in Uttar Pradesh 12 Kenya AFR No information 13 Kenya AFR 105 Bridge schools located in major cities of Kisumu in the west, the capital city of Nairobi in the center, and the eastern coastal city of Mombasa 14 Kenya AFR No information 15 Nepal SAR No information 16 Niger AFR Dosso and Zinder regions 17 Nigeria AFR Edo State (1,500 public primary and junior secondary schools) 18 Nigeria AFR Sokoto state 19 Nigeria AFR Borno and Yobe states 20 Nigeria AFR No information 21 Pakistan SAR five urban and rural areas of Islamabad 22 Pakistan SAR three districts of Punjab 23 Papua New EAP Rural Guinea elementary schools in two provinces, Madang and Simbu 24 Sierra Leone AFR No information 25 United States NA No information 26 Zambia AFR Chipata and Lundazi districts of Zambia’s Eastern Province 27 Zimbabwe AFR No information Project/Platform Name Implementor/Donor Mobile Literacy Project Afghan Institute of Learning (AIL), UNESCO, U.S. Afghan Women's Council, Creating Hope International Eneza Education No information Educate! No information Bangladesh Virtual Catholic Relief Interactive Classrooms Services/Niger, (BVIC) (CRS/Niger), Tufts University, the University of Oxford, and Systéme d’Information sur les, Marchés Agricoles (SIMA), Örebro University (Sweden) and Bangladesh Open University, with support from the Swedish Program for ICT in Developing Regions (Spider) at Stockholm University BBC Janala BBC in partnership with the UK Department for International, Development (DFID) No information Center for Global Development, University of Oxford, University of Colombia, RTI International and Young 1ove Allô Alphabet Jacobs Foundation Fellowship, Institute of Education Sciences, U.S. Department of Education, Eneza Education Viamo Viamo No information Pratham No information Central Square Foundation (CSF) and Saarthi Education Digital School Program Pratham and Education Above All (EAA) M-Shule No information No information Center for Reinventing Public Education (CRPE), Innovations for Poverty Action (IPA), NewGlobe Education, Brown University ElimuLeo Precision Development (PxD), Young 1ove, IPA, and the Kenya Institute for Curriculum Development (KICD) Low-tech Intervention for Ministry of Education, Foundational Education Science and Technology, (LIFE) Local Govt, Teach for Nepal Alphabétisation de Base Catholic Relief Services, par Cellulaire (ABC): Tufts University, Oxford Mobiles 4 Literacy University, UC-Davis Hitachi Fondation CITRIS EdoBest Program Ministry of Education, Universal Basic Education Board (SUBEB), LGEAs, World Bank Northern Education Creative Associates Initiative Plus (NEI+) International, USAID, Education Development Center, Florida State University, Overseas Strategic Consulting, Value Minds, Civil Society Action Coalition on Education for All, Reading Association of Nigeria, Federation of Muslim Women’s Associations in Nigeria. Addressing Education in FHI 360, USAID, Save the Northeast Nigeria Children, Viamo, Nigerian (AENN) Government, Local NGOs EdoBest No information Broad Class: Listen to The Communicators (Pvt.) Learn Limited, POWER99 Foundation, Marymount University Bunyad: Mobile-Based Punjab Department of Post Literacy Literacy and Non-Formal Programme Basic Education, Lahore; BUNYAD Foundation, Lahore; Dhaka Ahsania Mission Pakistan, Islamabad; Mobilink Pakistan; Nokia Pakistan SMS Story Australian Government, Voluntary Services Overseas, Papua New Guinea Department of Education Rising Academy Network Rising Academies Mobile Learning for All Cell-Ed, Centro Latino in (Cell-Ed) Los Angeles Makhalidwe Athu USAID, Creative Associates Viamo World Vision, Aktion Deutschland Hilft, Save The Children, ECHO & MoE Short Interactive Year of Message Voice Phone implementation Service Response Calls (SMS) (IVR) 2011 Yes No No 2013-present Yes No No No information Yes No No No information Yes No Yes 2008-2013 No Yes No 2020-present Yes No Yes Feb-May 2019 No Yes No Dec 2019-Feb 2020 No information Yes No Yes 2020 Yes No No 2020 No No Yes 2019-present Yes No Yes No information Yes No No October- No No Yes November 2020 (7 weeks) 2020 (two weeks) Yes No No November 2020- Yes No Yes present 2008-2010 Yes No No 2018-present Yes No No 2015-present No Yes Yes 2018-present Yes Yes Yes 2018-present Yes No No 2014-present No Yes No 2009-present Yes No No 2013 Yes No No May 2020-August Yes No Yes 2020 2014-present Yes Yes No 2015-2016 Yes Yes Yes No information No Yes No Phone-based F Mode of Assessment Number of (Synchronous/Async Grade/Level Participants (if hronous) mentioned) Asynchronous Adult education 50 participants (25 from each village) Synchronous Primary and No information Secondary Asynchronous Secondary No information education Synchronous Grade 9 - Higher No information Education Asynchronous Adult education No information Synchronous Grade 5 4,550 learners Synchronous Grade 5 No information Synchronous No information For English: 500 For Math: 300 Synchronous All Synchronous Ages 2-11 300 children Synchronous Secondary 2,000 learners education (Grade 8 and 10) Synchronous Grades 4-8 11-13K learners Synchronous Grade 3, 5, and 6 8,319 students Asynchronous Primary education No information Synchronous Grades 3-5 Baseline was conducted on 1800+ randomly selected students from full sample between Dec 2020- Jan-2021 Asynchronous Adult education 6700 Asynchronous Primary and Junior No information Secondary Synchronous Grades 1-3 40,000 No information Out of school 3,700 children Asynchronous K-12 No information Synchronous Kindergarten, 120 Grade 1 Asynchronous adolescent girls No information Asynchronous Grades 1-2 2,478 Synchronous Primary 4,399 students from 25 government primary schools Synchronous Adult education No information Asynchronous Grades 2-3 1,200 Synchronous ECCE 10,000 e-based Formative Assessments (SM Frequency of Communication Daily Self-paced No information No information Weekly/Self-paced Treatment arm 1: Weekly SMS with a "problem of the week" Treatment arm 2: Weekly live phone calls from instructors supplemented these SMS text messages. These calls averaged 15-20 minutes in length and provided a direct walkthrough of the learning activities sent via text message. No information Daily Daily No information Phone calls: daily (~20 mins) for first 3 weeks and then twice a week SMS: weekly starting 4th week No information Weekly No information No information Daily (Students attended classes five times a week, for three hours a day.) No information Four times a week No information Daily Daily Daily No information Three treatment arms: (1) SMS messages 2 times per week; (2) SMS messages two times per week plus live calls from a public teacher 1 per week on math and 1 per week in reading; (3) SMS messages two times per week plus live calls from a private teacher 1 per week on math and 1 per week in reading Self-paced Weekly- 3 messages a week Once a week rmative Assessments (SMS, IVR, Pho Information on Content Provided/Assessed AIL teacher trainers developed the curriculum for the texting class from June-Oct 2011. It provides curriculum aligned revision material in all subjects for primary and secondary learners on any device. A virtual tutor provides access to curriculum-aligned content in all subjects for learners in primary and secondary on SMS and on the Web. Locally developed and relevant content A learning management system (LMS) installed through mobile phones works to ensure an interactive, student- centered pedagogy. Using LMS, students interacted with their teachers and peers and could also self-assess, ask questions during classes, etc. Students took courses and learned lessons on various subjects, including the English language, through the provision of educational materials via SMS and seminars broadcast on television. BBC Janala launched at scale in 2008 with a primary delivery mechanism of 3-minute audio lessons, along with occasional Dual-Tone Multi-Frequency-based quizzes. At the time, a new lesson that coincided with the content of the TV component would be launched every week. In 2012, BBC Janala introduced a method whereby users can phone at any time to receive sequential lessons, with the system bookmarking users’ progress and allowing them to pick up from their last completed lesson at any point. Once a user completes the course, they can go online to receive a report recognising their completion of the course. No specific information on the content, but the study used data on the learning levels from the week 4 assessment to send tailored text messages to each student in the fifth week. For example, students who knew addition received subtraction problems to push them to a higher level of learning, whereas students who knew multiplication were sent division problems. Designed and implemented an early literacy curriculum on an interactive voice response (IVR) system, Allo Alphabet. The curriculum targets phonological awareness and print-sound mapping, gradually increasing in complexity and difficulty, from simple phoneme and syllable awareness, to mappings between letters, words, and sounds. At the start of each call, the system plays a welcome message, updates the user on their progress, and selects the next lesson based on the user’s prior mastery of concepts. Each lesson begins with an explanation of the concept in that lesson and an explanation of how to respond. For English: Used SMS and mobile phones: We offered a set of 20 audio English lessons delivered through mobile phones. Participants were given instructions in local languages, then they listened to a lesson recorded in English. They were able to test their speaking skills through a series of audio quizzes. We also sent out the lessons in two ways: - The first method was a “pull with a reminder.� We sent the participant an SMS message in the morning with a text version of the daily lesson. The participant was told to call in later that day to access the lesson via a voice message. - The second method was to “push� the lesson out automatically each morning through a phone call at a scheduled time. Participants were also able to call in to hear the content at any point in the day. For Math: Used SMS. We sent them one set of 10 math questions via SMS message. Content was developed by Pratham based on Indian national curriculum No information Content is developed by Pratham and aligned with the national Indian curriculum as well as learners’ contexts and learning levels. Content is developed by an internal team in line with the Kenyan national curriculum and is available in English and Swahili. Treatment group students in both arms of the intervention received weekly SMS text messages sharing practice problems that could then be referred back to during the calls themselves. The weekly SMS communication included a welcome message, a practice problem, guiding tips to master the topic, an additional challenge problem, and recommended mobile quizzes related to the content. All of the content for this intervention focused on mathematics. Students were in one of the three groups: (1) Accountability - five-minute teacher-student phone calls focused on accountability. Teachers were instructed to first gather information on whether the student attempted the SMS problems, answered the problems correctly, and completed the recommended interactive quizzes. Then, they discussed next steps including recommended additional mobile interactive quizzes and reminding them of the next time the teacher plans to call to check in. (2) Tutoring - 15-minute teacher-student phone calls focused on a mini-tutoring session in addition to basic accountability. No information Teachers were instructed to first gather The intervention is still ongoing and involves SMS messages and phone calls to support foundational skills in Math among Grade 3-5 students. The SMS includes multiple questions. Teachers/ facilitators adjust their teaching to the level of the child and would also practice with child additional questions (similar ones depending on the level of the child). a) Number operations task: Reading out loud the numeracy questions by assessor in ascending order of difficulty: place value, addition, subtraction, multiplication and division. b) Timed word problem task: Texting word problems to the student and asking them to read it out loud and solve them. c) Explanation of the solution: the students explaining their work to check for understanding. The program includes a mobile phone module as a complement to traditional literacy classes. The course taught students how to read and write in their native languages (Hausa and Zarma) and how to solve simple math problems. In addition, they learned about agricultural, environmental, and health issues. The state is incorporating digital self-study activity packages that are distributed online (with zero data internet access) and via applications such as WhatsApp. These packets contain hundreds of practice problems for all grades for different thematic areas and answer keys with feedback for parents or caregivers. Also, the state is developing a multi- channel approach that includes lessons through audio files for remote learning. These remote lessons are tailored to different grades and designed in an interactive way that includes health and sanitation messages. The project strategy focuses on strengthening the technical and administrative capacity, commitment, and accountability of federal, state, and Local Government Education Authorities (LGEAs) to provide practical English and mother- tongue-based Early Grade Reading (EGR) teaching and learning literacy, numeracy, and life-skills. No information 4 hours of daily content; created 7000 virtual, Zoom-based classrooms (30mins daily); 20min math & literacy digital self- study activity packets & storybooks for independent study; homeschooling support via 45-min learning guides with parent-led activities, distributed via WhatsApp & online. The content covers all basic skills in curriculum (including literacy, numeracy, English and life skills). Audio lessons guide teachers and students through activities, games, and exercises that teach carefully organized knowledge and skills which are culturally, linguistically and contextually relevant to marginalized populations. Audio lessons are led by a “radio teacher� who leads students through content and activities, pausing throughout the program to allow student responses. The one-hour broadcasts (30 minutes of interactive lessons and 30 minutes of health programming, stories, and games) are aired daily. Text messages containing literacy materials are sent via SMS to these mobile phones 6-8 times a day, and program participants are instructed to read the messages, practice writing them in their workbooks, and answer questions. The goal is to improve literacy retention among newly literate youths, particularly among young rural women, by delivering post-literacy materials via mobile phone. The project consisted of delivering daily short stories and complementary lesson plans via SMS for teachers designed to introduce children to reading English and followed an underlying phonics and key word based methodology. The aim of the project was to improve phonics through content delivery in a context where books are unavailable. SMS reminders asked students to tune in to the radio broadcast two times per week. The twice-weekly 15 minute telephone tutorial with a teacher covered content from the radio broadcast. Focus is on teaching literacy, language, and job skills, via three- minute micro lessons. They study through interactive text and audio voice lessons narrated by real teachers -- whether using an app or simply calling in. No internet or data plan is required. Cell-Ed also provides learners a live Cell-Ed coach who follows and supports each learner using texting and even phone calls to nudge, motivate, and provide feedback and further instruction. Creative Associates developed reading materials in ChiNyanja (the predominant local language). Over nine months, participant households received three SMS on their mobile phones each week. These three messages comprised a short story (e.g., 160 characters each). Children would read the stories with their families and respond to a question about the story. In addition, participants could call in for a pre-paid recorded voice message (IVR), which included comprehension questions and a recording of the story itself. 10 min/week IVR modules were prepared S, IVR, Phone Calls) Subject/Skills Modality of Assessment Tool Used Assessed Content Delivery (if any) Literacy SMS. The class No information met 6 days a week for an hour each day using Afghan literacy textbook. The students first learned the alphabet and sight words. Then they were given mobile phones and taught how to use them. Initially, in the classroom, teachers texted questions to students and students texted answers back. In addition, student wrote the answers All subjects SMS Kenya: SMS quizzes and Shupavu Mock Papers Socioemotional skills SMS, phone calls No information (such as and radio communication, teamwork, creativity, and grit) and enterprenuership training Literacy SMS, radio, TV, No information tutorial classes. English (Literacy) IVR No clear information about the tool, but the project developed its own, bespoke English testing tools specifically designed to be administered in dispersed settings with its target audience, in order to measure competence changes amongst service users. Numeracy SMS and phone ASER tool (adapted to call phone assessment) Literacy: Targeted IVR. The system No information phonological provides awareness and print- instructions, sound mapping, questions, and gradually increasing in feedback via voice complexity and messages difficulty, from simple recorded by an phoneme and syllable Ivorian researcher, awareness, to with answers input mappings between via touchtone letters, words, and (DTMF). The users sounds. In this study, call in to a users only specified number, experienced the first 2 which immediately of 8 units, which ask ends the call and learners to match calls the user back words or syllables that to avoid fees for share a sound or a the users. combination of sounds, to choose the word or sound that English (Literacy) For English: Used Audio quizzes SMS and phone calls For Math: Used SMS literacy, numeracy, SMS No information and life-skills early grade numeracy No information No information skills Language (English, SMS and phone No information Hindi), Mathematics call No information No information No information Numeracy/mathematic SMS + phone calls No information s Numeracy (Number SMS No information operations tasks) Numeracy SMS and Phone ASER tool (adapted to call phone assessment) Literacy and numeracy SMS No information No information SMS and No information WhatsApp literacy, numeracy, Lessons are No information and life-skills broadcasted by TV, radio, and IVR to students four times a week. A pedagogue follows up through live calls to check on the engagement of students. Parents can also request a call from a teacher if needed by responding to an IVR message No information No information No information Literacy and numeracy WhatsApp and No information Online Literacy and numeracy Radio No information Literacy SMS No information Literacy SMS EGRA Numeracy - counting SMS and Phone A mix of Early Grade and simple arithmetic call Reading and Mathematics Literacy - vocabulary, Assessments (EGRA and spelling, and aural EGMA), ASER comprehension. assessments, and items used orally in in-person assessments in urban India. Literacy, Job-skills SMS, IVR No information Literacy SMS and IVR. EGRA Literacy and numeracy IVR No information Language of Assessment Procedure Assessment Each student received a handset (that Dari ran on the standard 2G system), a phonecard, and a notebook. Teachers sent daily texts to the students, who read the incoming message and responded via return text message, demonstrating reading comprehension and writing skills. Students attended twice-monthly evaluation sessions to monitor progress and receive assistance, in addition to attending classes. Primarily, the messages comprised three types of questions: a) fill in the blank type sentences which students had to rewrite with the word filled in; b) open-ended questions to facilitate critical thinking and writing skills; c) reordering the jumbled sentence Rwanda: quizzes aligned to the No information Rwandan curriculum Kenya: Mock papers are designed to be as much like the real exam as possible (KCSE and KCPE exams). It is comprehensive, meaning that it tests all that has been taught in the previous classes Learning assessments are provided Local lannguage through SMS SMS was used to answer questions No information during lectures by students. They also were able to provide interaction through voice calls. No information English Student learning outcomes were English collected by directly assessing the child or children over the phone: a) Number operations task: Reading out loud the numeracy questions by assessor in ascending order of difficulty: addition, subtraction, multiplication and division; b) Timed word problem task: Texting word problems to the student and asking them to read it out loud and solve them; c) Explanation of the solution: the students explaining their work to check for understanding. The IVR plays a pre-recorded audio French message with the question and response options. Questions have either two or three responses, depending on the type of question, with most questions having three options. Students receive feedback upon responding; correct responses prompt new questions, while incorrect ones provide a hint and an opportunity to respond again to the question, focusing their attention on a particular part of the word or syllable. If incorrect, they receive the same question again, with a hint message explaining the concept or prompting the student to focus their attention on a particular part of the word or syllable. After one or two wrong attempts (depending on the question type), the answer is provided, with a Tested children's speaking skills English through a series of audio quizzes. Conducted through activties sent to One of the 11 mother children via SMS. Each SMS targeted tongues a competency. After parents were asked a few No information background questions, they were requested to pass the phone to the child for administering the phone- based assessment. Enumerators were trained to and were tasked with calling the parents, seeking their consent for the survey and for assessment of the child, and it was explained to them that these assessments would not adversely affect the child in any way. Remote assessments were conducted English and Hindi for students who could only read words. The assessment tool was sent to students as an image via WhatsApp, and facilitators assessed the students over a phone call. Student progress is tracked through English and Kiswahili continuous assessment. Students interact with the platform in a series of responses to questions that determine how the learner is progressing through a learning pathway. M-Shule uses a notification system to encourage learners to return to the application. A phone-based assessment took No information place in December 2020 with 2,552 students. It was conducted by hired enumerators and consisted of 14 questions, covering two predetermined sections on (1) core numeracy, and (2) curriculum-aligned standards based on what students would have been learning had schools been open and what they were supposed to be learning as part of the phone-based interventions. The curriculum-aligned questions varied across grades while the core numeracy section and survey questions were the same across grades. In addition, in-person assessments were conducted in February and March 2021. pilot sent trial math The two-week No information exercises to children to assess their skill level and, thereafter, sent them math exercises pitched to their abilities. As children mastered exercises of a certain difficulty, they received more challenging exercises. Conversely, if a child was struggling with their current set of exercises, they received easier exercises. As a part of the phone survey No information (Baseline assessment), a short Math test was administered (5 questions). This test is based on the ASER test and was adapted for phone delivery (Angrist et. al 2020). The test includes place value, addition, subtraction, multiplication and division questions that Grade 2 students are expected to answer. As per Angrist et al. (2020), student learning outcomes were collected by directly assessing the child or children over the phone: a) Number operations task: Reading out loud the numeracy questions by assessor in ascending order of difficulty: place value, addition, subtraction, multiplication and division. b) Timed word problem task: Texting word problems to the student and No information on assessment Hausa and Zarma procedure but the students were tested (1) in the beginning (January of each year), (2) at the end of the course, and (3) the following January (sought to determine whether the acquired literacy and numeracy skills had endured over time). Text-based assessments are No information distributed via SMS or WhatsApp. The state is developing interactive quizzes delivered to parents’ mobile phones. These quizzes are aligned with the state curriculum, and they interrelate with other resources such as digital storybooks and lesson guides delivered to parents. No information English, Mother-tongue SMS and IVR-based practice modules No information aimed at parents alongside their children Formative quizzes sent via SMS or No information WhatsApp. During short pauses built into radio English scripts, teachers and students participate in program, reacting verbally and physically to questions and exercises posed by radio characters. Monitoring the learners’ participation Urdu in the mobile-based programme is done by the web-based system which is used to send text messages to the learners. In addition, newly literates respond to questions/multiple-choice- questions (MCQs) or tests sent by SMS, and the results of these tests are summarised and recorded in the web-based system. Learners also report to literacy centres on regular bases. A monthly exam is given to learners at learning centres to track their retention rate and the development of their literacy skills. Project included a baseline reading English assessment, mid-point lesson and classroom observations, and an end- point reading assessment. Learning assessments are provided English through SMS for a small subsample. For the rest, it's an in-person assessment. SMS or IVR quizzes are prompted to Spanish participants. A correct response to the question triggers the beginning of the next micro-module, whereas an incorrect response leads to a repetition of the same micro-module until the user succeeds. Content is delivered at the learner’s pace and is able to track their responses and scores. Assessment was done by phone ChiNyanja through a questionnaire that tracked how many days did parents and children use the tools to read together. Using simple IVR formative Ndau assessments & retention surveys. For example, they hear a nursery rhyme about what sounds animals make, and then they are asked, “What sounds does this animal make?� So, they check via these mini assessments if students have listened and are still present. Implementation Results/Impact/Outcomes Challenges/Lessons Learned 83% of students were able to One of the challenges with pass two levels of AIL's literacy programs such as these is the course in just 4 months. This acceptance of mobile phone would normally take 18 months. usage in the Afghan culture. Mobile phones are seen as a potential threat to Afghan culture. People are afraid of what may happen when their family members have unlimited access to communicate with people all over the country. Reached over 6 million offline No information users at 8,000+ schools in Kenya, Ghana & the Ivory Coast No information No information No information No information BBC Janala has data that No information evidences a significant correlation between English language competency and income. The service estimates that their users experience a 20- 25% increase in income as a result of attaining an intermediate level of spoken English, compared to someone who speaks no English at all. 1. 56% of participants have shown competency increases 2. 28 million people have used the service on more than one platform - 80% of which are rural - 80% of which come from the lower 2 socioeconomic groups 3. In a country of 160 million, 1/3 The study found statistically No information significant learning differences between treatment and control groups. For the combined phone and SMS group, there was a 52 percent decrease in the share of students who could not do any numerical operations on an ASER test. The gains in average numerical skill were 24 percent, which translates to 0.29 standard deviations. For the SMS-only group, the study saw positive, statistically significant effects roughly half this size: a 0.16 standard deviation gain on the ASER test. Although the Allo Alphabet IVR No information was designed for a single user, adult supporters adapted their usage of it to fit their desire for collective support for children’s learning, much like other ICTD work on users’ appropriation of technologies beyond the intended usage. We find that adults engage in both simultaneous and sequential use of the IVR with their child. • People who registered tried at For Math: The main challenge of least one lesson: 91% rolling out this project on a larger • People who continued after the scale is students are not allowed first lesson: 94% to have cell phones at school. • People who finished 10 or more This is even more challenging for English lessons: 62% boarding school students who • People who finished all English only have access to phones lessons: 7% during vacations. No information No information No information 1. Only a limited set of competencies can be assessed through phone-based assessments. This is especially true for early grade students, for whom assessments are typically conducted verbally, face-to-face, and with help of visual aids. 2. There are tradeoffs in terms of the length, scope, and the type of assessments that can be administered through a phone. 3. There is a high likelihood of non-response. When the caller reaches the parent, they might not be at home or the child might not be willing to participate in the phone-based assessment. 4. To ensure that the child was comfortable while replying to the phone assessments, parents too On average, each facilitator a) About 10% of students don’t speaks to about 35 students have access to a phone and every week. Over 1000 students have, therefore, not been able to in DSP (89% of whom are benefit from the COVID-19 women) are in touch with their response outlined above. facilitators twice a week. 70% of b) Another 30% of students live students complete the activities further away from the village sent each week. center and have not been reachable since the beginning of this lockdown. c) Additionally, parents and spouses of students were initially unsupportive of the messages and phone calls made by facilitators due to cultural factors. Since most learners are young women, some of whom are married, long phone calls or those made during the evening were a challenge. M-Shule has been able to No information support the continuation of education for many, particularly during the pandemic. The team has found that the learners have begun to start using MShule as their primary means of education; therefore, the organization recognized the need to provide additional support to the parents: such as daily learner schedules and subject support, as well as developing digital literacy resources for parents. It was seen that although calls No information increased student perceptions that teachers cared, accountability checks had no effect on math performance up to four months after the intervention and tutoring decreased math achievement among students who returned to their schools after reopening. This was, in part, because the relatively low- achieving students most likely to benefit from calls were least likely to return and take in-person assessments. Tutoring substituted away from more productive uses of time, at least among returning students. No information No information Baseline Results: Overall, only No information 1.2 percent of grade 3 students, 3.8 percent of grade 4 students and 6.2 percent of grade 5 students were able to answer all questions on the mathematics assessment correctly. Only 16 percent of grade 3 students, 27 percent of grade 4 students and 38 percent of grade 5 students answered 3 or more questions correctly. Analysis by question type shows that student competencies are in fact increasing perceptibly by grade. However, even by grade 5, most students struggle with division, and to a lesser extent, 2- digit multiplication. At the other end, nearly 35 percent of Grade 5 and 49 percent of grade 4 Examining changes to reading No information and writing and maths scores over time, students, overall, increased their test scores from 0 to, on average, between 2 and 3, meaning that they could read and write sentences and complete addition and subtraction problems. However, the writing and maths test scores of ABC villages were between 20% and 25% higher than those of non- ABC villages in the short-term, and 20% higher in the longer- term (that is, seven months after the end of the programme). Although both groups experienced a decline in literacy and numeracy skills during the six months when classes were not Goingheld, the level forward, and depreciation ofwith World No information Bank support, the Edo Government intends to scale up EdoBest to cover both primary and secondary education and to integrate the EdoBest@Home program so that students can learn anytime, anywhere. By blending learning at school and learning at home and using digital technologies, Edo State is at the forefront of reimagining education. No information No information No information No information No information → Not enough data yet on whether the program is producing sufficient learning outcomes → Breaking the culture of falsifying data; putting in place QA processes → How to onboard more pupils into the program as fast and effectively as possible? (JSS next) → Low parental engagement in urban areas (even compared to remote); much more community sensitization still needed (physically going to marketplaces etc); but it’s eating into their resources. When schools resume they will really have to strengthen their urban school- based committees. The Program provides a high- No information quality interactive radio program to children attending formal, non- formal, religious schools and out- of-school children. The program has benefited 300,000 children in 2,386 classrooms, 3,400 heads and teachers in 838 schools, 172 educational supervisors, and 22,679 parents and community groups. Due to it being publicly broadcast, it reaches 10 million children who are currently not in school. 60 percent of the program beneficiaries are female Ten literacy centres were 1. Religious and cultural established in three districts of constraints and the authoritative the Punjab province and 250 attitude of males keep females learners completed the from receiving education. As far programme. Remarkable results as this particular programme is were found regarding learners’ concerned, family members – achievements during the mobile- especially males – were initially based programme. For example, very negative and hostile when at one of the districts, Sialkot, the they were approached to allow test results from the first month of their young female family the mobile-based programme members to participate in the showed that 90% of the learners programme. They strongly were in the 0–50% range and disagreed with the idea of giving none made it to the 70–100% mobile phones to young women range; however, results from the and doubted the effectiveness of last month of the programme the programme’s approach. In indicated only 14% of the order to overcome this problem, learners fell into the 0–50% the trust the community had in range and 39% of the learners the BUNYAD Foundation, a local reached the 70–100% range, NGO partner, was a great source showing The study clear benefit a found of the that children of Nohelp. information who did not receive the SMS Story were approximately twice as likely to be unable to read a single word of three sub-tests (decodable words, sight words, and oral reading). In other words, the intervention almost halved the number of children who could not read anything compared with the control schools. Three main takeaways: Difficulty of contacting children 1. Teacher phone calls had no through phone because phones effect on test scores (1) are often owned by parents 2. Private school teachers who need them for other things; worked harder than public school (2) are not always turned on teachers—but to no avail (sometimes only turned on 3. Learning assessments by during the weekend or during the phone may not be reliable. market days); (3) are sometimes owned by neighbors and friends (which made it hard to contact the child); (4) are sometimes not valid anymore. Data from the two field tests No information suggest that Cell-Ed has the potential to fill a need for on-the- go learning required by working adult learners. Cell-Ed’s interactive micro lessons provide an accessible ramp to digital learning for adults who need to study in short 5 to 10 minutes bursts. They are also accessible for learners with little to no digital literacy or experience with learning online. The MA project had a positive No information impact on three of the five EGRA subtasks evaluated. The study found positive and significant impacts for non-word reading, oral reading fluency (ORF), and reading comprehension, these effect sizes are, respectively 0.2, 0.27 and 0.23. No information No information Additional Link/Source details https://uil.unesco.or http://www.unesco g/case- .org/new/fileadmin study/effective- /MULTIMEDIA/HQ practices-database- /ED/ICT/pdf/Yaco litbase-0/mobile- obi.pdf literacy-programme- afghanistan- afghanistan https://enezaeducati on.com/#impact https://www.experie nceeducate.org/covi d-19-response http://citeseerx.ist.p su.edu/viewdoc/dow nload?doi=10.1.1.62 5.5535&rep=rep1&t ype=pdf https://www.gsma.c om/mobilefordevelo pment/resources/bb c-janala/ https://www.csae.ox .ac.uk/materials/pap ers/csae-wps-2020- 13.pdf https://dl.acm.org/do i/pdf/10.1145/33143 44.3332504 https://viamo.io/cas e-studies/classroom- palm-hand-lessons- via-mobile/ https://drive.google. com/file/d/1KAPEfO LASE6qxwsIL- 7oTlpL7mTv0J6Q/vi ew https://www.centrals https://riseprogra quarefoundation.org mme.org/blog/pho /wp- ne-based- content/uploads/202 assessments- 1/06/ASSESSMENT education- S-IN-THE-TIMES- systems OF-COVID-19.pdf https://resources.ed ucationaboveall.org/ sites/default/files/ng o/attachments/2020- 08/09.%20Pratham. pdf https://resources.ed ucationaboveall.org/ sites/default/files/ng o/attachments/2020- 08/07.%20M- Schule.pdf https://www.edworki ngpapers.com/sites/ default/files/ai21- 432.pdf https://2uy7xawu7lg 2zqdax41x9oc1- wpengine.netdna- ssl.com/wp- content/uploads/202 0/11/2020_Q3_PAD- report.pdf https://openknowled The intervention is ge.worldbank.org/bit designed as a stream/handle/1098 randomized 6/35384/Learning-in- control trial/ the-Time-of-COVID- impact evaluation. 19-Insights-from- Nepal.pdf?sequenc e=1&isAllowed=y https://uil.unesco.or g/case- study/effective- practices-database- litbase- 0/alphabetisation- base-cellulaire-abc- mobiles-4 https://blogs.worldb ank.org/education/le arning-despite-crisis- case-edo-state- nigeria Hale, J., & Udeh, A. (2020, June 8). NEI Plus COVID-19 Remote Learning Response. BEC EdTech Webinar. Koester, E. (2020, June 8). Addressing Education in Northeast Nigeria (AENN): COVID Response. BEC EdTech Webinar. http://pubdocs.world bank.org/en/685691 598013656403/Worl dBank-EdTech- Team-Knowledge- Pack- MobileDistance- HybridEducationSol utions-version2 https://www.globalin novationexchange.o rg/innovation/broadc lass-listen-to-learn https://uil.unesco.or g/case- study/effective- practices-database- litbase-0/mobile- based-post-literacy- programme- pakistan https://blogs.worldb ank.org/edutech/pro moting-literacy- mobile-phones-rural- papua-new-guinea https://www.cgdev.o https://www.cgdev rg/publication/teachi .org/blog/tech-plus- ng-and-testing- teachers-one-on- phone-pandemic one-phone- tutorials-didnt-help- kids-learn-better- during-school- closures https://edtech.world ed.org/wp- content/uploads/201 8/12/Cell-Ed-Report- final-012319-2-1.pdf https://pdf.usaid.gov /pdf_docs/PA00SZJ S.pdf https://pubdocs.worl dbank.org/en/68569 1598013656403/pdf /WorldBank-EdTech- Team-Knowledge- Pack- MobileDistance- HybridEducationSol utions-version2.pdf https://www.wvi.org/ stories/zimbabwe/vi amo-ensures- education-continues- midst-crises Programs/P Project/Platfor S.No. Country Region m Name 1 Afghanistan SAR Sandbox 2 Africa (Senegal, AFR All Children Malawi, and Reading Rwanda) 3 India SAR Gyan Vani and Gyan Dhara 4 Peru LAC No information 5 Philippines EAP All Children Reading 6 South Africa AFR FunDza ograms/Platforms that use SMS/IVR/ Interactive Implementor/Do Year of Short Message Voice nor implementation Service (SMS) Response (IVR) No information No information No No USAID and No information Yes No Human Network International Indira Gandhi 2001-present No No National Open University (IGNOU) MOE 2020 Yes No USAID and No information No No Education Development Center (EDC) FunDza Literacy No information Yes (using No Trust and Turn.io WhatsApp) MS/IVR/Phone calls (but the focus is n Number of Phone Calls Grade/Level participants (if mentioned) Yes Secondary No information No No information No information Yes Higher No information Education No K-12 No information No No information No information No No information No information ut the focus is not on student assess Modality of Frequency of Information on content Content communication provided/assessed Delivery Daily Each teacher spent Radio., TV, approximately 6.5 hours a day Feature phone on the phone, doing approximately 35-40 calls per day. Each call was approx 10 mins in length. Teachers talked to their own students (rather than a ‘call centre’ model). Each teacher had between 100-150 students. Calls were to focus on supplementing the national distance learning curricula No information No information No information Daily Higher education subjects Radio Lessons No information Developed by MOE TV, Radio, and Aprendo en Casa (online platform) No information No information No information No information No information No information assessment) Link/Source Additional details https://docs.goo Different countries are pursuing variations on the gle.com/docume theme of a ‘helpline’ using phones, based on nt/d/e/2PACX- local need and variations 1vRhEnDXEbzZ Coordinating with the government is instrumental. dbbC8SrkBqmfa SMS and WhatsApp groups (where smartphones ya1yb1ApX7WX are available) are being leveraged as the basic 7kx2viDRdYBd7 modalities, in addition to phone-calls. aOKEN2- In some areas, phones or even smartphones are 4V33VuK2HGO common, which gives more avenues to explore Opc1d6E2l0n3/p (e.g.BRAC Tanzania are exploring Whatsapp ub groups). It is more ideal to distribute phones rather than using the parents’ phone(s). Getting data on whether the helpline is working is difficult, for example on parental engagement and follow-through. Phone-calls for feedback have been the primary mode, but having a data- sharing agreement with a telecoms partner would enable much more ‘real time’ and reliable data. On-boarding the teacher is very important, for No information example Focus is notby providing priorities on student and scripts. assessment. Focus is to collect and analyse the student assessment results and aggregate them into a dashboard, allowing for real-time oversight. https://www.igno No information uonline.ac.in/gya ndarshan/ https://www.igno uonline.ac.in/gya ndhara/ http://pubdocs.w No information orldbank.org/en/ 6856915980136 56403/WorldBan k-EdTech-Team- Knowledge- Pack- MobileDistance- HybridEducation Solutions- version2 No information In the Philippines, EDC used SMS to enable the transmission and analysis of student National Achievement Test (NAT) scores at the school level. This helped school leadership become better informed about NAT data to guide decisions about addressing student learning gaps. No information FunDza’s goal is to promote reading and writing amongst teens and young adults. They have launched an initiative that leverages WhatsApp, the country’s most-used communications platform, as a way of connecting with their learners. WhatsApp allows FunDza to offer new reading content. The user needs to message “hello� to 0600 54 8676 on WhatsApp to unlock the content, which includes local stories, plays, blogs and poems all written for a young South African readership. There are plans to add reading challenges and competitions in the future to help drive engagement levels.