Systems Approach for Better Education Results The Learning Crisis in LAO PDR CHALLENGES AND POLICY PRIORITIES Angela Demas and Emiko Naka with Jennifer A. Mason June 27, 2018 Ministry of Education and Sports Cover photograph credit: Myra Murad Khan Table of Contents Executive Summary ................................................................................................................ 1 Understanding the Learning Crisis: A Need to Focus on Student Learning.............................. 5 Priorities for Improving Student Learning ............................................................................... 7 Improving Teacher Ability............................................................................................................................ 7 Reducing Lost Instructional Time ................................................................................................................. 8 Creating Accountability for Improving Student Learning .............................................................................. 8 Correcting Infrastructure Deficiencies.......................................................................................................... 8 Priority Recommendations ..................................................................................................... 9 I. Increasing the Effectiveness of the Existing Teaching Force .................................................................. 10 A. Building Leaders’ Capacity for Effective Coaching and Instructional Leadership .....................................10 B. Placing Student Learning at the Center of Training for In-Service Teachers ............................................11 II. Developing a Pipeline of High-Ability Teachers and Principals ............................................................... 13 A. Improving Pre-Service Teacher Training ................................................................................................13 B. Improving Initial Principal Leadership Training ......................................................................................14 III. Reducing Lost Instructional Time.......................................................................................................... 14 A. Finding Root Causes of Classrooms Lacking Teachers ............................................................................14 IV. Creating Accountability for Improving Student Learning ....................................................................... 15 A. District Supervision that Emphasizes Student Learning ..........................................................................15 B. Increasing Schools’ Accountability to Their Communities ......................................................................16 V. Correcting Infrastructure Deficiencies to Support a Better Learning Environment ................................ 17 A. Improving Health and Safety in Schools.................................................................................................17 B. Ensuring Access to Functioning Classrooms ...........................................................................................17 Improving Financial Transparency ........................................................................................ 18 Conclusions .......................................................................................................................... 19 Appendixes........................................................................................................................... 22 APPENDIX A: Background of SABER Service Delivery Survey .......................................................................... 22 APPENDIX B: SABER SD Service Delivery Protocols ........................................................................................ 24 APPENDIX C: Select Education Service Delivery Results ................................................................................. 27 This paper is based on the findings of a larger study: Delivery of Education Services in Lao PDR: Results of the SABER Service Delivery Survey, 2017, published in 2018 by Angela Demas, Myra Murad Khan, Gustavo Arcia, and Emiko Naka. It was prepared under the auspices of the World Bank and Lao PDR’s Ministry of Education and Sports. ii | The Learning Crisis in Lao PDR Executive Summary Improving student learning is now the most pressing education challenge facing the Lao People’s Democratic Republic. The SABER Service Delivery (SABER SD) survey conducted in 2017 reveals that student learning in Lao PDR is very low. On average, fourth-grade students can correctly answer only 23 percent of math questions and only 58 percent of Lao language questions (Demas et al. 2018).1 Insufficient student learning in primary school contributes to the low transition rates to secondary school, as the majority of students are not attaining the knowledge required for the secondary school curriculum. In addition, significant numbers of primary school students are not achieving even the most basic levels of literacy and numeracy, which contributes significantly to the high early dropout rate and the reduced but still concerning numbers of children never enrolled in school. While alarming results for student learning spanned all ethnicities and school types, the SABER SD survey revealed an urgent need to improve educational equity for students whose mother tongue is not Lao-Tai and for students in rural schools, as outcomes for these populations are particularly poor. Findings of the SABER SD survey are uniquely valuable for prioritizing policy actions with the greatest potential impact on student learning in Lao PDR. The most shocking finding about education service delivery is the low level of teacher ability. Fourth-grade teachers scored an average of 49 percent in math and 52 percent in Lao language, indicating that teachers have mastered only about half of the content they are teaching. Teacher scores on tests of pedagogical knowledge were even lower, with the average score at 27 percent.1 The survey also found that significant numbers of students were in classrooms without teachers on the study’s second unannounced visits. Approximately one in five classrooms of students were without a teacher (see table 1: LAO PDR Overview of Education Indicators). The highest priorities for improving student learning span five areas: I. Increasing the Effectiveness of the Existing Teaching Force To increase teacher quality, build principals’ and Pedagogical Advisors’ capacity for effective coaching and instructional leadership and launch universal in-service teacher training and coaching focused on learning outcomes. Efforts should address mastery of subject content knowledge, student-centered pedagogical content knowledge, and effective classroom application of a broad range of student-centered teaching strategies. To improve educational equity, principals and teachers at schools with high populations of non-Lao-Tai speakers should receive additional, intensive training in teaching strategies 1To facilitate convenient access to additional information on the specific SABER SD survey results cited in this paper, endnotes with page references will be used when referring to the technical report, “Delivery of Education Services in Lao PDR: Results of the SABER Service Delivery Survey, 2017” (Demas et al. 2018). The data cited in this paragraph appear on page 27. Challenges and Policy Priorities | 1 tailored to non-native language learners. Those at multi-grade schools should be trained in techniques for multi-grade instruction and classroom management. II. Developing a Pipeline of High-Ability New Teachers and Principals While the magnitude of low teacher ability makes in-service training the highest priority at present for improving teacher quality, achieving the greatest and most sustainable gains requires new teachers to have a higher level of ability upon their entrance into the classroom. Improving the quality of teacher education programs, setting high standards for entry into Teacher Education Institutions (TEIs), and equitable deployment of trained teachers across the country are all critical to raising the quality of entry-level teachers. Principals should also receive more rigorous initial school management and instructional leadership training, plus continuous professional development once on the job. III. Reducing Lost Instructional Time Efforts to improve teacher effectiveness will not improve student learning if students are in classrooms without teachers. The root causes of teacher absences should be examined to determine the most effective corrective actions. This includes addressing teacher management practices and processes for providing substitutes. IV. Creating Accountability for Improving Student Learning Ensuring that critical changes take place and are sustained in every school requires greater accountability for student learning outcomes to district offices and local communities. Recommended changes include reviewing the effectiveness of the district supervision processes, refocusing district supervision on student learning, sharing written recommendations from district supervisory visits with teachers and Village Education Development Committees (VEDCs) as well as with school management, providing VEDCs with training to better understand school performance and set appropriate goals, and creating consequences for persistent failures to address parents’ and VEDCs’ concerns. V. Correcting Infrastructure Deficiencies to Support a Better Learning Environment More than a third of principals reported that poor school infrastructure was a major constraint on the functioning of the school. Many of Lao PDR’s primary schools lack handwashing facilities, drinking water, an adequate number of toilets, and classrooms with sufficient lighting contrast for students to read the blackboard. Ensuring that every student has access to safe and sanitary conditions at school is an important component for creating an environment where better learning can take place. 2 | The Learning Crisis in Lao PDR In addition to identifying the actions with the greatest potential impact on student learning, the SABER SD survey also identified a need for greater financial transparency. The Ministry of Education and Sports (MoES) should adopt financial management procedures that ensure schools receive the full block grant funding to which they are entitled in a timely manner. Furthermore, each recipient school should be informed and aware of the amounts and transfer schedule. Other policy actions that are important even if they were not investigated by the SABER SD survey should also be included in the policy considerations. For example, we know from other studies that it is important to build political commitment around tangible national learning goals and build learning coalitions to sustain support for policies and reforms to achieve them (World Bank 2017a). Additionally, policy prioritizations on programs such as expansion of Early Child Education (ECE) services and introduction of effective digital technology systems to assist teaching and learning could largely promote learning outcomes. Challenges and Policy Priorities | 3 Table 1: Lao PDR Overview of Education Indicators Learning Outcomes and Preparedness Average student test score 41% Average student Lao language performance 58% Average student math performance 23% Students who ate breakfast 94% Teacher Effort and Ability Teacher school absence rate 16% Teacher classroom absence rate 26% Time spent teaching per day 4h 8 m Average test score 52% Minimum knowledge in math and Lao language 2.4% School Management Principal absence rate 20% Schools with VEDC/no training 95%/46% Schools with bank account 49% Information and Accountability Pedagogical Advisor reports shared with teachers 38% VEDC/parents can demand accountability 86% School takes action based on parent concern 25% Teacher Support and Motivation Feedback received in last 12 months 25% Delay in pay at least one time in school year 53% Key Education Inputs Students with textbooks 84% Student-teacher ratio 23:1 Minimum infrastructure availability 58% Minimum learning materials availability 47% Source: Demas et al. 2018. Data from SABER Service Delivery Survey, Lao PDR, 2017. 4 | The Learning Crisis in Lao PDR Understanding the Learning Crisis: A Need to Focus on Student Learning Improving student learning is now the most pressing education challenge facing Lao PDR. While Lao PDR has made impressive advancements in providing children with schooling, levels of student learning have remained low and the quality of education delivery is poor. In 2017, the World Bank conducted the SABER Service Delivery (SABER SD) survey to support the Ministry of Education and Sports (MoES) in its efforts to improve education outcomes. The survey collected data on what fourth-grade students in Lao PDR have learned in language and in math and the characteristics (such as ethnicity and mother tongue) of students in the sample. The survey also examined delivery of education services in areas spanning teacher effort and ability, classroom activities and conditions, principal leadership, school management and governance, and school resources and facilities. The research team then analyzed which student characteristics and service delivery factors were associated with higher or lower levels of student learning. The survey background and methodology are described in more detail in Appendixes A and B, and complete results of the study are presented and discussed in detail in “Delivery of Education Services in Lao PDR: Results of the SABER Service Delivery Survey, 2017” (Demas et al. 2018). Survey results reveal that student learning in Lao PDR is very low. On average, students can correctly answer only 23 percent of math questions and only 58 percent of questions in Lao language.2 Low levels of learning at school thus currently impact far more children in the country than lack of access to schools.3 Among children of primary school age, 98.8 percent of boys and 98.6 percent of girls were enrolled in school as of the 2016–17 school year (Government of Lao PDR, MoES 2017). However, as the SABER SD survey shows, only a minority of those enrolled students are achieving adequate levels of learning. SABER SD survey results indicate that insufficient student learning leaves the majority of students unprepared for secondary school, as most students are not attaining the knowledge required for the secondary school curriculum. Most students could not perform double-digit subtraction (74 percent); multiplication of single, double, or triple digits (77 percent, 84 percent, and 87 percent, respectively); and division of single or double digits (67and 86 percent, respectively).4 When asked to read an 83- word paragraph, students could read only about half of the words correctly and answer only about half of the comprehension questions on the passage.5 In addition, significant numbers of students are not achieving even the most basic levels of literacy and numeracy. Nearly 19 percent of fourth-grade students were not able to identify even three letters of the Lao alphabet, and more than a quarter of students were not able to read three words correctly.6 In mathematics, a quarter of students were not able to add single-digit numbers correctly, 45 percent were not able to arrange numbers from small to large, and 15 percent of students were not even able to identify at least three numbers.7 Challenges and Policy Priorities | 5 This data support the findings of previous studies that low levels of learning at the primary school level are a major reason parents allow children to drop out of school in early grades or never enroll them in school at all. In many communities, the benefits of attending school are not perceived to outweigh the direct and indirect cost of school attendance, such as spending for uniforms and transportation and lost wages and household labor (Cerdan-Infantes et al. 2016). While alarming results for student learning spanned all ethnicities, the SABER SD survey revealed an urgent need to improve educational equity for students whose mother tongue is not Lao-Tai. For grade 4 students whose mother tongue is not Lao-Tai, average scores in Lao language were between 24 and 44 percentage points lower than the average score for students of Lao-Tai background. In math the average score of non-native Lao-Tai speakers was approximately 40 to 50 percent lower than the average score of Lao-Tai students8 (see table 2). Their difficulties in reading and understanding Lao-Tai further compound learning in other subjects. Table 2: Average Student Performance in Math and Lao Language by Ethnicity Lao-Tai Mon-Khmer Chine-Tibet Hmong-Mien Average Math Score 29.3 15.4 17.4 16.7 Average Lao Language Score 72.5 48.1 33.0 28.7 Source: Demas et al. 2018. Data from SABER Service Delivery Survey, Lao PDR, 2017. Non-Lao-Tai students were also significantly more likely to lack basic skills in literacy and numeracy. In the regression analysis of survey data, having Lao-Tai as mother tongue language is positively associated with better learning performance.9 While 67 percent of Lao-Tai students could correctly arrange numbers from small to large, for example, success rates at this task for students from other ethnic backgrounds ranged from 24 to 49 percent. 10 The success rate on the task of reading three letters correctly was 91 percent for Lao-Tai students, yet only between 51 and 82 percent for other ethnicities, with most groups falling on the lower end of this range.11 The gap in performance between students in rural schools and urban schools is also significant. Students in rural schools scored approximately 14 percentage points lower in math and 12 percentage points lower in language than student in urban schools (17.3 vs. 31.7 percent and 52.9 vs. 65.3 percent, respectively).12 Some of this difference reflects the higher performance of students at private schools, which are located primarily in urban areas. However, even within public schools, students in rural schools have significantly lower test scores. Students in rural public schools scored approximately nine percentage points lower in math than students in urban public schools (17.6 vs. 27.2 percent) and eight percentage points lower in language (53.3 vs. 61.8 percent).13 In summary, continuing to expand school provision will address some of the needs of Lao PDR children, but achieving goals for enrollment, retention, and learning outcomes in those schools requires concerted focus by all education service providers and new investments in improving learning for all children. 6 | The Learning Crisis in Lao PDR Priorities for Improving Student Learning Findings of the SABER Service Delivery survey are uniquely valuable for prioritizing policy actions with the greatest potential impact on student learning in Lao PDR. The survey identified areas of accomplishment as well as many areas in which education service delivery warrants improvement (Appendix C). The research team used regression analysis to identify which factors were associated with significantly stronger or weaker student learning outcomes. Results from other studies of the Lao PDR education system were also reviewed and used to inform recommendations. Improving Teacher Ability The most shocking finding about education service delivery in Lao PDR is the low level of teacher ability. As part of the SABER SD survey, teachers’ subject content knowledge and pedagogical skills were assessed. Results show the average teacher score in math and language content was 52.2 percent, indicating that teachers have mastered only about half of the content they are teaching. Only 2.4 percent of fourth-grade teachers demonstrated proficiency in math and language content (with proficiency defined as a score of 80 percent or better on the exam). Results for pedagogical knowledge were also extremely poor. Only 1.8 percent of teachers achieved a score of 80 percent or better, and the average pedagogy score was 27.4 percent.14 Improving the abilities of teachers is an urgent priority for improving student learning. Students cannot succeed in learning if their teachers do not have sufficient knowledge of the content that they are teaching or if they lack the pedagogical skills to help students learn what is presented in the curricula. Analysis of the Lao PDR SABER SD survey data showed that teacher test scores were strongly and significantly associated with the test scores of their students. Teachers with higher scores were associated with students with higher scores, indicating that improving teacher knowledge would have a significant impact on student learning.15 Teacher professional development programs that do the following are more likely to result in more gains in student learning: (i) link incentives such as career opportunities, promotion, or salary increases, (ii) offer follow-up monitoring visits to teachers to provide in-class pedagogical support, and (iii) provide time for teachers to practice with one another (Popova et al. 2017). The Más Tecnología program in Ecuador and Pratham professional development in India are among successful in- service training programs that offer career incentives and opportunities for teachers who participate in the programs. Early Grade Reading Assessment programs in the Liberia and Northern Uganda Literacy Projects, which include follow-up visits to teachers to provide pedagogical support in class, are examples of programs that demonstrated significant effect on student learning outcomes (Popova et al. 2017). Challenges and Policy Priorities | 7 The extent of teacher deficiencies in content and pedagogical knowledge point to urgent needs to improve both the abilities of the existing teaching force and the quality of pre-service teacher education. Achieving timely and lasting impact on student learning outcomes requires addressing the skills of in-service and pre-service teachers in coordination. The improvement of both programs must happen in tandem and be aligned in their promotion of strong content knowledge, pedagogical content knowledge, and skill in delivery of the curricula via student-centered learning. Reducing Lost Instructional Time The survey also found that significant numbers of students were in classrooms without teachers on the study’s second unannounced school visits. Up to 10 teachers at each school were identified on the first visit, and on the second visit researchers observed whether or not these teachers were at the school and in their classrooms teaching. On average, 16.3 percent of the teachers were absent from school and another 9.2 percent were at school but not in their classrooms, bringing the total share of teachers absent from their classrooms to 25.5 percent.16 Approximately a quarter of teachers absent from the school were on maternity leave, but in Lao PDR, teachers on maternity leave are usually replaced by a substitute teacher. Adjusted to exclude maternity leave, the teacher classroom absence rate is 21.6 percent, meaning approximately one in five classrooms was without a teacher.17 In addition to documenting the locations of the 10 teachers at each school selected on the first visit, researchers also inspected every classroom at each school on their second unannounced visit and noted whether or not the classrooms contained students and, if they did, whether or not a teacher was present with the students. Across all school types, 16 percent of classrooms were “orphaned,” meaning they contained students but no teacher.18 These data indicate a widespread problem that needs to be addressed to ensure regular access to instruction. Creating Accountability for Improving Student Learning The SABER SD survey also identified lack of effective accountability mechanisms at the district and school levels for ensuring and supporting improvements in school performance and student learning. While nearly all schools have district administrative supervision and community participation mechanisms in place, their supervision and involvement in school affairs currently does not include adequate accountability mechanisms for improving school performance and student learning outcomes. Analysis of survey data showed that there is no significant association between supervision activity and student test scores. The poor capacity of VEDCs to understand the state of their school, be involved in school planning, and fully advocate for improving the quality of teaching and learning also prevents a local accountability mechanism within the school. It is critical to focus VEDC trainings on understanding school and student performance to empower them to support and hold schools accountable to achieve their educational goals. Correcting Infrastructure Deficiencies Finally, the SABER SD survey revealed that many of Lao PDR’s primary schools have infrastructure deficiencies that prevent suitable learning environments. Most notable are makeshift classrooms, lack of handwashing facilities and adequate sources of drinking water, an insufficient number of toilets, and electricity supply, especially for classrooms with insufficient lighting contrast for students to read the blackboard. Ensuring that every student has access to a safe and sanitary environment where learning can take place is another important component in support of better student learning outcomes. 8 | The Learning Crisis in Lao PDR Priority Recommendations Actions with the greatest potential impact on student learning span five areas: increasing the effectiveness of the existing teaching force, developing a pipeline of high-ability teachers and principals, reducing lost instructional time, creating accountability for student learning, and improving infrastructure (see box 1). Box 1: Priority Actions for Improving Student Learning in Lao PDR I. Increasing the Effectiveness of the Existing Teaching Force A. Building Leaders’ Capacity for Effective Coaching and Instructional Leadership 1. In-Service Principal Training 2. Review of Pedagogical Advisor Program B. Placing Student Learning Outcomes at the Center of Training for In-Service Teachers 1. Universal Training and Coaching Focused on improving Content Knowledge, Pedagogy, and Effective Classroom Application 2. Teaching Non-Native Lao-Tai Speakers 3. Multi-grade Instruction II. Developing a Pipeline of High-Ability Teachers and Principals A. Improving Pre-Service Teacher Education 1. Improved Teacher Education Curriculum 2. Required Practicum on Student-Centered Teaching 3. Professional Development for Teacher Education Instructors 4. PA-TEI Collaboration B. Improving Initial Principal Leadership Training 1. Robust Training for Principal Candidates in School Management and Instructional Leadership III. Reducing Lost Instructional Time A. Finding Root Causes of Classrooms Lacking Teachers 1. Draw on Best Practices for Teacher Attendance Management 2. Review of Substitute Teacher Policies IV. Creating Accountability for Improving Student Learning A. District Supervision that Emphasizes Student Learning 1. Review of Effectiveness of Administrative Supervision Processes 2. Sharing Administrative Supervision Visit Reports with School Stakeholders B. Increasing Schools’ Accountability to Their Communities 1. Building Capacity of VEDCs 2. Consequences for Failure to Address Community Concerns V. Correcting Infrastructure Deficiencies for a Better Learning Environment A. Improving Health and Safety in Schools 1. Adequate Numbers of Toilets 2. Access to Handwashing Facilities and Drinking Water B. Ensuring Access to Functioning Classrooms Challenges and Policy Priorities | 9 I. INCREASING THE EFFECTIVENESS OF THE EXISTING TEACHING FORCE A. Building Leaders’ Capacity for Effective Coaching and Instructional Leadership Building principals’ and Pedagogical Advisors’ (PAs’) capacity for effective coaching and instructional leadership2 is critical for supporting and improving the effectiveness of existing teachers. Often this includes setting clear learning goals, managing the curriculum, monitoring lesson plans, allocating resources, and evaluating teachers regularly to promote student learning and growth. Although principals at public schools report that they and the Pedagogical Advisors deliver feedback based on classroom observations or other reviews to a high percentage of teachers every year, only a small percentage of teachers report that they received feedback in the past year. Principals reported they met with an average of 72 percent of their teachers in the past year, yet only 14 percent of teachers report receiving feedback from their principal.19 Public school principals report that PAs meet with 69 percent of teachers at their schools on average, yet only 6.5 percent of teachers say they received feedback from a PA in the past 12 months.20 Even more concerning, student performance was negatively associated with teachers receiving feedback. That is, teachers who received feedback were significantly more likely to have students with lower test scores. This may indicate that feedback was given selectively to only the lowest performing teachers, or it may indicate that feedback being given is ineffective.21 In either case, there is a pressing need for change, as the vast majority of teachers have a desperate need for effective professional development and they are not currently receiving it. 1. In-Service Principal Training Most principals will need additional training to become effective at teacher coaching and instructional leadership, especially in the area of student-centered teaching. While more than two- thirds of principals have received some formal training in school management, only about four percent have completed training lasting a month or more.22 Principals are generally former teachers; since most teachers have weak knowledge of student-centered teaching (LADLF 2016), principals are likely to need additional training to be able to cultivate these skills in their staff. The MoES should investigate options for providing modular training for in-service principals, including partnering with programs that deliver management training to other public servants. Training should be delivered first to principals who did not receive any initial school management training, those who are newest in their roles, and those at rural schools and schools serving large numbers of students whose mother tongue is not Lao-Tai. 2. Review of Pedagogical Advisor Program A review of the Pedagogical Advisor (PA) program is recommended to assess ways to improve its effectiveness. The review should identify the key areas of support the PAs currently give to teachers and document any measurable improvements in teacher practices or student learning that result from PA consultations. Clear qualifications for PAs should also be established or revisited to ensure that PAs 2Instructional leadership is the set of practices that school principals use to improve teaching and learning, and it is a strong predictor of how teachers collaborate and engage in a reflective dialogue about their practice (OECD 2016). 10 | The Learning Crisis in Lao PDR have the necessary high-level pedagogy skills and content knowledge to be able to demonstrate, advise, and support teachers in effective teaching for better student learning. PAs should be well-trained staff who can help teachers learn effective teaching practices and use them successfully in their classrooms. Training or retraining of PAs is necessary to build their capacity to support teachers more effectively. Training should address how to observe classrooms, engage in dialogue with teachers, and deliver feedback and demonstrations of teaching methods that improve teacher performance and student learning outcomes. Elimination or reassignment of PAs to other positions who do not improve and cannot effectively provide support to schools is essential for ensuring high quality and cost effectiveness of the PA program. PAs should work closely with principals to make sure pointed feedback (written or face-to-face meeting) is delivered to observed teachers. Currently, teachers are receiving the final report from the pedagogical advisor at only 38 percent of schools, according to principals’ reports.23 B. Placing Student Learning at the Center of Training for In-Service Teachers 1. Universal Training and Coaching on Content Knowledge, Pedagogy, and Effective Classroom Application Results of the SABER SD survey indicate that nearly every in-service teacher needs outcomes-focused professional development in both subject content knowledge and pedagogical skills. Regular, ongoing trainings and coaching for teachers are recommended, but these efforts should not follow the common “cascade” model, in which first leaders and then teachers are pulled out of their daily duties for stand-alone instructional modules. This method has been shown to be ineffective in changing either teachers’ pedagogical practices or improving outcomes for student learning. Training modules are recommended, but trainings should be targeted, ongoing, and focused on effective classroom application of the content, including hands-on demonstrations. Training sessions must also be accompanied by high-quality coaching informed by classroom observation in which teachers receive individualized feedback on their performance in their classrooms. Coaches should be assessed and selected carefully prior to the launch of programming. Other studies have shown that a combination of training and coaching produces good results in Lao PDR teacher performance, but coaching is effective only when delivered by coaches who are technically strong.24 Along with subject content knowledge, training and coaching should focus on the concepts and methods of student-centered teaching. Teachers should be reoriented to focus on student learning and allowed to make adjustments to their teaching methods and pace of delivery of the curricula to ensure that students grasp concepts before moving forward to advanced topics. Teachers should be introduced to the concept of feedback loops so they can understand how to adapt their instruction in response to shortcomings in student learning. Trainings should address effective strategies for testing students in order to identify where individual students are weak and need additional or different instruction. Teachers will need to receive feedback on their performance as they learn to implement these methods. Principals should also encourage action-research by teachers. This means giving teachers the opportunity to continuously learn new and varied teaching methods and then apply them in their own classrooms. This approach allows teachers to learn which teaching methods show an improvement in Challenges and Policy Priorities | 11 their students’ learning — and which do not. These methods could be shared among teachers at a school or even in school clusters. In-service teacher training and coaching should be strategically rolled out to tackle the areas showing the lowest returns on subject content knowledge for teachers and students. Results of the survey show that subject mastery is low across the board, but that teacher and student mastery of mathematics is lower than of Lao language. However, for students who are non-native Lao-Tai speakers, performance is extremely low in both language and math. Geographic areas with the worst performance (rural, high ethnic concentrations, and multi-grade schools) should be prioritized, and these areas may need more intensive coaching than others. 2. Teaching Non-Native Lao-Tai Speakers To improve educational equity, teachers at schools with high populations of students from ethnicities other than Lao-Tai should receive additional pedagogical and instructional support to effectively teach students who are learning in a language that is not their mother tongue. The MoES should also consider providing teachers at these schools with a more flexible curriculum that can be more easily adapted to the needs of non-native Lao-Tai speakers (LADLF 2016). The effort to improve Lao language skills for non Lao-Tai children should focus particularly on teachers working with the youngest children. Delays in development of Lao language skills impact achievement in all other areas of education since students who have not learned to read cannot read to learn. The results from direct student assessment suggest that the level of early literacy and numeracy skills in young children two to five years of age is very poor, with significant disparities across ethnicities (Brinkman et al. 2016a). Along with ensuring that early-grade teachers are well prepared for meeting the needs of students whose mother tongue is not Lao-Tai, the MoES should also examine options for supporting pre-school-age children’s’ development of skills for school and reading readiness. Skills developed in early childhood can position students whose mother tongue is not Lao- Tai to realize greater and faster gains from effective teaching in the early primary grades. 3. Multi-Grade Instruction Teachers at multi-grade schools should receive targeted professional development focused on effective multi-grade instruction. Multi-grade teaching has been shown to produce learning outcomes equal to or greater than single-grade instruction. However, as a recent study has observed, “to be effective, multi-grade teaching requires specific teaching resources and better skilled teachers than monograde instruction, which are less likely to be available in rural and remote areas where multi- grade classes in Lao PDR are more prevalent” (LADLF 2017). In addition, the MoES should develop a communication campaign to reorient teacher and parent perspectives on multi-grade instruction, as both groups’ perceptions of multi-grade instruction are very negative (LADLF 2017). Case studies and data can be used to demonstrate that multi-grade instruction can be a successful teaching practice when implemented well. 12 | The Learning Crisis in Lao PDR II. DEVELOPING A PIPELINE OF HIGH-ABILITY TEACHERS AND PRINCIPALS Achieving the greatest and most sustainable gains in student learning requires new teachers to have a higher level of ability upon their entrance into the classroom. Since the majority of teachers (85 percent) have completed a teacher training/vocational degree yet have very low content and pedagogical knowledge, an in-depth examination and reform of teacher pre-service education is needed.25 Setting high standards for entry into Teacher Education Institutions (TEIs) and equitable deployment of trained teachers across the country are similarly critical to raising the quality of entry- level teachers. Principals should also receive more rigorous initial school management training. A. Improving Pre-Service Teacher Training 1. Improved Teacher Education Curriculum Teacher Education Institutions (TEIs) should review their curricula and methods of assessing student- teachers’ knowledge and enforce minimum knowledge requirements for entry into the program. There should be a stronger focus on content knowledge for key subject areas and on pedagogical skills. Since other studies have found widespread lack of understanding of the theory and applications of student-centered pedagogy among in-service teachers, efforts should pay particular attention to this area (LADLF 2016). Applicants to TEIs should also be evaluated carefully to ensure they have achieved sufficient mastery of subject content knowledge prior to entrance to the program. TEIs should set higher minimum entry requirements and admit only those students meeting the criteria. The deficiencies in teachers’ subject content knowledge of Lao language and math that were assessed in the SABER SD survey reflect knowledge that should have been acquired long before entry into a TEI. 2. Required Practicum on Student-Centered Teaching A substantive teaching practicum should be a required component of pre-service teacher education. Effective teaching cannot be learned from lectures and coursework only: student teachers must have opportunities to practice their pedagogical skills, receive intensive feedback on their efforts, and see effective student-centered teaching concepts demonstrated. The SABER SD survey revealed that students are more likely to have higher test scores if their teachers participated in a teaching practicum.26 3. Professional Development for Teacher Education Instructors TEI instructors should receive professional development and coaching needed to employ student- centered teaching practices in their own classrooms, and program supervisors should observe classes to ensure instructors are using these methods consistently and effectively. Students in TEIs in Lao PDR currently have few opportunities to experience student-centered teaching, as program instructors generally rely on teacher-centered practices (LADLF 2016). 4. PA-TEI Collaboration The Pedagogical Advisor program and Teacher Education Institutions should collaborate to improve pre-service training and in-service professional development. PAs should regularly report the subject content and pedagogical areas in which in-service teachers are struggling so that the TEI curriculum Challenges and Policy Priorities | 13 can be adjusted accordingly. TEIs should act as a liaison to in-service teacher training, sharing the latest proven teaching techniques to improve classroom teaching and learning outcomes. B. Improving Initial Principal Leadership Training 1. Robust Training for Principals in School Management and Instructional Leadership Achieving long-term gains in principals’ capacity for effective leadership requires more robust initial principal training. Twenty-nine percent of principals have not completed any formal training in the management of a school. A large proportion of principals without any formal school management training oversee multi-grade schools. For the vast majority of those with formal training (95 percent), their training was only one to three weeks long.27 Regression analysis of the SABER SD survey data indicates that in initial school management training for principals is associated with small gains in student learning in comparison to schools where principals did not have school management training. Higher student test scores were associated with principals having received at least some school management training.28 Options for developing comprehensive initial principal training programs and making them accessible should be investigated and put in place. Completion of a robust formal, initial training for principal candidates with modules in school management and instructional leadership of minimum length should be a prerequisite for all principals before taking up their new post. This initial training should be followed by coaching from the district level and peer sharing among principals. When rolling out the initial principal training programs, steps should be taken to ensure they are accessible to women. Schools with female principals have student test scores that are higher than schools with male principals, and regression analysis showed that this association is large and highly significant.29 Among the factors that should be considered to ensure equal access for female candidates are travel requirements, scheduling of classes in evenings and on weekends, and availability of child care. III. REDUCING LOST INSTRUCTIONAL TIME A. Finding Root Causes of Classrooms Lacking Teachers 1. Draw on Best Practices for Teacher Attendance Management The root causes of teacher absences in Lao PDR should be examined to determine the most effective corrective actions, as efforts to improve teacher effectiveness will not improve student learning if students are in classrooms without teachers. Examination of teacher management practices at private schools may reveal best practices that could be applied in public schools. Teacher school absence rates are significantly lower at private schools than public schools. After adjusting for maternity leave,13.4 percent of teachers at public schools were absent from the school on a second unannounced visit, while only 1.9 percent of teachers at private schools were absent.30 Global research may also inform remedies for Lao PDR. High rates of teacher absenteeism are a common problem that is detrimental to learning in many countries. Pakistan is an example of a country 14 | The Learning Crisis in Lao PDR that has successfully adopted a school monitoring system to improve governance, accountability, and service delivery. By using digital technology in the form of smart phones and biometric devices that record thumb prints, the system allows real-time reporting to a centralized government dashboard. It is a scalable solution to address teacher absenteeism, missing facilities, and student attendance and enrollment. The system acknowledges teachers who are at work regularly and identifies those who are not. To date, more than 26,200 schools in Sindh Province and 210,000 teachers have been monitored. The system has enabled the Sindh government, through review meetings with districts, to plan and take necessary actions on school management and teacher absenteeism (World Bank 2017b). 2. Review of Substitute Teacher Policies It may also be useful to examine policies and processes for providing substitute teachers when teachers are absent for reasons other than maternity leave. The continuity and quality of instruction will be highest when students are taught by their regular teachers, and improving teacher attendance rates is the most critical strategy for addressing the high share of students in classrooms without teachers. Nonetheless, instruction from a substitute teacher is clearly preferable to students missing instruction entirely. In cases of planned absences for events, such as trainings or community festivals, schools should require teachers to request approval for their absence far enough in advance to leave ample time for securing a substitute instructor. Schools should also have guidelines for providing notification in as timely a manner as possible in cases of unplanned teacher absences, such as illness and family emergencies. Schools should maintain a pool of teachers who are able and qualified to serve as substitutes and follow protocols for timely scheduling of substitutes in cases of both planned and unplanned teacher absences. IV. CREATING ACCOUNTABILITY FOR IMPROVING STUDENT LEARNING A. District Supervision that Emphasizes Student Learning 1. Review of Effectiveness of Administrative Supervision Processes Processes for administrative supervision should be reviewed to determine how to increase their effectiveness at improving school and student performance. About three-quarters of public schools receive at least one administrative visit a year, and the average annual frequency of administrative supervision visits was 2.6, which is acceptable and indicates that there are funds to support administrative school visits annually.31 However, no statistically significant association was found between schools receiving visits and student test scores.32 The fact that administrative visits do not affect student learning outcomes indicates a need to review the nature of the supervision provided to schools and the characteristics of the supervisors. Oversight from district-level supervisors should be refocused on accountability for school performance, including improving student learning outcomes and monitoring the execution of school-level recommendations. 2. Sharing Administrative Supervision Visit Reports with School Stakeholders Recommendations from district administrative visits should be documented in writing, and these written reports should be shared with school management, VEDCs, and teachers at the school for better monitoring of school performance and to enable actionable responses. Currently only 13 Challenges and Policy Priorities | 15 percent of principals say they share written reports from administrative supervision visits with teachers.33 Administrative supervision reports are often filed at the district office but not used as a tool for helping to improve school procedures and processes that support better learning environments and outcomes. B. Increasing Schools’ Accountability to Their Communities 1. Building the Capacity of VEDCs All VEDCs should receive training in the knowledge required to hold schools accountable for student learning outcomes. Nearly all public primary schools have a functioning Village Education Development Committee (VEDC), but only half of VEDCs have received any type of training. Of that half, only 15 percent have been trained in understanding school and student performance at their school.34 Without the knowledge to understand how well their students are performing, it is difficult to set goals for school improvement, monitor progress against targets, and hold the school accountable for achieving set goals. It also makes it difficult for VEDCs and parents to advocate for support for their schools to the district level. Building capacity and making such training available to all VEDCs is an important step in empowering them to hold schools accountable for change and actively supporting their schools in setting the right goals and achieving the targets set forth in their school improvement plans. 2. Consequences for Failure to Address Community Concerns District supervisors should ensure that schools are held accountable for addressing demands of parents and VEDCs. Although most schools allow parents or the VEDC to demand changes, demands typically do not lead to any action by the schools. Nearly 89 percent of public school principals say that the VEDC and parents can use student performance results to demand accountability; only 26 percent of principals report that the school takes any action on those demands.35 School responsiveness to community demands should be addressed in every administrative supervision visit. District supervisors should contact VEDCs directly to ask them what issues they have raised with school administration and whether the issues have been addressed to their satisfaction. In addition, district offices may conduct periodic surveys of parents. District supervisors should address responsiveness to community demands through administrative supervision visits and deliver feedback to principals on their performance in this area. Principals and schools that consistently fail to address reasonable demands from their communities should face documented consequences for their inaction and be coached or receive resources if needed to make the necessary improvements. 16 | The Learning Crisis in Lao PDR V. CORRECTING INFRASTRUCTURE DEFICIENCIES TO SUPPORT A BETTER LEARNING ENVIRONMENT A. Improving Health and Safety in Schools 1. Adequate Numbers of Toilets The SABER SD survey revealed that many school facilities lack basic provisions for student health and safety, including access to an adequate number of toilets. While over 98 percent of schools had at least one functioning toilet, there are often too few toilets for the number of students enrolled. The MoES standards specify one toilet for every 45 to 75 students, but 40 percent of schools do not meet that standard. Across all schools, the average number of students for each toilet is 86. However, some schools have ratios greater than 200 students per toilet, and the recorded ratios reached as high as 305:1. Additional toilet facilities should be added to campuses, with priority given to schools most significantly outside MoES targets. In addition, separate toilets should be designated for boys and girls, per MoES guidelines. Currently only 23 percent of schools have separate toilets for girls and boys.36 2. Access to Handwashing Facilities and Drinking Water Handwashing facilities and drinking water should be made available at all schools. More than two- thirds of schools lack handwashing facilities, and more than a quarter do not have access to drinking water.37 Along with insufficient numbers of toilets, lack of handwashing facilities and adequate sources of potable water threatens student health and safety and increases the risk of students missing school from illness. Lack of these basic provisions also impacts staff. These factors impact student well-being, students’ pride in their schools, and general school-day efficiency. B. Ensuring Access to Functioning Classrooms The MoES should conduct an audit of school facilities to identify those with classrooms not configured for student learning and then target upgrading to those facilities. Approximately 41 percent of principals reported poor school infrastructure as a major constraint to the functioning of the school.38 These data align with findings of other studies that, particularly in rural areas, some schools are housed in inappropriate buildings or makeshift classrooms that are not configured for learning (LADLF 2017). The MoES should set goals for electrification to improve general school functioning and workday efficiency. Only 29 percent of observed classrooms had electricity.39 Upgrade campaigns should target the 16 percent of classrooms that did not have sufficient lighting contrast for students to read text written on blackboards. In some cases, classrooms were too dimly lit to read the board. In others, the high reflection from natural light made it difficult for students to read the text written on the board from all angles of the classroom. Problems with blackboard visibility are greatest in rural schools, where nearly one in five classrooms lack sufficient lighting contrast to read the board.40 In addition to allowing better classroom lighting, electricity makes it possible to introduce learning supported through technology and electronic management of school data and school development planning. Challenges and Policy Priorities | 17 Improving Financial Transparency In addition to identifying the actions with the greatest potential impact on student learning, the SABER SD survey also identified a need for greater financial transparency, which can have a significant impact on resources available to schools. Survey results raised questions as to whether schools are receiving all of the block grant funding to which they are entitled, and this issue should be investigated. More than three-quarters of principals reported that the annual per-student amount they were entitled to receive was LAK 50,000 (US$6.02 equivalent), but at the time of the survey the actual amount they were entitled to was LAK 70,000.41 (This amount increased to LAK 100,000 per student in the 2017–18 school year.) The average amount principals reported receiving for the 2015–16 school year is LAK 54,883, which is LAK 15,167 below that year’s allocation.42 Principals, VEDCs, the local community, parents, and teachers should all have visibility into how much money the school is supposed to receive, how much it is actually receiving, and how the money is being spent. This transparency and accountability will help ensure that schools are receiving all the block grant funding to which they are entitled in a timely manner. The SABER SD survey also documented that, despite widespread perception of public education being free, the cost of education falling to parents is very high. The total estimated average annual per-child cost to parents of children in public primary schools is LAK 540,602 (roughly the equivalent to US$65).43 Thus parental spending is more than five times higher than the per-child school block grant provided by the government. The largest portions of parental spending go to food, transportation, and uniforms. Other expenses include school maintenance, school events, and after- school classes.44 Lessening the parental cost of education has the potential to improve enrollment and retention as well as the health and academic performance of students who remain enrolled. High parental costs for education contribute to parents never enrolling their children in school, delaying their enrollment, or allowing them to drop out in early grades. Delayed enrollment has been shown to be strongly associated with early drop out, as students do not have time to achieve a level of education that would appear to justify their continued enrollment by the age when their direct and indirect economic contributions to their families reach heightened importance (Cerdan-Infantes et al. 2016). Even if parents keep children enrolled, high parental costs for education negatively impact both the family unit and students’ learning, as money spent on schooling cuts into funds available for providing children with the nutrition critical for day-to-day learning and proper brain development. More than a quarter of children under age five in Lao PDR are underweight, and 44 percent are estimated to be stunted (Cerdan-Infantes et al. 2016). 18 | The Learning Crisis in Lao PDR Conclusions In Lao PDR, like many other countries, although the government succeeded in getting more children in school, the education system is still unable to provide the most basic foundational skills as children reach the end of basic education. The level of student learning is low, and the gaps remain significant across geographical, ethnic, and socioeconomic groups. Although more children remain in school, substantial numbers are not achieving even the most basic levels of literacy and numeracy to enable them to pursue their education in higher grades. In the grander scheme, it also prevents Lao PDR from reaching its economic and social goals due to low human capital development. The SABER SD findings in Lao PDR provide a range of valuable evidence and inputs to the prioritization of policy actions to improve student learning. The highest priorities for improving student learning span five areas: 1. Increasing the effectiveness of the existing teaching force 2. Developing a pipeline of high-ability and motivate new teachers and principals 3. Reducing lost instructional time 4. Creating accountability for improving student learning 5. Correcting infrastructure deficiencies to support a better learning environment Effective teaching is among the critical inputs to student learning. In Lao PDR the quality of basic education is hindered largely by teacher absenteeism and teacher deficiencies in subject content and pedagogical knowledge. According to the SABER SD results, one in five classrooms in Lao PDR was without a teacher, and teachers have mastered only half of the content they are teaching. This suggests that teacher education programs should be reviewed and modernized to adopt evidence-based research approaches that have been successful such as continued follow-up approach, in-class pedagogical support, and ongoing feedback, not just a one-off training (Popova et al. 2017). Teacher training with continued follow-up support leads to higher learning gains (World Bank 2018). Other important areas that have not been examined directly or in depth through the SABER SD tool but are critical to policy considerations also exist and can be informed from other studies. Learner preparedness is one such area. Children in Lao PDR are not receiving sufficient health services and nutrition, and the stunting rate is considered a high public health concern (Brinkman et al. 2016b). The availability of Early Child Education (ECE) services is limited, with coverage varying significantly across geographic location and socioeconomic and ethnic groups. Increasing access to quality ECE services together with improving child nutrition will enhance child development and their readiness to learn at school. Challenges and Policy Priorities | 19 The most effective way to build fundamental skills for children is at the early stage of their lives (first 1,000 days of life), which can be done through improving maternal and child health, nutrition, and provision of quality early child development programs. Cognitive and socioemotional skills are the foundational skills that are most effective to be invested at early age (World Bank 2017a). Effective programs such as early child care and training of caregivers could promote healthy child development and their cognitive and socioemotional skills for future educational outcomes (World Bank 2018). If children are prepared to attend school, they are more likely to benefit from learning and less likely to repeat or dropout. While SABER SD does not collect data in this area, teacher support through effective use of digital technology could also be considered a mechanism to help complement teaching and assist teachers in teaching to the level of the students. New technologies, such as learner-assisted subject software or self-paced learning software that complements teaching, when well utilized, could assist teachers to reach students at varied learning levels and improve learning outcomes at scale. 20 | The Learning Crisis in Lao PDR References Brinkman, Sally, Alanna Sincovich, Plamen Nikolov Danchev, and Pedro Cerdan-Infantes. 2016a. Lao PDR Early Childhood Development Project. Snapshot Three: Child Development. World Bank, Lao, PDR. _____. 2016b. Lao PDR Early Childhood Development project. Snapshot Two: Child Health and Nutrition. World Bank, Lao, PDR. Cerdan-Infantes, Pedro, Jeffrey H. Marshall, and Emiko Naka. 2016. Reducing Early Grade Dropout and Low Achievement in Lao PDR: Root Causes Research and Possible Interventions. Care International, Dubai Cares, and World Bank, Vientiane, Lao PDR. Demas, Angela, Myra Murad Khan, Gustavo Arcia, and Emiko Naka. 2018. Delivery of Education Services in Lao PDR: Results of the SABER Service Delivery Survey, 2017. World Bank, Washington, DC. Government of Lao PDR, Ministry of Education and Sports. 2017. Statistics and Information Technology Center for Education and Sport Annual Report 2016–2017: Annex Matrix for School Year 2016–2017. Vientiane, Lao PDR. LADLF. 2016. The Readiness and Motivation of Teachers and Principals to Change Basic Education Teaching Practices in Lao PDR. Australian Aid, Canberra. LADLF. 2017. Rapid Appraisal of Literature on Multi-Grade Teaching in ASEAN and Other Countries. Australian Aid, Canberra. OECD. 2016. School Leadership for Developing Professional Learning Communities. Teaching in Focus, No. 15. OECD Publishing, Paris. Popova, Anna, David K. Evans, Mary E. Breeding, and Violeta Arancibia. 2017. Global Landscape of Teacher Professional Development Programs: The Gap between Evidence and Practice. World Bank, Washington, DC. World Bank. 2017a. Africa’s Pulse: An Analysis of Issues Shaping Africa’s Economic Future. October. Volume 16. World Bank, Washington, DC. World Bank. 2017b. Feature Story: Smart Solutions to Improve Pakistan’s Education. February 13. World Bank, Washington, DC. World Bank. 2018. World Development Report: Learning to Realize Education’s Promise. World Bank, Washington, DC. Challenges and Policy Priorities | 21 Appendixes APPENDIX A: BACKGROUND OF SABER SERVICE DELIVERY SURVEY The SABER Service Delivery survey tool was developed in 2016 in the Global Engagement and Knowledge Unit of the Education Global Practice (GP) at the World Bank, as an initiative to uncover bottlenecks that inhibit student learning in low-income and middle-income countries and to better understand the quality of education service delivery in a country as well as gaps in policy implementation. In alignment with the World Development Report (WDR 2018), the SABER SD instrument examines four key elements in an education system identified as the main determinants for student learning. These are learner preparedness, teacher knowledge and motivation, availability of inputs, and school management and governance. These elements are interactive and directly affect student learning. One of the main premises of the WDR 2018 is that schooling is not the same as learning. An education system that lacks one or more of these elements would struggle to make learning happen. The SABER SD instrument, through its comprehensive, school-based modules, captures information in these four elements to examine the education system of a country and identifies bottlenecks that prevent student learning as well as gaps in policy implementation. The SABER SD survey collects strategic information on school inputs and processes that influence learning outcomes. The data collected aim to uncover the extent to which policies translate into implementation and practice. As a global initiative, SABER SD provides data for the new global lead indicator on learning, which makes it easier to monitor the Sustainable Development Goal of achieving universal primary education. SABER SD was created using knowledge and expertise from two major initiatives at the World Bank: SABER (Systems Approach for Better Education Results) and the SDI (Service Delivery Indicators) tools. The SABER program conducts research and knowledge from leading expertise in various themes of education. Using diagnostic tools and detailed policy information, the SABER program collects and analyzes comparative data and knowledge on education systems around the world and highlights the policies and institutions that matter most to promote learning for all children and youth. The SDI program is a large-scale survey of education and health facilities across Africa. The new SABER SD tool builds on and contributes to the growing SABER evidence base by capturing policy implementation measures identified as important in the frameworks of the core SABER domains of School Autonomy and Accountability, Student Assessment, Teachers, Finance, Education Management Information Systems, and Education Resilience. The SABER SD instrument collects data at the school level and asks questions related to the roles of all levels of government (including local and regional). The tool provides comprehensive data on teacher effort and ability; principal leadership; 22 | The Learning Crisis in Lao PDR school governance, management, and finances; community participation; and student performance in math and language and includes a classroom observation module. In addition to this, SABER SD: • Provides policy makers with a quick, comprehensive overview of how their education system is currently functioning • Complements existing SABER domain research to deepen the knowledge and understanding of education systems • Identifies breakdowns in policy implementation at various levels of government • Provides metrics to assess the quality of education service delivery in a country • Links gaps in policy and policy implementation to real consequences in the classroom • Provides data that could help bridge dialogue among financiers, policy makers, and frontline education service providers and • Generates information on the implementation of country policies benchmarked by SABER. Challenges and Policy Priorities | 23 APPENDIX B: SABER SD SERVICE DELIVERY PROTOCOLS Methodology Fieldwork Protocols, Duration and Months of Implementation, Standards of Data The SABER SD survey in Lao PDR was implemented from December 2016 to March 2017 following a customization of the survey instrument for the country and translation into the national language. The SABER SD team worked closely with IndoChina Research (IRL), a local survey firm that was hired to implement the field survey. The two teams coordinated the work throughout the stages of survey preparation, survey implementation training, and data collection as part of the quality assurance process. The questions in each of the six SABER SD survey modules were reviewed carefully prior to the training for data collection and adapted to the Lao context.45 The questionnaires were further revised after field testing them in 23 schools in and near the capital of Vientiane during the training of team leaders and enumerators. As part of the quality control measures, the Bank team delivered training to the IRL supervisors, team leaders, enumerators, and auditors. The training for data collection was delivered in two parts: (i) 10-day team leaders training in December 2016 and (ii) 14-day enumerators and auditors training in February 2017. In the second training, IRL supervisors and team leaders had the opportunity to practice the skills they learned from the first training as they joined the Bank team in delivering the training to the enumerators and auditors. The enumerator training was expanded to allow for additional practice time and testing in 20 schools near and in Vientiane. The final group of enumerators were selected at the end of the training, and only those qualified proceeded to the fieldwork. The actual data collection was conducted during the months of February to March 2017 in 200 schools across all 18 provinces in Lao PDR. In an effort to streamline data collection and reduce error, SABER SD piloted the use of tablets with the questionnaires programmed in Computer-Assisted Personal Interviewing (CAPI) form. During the field work, enumerators entered data directly into the tablets, which were able to instantaneously report data through a mobile internet connection to the central server for review. Through enumerator training sessions, enumerators were able to familiarize themselves with the questionnaires programmed in the tablets and trial the CAPI questionnaires in schools. This also helped strengthen the accuracy and functionality of the program. To reinforce standards of data, a set of monitoring procedures was designed by the technical team and conducted by the survey supervisors on a day-to-day basis, including observation of data collection and back-checks of data entry done by enumerators. Each survey team performed data checks and debriefing at the end of the day and transferred data from tablets to the IRL office in Vientiane. The data-processing team in Vientiane then conducted further verification to ensure the completion and logicality of data entry for all questionnaires. A complete set of SABER SD data was submitted to the Bank team in May 2017 for verification and analysis. The details of fieldwork protocol are described in the technical report, “Delivery of Education Services in Lao PDR: Results of the SABER Service Delivery Survey, 2017” (Demas et al. 2018). 24 | The Learning Crisis in Lao PDR Sampling The SABER SD Survey was implemented in 200 schools across all 18 provinces in Lao PDR. Since the survey focuses on grade 4 students, sampling was conducted using probability proportional to size (PPS) on the grade 4 student population reported by schools in the national Education Management Information System (EMIS). The 200 schools that were chosen for the final sample were categorized into four different strata for comparative analysis: urban/rural, public/private, single grade/multi- grade, and priority/non-priority districts. The analysis in the report examines the differences between these categories of schools. Four different sampling scenarios were drawn up, after which the final sample was selected that would enable enough statistical power for each of the divisions in the strata as well as keep errors to a minimum. For the final sample of schools in each stratum and detailed sampling information, see the technical report (Demas et al. 2018). Instrument Structure and Summary of the Instrument, Description of the Modules The SABER Service Delivery Survey comprises six modules. Each module has its own questionnaire. Table B1 summarizes the different modules, including the interviewee and description of data that the module collects. The survey was administered in-person by teams of enumerators. For most of the survey, enumerators read questions to the respondent and recorded answers on the electronic questionnaires. In specific parts of the survey, respondents wrote down their own answers on paper, and those answers were transferred to the electronic questionnaires. The specific sections included the student assessment (answers were immediately transferred to tablet) and the teacher assessment (answers transferred later and trained markers used the paper assessments to review answers). All questionnaires were in Lao language. Challenges and Policy Priorities | 25 Table B1: Structure of the SABER Service Delivery Survey Module Module Title Interviewee Description Module 1 School Principal and Direct Administered to the principal to collect basic school Information Observation information (location, type of school, etc.), state of facility, student enrollment per grade, school hours and shifts. Module 2A, Principal and Principal and 2A Administered to the principal to collect a staff 2B and 2C Teacher Roster Teachers roster of all teachers at the school. 2B. Measures principal and assistant principal absence. Collects information on principal and assistant principal remuneration. 2C. Measures teacher absence from classroom and school. Administered to 10 randomly selected teachers and the grade 4 observed teacher. Captures teacher characteristics, as well as information about teacher pay, initial training, professional development, and teacher supports. Module 3 School Principal Administered to the principal to collect information Governance, about school leadership and management, Management governance, community participation, school and Finance assessment and supervision, student assessment, information for accountability, decision-making and improvements, and school financing. Module 4 Classroom Direct Observation Classroom observation of a grade 4 language or Observation math class that assesses teacher delivery of a lesson. It records time-on-task and student-teacher 4A: Time on Task interactions. The module also captures information 4B: Classroom about available teaching and learning supplies, Environment classroom conditions, and conducts a brief interview 4C: Teaching with the teacher at the end of the class. 4D: Questions to Teachers 4E: Students Module 5 Student Grade 4 Students Assesses student performance in Lao language and Assessment mathematics from the same grade 4 classroom that was observed. Tests up to 10 students on both mathematics and Lao language. Module 6 Teacher Teachers Assesses teacher performance in three areas: (i) Assessment content knowledge of Lao language, (2) content knowledge of mathematics, and (3) pedagogical content knowledge. It is a two-part test administered to relevant teachers. 26 | The Learning Crisis in Lao PDR APPENDIX C: SELECT EDUCATION SERVICE DELIVERY RESULTS LAO PDR SABER Service Delivery Survey, 2017 Areas of Accomplishment and Progress Teacher Credentials and Teachers with technical/vocational degree or higher 87% Time Spent Teaching Classroom time spent on teaching 78% Average Class Size Observed student-teacher ratio 24.9 Delivery of Assessments Schools with annual student assessment exams 97.1% Access to Learning Students with a pencil 98.3% Materials Students with a textbook 88.2% Classrooms with a blackboard 100% Classrooms with chalk 99.3% Governance Structures Schools with functioning VEDC 99.4% (Public Schools) Average number of VEDC meetings/year 3.4 VEDCs that made recommendations in the past year 94.6% Schools receiving one or more administrative supervision visit a year 78.2% Average number of administrative visits/year 2.6 Schools receiving at least one Pedagogical Advisor visit a year 70.9% Average number of pedagogical advisor visits/year 2.0 Schools holding at least one community meetings/year on School Development Plan 92.1% Areas of Need Teacher Knowledge Grade 4 teachers achieving minimum score of 80% correct on language and math 2.4% Average teacher score in math 49.2% Average teacher score in Lao language 55.4% Grade 4 teachers with passing score (80%) for pedagogical knowledge 1.8% Average teacher score in pedagogical knowledge 27.4% Access to Instruction Classrooms without a teacher 1 in 5 Teacher Coaching Public school teachers reporting they received feedback from the principal 14.0% and Professional Public school teachers reporting they received feedback from a Pedagogical Advisor 6.5% Development Principal time spent on teacher management 19% Accountability for Principals sharing recommendations from administrative visits with staff in writing 13.3% Student Learning VEDCs receiving training of any kind 50% (Public Schools) VEDCs with training in understanding school and student assessments (among those 15% receiving any training) Principals reporting school takes any action when VEDC/parents demand accountability for 26% student results Funding Transparency Principals citing amount of block grant as LAK 50,000 per student (vs. actual amount of 78% and Access LAK 70,000) Average gap between per-student block grant funding principals report receiving and -LAK amount stipulated by national policy 15,167 Access to Adequate Principals reporting poor infrastructure as a major constraint to school functioning 41% Facilities Schools without at least 1 toilet for every 75 students 40% Schools without handwashing facilities 65% Schools without potable water 28% Schools without electricity 71% Classrooms without sufficient lighting contrast to see the board (public schools) 15.4% Challenges and Policy Priorities | 27 1 Demas et al., 39. 2 Demas et al., 22. 3 It should be noted that while most children are enrolled in primary school, many still lack access to complete schools. A recent study on reducing early-grade dropout found that 40 percent of primary school students who dropped out left schooling at a grade that was not offered in their village. However, the same study concluded that “the main reasons for never attending or dropping out early are related to perceptions [that] the quality of education is low or that it lacks relevance” (Cerdan-Infantes et al. 2016, 36, 14). 4 Demas et al., 25. 5 Demas et al., 23. 6 Demas et al., 23. 7 Demas et al., 25. 8 Demas et al., 22. 9 Demas et al., 76–9. 10 Demas et al., 27. 11 Demas et al., 25. 12 Demas et al., 25, 23. 13 Demas et al., 115. 14 Demas et al., 39. 15 Demas et al., 76. 16 Demas et al., 35. 17 Demas et al., 33–5 18 Demas et al., 37. 19 Demas et al., 50. 20 Demas at al., 61, 49–50. 21 Demas et al., 77. 22 Demas et al., 56. 23 Demas et al., 61. 24 Demas et al., 82. The Room to Read program has achieved better student outcomes in Lao PDR by placing literacy coaches in classrooms alongside teachers and implementing an approach to teaching children how to read, write, and develop the habit of reading rooted in scientifically based instructional methods. Second-grade students in the program read 3.5 times as fast as students in nearby control schools (https://www.roomtoread.org/countries/laos/?tab=country percent20summary). 25 Demas et al., 30–2. 26 Demas et al., 79. 27 Demas et al., 55–6. 28 Demas et al., 103. 29 Demas et al., 77. 30 Demas et al., 35. 31 Demas et al., 59. 32 Demas et al., 79. 33 Demas et al., 59. 34 Demas et al., 63–4. 35 Demas et al., 63–4. 36 Demas et al., 71–2. 37 Demas et al., 71–2. 38 Demas et al., 58. 39 Demas et al., 70–1. 40 Demas et al., 70–1. 41 Exchange rate: US$1 = LAK 8,300 (Laotian kip) as of January 24, 2018. 42 Demas et al., 66. 43 Exchange rate: US$1 = LAK 8,300 (Laotian kip) as of January 24, 2018. 44 Demas et al., 67–8. 45Details of the SABER SD modules are described in Section 1.1.3 of Demas et al. 28 | The Learning Crisis in Lao PDR Systems Approach for Better Education Results saber.worldbank.org What is the real quality of education in low- and middle-income countries — not according to policy statements, but measured by what students actually experience in schools and classrooms? SABER Service Delivery (SABER SD), an initiative of the Education Global Practice, answers this question by fielding school surveys to collect data on the inputs and processes that most affect student learning. With this new data, SABER SD uncovers how well education policies are translating into results on the ground, and it flags school-level bottlenecks that need to be addressed to improve learning. The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the International Bank for Reconstruction and Development/The World Bank Group and its affiliated organizations, or those of the Executive Directors of the World Bank Group or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Ministry of Education and Sports