TEACHER PRACTICES IN INDONESIA Results of the Teach Primary Classroom Observation Study World Bank June 2024 Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study This work is a product of the staff of The World Bank. This work was supported by the Foundational Learning Compact multidonor umbrella trust fund. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions © 2024 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for non-commercial purposes as long as full attribution to this work is given. All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Attribution Please cite the work as follows: Dini, Indah; Kim, Seil; Nomura, Shinsaku. 2024. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study. World Bank, Jakarta. © World Bank. Contact Information The authors can be contacted at snomura@worldbank.org. Photo & Graphics Credit: Envato TABLE OF CONTENTS Table of Contents 03 3 List of Acronyms 04 Acknowledgements 04 Executive Summary 05 Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 1 INTRODUCTION 09 2 THEORETICAL FRAMEWORK: 11 CAPTURING TEACHER PRACTICES 3 TEACH RESULTS: 14 INSIGHTS INTO TEACHER PRACTICES 3.1. Overview of Teach Results in Indonesia 14 3.2 Time on Task 16 3.3 Area A: Overview of the Classroom Culture Area 17 Result 3.4 Area B: Overview of the Instruction Area Result 20 3.5 Area C: Overview of the Socioemotional Skills 25 Area Result 4 COMPARATIVE ANALYSIS OF TEACHING PRACTICES 29 4.1. Teaching Practices Analysis: MoECRT and MoRA 29 4.2. Teaching Practices Analysis: Urban and Rural 30 4.3. Teaching Practices Analysis by Teacher 32 Characteristics 4.4. Teaching Practices Analysis by Academic 34 Subject and Curriculum 5 CONCLUSION AND RECOMMENDATION 36 References 39 Appendices 41 A1. Teach Sampling and Data Collection in Indonesia 41 A2. Summary Statistics 44 A3. Inter-item Correlations 46 A4. Comparative analysis of Teach Scores Across Groups 47 A5. Teach Score Distribution and Rubrics at A Glance 53 List of Acronyms CLASS Classroom Assessment Scoring System FFT Framework for Teaching KTSP Kurikulum Tingkat Satuan Pendidikan (School- Based Curriculum) MoECRT Ministry of Education, Culture, Research, and Technology MoRA Ministry of Religious Affairs SDI Service Delivery Indicator SD Standard Deviation Acknowledgements This report was written by Indah Dini, Seil Kim, and Shinsaku Nomura from the Education Global Practice, East Asia and Pacific Region (HEAED). Alexander Michael Tjahjadi and Delbert Lim provided support to data cleaning and Anna Hata provided valuable feedback to improve this report. The authors are grateful for the overall guidance provided by Cristian Aedo (Practice Manager, HEAED). The report benefited from peer review comments from Harry Patrinos (Senior Advisor, HEDDR), Marie-Helene Cloutier (Senior Economist, HEDGE), and Adelle Pushparatnam (Senior Education Specialist, HMNED). This work received support from the Teach/Coach Grants: Scaling-Up National Support for Effective Teaching (SUNSET) Trust Fund, which is co-financed by the World Bank and the LEGO Foundation. The data was collected as part of the Learning Loss Survey in 2023 funded by the Government of Australia’s Department of Foreign Affairs and Trade through the Australia-World Bank Indonesia Partnership. Data collection was undertaken by SurveyMeter. Data was collected by ten enumerators who were chosen by SurveyMeter and passed the Teach Primary certification exam, a prerequisite to take part in the study. Finally, the team would like to extend its sincerest gratitude to all the teachers who allowed us to record their classrooms for the Teach Primary training. This publication was designed by Nuriza Saputra. 5 EXECUTIVE SUMMARY Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study What is Teach and How It Used in This Study Teach Primary (hereafter “Teach”) is a classroom observation tool developed by the World Bank that has been used in over 30 middle income countries across the world. It captures the quality of teaching practices by measuring (i) time on task: the time teachers spend on learning and the extent to which students are on task, and (ii) the quality of teaching practices measured by three primary areas: Classroom Culture, Instruction, and Socio-emotional Skills. The tool underwent a rigorous development and validation process which has met the appropriate psychometric criteria of reliability.1 As part of the Time on Task component, three “snapshots” of 1–10 seconds are used to record both the teacher’s actions and the number of students who are on task throughout the observation. The quality of teaching practices is evaluated in three areas: Classroom Culture, Instruction, and Socio-emotional Skills. These areas have nine corresponding elements that point to twenty-eight behaviors. The behaviors are characterized as low, medium, or high, based on the evidence observed in this classroom. These preliminary scores are translated into a five- point scale, which quantifies the teacher’s practices as captured in two, 15-minute observations. This study employs Teach to investigate the current landscape of teaching practices in Indonesia. A total of 993 observations were collected, encompassing 501 teachers at two time points during their classes—the first and last 15 minutes. It included 405 primary schools, with 54 percent from the Ministry of Education, Culture, Research, and Technology (MoECRT) and 46 percent from the Ministry of Religious Affairs (MoRA), strategically chosen to ensure national representativeness. Grade 4 primary school classrooms were observed across subjects, including Mathematics (46%), Language (Bahasa Indonesia, 31%), and other subjects (23%). These sample schools were selected based on the 2019 Service Delivery Indicators (SDI) survey, with necessary adjustments to ensure a nationally representative sample (See Appendix 1 for details on the sampling process and the sample breakdown) (World Bank, 2023). 1 Teach Primary: Helping Countries Track and Improve Teaching Quality 6 Summary of Key Findings Time on Task Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 1 Teachers spend their time in the classroom on teaching activities although students pay attention only about half of the time during classes. Teachers in Indonesia provide a learning activity to students for 96 percent of the time. However, when teachers provide a learning activity, all students are on task only 48 percent of the time. Quality of Teaching Practices Indonesian teachers generally perform well on Classroom Culture (88 2 percent of teachers score three or higher) but moderate on Instruction (26 percent score three or higher) and low on Socioemotional Skills (10 percent of teachers score three or higher). This pattern aligns with findings from other developing countries worldwide (World Bank, 2021) and the East Asia and Pacific Region where less than one-third of teachers achieve a score of three or above, indicates ineffective teaching practices, resulting in inferior learning outcomes (Afkar et.al., 2023). • Classroom Culture (scoring 3.4 out of 5): Teachers were somewhat effective in creating a supporting learning environment (3.5/5) and somewhat effective in setting positive behavioral expectations (3.4/5). • Instruction (scoring 2.5 out of 5): Teachers were somewhat effective at facilitating lessons (3.2/5), less effective at checking for understanding (2.7/5), less effective in encouraging students to think critically (2.4/5), and poor at providing feedback (1.8/5). • Socioemotional Skills (scoring 2.1 out of the 5): Teachers were less effective at Autonomy (2.5/5), also less effective at Perseverance (2.2/5), but were poor at Social and Collaborative Skills (1.6/5 – lowest among nine elements). Disparity in Teaching Quality Among Different Groups 3 MoECRT educators displayed slightly stronger teaching practices than MoRA educators across multiple dimensions, with notable expertise in providing clear and constructive feedback in the Instruction area. • MoECRT teachers achieved slightly higher average Teach scores (2.7) compared to MoRA teachers (2.6). • In the Instruction area, MoECRT teachers showed a relatively higher proficiency in providing feedback, with a difference in average score of 0.16, translating into 0.2 standard deviation (SD). This indicates that MoECRT teachers tend to deliver clearer and more constructive feedback. 7 4 Urban teachers outperform their rural counterparts in several teaching elements, including Classroom Culture (Urban 3.6, Rural 3.4) and Instruction (Urban 2.7, Rural 2.5) indicating disparities in the classroom environment and instructional approaches. • The most notable difference is observed in the critical thinking component, Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study where urban teachers (2.8) outpace rural teachers (2.3). This highlights the urban teachers’ ability to encourage open-ended thinking and thought- provoking tasks, indicating a potential area for improvement in rural educational settings. 5 Female and highly educated teachers outperform other groups. • Female teachers outperform male teachers, particularly in Socioemotional Skills. This suggests that female educators excel in fostering social and collaborative learning among students (Female 1.7, Male 1.5). • Teachers with higher education levels also consistently achieve higher Teach scores. This trend is especially noticeable in the areas of Instruction and Socioemotional Skills. It emphasizes the importance of targeted teacher training programs to improve teaching practices, particularly for educators with lower levels of education. 6 Curriculum and type of subject influence teaching practices. • Some schools in Indonesia continue to implement the old Kurikulum Tingkat Satuan Pendidikan (KTSP) 2013 Curriculum (i.e. School-Based Curriculum), while others have updated to the latest curriculum called the Merdeka Curriculum. • Our study finds that curriculum choices correlate with teaching effectiveness, with teachers under the Merdeka Curriculum showing strengths in particular areas. Specifically, they show enhanced capabilities in the Teach elements of lesson facilitation (0.3 points higher, 0.25 SD) and critical thinking (0.4 points higher, 0.40 SD) compared to those adhering to the KTSP 2013 Curriculum. • Mathematics classes exhibit higher performance compared to language and other subjects, with higher scores in areas such as Classroom Culture and Instruction. Given that a single classroom teacher typically handles all subjects in primary schools in Indonesia, the significant variations in Teach scores across academic subjects underscores the need to enhance teaching methodologies. 8 Suggestions and Recommendations from Teach Results 1 Tailored teacher development is essential for Indonesia. • In line with trends in similar countries, Indonesian teachers excel in Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study fostering a positive classroom culture but require significant improvements in Instruction and Socioemotional skills areas. This underscores the need for targeted teacher training programs aimed at enhancing instruction and socioemotional skills, ultimately benefiting students’ cognitive and socioemotional development. 2 Curriculum and school differences should align more closely with modern teaching methods. • The substantial differences in Teach scores across curriculum choices, school types (e.g., MoECRT and MoRA), and academic subjects may suggest the need for educational policymakers to consider aligning curricula with modern teaching methodologies. Additionally, teacher training programs should be tailored to specific curricular requirements, focusing on areas where each curriculum may need improvement. This alignment can enhance overall teaching quality and improve students’ critical thinking abilities. 3 Educational divides need to be bridged. • The notable disparities between urban and rural teachers, particularly in terms of fostering critical thinking among students, highlight the need for targeted support and professional development initiatives in rural educational settings. Policymakers should invest in training programs that equip rural teachers with strategies to encourage open-ended thinking and thought-provoking tasks, bridging the urban-rural divide in teaching quality. • The positive correlation between higher education levels and teaching proficiency emphasizes the significance of teacher education. Policies should encourage teachers to pursue advanced degrees and provide opportunities for ongoing professional development to elevate teaching practices, particularly among educators with lower levels of education. 9 1 INTRODUCTION Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study School enrollment has increased substantially over the last 25 years in low and middle-income countries. However, despite this growth, the quality of schooling, such as basic skills like reading, writing, and arithmetic, remains a challenge (World Development Report, 2018). The expansion of access to school has also seen a rise in Indonesia over the last three decades, but the concern for the quality of education persists. According to the World Bank Human Capital Index (2020), although Indonesian students spend an average of 12.4 years in school, they acquire only 7.8 years’ worth of learning, indicating a gap between time spent in school and actual learning outcomes. Furthermore, 35 percent of Indonesian children at late primary age struggle with reading proficiency. This is higher compared to neighboring countries such as Malaysia (25 percent), the Philippines (20 percent), Thailand (15 percent), and Vietnam (10 percent) (World Bank, 2019). Around the world, the learning crisis is, at its core, a teaching crisis (Bold et al., 2017). This report details the nature of teaching practices across Indonesia as captured by the high-inference classroom observation tool, Teach. Identifying effective teaching is not straightforward. Research indicates teacher characteristics such as formal education, years of experience (beyond the first two), cognitive skills, and entry exam performance scores only explain a small fraction of the variation in teacher effectiveness (Staiger & Rockoff, 2010; Araujo et al., 2016; Bau & Das, 2017; Cruz-Aguayo et al., 2017). Variation in student learning is better explained by teachers’ practices in the classroom. For example, a seminal study in Ecuador found a one standard deviation (SD) increase in teacher quality, as measured by teachers’ scores on the CLASS observation tool2, is associated with a 0.18 SD increase in learning outcomes (Araujo et al., 2016). Moreover, teachers’ scores on classroom observation tools in the United States are positively associated with student achievement gains (Kane & Staiger, 2009; Kane & Staiger, 2012; Hamre et al., 2014; Holtzapple, 2003; Milanowski, 2004). However, it is not simply teacher practices that exhibit positive effects, as the improvement of their practices also has positive effects on student outcomes. For instance, students of Chilean teachers who were given access to classroom observation feedback and coaching performed .05-.09 SD higher on state tests and .04-.06 SD higher on national tests than those whose teachers did not receive such feedback (Bruns et al., 2016). Moreover, a study of over 60 coaching programs found those designed to advance teacher practices (0.58 SD) also resulted in increased student learning (0.15 SD) (Kraft et al., 2018). 2 The Classroom Assessment Scoring System (CLASS) is a classroom observation tool for analyzing the quality of teacher-student interactions in the classroom on a scale from 1-7 across three broad domains: emotional support, classroom organization, and instructional support. 10 Background of This Study There are over three million teachers in Indonesia who are devoted to educating approximately 54 million students (World Bank, 2023). This study aims to comprehensively analyze teaching practices using Teach classroom observation tool Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study in Indonesia classrooms. This report is part of the World Bank’s broader Indonesia Learning Loss research initiative. In the context where the education system faces diverse challenges, ranging from education governance between ministries to teacher quality and resource disparities in rural and urban areas, understanding and measuring teaching practices can recommend effective strategies, ultimately contributing to the ongoing efforts to recover from learning loss, improving student learning outcome and enhancing overall educational quality. This Report This report is organized as follows: Section 2 describes the Teach theoretical framework, content, and development process. Section 3 provides an overview of teaching practices in Indonesia using Teach, while Section 4 is a comparative analysis across school types, environments, teacher characteristics, academic subjects, and curricula. 11 2 THEORETICAL FRAMEWORK: CAPTURING TEACHER PRACTICES Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Over the course of a teacher’s lesson, Teach measures: (i) the time teachers spend on learning and the extent to which students are on task, and (ii) the quality of teaching practices that help develop students’ socio-emotional and cognitive skills. As part of the Time on Task component, three “snapshots” of 1–10 seconds are used to record both the teacher’s actions and the number of students who are on task throughout the observation. The Quality of Teaching Practices component, on the other hand, is organized into three primary areas: Classroom Culture, Instruction, and Socio-emotional Skills.3 Figure 2.1: Teach Framework TIME ON TIME ON LEARNING TASK + SUPPORTIVE LEARNING POSITIVE BEHAVIORAL QUALITY OF ENVIRONMENT EXPECTATIONS CLASSROOM TEACHING CULTURE PRACTICES LESSON CHECKS FOR FEEDBACK CRITICAL FACILITATION UNDER- THINKING INSTRUCTION STANDING AUTONOMY PERSEVERANCE SOCIAL & SOCIOEMOTIONAL COLLABORATIVE SKILLS SKILLS Source: World Bank (2022) 3 It should be noted that it is impossible to draw a clear line between teacher practices linked to academic versus socio-emotional learning. Many teacher practices included in common professional teaching frameworks do impact student’s socio-emotional development, though are usually thought of in terms of academic rather than socio-emotional learning. Explicitly linking teacher practices with socio-emotional outcomes in measures used for assessment will serve to increase the salience of student’s socio-emotional skills to teachers, as well as to other stakeholders and policymakers, thus ensuring a focus on both academic and socio-emotional learning in the classroom. Table 2.1. Description of Teach Instrument 12 Area Element Behavior (Score: 1-5) (Score: Low, Medium, High) Time on Task 0. Time on 0.1 Teacher provide activities to most students Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Learning 0.2 Students are on task Quality of Teaching Practices A. 1. Supportive 1.1 Teacher treats all students respectfully Classroom Learning 1.2 Teacher uses positive language with students culture Environment 1.3 Teacher responds to students’ needs 1.4 Teacher does not exhibit bias and challenges stereotypes in the classroom 2. Positive 2.1 Teacher sets clear behavioral expectations for Behavioral classroom activities Expectations 2.2 Teacher acknowledges positive student behavior 2.3 Teacher redirects misbehavior and focuses on the expected behavior, rather than the undesired behavior B. 3. Lesson 3.1 Teacher explicitly articulates the objectives of Instruction Facilitation the lesson and relates classroom activities to the objectives 3.2 Teacher explains content using multiple forms of representation 3.3 Teacher makes connections in the lesson that related to other content knowledge or students’ daily lives 3.4 Teacher models by enacting or thinking aloud 4. Checks for 4.1 Teacher uses questions, prompts or other Understanding strategies to determine students’ level of understanding 4.2 Teacher monitors most students during independent/group work 4.3 Teacher adjusts teaching to the level of students 5. Feedback 5.1 Teacher provides specific comments or prompts that help clarify students’ misunderstandings 5.2 Teacher provides specific comments or prompts that help identify students’ successes 6. Critical 6.1 Teacher asks open-ended questions thinking 6.2 Teacher provides thinking tasks 6.3 Students ask open-ended questions or perform thinking tasks C. 7. Autonomy 7.1 Teacher provides students with choices Socio-emotional 7.2 Teacher provides students with opportunities to skills take on roles in the classroom 7.3 Students volunteer to participate in the classroom 8. Perseverance 8.1 Teacher acknowledges students’ efforts 8.2 Teacher has a positive attitude towards students’ challenges 8.3 Teacher encourages goal setting 9. Social and 9.1 Teacher promotes student collaboration through collaborative peer interaction skills 9.2 Teacher promotes students’ interpersonal skills 9.3 Students collaborate with one another through peer interaction Source: World Bank (2022) The three quality of teaching practices’ areas have nine corresponding elements 13 that point to 30 behaviors (See Figure 2.1). The behaviors are characterized as low, medium, or high, based on the quality of teacher practices observed. These behavior scores are translated into a five-point scale that quantifies teaching practices as captured in a series of two, 15-minute lesson observations. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Classroom Culture: The teacher creates a culture that is conducive to learning. The focus here is not on the teacher correcting students’ negative behaviors but rather on the extent to which the teacher creates: (Element 1) a supportive learning environment by treating all students respectfully, consistently using positive language, responding to students’ needs, and both challenging gender stereotypes and not exhibiting bias (against gender or students with disabilities) in the classroom; and (Element 2) positive behavioral expectations by setting clear behavioral expectations, acknowledging positive student behavior, and effectively redirecting misbehavior. Instruction: The teacher instructs in a way that deepens student understanding and encourages critical thinking and analysis. The focus here is not on content-specific methods of instruction, but rather the extent to which the teacher: (Element 3) facilitates the lesson by explicitly articulating lesson objectives that are aligned to the learning activity, clearly explaining content using multiple forms of representation, and connecting the learning activity to other content knowledge or students’ daily lives, and by modeling the learning activity through enacting or thinking aloud; (Element 4) does not simply move from one topic to the next but checks for understanding by using questions, prompts, or other strategies to determine students’ level of understanding, by monitoring students during group and independent work, and by adjusting his/her teaching to the level of students; (Element 5) gives feedback by providing specific comments or prompts to help clarify students’ misunderstandings or identify their successes; and (Element 6) encourages students to think critically by asking open-ended questions and providing students with thinking tasks that require them to actively analyze content. Students exhibit critical thinking ability by asking open-ended questions or performing thinking tasks. Socio-emotional Skills: The teacher fosters socio-emotional skills that encourage students to succeed both inside and outside the classroom. To develop students’ social and emotional skills, the teacher: (Element 7) instills autonomy by providing students with opportunities to make choices and take on meaningful roles in the classroom. Students exhibit their autonomy by volunteering to participate in classroom activities; (Element 8) promotes perseverance by acknowledging students’ efforts, rather than focusing solely on their intelligence or natural abilities, showing a positive attitude toward students’ challenges by framing failure and frustrations as part of the learning process, and by encouraging students to set short- and long-term goals; and (Element 9) fosters social and collaborative skills by encouraging collaboration through peer interaction and by promoting interpersonal skills, such as perspective taking, empathizing, emotion regulation, and social problem solving. Students exhibit social and collaborative skills by collaborating with one another through peer interaction. 14 3 TEACH RESULTS: INSIGHTS INTO TEACHER PRACTICES Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 3.1 Overview of Teach Results in Indonesia The study encompassed 500 teachers from 405 primary schools, comprising 54 percent MoECRT and 46 percent MoRA schools. Grade 4 primary school classrooms were observed across subjects, including Mathematics, Language (Bahasa Indonesia), and other subjects (e.g., science, religion). These sample schools were selected based on the 2019 Service Delivery Indicators (SDI) survey, with necessary adjustments to ensure a nationally representative sample (World Bank, 2023) (See Appendix 1 for more details about the sampling design). Data collected from Teach indicates that a significant portion of Indonesian teachers face challenges in various aspects of their teaching practice areas, such as creating a conducive classroom culture, delivering engaging instruction, and fostering socioemotional skills in students. Figure 3.1.1, displayed below, illustrates that only 20 percent of teachers throughout Indonesia manage to achieve scores exceeding three out of a possible five points. This result is notably lower compared to the performance of teachers who took specific teacher training program in Indonesia as observed through a teacher training impact evaluation study conducted by the World Bank among selected high-performing teachers in Java, where 64 percent attain scores of three or higher (Khairina, et.al., 2024). However, data from regions such as Guangdong, Mongolia, the Philippines, and Vietnam also indicates a similar pattern, where less than one-third of teachers score three and above. These scores translate into the use of ineffective and weak teaching practices, which lead to poorer learning outcomes (Afkar et al., 2023). Figure 3.1.1. Teach Score in All Areas (Overall Teach Score) in Indonesia and Other Countries 100% 73% 75% 50% 20% 25% 7% 0% 0% Low Low Medium Medium High High Distribution of Scores 15 a. Guangdong, China b. Mongolia 100% 100% Share of teachers Share of teachers 75% 67% 75% 67% observed (%) observed (%) 50% 50% 20% 24% 25% 25% 7% 7% 9% Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 0% 0% 0% Low Low Medium High Low Low Medium High Medium High Medium High Distribution of scores Distribution of scores c. Philippines d. Vietnam 100% 100% 70% Share of teachers Share of teachers 75% 66% 75% observed (%) observed (%) 50% 50% 28% 25% 19% 15% 25% 0% 1% 0% 0% 0% Low Low Medium High Low Low Medium High Medium High Medium High Distribution of scores Distribution of scores Source: Teach Indonesia 2023 and Afkar et. al., 2023 Figure 3.1.2 below describes the teachers’ competency related to classroom culture, instruction, and socio-emotional skills. Teachers in Indonesia are somewhat effective (score 3.4 out of the 5 possible points) at creating a positive classroom culture. With regard to Instruction, Indonesian teachers score around the medium range in facilitating the lesson; but are less skilled at checking for understanding, encouraging students to think critically and poor at providing feedback. Lastly, in the Socioemotional Skills area, teachers are less skilled at promoting student autonomy and fostering perseverance. Furthermore, they also scored low (1.6) in providing students opportunities to practice social and collaborative skills. Similarly, the Teach findings in other countries (Molina et al., 2021; 2020) also found that teachers have strong ability in Classroom Culture and weak ability in Instruction and Socio- emotional Skills. Figure 3.1.2. Distribution of Average Teach Scores by Element Overall Teach Score 2.7 A. Classroom Culture 3.4 1. Supportive Learning Environment 3.5 2. Positive Behavioral Expectations 3.4 B. Instruction 2.5 3. Lesson Facilitation 3.2 4. Checks for Understanding 2.7 5. Feedback 1.8 6. Critical Thinking 2.4 C. Socioemotional Skills 2.1 7. Autonomy 2.5 8. Perseverance 2.2 9. Social and Collaborative Skills 1.6 1 2 3 4 5 Notes: Scores are weighted to account for student and school populations across different educational institutions. The vertical line shows the average Teach element score (Overall Teach Score). 16 3.2 Time on Task As discussed, Teach captures (i) the time teachers spend on learning and the extent to which students are on task, and (ii) the quality of teaching practices. The Time on Task element measures the quantitative aspect of teaching practices and records Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study both the teacher’s actions and the number of students who are on task throughout the observation. Teachers on Task: For the time on learning section (See Figure 3.2), teachers in Indonesia provide learning activities to students 96 percent of the time. These observations encompass any learning activity related to class content, regardless of quality. Teach took three snapshots during the observation at minutes 4, 9, and 14, to determine whether teachers were providing activities or not. In our findings, it was observed that 96 percent of the time, teachers were engaging students in various learning activities, such as lecturing, assigning worksheets, and facilitating independent or group work. In the remaining four percent of the time, teachers did not engage in any learning activities, which included tasks such as taking attendance, silently reading or writing on the board without instructing students to copy, addressing misbehavior, or other non-learning activities that left students waiting. Students on Task: When teachers provide a learning activity, all students are on task only 48 percent of the time. During these instances, at least two students did not participate in the task assigned by the teachers. This can be attributed to either students being quiet but distracted or because they are causing disruptions in the classroom. For instance, in the first scenario, students may be seen gazing out of the window, resting their heads on their desks, looking down at the floor or towards the observer, or even sleeping. In the second scenario, students might be passing notes, whispering, conversing with a peer when the activity does not require discussion, moving around the classroom, shouting, or engaging in any other behavior that disrupts the class. Figure 3.2. Distribution of Time on Task variables Teacher No Yes provides learning activity 4% 96% Time on Low Medium High Learning 6 or more 2 to 5 0 or 1 Students are students are students are students are on task off task off task off task 10% 39% 48% 17 3.3 Area A: Overview of the Classroom Culture Area Result Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Indonesian teachers are somewhat effective in Classroom Culture. Figure 3.1.2 indicates that 88 percent of teachers score three and above. On average, they score 3.4 points out of the 5 points possible in this element (See Figure 3.1.1). Among the Classroom Culture Area elements, teachers performed consistently well and were more effective in providing students with a supportive learning environment (3.5) than setting positive behavioral expectations (3.4). Overall, compared to the other two areas explained later, teachers did perform somewhat effectively in the Classroom Culture area. Figure 3.3.1. Average of Classroom Culture Area and Elements Score A. CLASSROOM CULTURE 3.4 1. SUPPORTIVE LEARNING ENVIRONMENT 3.5 2. POSITIVE BEHAVIORAL EXPECTATIONS 3.4 1 2 3 4 5 Figure 3.3.2. Distribution of Classroom Culture Area Score 100% 75% 58% 50% 30% 25% 12% 0% 0% Low Low Medium Medium High High (1-2) (2-3) (3-4) (4-5) Distribution of Scores Figure 3.3.3 below shows the score distribution within the Classroom Culture 18 area. The Supporting Learning Environment element reflects a generally positive trend, emphasizing medium (38 percent) to medium-high (52 percent) levels of supportiveness. Likewise, the Positive Behavioral Expectation element distribution highlights a cumulative 84 percent at medium to high levels, indicating success in clearly defining expected student behavior in the classroom. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Figure 3.3.3. Distribution of Classroom Culture Elements Scores 1. SUPPORTIVE LEARNING ENVIRONMENT 10% 38% 52% 2. POSITIVE BEHAVIORAL EXPECTATIONS 6% 10% 33% 48% 3% Low Low Medium Medium Medium High High (1) (2) (3) (4) (5) The following section will provide a detailed examination of the two pivotal elements influencing the Classroom Culture score. Element 1: Supportive Learning Environment. Figure 3.3.4 Distribution of Supportive Learning Environment by Behaviors 1.1 RESPECTS STUDENTS 3% 18% 80% 1.2 USES POSITIVE LANGUAGE 61% 26% 13% 1.3 RESPONDS TO STUDENTS NEEDS 6% 2% 17% 75% 1.4 BIAS AND STEREOTYPES 1% 98% 1% 1.4a GENDER BIAS AND STEREOTYPES 1% 97% 1% 1.4b DISABILITY BIAS AND STEREOTYPES 100% Low Medium High N/A Figure 3.3.4 displays the distribution of scores for supportive learning environment 19 and its respective behaviors. Indonesian teachers treat all students respectfully (80 percent). They call students by their name, and in some areas of Java teachers use the polite female and male prefixes of “Mbak” and “Mas” before students’ names. Most teachers also say “please” and “thank you” to students when students answer questions which shows sign of respect. However, many teachers (61 percent) do not Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study use positive language when they communicate with students. Some teachers (26 percent) say “good job” or “nice”, although this happens infrequently, and only a few teachers (13 percent) consistently use positive language when the students show their work or encourage the class with phrases such as, “you are such a talented group of students” or “you can do this, I’m so proud of you”. In scenarios where students communicate their needs, teachers promptly address them (1.3), ensuring a supportive environment. 75 percent scored as N/A (not applicable) because there are no observable emotional, material, or physical needs and no students ask help from teachers. Finally, concerning bias and stereotypes (behavior 1.4), 1 percent of teachers exhibit instances of bias or stereotyping. The majority of teachers refrain from such behavior, providing equal opportunities for students of all genders and abilities, thus fostering inclusive and equitable learning. Element 2: Positive Behavioral Expectations. Figure 3.3.5 Distribution of Positive Behavioral Expectations by Behaviors 2.1 CLEAR BEHAVIOR EXPECTATIONS 12% 14% 72% 2.2 ACKNOWLEDGES POSITIVE BEHAVIOR 89% 6% 7% 2.3 REDIRECTS MISBEHAVIOR 9% 12% 77% Low Medium High N/A Figure 3.3.5 shows the distribution of scores for the positive behavioral expectation element and respective behaviors. Teachers generally set clear expectations (72 percent), but 90 percent do not acknowledge students’ positive behavior. For instance, at the beginning of session, teachers explain that they want students to do the worksheet in silence by themselves. However, once students finish the task and follow the expected behavior by not talking to friends, the teachers do not acknowledge the positive behavior that meets or exceeds their expectations. In the case of classrooms where students misbehave, many teachers (78 precent) effectively redirect the misbehavior. For instance, teachers address the problem at hand and focus on the expected behavior such as saying “remember, we need to keep quiet while working on the worksheet” instead of “stop talking and don’t be noisy”. In this example, the teacher is stating the expected behavior rather than focusing on the misbehavior. 20 3.4 Area B: Overview of the Instruction Area Result Indonesian teachers are less effective in Instruction (26 percent of teachers score Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study medium high to high level). On average, they score 2.5 points out of the 5 points possible in this element. In the Instruction area, teachers were most effective at facilitating lessons (3.2), somewhat effective at checking students’ understanding (2.7), less effective in encouraging students to think critically (2.4), and poor at providing feedback (1.8). Figure 3.4.1. Average Instruction Area Elements Scores B. INSTRUCTION 2.5 3. LESSON FACILITATION 3.2 4. CHECKS FOR UNDERSTANDING 2.7 5. FEEDBACK 1.8 6. CRITICAL THINKING 2.4 1 2 3 4 5 Figure 3.4.2. Distribution of Instruction Area Scores 100% 75% 54% 50% 26% 20% 25% 1% 0% Low Low Medium Medium High High (1-2) (2-3) (3-4) (4-5) Distribution of Scores Figure 3.4.3 below shows the score distribution within the Instruction area. The lesson facilitation element displays a positive trend towards higher levels, indicating proficient topic explanation in the classroom. Conversely, feedback scores are notably skewed towards the lowest end of the scale, suggesting a need for teacher interventions or training to enhance feedback provision skills, as only a minority of teachers demonstrate exemplary practices. Meanwhile, the check for understanding and critical thinking shows varied teacher performance. Roughly half of the teachers score 1 or 2, signaling a lower proficiency in ensuring students understand the topic discussed and use critical thinking skills, and half of teachers received a score of 4 or 5, demonstrating a relatively higher degree of success in checking student’s understanding and encouraging them to use critical thinking skills. Figure 3.4.3. Distribution of Instruction Elements Scores 21 3. LESSON FACILITATION 2% 28% 30% 36% 3% 4. CHECKS FOR UNDERSTANDING Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 20% 27% 28% 23% 2% 5. FEEDBACK 54% 24% 17% 3% 1% 6. CRITICAL THINKING 24% 34% 26% 15% 2% Low Low Medium Medium Medium High High (1) (2) (3) (4) (5) Element 3: Lesson Facilitation. Figure 3.4.4. Distribution of Lesson Facilitation by Behaviors 3.1 ARTICULATES LESSON OBJECTIVES 6% 53% 39% 3.2 CLEAR EXPLANATIONS 11% 54% 34% 3.3 CONNECTS LESSON 72% 14% 12% 3.4 MODELS BY ENACTING OR THINKING ALOUD 35% 26% 37% Low Medium High N/A Figure 3.4.4 shows the distribution of teacher’s scores for the lesson facilitation element and the respective behaviors. Teachers are moderately good at articulating lesson objectives. More than half of the teachers (54 percent) explicitly state a general lesson objective such as, “Today we are going to learn about two dimensional shapes” without further explanation. Others do not state it explicitly, but it can be inferred from the lesson. For example, after giving an example of different type of formula to calculate the area of different shapes the teachers clearly explain the activity when she says, “Now I will give you the quantity of width and height of each shape and you have to apply the formula”. From this, it can be inferred they are working on calculating the areas of two-dimensional figures; however, the teacher does not make an explicit lesson objective statement. 39 percent teachers explicitly state a specific lesson objective, and the lesson activities align to the stated objective, while seven percent do not state the lesson objective, nor can one be inferred from the lesson activities. Half of teachers are providing somewhat clear explanations of the lesson. Although 22 part of these explanations may be clear, others are confusing or superficial. For example, while explaining fractions, teachers explain verbally and write the sample on the board. Only 34 percent use other forms of representation in addition to verbal and written such as using figures, visuals, and others to explain the topic. Meanwhile 12 percent of teachers use only one form of explanation, or content is Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study simply not being explained. Moreover, many Indonesian teachers (73 percent) do not connect the lesson taught to other content knowledge or students’ daily lives. While 15 percent of teachers may attempt to connect the lesson to other content knowledge or students’ daily lives, the connections are superficial, confusing, or unclear. For example, when introducing a lesson on types of triangles, the teacher says, “Yesterday we learned how to calculate the area of a rectangle, today we will learn different type of triangle shapes” and go on to explain triangles and present the formula. The connections to other content knowledge or students’ lives are superficial and nonspecific. Lastly, 35 percent of teachers model by enacting procedures or thinking aloud, but 37 percent do not model at all. Although they ask students to read a text, answer specific questions, or complete activities, they rarely walk them through the process of how to solve a task. Element 4: Check for Understanding. Figure 3.4.5. Distribution of Checks for Understanding by Behaviors 4.1 USES QUESTIONS AND PROMPTS 43% 37% 19% 4.2 MONITORS DURING INDEPENDENT/GROUP WORK 19% 19% 30% 31% 4.3 ADJUSTS TEACHING 66% 21% 12% Low Medium High N/A On average, teachers score 2.6 points out of the 5 points possible in this element (See Figure 3.4.1). Figure 3.4.5. shows the distribution of teacher’s scores for the checks for understanding the element and its respective behaviors. When teachers explain a topic, almost half of the teachers (43.2 percent) do not ask questions, prompt, or use other strategies to clarify students understanding. When they do ask, “is this correct?” students chorus “yes”, which is accepted without further clarification for understanding. However, when students work independently or in groups, many teachers (30.6 percent) monitor students systematically by circling the classroom and approaching individual students or groups to check their understanding. Finally, when many students get the wrong answer or misunderstand a concept, most teachers (66.5 percent) may notice but do not re-explain the concept or provide additional opportunities to learn by adjusting the lesson. 23 Element 5: Feedback. Figure 3.4.6. Distribution of Feedback by Behaviors 5.1 PROVIDES COMMENTS TO CLARIFY MISUNDERSTANDINGS Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 60% 24% 14% 5.2 PROVIDE COMMENTS TO IDENTIFY SUCCESSES 85% 9% 4% Low Medium High N/A On average, teachers score 1.7 points out of the 5 points possible in this element (See Figure 3.4.1.). Figure 3.4.6 shows the distribution of teacher’s scores for the feedback element and its respective behaviors. A notable 60 percent of teachers landed in the ‘Low’ proficiency range. For instance, when a student answers a question incorrectly, these teachers might simply state, “That is not the correct answer,” and move on without offering further guidance. The second criteria evaluated the ability of teachers to provide comments that identify student successes. Alarmingly, 85 percent scored in the ‘Low’ category, often giving feedback as basic as, “That is correct,” without elaboration on a student’s accurate response. A mere 4 percent excelled in offering detailed feedback. In essence, while many educators may find it challenging to provide constructive feedback for clarifications, the result suggests they face even greater challenges in highlighting student successes. The following figure presents the distribution of overall feedback scores, ranging from 1 to 5. More than half (54 percent) of educators, received the lowest feedback score of 1, suggesting that there is a predominant area of concern or deficiency in their feedback mechanisms. 24 percent of the educators were scored at 2, indicating that while they performed better than the majority, there is still considerable room for improvement in their feedback provision. Approximately five percent of the surveyed educators achieved high feedback scores, underscoring the rarity of top-tier feedback practices. Element 6: Critical Thinking. Figure 3.4.7. Distribution of Critical thinking by Behaviors 6.1 ASKS OPEN-ENDED QUESTIONS 77% 15% 6% 6.2 PROVIDES THINKING TASKS 34% 47% 18% 6.3 STUDENTS ASK OPEN-ENDED QUESTIONS AND/OR PERFORM THINKING TASKS 32% 53% 14% Low Medium High N/A On average, teachers scored 2.4 out of a possible 5 points in the critical thinking 24 element. This average is comparably higher than their scores in the feedback element. Figure 3.4.7 shows the distribution of teachers’ scores for the critical thinking element and its respective behaviors. Overall, there is a varied distribution in critical thinking proficiencies among participants. 78 percent demonstrated a ‘Low’ proficiency in asking open-ended questions, which suggests a tendency Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study to opt for simpler, closed-ended inquiries. This pattern contrasts sharply with the 15 percent who achieved a ‘Medium’ proficiency and the even smaller seven percent who excelled in this domain. Delving into the provision of thinking tasks, the landscape appears slightly more diverse. About 34 percent of educators were in the ‘Low’ category, often presenting students with straightforward tasks. Still, a notable 48 percent fell into the ‘Medium’ bracket, hinting at periodic challenges posed to students. An encouraging 18 percent consistently performed thinking tasks. However, when looking at how often students asked open-ended questions or took on challenging tasks, 32 percent of classrooms were in the ‘Low’ category, showing that students rarely ask questions. Slightly over half of classrooms (54 percent) were in the ‘Medium’ range, where students sometimes took part actively. Only 14 percent of classrooms were in the ‘High’ category, where students frequently asked deep questions and tackled hard tasks. 3.5 Area C: Overview of the Socioemotional Skills Area Result Indonesian teachers exhibit poor ability in Socioemotional Skills (10 percent of teachers score 3 and above). On average, they score 2.1 out of the 5 points. Among the elements of Socioemotional Skills, they were less effective at instilling autonomy (2.5) and in promoting perseverance (2.2), and poor in fostering social and collaborative skills in their students (1.6). Overall, teachers showed the greatest room for improvement in Socioemotional Skills. Figure 3.5.1. Average of Socioemotional Skills Area and Elements Scores C. SOCIOEMOTIONAL SKILLS 2.1 7. AUTONOMY 2.5 8. PERSEVERANCE 2.2 9. SOCIAL AND COLLABORATIVE SKILLS 1.6 1 2 3 4 5 Figure 3.5.2. Distribution of Socioemotional Skills Area Scores 25 100% Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 75% 49% 50% 41% 25% 10% 0% 0% Low Low Medium Medium High High (1-2) (2-3) (3-4) (4-5) Distribution of Scores Figure 3.5.3 below illustrates the score distribution within the Socioemotional Skills area. The distribution of scores for the Autonomy element reveals a complex pattern of teacher performance. Among the assessed educators, more than half (54 percent) received a score of 1 or 2, indicating a lower proficiency in nurturing students’ autonomy in learning. In the Perseverance element, an overwhelming 86 percent of teachers received a score of 1 or 2, signaling a low level of proficiency in fostering students’ perseverance. Similarly, 73 percent of teachers received a score of 1 in social and collaborative skills element, highlighting challenges in teacher proficiency to cultivate collaborative and social abilities among students. In summary, the distribution scores among the three elements in this area emphasize the significant need for improvement in teachers’ approaches to nurturing autonomy, perseverance, and social and collaborative skills in students. Figure 3.5.3. Distribution of Socioemotional Skills Elements Scores 7. AUTONOMY 23% 31% 33% 13% 1% 8. PERSEVERANCE 3% 83% 13% 2% 9. SOCIAL AND COLLABORATIVE SKILLS 73% 9% 8% 10% 1% Low Low Medium Medium Medium High High (1) (2) (3) (4) (5) The following section will provide a comprehensive examination of the three 26 elements influencing the overall Socioemotional Skills Area score. Element 7: Autonomy. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Figure 3.5.4. Distribution of Autonomy by Behaviors 7.1 PROVIDES STUDENTS WITH CHOICES 87% 6% 6% 7.2 PROVIDES OPPORTUNIITIES TO TAKE ON ROLES 49% 28% 21% 7.3 STUDENTS VOLUNTEER TO PARTICIPATE 39% 38% 21% Low Medium High N/A On average, teachers achieve a score of 2.4 out of 5 possible points in this particular element, which stands out as relatively high compared to the other Socioemotional elements (See Figure 3.5.1). Figure 3.5.4. illustrates the distribution of teacher’s scores for the autonomy element and its respective behaviors. In terms of providing students with choices, a significant number of teachers (87 percent) fell into the ‘Low’ category, indicating a prevalent tendency to offer limited decision-making opportunities, thereby potentially hindering student autonomy. In contrast, only a small portion (6 percent) demonstrated a ‘Medium’ proficiency, and only seven percent of teachers fostered a more independent learning environment. Examining “Provides opportunities to take on roles,” approximately half of the teachers (49 percent) were in the ‘Low’ range, implying a lack of diverse roles for students in the classroom. However, a substantial 29 percent embraced a ‘Medium’ approach, suggesting intermittent involvement of students in meaningful roles. About 22 percent achieved a ‘High’ level, effectively promoting active participation and responsibility among students. In terms of “Students volunteer to participate”, 40 percent of teachers encountered a ‘Low’ scenario, with limited student initiative. Notably, 38 percent experienced a ‘Medium’ scenario, indicating a more active involvement, while a commendable 22 percent achieved a ‘High’ level, reflecting proactive student engagement. These results underscore the varied landscape of teachers’ ability to cultivate autonomy, role participation, and student engagement within the Socioemotional Skills context. 27 Element 8: Perseverance. Figure 3.5.5. Distribution of Perseverance by Behaviors 8.1 ACKNOWLEDGES STUDENTS EFFORTS Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 86% 11% 1% 8.2 POSITIVE ATTITUDE TOWARD STUDENTS CHALLENGES 2% 89% 8% 8.3 ENCOURAGES GOAL SETTING 93% 5% 1% Low Medium High N/A On average, teachers score 2.1 points out of the 5-points possible in this element (See Figure 3.5.1). Figure 3.5.5. shows the distribution of teacher’s scores for the perseverance element and its respective behaviors. The evaluation of the Perseverance behaviors reveals a diverse range of teacher behaviors. In the context of behavior of “Acknowledges students’ efforts,” a majority of teachers (87 percent) received a ‘Low’ rating, indicating a deficiency in cultivating students’ perseverance. This shows that there is a tendency to praise natural talents rather than acknowledging hard work and effort. Shifting our focus to the behavior of a “Positive attitude towards students’ challenges,” it is worth noting that few (3 percent) of teachers showed a ‘Low’ proficiency in this area. In contrast, an overwhelming 89 percent demonstrated a ‘Medium’ rating, reflecting a neutral stance without actively fostering resilience. Finally, a significant 94 percent of teachers are in the ‘Low’ category of “Encourages goal setting,”, suggesting a missed opportunity to foster perseverance through goal-oriented strategies. In contrast, a mere five percent demonstrated a ‘Medium’ proficiency by either discussing short- or long-term goals. In summary, the evaluation highlights the lack of teacher efforts to promote student perseverance and highlights areas for improvement. 28 Element 9: Social and Collaborative Skills. Figure 3.5.6. Distribution of Social and Collaborative Skills by Behaviors 9.1 PROMOTES SOCIAL COLLABORATION Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 81% 7% 11% 9.2 PROMOTES STUDENTS INTERPERSONAL SKILLS 87% 9% 3% 9.3 STUDENTS COLLABORATE 78% 10% 11% Low Medium High N/A On average, their score in this element is 1.5 out of a possible 5 points, marking the lowest score among all the elements (See Figure 3.5.1). Figure 3.5.6 shows the distribution of teachers’ scores for the social and collaborative skills element and its respective behaviors. Regarding promoting student collaboration through peer interaction, a substantial portion of teachers (81 percent) received a ‘Low’’ rating. This indicates a lack of emphasis on fostering collaborative efforts among students within these classrooms. In contrast, a modest seven percent demonstrated a ‘‘Medium’ proficiency, suggesting sporadic instances of superficial collaboration such as asking students to share materials to the person next to them. Similarly, a significant 87 percent of teachers scored ‘low’ on ‘Moving to promote students’ interpersonal skills’, signaling a missed opportunity to develop students’ interpersonal skills. These classrooms may lack strategies for nurturing skills like perspective taking, empathy, emotion regulation, and problem solving. Lastly, considering the way students collaborate through interacting with their peers, 78 percent of students were categorized as ‘Low’, indicating either a lack of collaboration or instances of negative behaviors during interactions. A smaller group of ten percent exhibited a ‘Medium’ proficiency, showcasing instances of surface-level collaboration combined with minor occurrences of negative behavior. In summary, this assessment illuminates the need to cultivate students’ social and collaborative skills during class. 29 4 COMPARATIVE ANALYSIS OF TEACHING PRACTICES Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 4.1. Teaching Practices Analysis: MoECRT and MoRA4 Figure 4.1 displays average Teach scores between MoECRT and MoRA schools, highlighting category score differences from regression analysis (See Table 4.1 in Appendix 4). Upon reviewing teaching practices between MoECRT and MoRA schools, MoECRT teachers averaged a score of 2.7, slightly higher than MoRA’s 2.64. This 0.06-point difference indicates that MoECRT teachers scored approximately 0.03 SD higher than MoRA teachers. In the area of Classroom Culture, MoECRT teachers led with a mean score of 3.46 compared to MoRA’s 3.37, a difference of 0.09 points. Within this area, MoECRT teachers exhibit a higher average score for fostering a supportive learning environment, with a mean of 3.55, which is 0.12 points above MoRA’s average of 3.43. Shifting focus to the Instruction area, MoECRT teachers attained an average score of 2.54, slightly higher than MoRA teachers, who averaged 2.46, marking a difference of 0.08 points. A difference is evident in the feedback element, with MoECRT teachers scoring 1.84, compared to MoRA’s 1.68. Lastly, in the Socioemotional Skills area, MoECRT scored 2.10 and MoRA, 2.08. In the social and collaborative skills component, MoECRT teachers scored 1.64 and MoRA, 1.60. 4 For enhanced analytical precision, this section presents group differences to two decimal places. Figure 4.1. MoECRT and MoRA Scores Across the Teach Areas 30 Overall Teach Score 2.70 2.64 0.06*** A. Classroom Culture 3.46 3.37 0.09*** Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 1. Supportive Learning Environment 3.55 0.12*** 3.43 2. Positive Behavioral Expectations 3.37 3.32 0.06*** B. Instruction 2.54 2.46 0.07*** 3. Lesson Facilitation 3.18 0.00 3.18 4. Checks for Understanding 2.71 0.11*** 2.60 5. Feedback 1.84 0.16*** 1.68 6. Critical Thinking 2.41 0.03*** 2.38 C. Socioemotional Skills 2.10 0.02*** 2.08 7. Autonomy 2.51 0.05*** 2.47 8. Perseverance 2.15 -0.01*** 2.16 9. Social and Collaborative Skills 1.64 0.04*** 1.60 MoECRT 1 2 3 4 5 MoRA Note. The score difference between MoECRT and MoRA represents the disparity between groups as measured by a weighted regression analysis. p<0.05, ** p<0.01, *** p<0.001. See Table 4.1 in Appendix 4. 4.2. Teaching Practices Analysis: Urban and Rural Figure 4.2 offers an analysis of average scores across Teach elements for two distinct school environments: Urban and Rural. When observing the teaching practices between these school environments, urban teachers excelled in several elements. Overall, urban educators have an average score of 2.76, slightly surpassing their rural counterparts who recorded 2.66. This difference of 0.11 points translates to urban educators being approximately 0.06 SD more effective in this aspect. In the area of Classroom Culture, urban educators have an edge with a score of 3.55, as opposed to rural teachers’ 3.40. This difference suggests potential disparities in the classroom environment between the two settings. Urban teachers demonstrate a higher average score in fostering a supportive learning environment, with a score of 3.63 compared to rural teachers’ 3.48. It implies urban teachers are treating all students with respect, and employing positive language, and addressing students’ diverse needs. Figure 4.2. Urban and Rural Scores Across the Teach Areas 31 Overall Teach Score 2.76 0.11*** 2.66 A. Classroom Culture 3.55 0.14*** 3.40 Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 1. Supportive Learning Environment 3.63 0.14*** 3.48 2. Positive Behavioral Expectations 3.47 0.15*** 3.32 B. Instruction 2.69 0.23*** 2.46 3. Lesson Facilitation 3.40 0.30*** 3.10 4. Checks for Understanding 2.61 -0.10*** 2.71 5. Feedback 1.92 0.16*** 1.76 6. Critical Thinking 2.83 0.57*** 2.27 C. Socioemotional Skills 2.05 0.06*** 2.11 7. Autonomy 2.53 0.04*** 2.49 8. Perseverance 2.19 0.04*** 2.15 9. Social and Collaborative Skills 1.44 -0.26*** 1.69 1 2 3 4 5 Urban Rural Note. The score difference between MoECRT and MoRA represents the disparity between groups as measured by a weighted regression analysis. p<0.05, ** p<0.01, *** p<0.001. See Table 4.2 in Appendix 4 for the regression results. Turning our attention to Instruction, urban teachers, with a score of 2.69, significantly led by 0.23 points over rural teachers’ 2.46. A notable disparity is observed in the critical thinking component, where urban teachers score 2.83 and rural teachers, 2.27. It suggests that urban teachers have a higher tendency to encourage open- ended questions and provide thought-provoking tasks compared to rural educators. In the area of Socioemotional Skills, urban educators scored slightly lower, with a score of 2.05 compared to rural teachers’ 2.11. Delving deeper, the autonomy element shows urban educators marginally ahead at 2.53 compared to rural teachers’ 2.49. The perseverance element is closely matched between the two. However, when it comes to fostering a collaborative classroom ethos, rural educators take the lead. Their score of 1.69 in social and collaborative skills, underlines the emphasis on peer interactions and interpersonal skill cultivation, surpassing urban’s 1.44. In summary, while urban teachers generally performed better in several elements, especially in Classroom Culture and Instruction areas, the results spotlight distinct teaching practices and suggest areas of potential enhancement for both school environments. 32 4.3 Teaching Practices Analysis by Teacher Characteristics This section discusses Teach scores in relation to teacher characteristics. With respect to the Teach score scores across teachers’ gender, female teachers consistently edge Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study out their male counterparts across all Teach elements (See Table 4.5 in Appendix 4). In Classroom Culture, female educators lead with an average of 3.45 compared to 3.39 by males. This suggests that they might be more effective at fostering a supportive environment and setting positive behavioral expectations. Regarding the Instruction area, scores between male and female teachers are closely matched, indicating they share similar instructional approaches. In the Socioemotional Skills area, female educators scored 2.12 compared to the male score of 2.01. This difference hints at female teachers’ tendency to support student autonomy and foster social and collaborative skills. However, when it comes to instilling perseverance, the scores are closely matched. Overall, female teachers show a slightly higher proficiency in multiple teaching domains, especially within Classroom Culture and Socioemotional Skills. However, the slight differences in scores between genders illuminate opportunities for improvement of both groups, particularly in the area of Instruction and Socioemotional Skills. When analyzing Teach scores across various levels of teacher education, consistent patterns emerge. Teachers with postgraduate education tend to achieve the highest Teach scores, with undergraduate teachers following. Those with a diploma or equivalent education generally record lower scores. The differences in Teach scores are most marked within the domains of Instruction and Socioemotional Skills, with lower scores observed among teachers with less education. (See Table 4.7 in Appendix 4). It reveals a positive relationship between higher education levels and higher Teach scores, implying that teachers with more advanced degrees tend to exhibit stronger teaching practices across various domains. The results shed light on the importance of considering the design and implementation of targeted teacher training programs, particularly for educators with lower levels of education. Figure 4.3.1. Teach Area Scores by Teacher’s Gender 33 5 Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 4 0.06*** 3.45 3.39 0.06*** 3 0.02*** 2.70 2.63 2.52 2.50 0.11*** 2.12 2.01 2 1 Overall A B C Teach Score Classroom Instruction Socioemotional Culture Skill Female Male Note. The Teach score difference represents the disparity between genders as measured by a weighted regression analysis. * p<0.05, ** p<0.01, *** p<0.001. Figure 4.3.2. Teach Area Scores by Teacher’s Education Level 5 4 3.5 3.44 3.37 3 2.72 2.69 2.49 2.51 2.53 2.24 2.14 2.1 2 1.86 1 Overall A B C Teach Score Classroom Instruction Socioemotional Culture Skill Postgraduate Undergraduate Diploma or less Note. The figure present mean Teach scores across academic subjects See the details on Table 4.6 & 7 in Appendix 4. 34 4.4 Teaching Practices Analysis by Academic Subject and Curriculum Figures 4.4.1 and 4.4.2 examine the Teach scores across academic subjects and Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study academic curricula. First, when examining teaching practices across academic subjects, mathematics classes, with an average score of 2.72, are ahead in their general teaching practices compared to their counterparts in language (2.6) and other subjects (2.71). In the area of Classroom Culture, teachers tend to show better teaching practices when teaching math classes that treat students respectfully, use positive language and avoid bias. With respect to the Instruction area, teachers in mathematics classes tend to perform better, especially in lesson facilitation and feedback. It is pronounced in feedback practice, where they are more likely to provide specific comments that help clarify students’ misunderstandings, scoring 1.98 compared to the 1.58 in language (0.4 mean difference, 0.2 SD). However, in socioemotional skills, teachers who taught other subjects seem to had a better score of 2.22. They particularly performed better in fostering student autonomy and promoting perseverance, evident from their practices of providing students with choices and acknowledging their efforts (See Table 4.8 in Appendix 4). In essence, the variation in the Teach scores across academic subjects is especially noteworthy given that a single homeroom teacher typically covers all subjects. Such differences in scores across subjects highlight the importance of a subject- specific evaluation of teaching practices, emphasizing the need for improved teaching methods. Figure 4.4.2 presents a comparison of Teach scores across different academic curriculums, shedding light on how curriculum choices are associated with teaching practices. Notably, the Merdeka Curriculum5 outperforms the widely used curriculum, Kurikulum Tingkat Satuan Pendidikan (KTSP; translated as School-Based Curriculum) 20136, demonstrating an advantage in lesson facilitation (0.29 mean difference, 0.24 SD) and a substantial advantage in critical thinking (0.4 difference, 0.38 SD). These findings underscore Merdeka Curriculum’s strong commitment to effective classroom guidance and the cultivation of critical thinking abilities (See Table 4.11 and 4.12 in Appendix 4). These findings suggest that curriculum choices play a significant role in shaping teaching practices. To leverage these insights, educational policymakers and institutions should consider tailoring teacher training programs and support based on the specific curriculum in use, addressing areas where each curriculum may require additional focus or improvement to enhance overall teaching quality. 5 Kurikulum Merdeka, translated as the emancipated curriculum, is part of Indonesia’s latest educational reform introduced in 2022. This curriculum prioritizes flexibility and adaptability, steering away from standardized testing and rote memorization towards the cultivation of students’ holistic competencies and character. For more information, visit https://kurikulum.kemdikbud. go.id/kurikulum-merdeka/ 6 Also known as Kurikulum 2013, it was Indonesia’s education curriculum from 2013 to 2022. It introduced a competency-based approach and placed a strong emphasis on assessment. For more information, visit https://kurikulum.kemdikbud.go.id/kurikulum-2013. In conclusion, these findings emphasize the importance of tailoring teacher 35 development and curriculum design to specific subject areas and curricular choices. By addressing the specific needs and strengths associated with each subject and curriculum, educational institutions can enhance overall teaching quality and contribute to better learning outcomes for students. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Figure 4.4.1. Teach Area Scores by Academic Subject 5 4 3.46 3.4 3.43 3 2.72 2.6 2.71 2.63 2.38 2.46 2.22 2.08 2.02 2 1 Overall A B C Teach Score Classroom Instruction Socioemotional Culture Skill Mathematics Language Other Note. The figure present mean Teach scores across academic subjects (See Table 4.8 in Appendix 4). Figure 4.4.2. Teach Area Scores by Academic Curriculum 5 4 3.55 3.37 3.32 2.8 3 2.61 2.66 2.58 2.43 2.35 2.19 2.06 2.03 2 1 Overall A B C Teach Score Classroom Instruction Socioemotional Culture Skill Merdeka KTSP 2013 Others Curriculum Curriculum Note. The Teach score difference represents the disparity between genders as measured by a weighted regression analysis. * p<0.05, ** p<0.01, *** p<0.001. 36 5 CONCLUSION AND RECOMMENDATION Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 5.1 Conclusion This report has shown that Indonesia primary school teachers demonstrate a number of pedagogical strengths and weaknesses in their classrooms. Results from Teach demonstrate that teachers have strong ability in classroom culture (88 percent score three and above), but less effective in giving instruction (26 percent score three and above) and are less skilled in socioemotional skills (only 10 precent score three and above). Within the Classroom Culture area, teachers excel at creating a supportive learning environment by treating all students respectfully, using positive language, and being responsive to student needs (with an average score of 3.5 out of 5). They are also relatively effective in setting positive behavioral expectations, particularly in recognizing positive student behavior (3.4/5). Conversely, they exhibit less effectiveness in the Instruction area, defined by Teach as facilitating lessons (3.2/5), checking for understanding (2.7/5), providing feedback (1.8/5), and encouraging students to exercise critical thinking (2.4/5). Likewise, they demonstrate lower proficiency in Socioemotional Skills area, defined as instilling autonomy (2.5/5), promoting perseverance (2.2/5), and fostering social and collaborative skills (1.6/5) (See Figure 3.2). The paper also reveals disparities in teaching quality among different groups. MoECRT teachers marginally outperform MoRA teachers, particularly in the area of providing clear and constructive feedback. In addition, urban teachers display better teaching practices compared to their rural counterparts, emphasizing the need for support and professional development initiatives in rural educational settings. Teacher characteristics, including gender and education level, are also related to teaching proficiency. Female teachers and teachers with higher education levels tend to have stronger teaching practices, particularly in instruction and socioemotional skills. Curriculum choices and academic subjects also play a significant role in shaping teaching practices. The Merdeka Curriculum outperforms the widely-used KTSP 2013 Curriculum indicating the importance of aligning curricula with modern teaching methodologies. 37 5.2 Recommendation Based on the key findings from the research on teacher classroom observation in Indonesia, the following recommendations and suggestions are proposed for the government and relevant stakeholders to improve the quality of education: Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 1. Tailored teacher development is essential for Indonesia. Indonesian teachers excel in fostering a positive classroom culture but require significant changes in improving Instruction and Socioemotional skill aspects in their teaching practice. This trend, seen in similar countries such as Vietnam, Mongolia, Philippines and China, underscores the need for targeted teacher training programs aimed at enhancing these areas, benefiting students’ cognitive and socioemotional development. Below are some actionable steps to achieve this: a. Specialized Training Programs: create and implement training modules specifically focused on improving instructional techniques and socioemotional skills. Partner with educational institutions/ colleges (LPTK) and NGOs to deliver these programs. b. Mentorship and Peer Learning: Ensure the sustainability of mentorship programs such as the Guru Penggerak Program, where experienced teachers can guide and support less experienced peers, particularly in areas of instruction and socioemotional development. Unfortunately, the program is currently limited to teachers working under MoECRT and does not include MoRA teachers. c. Regular Workshops and Communities: Organize regular workshops and community platforms to provide continuous professional development opportunities for teachers, ensuring they stay updated with the latest educational practices and methodologies. 2. Curriculum and School Differences Should Align More Closely with Modern Teaching Methods The variation in the quality of teaching practices across different curricula, ministry oversight, and subjects indicates a need for educational policymakers to align curricula with modern teaching methodologies. In addition, teacher training programs should be tailored to specific to these needs. Here are some recommended actions: a. Curriculum Review and Alignment: Conduct a comprehensive review of existing curricula (Merdeka Curriculum and KTSP 2013 Curriculum) and align them with modern teaching methodologies. Engage educational experts to identify gaps and suggest 38 improvements. b. Customized Teacher Training: Develop training programs tailored to specific curricular requirements. Focus on areas where each curriculum may need improvement, such as lesson facilitation, Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study critical thinking, and student engagement. 3. Educational Divides Need to be Bridged Disparities between urban and rural teachers, especially in fostering critical thinking, highlight the need for targeted support in rural areas. Unfortunately, the current training targeted for rural area such as Guru Penggerak for special region is limited to MoECRT teachers only. Policymakers should invest in accessible training programs to equip rural teachers. In addition, encouraging teachers to pursue advanced degrees and providing opportunities for ongoing professional development are crucial to elevate teaching practices, particularly for those with lower education levels. The following steps can help bridge this divide: a. Targeted Rural Teacher Support: Provide additional resources and support to rural teachers through specialized training programs aimed at enhancing critical thinking and problem- solving skills. Utilize technology to deliver remote training sessions and resources. b. Incentive Programs: Currently, NGOs such as Indonesia Mengajar, which provide teaching internship opportunities in rural areas for fresh graduates, have a low conversion rate of these interns becoming actual teachers after completing the program. The government can introduce incentive programs for teachers working in rural areas to encourage retention and quality talents to become teachers. Offer scholarships and financial support for teachers pursuing advanced degrees can also support continuous learning for teachers. c. Collaboration with Higher Education Institutions: Partner with universities and teacher training colleges to provide advanced degree programs and professional development courses tailored to the needs of teachers in different regions. The government can collaborate with teacher colleges or pre-service teacher training institutions to attract quality teachers to rural areas. In conclusion, the findings from Teach in Indonesia highlight the need for targeted teacher development, curriculum alignment, and bridging educational divides to enhance teaching quality and improve the overall quality of education in Indonesia. These recommendations can guide educational policymakers and educational institutions in addressing the specific challenges and strengths associated with teaching practices in Indonesia. 39 REFERENCES Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Afkar, R., Béteille, T., Breeding, M., Linden, T., Mason, A., Mattoo, A., Pfutze, T., Sondergaard, L., and Yarrow, N. 2023. Fixing the Foundation: Teachers and Basic Education in East Asia and Pacific. World Bank East Asia and Pacific Regional Report. Washington, DC: World Bank. doi:10.1596/978-1-4648-1904-9. Araujo, Caridad, Pedro Carneiro, Yyannu Cruz-Aguayo, and Norbert Schady (2016). “Teacher-Quality and Learning Outcomes in Kindergarten.” Quarterly Journal of Economics: 1415-53. Bau, Natalie, and Jishnu Das (2017). “The Misallocation of Pay and Productivity in the Public Sector: Evidence from the Labor Market for Teachers.” Policy Research Working Paper 8050. World Bank. Washington, D.C. Bruns, B., De Gregorio, S., & Taut, S. (2016). Measures of effective teaching in developing countries. Research on Improving Systems of Education (RISE) Working Paper, 16(009). Cruz -Aguayo, Ibarraran, Pablo, Schady, Norbert (2017). Do tests applied to teachers predict their effectiveness? Economic Letters (159), 108-111. Hamre, Bridget, Robert Pianta, Bridget Hatfield, and Faiza Jamil (2014). Evidence for general and domain-specific elements of teacher-child interactions: associations with preschool children’s development. Child Development. (85), 1257-1274. Holtzapple, E. (2004). Criterion-related validity evidence for a standards-based teacher evaluation system. Journal of Personnel Evaluation in Education. 17(3), 207-219. Kane, T and Staiger, D. (2009). Estimating Teacher Impacts on Student Achievement: An Experimental Evaluation. NBER Working Paper. Washington, D.C. Kane, T., & Staiger, D. (2012). Gathering Feedback for Teaching: Combining High- Quality Observations with Student Surveys and Achievement Gains. Research Paper. MET Project. Bill & Melinda Gates Foundation. Khairina, N., Yarrow, N. Cilliers, J. Dini, I. (2024). Improving Teachers and School Leadership in Indonesia: Impact Evaluation of Guru Penggerak Program at the Primary Level. World Bank, Jakarta. © World Bank Kraft, Matthew A., David Blazar, and Dylan Hogan (2018). The effect of teaching 40 coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research. Li, C. (2013). Little’s test of missing completely at random. The Stata Journal, 13(4), 795-809. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Molina, E., Carter, E., Luna-Bazaldua, D., Pushparatnum, A., & Singal, N. (2021). Teaching for All? Measuring the Quality of Inclusive Practices Across 8 Countries. Working Paper. World Bank, Washington, DC Molina, E., Fatima, S. F., Ho, A., Melo, C., Wilichowksi, T., & Pushparatnam, A. (2020). Measuring the Quality of Teaching Practices in Primary Schools: Assessing the Validity of the Teach Observation Tool in Punjab, Pakistan. Teaching and Teacher Education. 96, 103171. Milanowski, Anthony (2004). “The Relationship between teacher performance evaluation scores and student achievement: evidence from Cincinnati.” Peabody Journal of Education. 79 (4), 33-53 Staiger, O., Rockoff, E. (2010). Searching for effective teachers with imperfect information. Journal of Economic Perspectives. 24 (3), 97-118. Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International journal of medical education, 2, 53. World Bank (2018). World Development Report 2018: Learning to Realize Education’s Promise. World Bank. World Bank (2019). Indonesia. Learning Poverty Brief. © World Bank, Washington, DC. World Bank. (2020). The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19. © World Bank, Washington, DC. World Bank (2022). Teach Primary : Observer Manual (Vol. 2) : Observation Sheet (English). Teach Primary Washington, D.C. : World Bank roup. https://documents1. worldbank.org/curated/en/223581641964111529/pdf/Observation-Sheet.pdf World Bank (2023). Indonesia Economic Prospects (IEP) June 2023: The Invisible Toll of Covid-19 on Learning. © World Bank, Washington, DC. Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior research methods, 51, 409-428. 41 APPENDICES - A1 A1. Teach Sampling and Data Collection in Indonesia Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 1 Teach Sampling Process This study was conducted as part of the Learning Loss Survey in 2023. It was designed Table A1.1. as a panel survey to the the 2019 Service Teach Sampling Delivery Indicators (SDI) survey established a in Indonesia nearly nationally representative benchmark of student learning outcomes in language Number of schools 405 (Bahasa Indonesia) and mathematics at the fourth-grade level, providing a pre-pandemic baseline for educational attainment. Therefore, % of SD (MoECRT) 54% sampling frame for this Teach study was built on the sampling frame of 2019 SDI, with expanded % of Madrasah (MoRA) 46% coverage to make the sample nationally representative by covering both MoECRT and % of Urban Schools 16% MoRA schools and educational institutions. The study incorporated a carefully selected % of Rural Schools 84% sample of 405 primary schools, representing a balance of 54 percent MoECRT schools and 46 percent MoRA schools, chosen to ensure a Number of Teachers 500 representative cross-section of the national school distribution. % of Male Teachers 29% In this study, a total of 501 teachers were observed % of Female Teachers 71% using the Teach classroom observation tool at two time points—the first and last 15 minutes of their class—resulting in 1,002 observations. # of Teach Observations 993 Of these, nine observations were excluded due to missing data, yielding a final dataset of 993 Medium Class Size 19 observations from 500 teachers. Observations were carried out in Grade 4 classrooms, focusing Number of Students 9644 on a range of subjects including Mathematics, Language (Bahasa Indonesia), and additional areas like science and religion. To accurately % of Male Students 49% reflect the national educational environment, the study also implemented school weights % of Female Students 51% helped to normalize for variations in student body sizes and school characteristics across Subject Distribution the different institutions, thereby ensuring that the findings are representative of the broader Math 46% educational landscape in Indonesia. However, it should be noted that while our dataset is nearly nationally representative, it does not provide Language 31% provincial or district-level representation due to the sampling design. Therefore, we have limited Others 23% our work to only the national level results. APPENDICES - A1 42 2 Teach Reliability Test In our sample, nine observations were excluded due to missing data, resulting in an attrition rate of less than one percent. The pattern of missing data was found to be completely random, indicating that it is unlikely to skew the results, as confirmed by Little’s Missing Completely At Random (MCAR) test (p-value > 0.1) (Little, 2013). In Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study addition, to assess the reliability of the Teach elements alongside their related behaviors, we conducted a Cronbach’s alpha test. This test measures the internal consistency of the Teach elements, specifically how well these elements correlate with each other. In other words, it helps us determine if the items within the Teach elements are consistently measuring the same underlying the construct of teaching quality. The outcome affirmed a robust internal consistency, evidenced by a Cronbach’s alpha coefficient of .84 at the aggregate level (Tavakol & Dennick, 2011). Lastly, we conducted a confirmatory factor analysis (CFA) to verify the factor structure of Teach. The CFA determines whether these Teach elements are all related to and influenced by the underlying concept of Teach. The results show that the RMSEA suggests a reasonable fit (<0.07), both the CFI and TLI are below the commonly accepted thresholds for good or even acceptable fit (Xia, Y., & Yang, Y., 2019). Overall, both the Cronbach’s alpha test and CFA results affirm the reliability of Teach score in Indonesia, demonstrating strong internal consistency and relationships among the elements. 3 Teach Training This section further validate the use of Teach scores in the context of Indonesia. We began by translating all documents (modules, manuals, training PowerPoints, and Teach tools) into Bahasa Indonesia. For the practice videos, we provided Indonesian subtitles if the video is coming from other countries and produced a total of 16 videos of Indonesian classrooms, which were used to practice and examination during observer training for our enumerator of this study. Prior the training, we held discussions with teachers, researchers, and experts on the tools and how to translate the documents accurately. We also provided examples of each case based on the Indonesian local context in Teach manual Indonesia version. For instance, what is considered “positive learning behavior” or “positive language” in Indonesia might differ from that in Afghanistan or Tanzania, so we provided a specific sample of what is considered Low, Medium, or High for Indonesia classrooms. We continued by showing the performance of observers after a 6-day training course. Then, we investigate Teach reliability in the field. A total of forty-four classroom observers participated in observer training to be raters. The observation team comprised professionals with a minimum of bachelor’s degree, experienced enumerators and surveyors who have collected data related to the education field in the past. Observers participated in a six-day training that required them to practice coding using recorded videos, participate in a live field visit to MoECRT and MoRA schools in the Salatiga region, and pass the Teach reliability exam. The reliability exam required them to code 15-minute classroom observation segments in accordance with the Teach manual’s rubric. After watching the 15-minute segment, observers were given 15 minutes to score the video. To pass the exam, they must be accurate within one of the master codes in eight of the ten scores (nine quality of teaching practices elements and the time on task element) for each segment. Observers were given two attempts to pass the exam. Of the 44 observers APPENDICES - A1 that were trained and took the exam, 36 passed the exam on the first attempt. Those who 43 did not pass (8) received a second attempt exam opportunity and 4 passed the exam. At the end a total of 40 enumerators were able to become Teach observers (See Table A1.2). Only certified observers who passed the exam then administered Teach during the data collection, with a total of 500 classes observed during the implementation. Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Table A1.2. Success Rate on Teach Training Exam Second Final Attempt Result Number of observers that took the exam 44 8 44 Number of observers that passed the exam 36 4 40 Success rate 82% 50% 91% 44 APPENDICES - A2 A2. Summary Statistics Variable Obs Mean SD Min Max School location (Rural=1, Urban=2) 993 1.25 0.36 1 2 Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study School type (MoRA=1, MoECRT=2) 993 1.74 0.50 1 2 School province 993 10.63 7.41 1 28 Teacher sex (Male=0, Female=1) 993 0.77 0.46 0 1 Teacher experience (0-5 years=1, 6-10 years=2, 10- 993 2.70 1.02 1 4 20 years=3, Above 20 years = 4) Teacher education level (High school=1, Tertiary 993 2.99 0.49 1 4 equivalent=2, Graduate=3, Master’s degree=4) Overall Teach Score (1-5 scale) 993 2.68 0.42 1.42 4.08 0.1. Provide learning activity (S1) (Yes=1, No=0) 993 0.96 0.17 0 1 0.2. Students are on Task (S1) 993 3.36 0.76 1 4 (Not on Task =1, Low=2, Medium=3, High=4) 0.1. Provide learning activity (S2) (Yes=1, No=0) 993 0.96 0.17 0 1 0.2. Students are on Task (S2) 993 3.30 0.77 1 4 (Not on Task =1, Low=2, Medium=3, High=4) 0.1. Provide learning activity (S3) (Yes=1, No=0) 993 0.95 0.22 0 1 0.2. Students are on Task (S3) 993 3.21 0.85 1 4 (Not on Task =1, Low=2, Medium=3, High=4) Classroom Culture (1-5 scale) 993 3.44 0.57 1.5 4.5 Supportive Learning Environment (1-5 scale) 993 3.52 0.67 2 5 1.1. Respects students (1-4 scale) 993 3.82 0.47 2 4 1.2. Uses positive language 993 2.62 0.71 2 4 (Low=2, Medium=3, High=4) 1.3. Responds to students’ needs 993 1.83 1.15 1 4 (N/A=1, Low=2, Medium=3, High=4) 1.4a. Gender bias and stereotypes 993 3.00 0.16 2 4 (Low=2, Medium=3, High=4) 1.4b. Disability bias and stereotypes 993 3.00 0.00 3 3 (Low=2, Medium=3, High=4) 1.4. Bias and stereotypes 993 2.99 0.15 2 4 (Low=2, Medium=3, High=4) Positive Behavioral Expectations (1-5 scale) 993 3.36 0.91 1 5 2.1. Clear behavioral expectations 993 3.59 0.70 2 4 (Low=2, Medium=3, High=4) 2.2. Acknowledges positive behavior 993 2.19 0.45 2 4 (Low=2, Medium=3, High=4) 2.3. Redirects misbehavior 993 3.69 0.64 2 4 (Low=2, Medium=3, High=4) Instruction (1-5 scale) 993 2.52 0.61 1 4.25 Lesson Facilitation (1-5 scale) 993 3.18 0.92 1 5 3.1. Articulates lesson objectives 993 3.34 0.60 2 4 (Low=2, Medium=3, High=4) 3.2. Clear explanations 993 3.31 0.64 2 4 (Low=2, Medium=3, High=4) 45 APPENDICES - A2 Variable 3.3. Connects lesson Obs Mean SD Min Max 993 2.46 0.70 2 4 (Low=2, Medium=3, High=4) 3.4. Models by enacting or thinking 993 3.12 0.86 2 4 (Low=2, Medium=3, High=4) Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Checks for Understanding (1-5 scale) 993 2.68 1.09 1 5 4.1. Uses questions and prompts (Low=2, Medium=3, High=4) 993 2.77 0.75 2 4 4.2. Monitors during independent/group work (N/A=1, Low=2, Medium=3, High=4) 993 2.53 1.22 1 4 4.3. Adjusts teaching (Low=2, Medium=3, High=4) 993 2.53 0.70 2 4 Feedback (1-5 scale) 993 1.80 0.94 1 5 5.1. Provides comments to clarify misunderstanding (Low=2, Medium=3, 993 2.56 0.74 2 4 High=4) 5.2. Provides comments to identify successes (Low=2, Medium=3, High=4) 993 2.22 0.49 2 4 Critical Thinking (1-5 scale) 993 2.41 1.05 1 5 6.1. Asks open-ended questions (Low=2, Medium=3, High=4) 993 2.37 0.59 2 4 6.2. Provides thinking tasks (Low=2, Medium=3, High=4) 993 2.82 0.71 2 4 6.3. Students ask open-ended quest (Low=2, Medium=3, High=4) 993 2.83 0.66 2 4 Autonomy (1-5 scale) 993 2.50 1.00 1 5 7.1. Provides students with choice (Low=2, Medium=3, High=4) 993 2.26 0.54 2 4 7.2. Provides opportunities to take on roles (Low=2, Medium=3, High=4) 993 2.78 0.80 2 4 7.3. Students volunteer to participate (Low=2, Medium=3, High=4) 993 2.89 0.76 2 4 Socioemotional Skills (1-5 scale) 993 2.10 0.55 1 4 Perseverance (1-5 scale) 993 2.16 0.45 1 4 8.1. Acknowledges students’ effort (Low=2, Medium=3, High=4) 993 2.20 0.39 2 4 8.2. Positive attitude toward students’ challenges (Low=2, Medium=3, High=4) 993 3.05 0.33 2 4 8.3. Encourages goal setting (Low=2, Medium=3, High=4) 993 2.08 0.31 2 4 Social and Collaborative Skills (1-5 scale) 993 1.63 1.04 1 5 9.1. Promotes student collaboration (Low=2, Medium=3, High=4) 993 2.33 0.66 2 4 9.2. Promotes students’ interpersonal skills (Low=2, Medium=3, High=4) 993 2.19 0.45 2 4 9.3. Students collaborate (Low=2, Medium=3, High=4) 993 2.36 0.67 2 4 Note. Scale for Teach Behaviors (1to 4): 1 Not Applicable, 2 Low, 3 Medium, 4 High. Weighted statistics, encompassing means, standard deviations, minimum, and maximum values, have been utilized to account for variations in student and school populations. 46 APPENDICES - A3 A3. Inter-item Correlations Inter-item correlations are also computed to examine associations between the different elements of the Teach scale. Table A3 presents the descriptive statistics and inter-item correlations of the nine quality of teaching practices elements with means ranging from 1.63 to 3.52, and the time Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study on task variables, with means of 3.4 and 3.3, and inter-item correlations ranging from 0 to 0.23. Table A3. Teach Inter-Element Correlations Variable SLE PBE LF CFU F CT A P SCS Supportive Learning 1.00 Environment (SLE) Positive Behavioral 0.03 1.00 Expectations (PBE) Lesson Facilitation (LF) 0.18* 0.20* 1.00 Checks for 0.24* 0.09* 0.05 1.00 Understanding (CFU) Feedback (F) 0.06* 0.03 0.00 0.30* 1.00 Critical Thinking (CT) 0.17* 0.13* 0.15* 0.33* 0.10* 1.00 Autonomy (A) 0.24* 0.14* 0.17* 0.18* 0.06 0.16* 1.00 Perseverance (P) 0.18* 0.01 0.08* 0.16* 0.20* 0.11* 0.08* 1.00 Social and Collaborative 0.16* -0.02 0.02 0.17* 0.01 0.14* 0.12* 0.09* 1.00 Skills (SCS) Note. It shows pairwise correlations among the Teach elements, with sample weights applied to accommodate differences in student and school populations. The abbreviations are utilized to represent Teach elements. *p < 0.05. 47 APPENDICES - A4 A4. Comparative analysis of Teach Scores Across Groups Table 4.1. Regression Analysis of Teach Scores: MoECRT versus MoRA Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study MoECRT (1) MoRA (2) Diff (1-2) Mean (SD) Mean (SD) Mean (SD) Overall Teach Score 2.70 (0.43) 2.64 (0.43) 0.06*** (0.00) A. Classroom Culture 3.46 (0.54) 3.37 (0.6) 0.09*** (0.00) 1. Supportive Learning Environment 3.55 (0.62) 3.43 (0.64) 0.12*** (0.00) 2. Positive Behavioral Expectations 3.37 (0.84) 3.32 (0.98) 0.06*** (0.00) B. Instruction 2.54 (0.6) 2.46 (0.62) 0.08*** (0.00) 3. Lesson Facilitation 3.18 (0.87) 3.18 (0.93) -0.00 (0.00) 4. Checks for Understanding 2.71 (1.06) 2.60 (1.09) 0.11*** (0.00) 5. Feedback 1.84 (0.97) 1.68 (0.97) 0.16*** (0.00) 6. Critical Thinking 2.41 (1.03) 2.38 (1.08) 0.03*** (0.00) C. Socioemotional Skills 2.10 (0.58) 2.08 (0.57) 0.02*** (0.00) 7. Autonomy 2.51 (1.05) 2.47 (0.99) 0.05*** (0.00) 8. Perseverance 2.15 (0.51) 2.16 (0.46) -0.01*** (0.00) 9. Social and Collaborative Skills 1.64 (1.08) 1.60 (1.05) 0.04*** (0.00) Note. Weighted statistics account for population variations in student and school demographics across diverse educational institutions (MoRA and MoECRT). Regression analysis employs weighted methodology to address these variations, with standard errors reported in parentheses, signifying statistical significance as * p<0.05, ** p<0.01, *** p<0.001. Table 4.2. Regression Analysis of Teach Urban (1) Rural (2) Diff (1-2) Mean (SD) Mean (SD) Mean (SD) Overall Teach Score 2.76 (0.4) 2.66 (0.44) 0.11*** (0.00) A. Classroom Culture 3.55 (0.59) 3.4 (0.54) 0.14*** (0.00) 1. Supportive Learning Environment 3.63 (0.55) 3.48 (0.64) 0.14*** (0.00) 2. Positive Behavioral Expectations 3.47 (0.93) 3.32 (0.86) 0.15*** (0.00) B. Instruction 2.69 (0.58) 2.46 (0.61) 0.23*** (0.00) 3. Lesson Facilitation 3.4 (0.83) 3.1 (0.89) 0.30*** (0.00) 4. Checks for Understanding 2.61 (1.06) 2.71 (1.07) -0.10*** (0.00) 5. Feedback 1.92 (1.04) 1.76 (0.95) 0.16*** (0.00) 6. Critical Thinking 2.83 (1.08) 2.27 (1) 0.57*** (0.00) C. Socioemotional Skills 2.05 (0.49) 2.11 (0.6) -0.06*** (0.00) 7. Autonomy 2.53 (1.02) 2.49 (1.04) 0.04*** (0.00) 8. Perseverance 2.19 (0.57) 2.15 (0.47) 0.04*** (0.00) 9. Social and Collaborative Skills 1.44 (0.89) 1.69 (1.12) -0.26*** (0.00) Note. Weighted statistics account for population variations in student and school demographics across diverse educational institutions (MoRA and MoECRT). Regression analysis employs weighted methodology to address these variations, with standard errors reported in parentheses, signifying statistical significance as * p<0.05, ** p<0.01, *** p<0.001. APPENDICES - A4 48 Table 4.3. Regression Result on Teach Global Score by School Characteristics Variable (1) (2) (3) (4) Type Location Both Specified School type 1.00 0.06*** Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study (MoECRT=1, MoRA=0) 0.03 (0.001) Urban 0.18* 0.11*** 0.11*** (Urban=1, Rural=0) (0.001) (0.001) School type and location (MoRA school in rural areas as a reference) MoECRT in Rural 0.08*** (0.001) MoRA in Urban 0.18*** (0.002) MoECRT in Urban 0.16*** (0.001) 2.58*** 2.55*** 2.45*** 2.60*** Constant (0.002) (0.001) (0.002) (0.001) [2.40,2.54] [2.36,2.61] [2.23,2.46] [2.64,2.87] R2 0.02 0.01 0.02 0.03 N 993 993 993 993 Note. Standard errors are clustered at the provincial level. 95% confidence intervals in brackets. * p<0.05, ** p<0.01, *** p<0.001. Reference groups: MoRA school observations in rural areas (N=459) In our sample, MoRA schools in rural areas have 459 observations (46%). MoECRT schools in rural regions account for 378 observations (38%). In urban areas, MoRA and MoECRT schools consist of 82 observations (8%) and 74 observations (7%) respectively. Table 4.4. Distribution of Observations for MoRA and MoECRT Schools in Rural and Urban Areas Authority Location Number of Observation % MoRA Rural 459 46% MoRA Urban 82 8% MoECRT Rural 378 38% MoECRT Urban 74 7% APPENDICES - A4 Table 4.5. Regression Analysis of Teach Scores: Female versus Male Teachers 49 Female Male Diff (1) (2) (1-2) Mean (SD) Mean (SD) Mean (SD) Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Overall Teach Score 2.7 (0.43) 2.63 (0.42) 0.06*** (0.00) A. Classroom Culture 3.45 (0.55) 3.39 (0.59) 0.06*** (0.00) 1. Supportive Learning Environment 3.53 (0.61) 3.49 (0.69) 0.04*** (0.00) 2. Positive Behavioral Expectations 3.38 (0.88) 3.3 (0.87) 0.08*** (0.00) B. Instruction 2.52 (0.61) 2.5 (0.87) 0.02*** (0.00) 3. Lesson Facilitation 3.18 (0.88) 3.17 (0.92) 0.02*** (0.00) 4. Checks for Understanding 2.68 (1.06) 2.67 (1.07) 0.01*** (0.00) 5. Feedback 1.78 (0.99) 1.86 (0.94) -0.08*** (0.00) 6. Critical Thinking 2.44 (1.03) 2.29 (1.08) 0.15*** (0.00) C. Socioemotional Skills 2.12 (0.58) 2.01 (0.57) 0.11*** (0.00) 7. Autonomy 2.56 (1.03) 2.29 (1.02) 0.27*** (0.00) 8. Perseverance 2.14 (0.48) 2.21 (0.56) -0.07*** (0.00) 9. Social and Collaborative Skills 1.66 (1.1) 1.54 (0.99) 0.11*** (0.00) Note. Weighted statistics account for population variations in student and school demographics across diverse educational institutions (MoRA and MoECRT). Regression analysis employs weighted methodology to address these variations, with standard errors reported in parentheses, signifying statistical significance as * p<0.05, ** p<0.01, *** p<0.001. Table 4.6. Teach Score by the Level of Teacher Education Postgraduate Undergraduate Diploma or less (1) (2) (3) Mean (SD) Mean (SD) Mean (SD) Overall Teach Score 2.72 (0.56) 2.69 (0.42) 2.49 (0.35) A. Classroom Culture 3.5 (0.67) 3.44 (0.55) 3.37 (0.58) 1. Supportive Learning Environment 3.48 (0.64) 3.52 (0.62) 3.42 (0.66) 2. Positive Behavioral Expectations 3.52 (0.97) 3.35 (0.87) 3.32 (0.91) B. Instruction 2.51 (0.69) 2.53 (0.6) 2.24 (0.52) 3. Lesson Facilitation 3.21 (0.97) 3.18 (0.87) 3.12 (1) 4. Checks for Understanding 2.46 (0.94) 2.72 (1.07) 2.2 (1.1) 5. Feedback 2 (1.06) 1.8 (0.98) 1.56 (0.67) 6. Critical Thinking 2.35 (1.01) 2.43 (1.05) 2.08 (0.95) C. Socioemotional Skills 2.14 (0.61) 2.1 (0.58) 1.86 (0.43) 7. Autonomy 2.65 (1.05) 2.5 (1.03) 2.27 (0.94) 8. Perseverance 2.13 (0.37) 2.16 (0.51) 2.07 (0.32) 9. Social and Collaborative Skills 1.64 (1.02) 1.65 (1.09) 1.23 (0.69) Note. Weighted statistics account for population variations in student and school demographics across diverse educational institutions. The sample class for teachers with a “Diploma or less” education level is 68, while there are 883 teachers with an “Undergraduate” degree and 42 teachers with a “Postgraduate” degree. APPENDICES - A4 50 Table 4.7. Regression Result on Teach Global Score by Teacher’s education level (1) (2) (3) (4) Overall Teach Classroom Instruction Socioemotional Score Culture Skills Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Undergraduate 0.20*** 0.07*** 0.29*** 0.25*** (0.002) (0.003) (0.002) (0.002) Postgraduate 0.23*** 0.14*** 0.27*** 0.28*** (0.003) (0.004) (0.003) (0.003) Constant 2.49*** 3.37*** 2.24*** 1.86*** (0.002) (0.003) (0.002) (0.002) R2 0.01 0.00 0.01 0.01 N 993 Note. Reference groups: Teachers with education below diploma (N=68). Regression analysis employs weighted methodology to account for variations in student and school populations, with standard errors reported in parentheses. * p<0.05, ** p<0.01, *** p<0.001. Table 4.8. Teach Score by Academic Subject Mathematics Language Other (N=230) (N=154) (N=117) (1) (2) (3) Mean (SD) Mean (SD) Mean (SD) Overall Teach Score 2.72 (0.41) 2.6 (0.41) 2.71 (0.48) A. Classroom Culture 3.46 (0.57) 3.4 (0.56) 3.43 (0.54) 1. Supportive Learning Environment 3.59 (0.58) 3.45 (0.66) 3.45 (0.65) 2. Positive Behavioral Expectations 3.33 (0.92) 3.35 (0.9) 3.41 (0.77) B. Instruction 2.63 (0.58) 2.38 (0.62) 2.46 (0.62) 3. Lesson Facilitation 3.33 (0.82) 3.02 (0.93) 3.08 (0.91) 4. Checks for Understanding 2.84 (1.08) 2.42 (1) 2.69 (1.06) 5. Feedback 1.98 (1.05) 1.58 (0.83) 1.7 (0.91) 6. Critical Thinking 2.36 (1.04) 2.5 (1.1) 2.38 (0.97) C. Socioemotional Skills 2.08 (0.56) 2.02 (0.49) 2.22 (0.68) 7. Autonomy 2.57 (1.04) 2.35 (0.95) 2.55 (1.09) 8. Perseverance 2.17 (0.54) 2.1 (0.42) 2.21 (0.49) 9. Social and Collaborative Skills 1.51 (0.95) 1.6 (1.04) 1.9 (1.27) Note. Others refer to classes, including Arabic, English, Quran/ Islamic Studies, Science, etc APPENDICES - A4 51 Table 4.9. Regression Result on Teach Score by Academic Subject (1) (2) (3) (4) Overall Teach Classroom Instruction Socioemotional Score Culture Skills Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Language -0.13*** -0.06*** -0.25*** -0.07*** (0.001) (0.001) (0.001) (0.001) Others -0.02*** -0.03*** -0.17*** 0.14*** (0.001) (0.001) (0.001) (0.001) Constant 2.72*** 3.46*** 2.63*** 2.08*** (0.001) (0.001) (0.001) (0.001) R2 0.02 0.00 0.03 0.02 N 993 Note. Reference groups: Mathematic teachers (N=456). Regression analysis employs weighted methodology to account for variations in student and school populations, with standard errors reported in parentheses. * p<0.05, ** p<0.01, *** p<0.001. Table 4.10. Teach Score by Academic Curriculum Curriculum Curriculum Others Merdeka KTSP 2013 (N=30) (N=263) (N=700) (3) (1) (2) Element Mean (SD) Mean (SD) Mean (SD) Overall Teach Score 2.8 (0.42) 2.61 (0.42) 2.58 (0.29) A. Classroom Culture 3.55 (0.55) 3.37 (0.55) 3.32 (0.53) 1. Supportive Learning Environment 3.64 (0.54) 3.43 (0.66) 3.58 (0.61) 2. Positive Behavioral Expectations 3.45 (0.91) 3.3 (0.85) 3.05 (0.83) B. Instruction 2.66 (0.58) 2.43 (0.61) 2.35 (0.47) 3. Lesson Facilitation 3.35 (0.86) 3.07 (0.88) 2.82 (0.88) 4. Checks for Understanding 2.72 (0.98) 2.66 (1.12) 2.69 (0.93) 5. Feedback 1.9 (1.02) 1.73 (0.94) 1.78 (0.75) 6. Critical Thinking 2.65 (1.01) 2.25 (1.04) 2.11 (0.79) C. Socioemotional Skills 2.19 (0.6) 2.03 (0.55) 2.06 (0.46) 7. Autonomy 2.65 (1.1) 2.4 (0.97) 2.57 (0.87) 8. Perseverance 2.22 (0.56) 2.11 (0.45) 2.13 (0.44) 9. Social and Collaborative Skills 1.71 (1.09) 1.58 (1.06) 1.48 (0.77) Note. Weighted statistics account for population variations in student and school demographics. The sample size for Merdeka Curriculum observations is 263, while for KTSP 2013, it is 700, and for the “Other” category, it is 30. Others refer to curriculums, including national emergency, KTSP 2006, etc. APPENDICES - A4 52 Table 4.11. Regression Result on Teach Score by Academic Curriculum (1) (2) (3) (4) Overall Teach Classroom Instruction Socioemotional Score Culture Skills Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study Others -0.03*** -0.05*** -0.08*** 0.03*** (0.003) (0.005) (0.004) (0.004) Merdeka Curriculum 0.19*** 0.18*** 0.23*** 0.16*** (0.001) (0.001) (0.001) (0.001) Constant 2.61*** 3.37*** 2.43*** 2.03*** (0.000) (0.001) (0.001) (0.001) R2 0.05 0.03 0.04 0.02 N 993 Note. Reference groups: Mathematic teachers (N=456). Regression analysis employs weighted methodology to account for variations in student and school populations, with standard errors reported in parentheses. * p<0.05, ** p<0.01, *** p<0.001. Table 4.12. Regression Analysis of Teach Scores between Kurikulum Merdeka and Kurikulum KTSP 2013 Curriculum Curriculum Diff Merdeka KTSP 2013 (1-2) (1) (2) Element Mean (SD) Mean (SD) Mean (SD) Overall Teach Score 2.8 (0.42) 2.61 (0.42) 0.19*** (0.001) A. Classroom Culture 3.55 (0.55) 3.37 (0.55) 0.18*** (0.001) 1. Supportive Learning Environment 3.64 (0.54) 3.43 (0.66) 0.22*** (0.001) 2. Positive Behavioral Expectations 3.45 (0.91) 3.3 (0.85) 0.15*** (0.002) B. Instruction 2.66 (0.58) 2.43 (0.61) 0.23*** (0.001) 3. Lesson Facilitation 3.35 (0.86) 3.07 (0.88) 0.29*** (0.002) 4. Checks for Understanding 2.72 (0.98) 2.66 (1.12) 0.06*** (0.002) 5. Feedback 1.9 (1.02) 1.73 (0.94) 0.18*** (0.002) 6. Critical Thinking 2.65 (1.01) 2.25 (1.04) 0.40*** (0.002) C. Socioemotional Skills 2.19 (0.6) 2.03 (0.55) 0.16*** (0.001) 7. Autonomy 2.65 (1.1) 2.4 (0.97) 0.26*** (0.002) 8. Perseverance 2.22 (0.56) 2.11 (0.45) 0.10*** (0.001) 9. Social and Collaborative Skills 1.71 (1.09) 1.58 (1.06) 0.13*** (0.002) Note. Weighted statistics account for population variations in student and school demographics across diverse educational institutions (MoRA and MoECRT). Regression analysis employs weighted methodology to address these variations, with standard errors reported in parentheses. * p<0.05, ** p<0.01, *** p<0.001. 53 APPENDICES - A5 A5. Teach Score Distribution and Rubrics at A Glance Teacher No Yes Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study provides learning activity 4% 96% Time on Low Medium High Learning 6 or more 2 to 5 0 or 1 Students are students are students are students are on task off task off task off task 10% 39% 48% Low Score & Medium Score High Score & Area Element Behaviors N/A Description & Description Description SUPPORTIVE 18% 2% 80% LEARNING Treats all 1.1: Respect Does not treat Treats all 0% somewhat ENVIRONMENT: all respectfully respectfully respectfully The teacher creates a classroom 61% 26% 13% environment where 1.2: Positive Does not Uses some Consistently 0% Language use positive positive uses positive students can feel language language language emotionally safe and supported. Moreover, 6% 2% 17% all students feel Is not aware Responds 1.3: Responds Responds & or does not but does not 75% welcome, as the to Needs addresses the respond to address the problem teacher treats all needs problem Classroom Culture students respectfully. 98% 1% 1% Does not exhibit Does not 1.4: Bias and Exhibits bias bias but does exhibit bias 0% Stereotypes or reinforces not challenge and challenge stereotypes stereotypes stereotypes either 98% 1% Does not 1% 1.4a: Gender Exhibits exhibit gender Does not exhibit Bias and gender bias bias but does gender bias 0% Stereotypes or reinforces not challenge and challenge stereotypes stereotypes stereotypes either 100% 0% Does not exhibit 0% 1.4b: Disability Exhibits disability bias Does not exhibit bias and gender bias but does not disability bias 0% challenges or reinforces challenge and challenge stereotypes stereotypes stereotypes stereotypes either 54 APPENDICES - A5 Low Score & Medium Score High Score & Area Element Behaviors N/A Description & Description Description POSITIVE BEHAVIORAL 12% 18% Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study 80% 2.1: Behavioral Does not Treats all EXPECTATIONS: The Treats all 0% Expectations set clear somewhat teacher promotes respectfully expectations respectfully positive behavior Classroom Culture by acknowledging students’ behavior 90% 2.2: 3% Does not 7% that meets or exceeds Acknowledges Acknowledges acknowledge Acknowledges 0% expectations. Positive positive positive some behavior Behavior behavior Moreover, the teacher behavior sets clear behavioral expectations for 12% 78% different parts of the 10% Effectively Effectively 2.3: Redirects Ineffectively redirects or redirects or 0% lesson. Misbehavior redirects somewhat students are effective well-behaved LESSON 7% 40% 53% FACILITATION: 3.1 Articulates Does not state States specific States broad Lesson objective or objective that 0% The teacher facilitates objective or can Objectives cannot be is aligned to be inferred the lesson to promote inferred activities comprehension by explicitly articulating 34% 12% 54% 3.2 Clear Clear and the objectives, Confusing or no Somewhat clear 0% Explanations straightforward explanation explanation providing clear explanation explanations of 15% concepts, and 73% 12% 3.3 Connects Superficially connecting the Does not Meaningfully 0% Lesson or unclearly connect connects lesson with other connects content knowledge or students’ experiences. 3.4 Models by 36% 27% 37% Enacting or Does not Partially Completely 0% Instruction Thinking Aloud model models models CHECKS FOR 4.1 Uses 43% 37% 20% UNDERSTANDING: Questions & Either does not Asks effectively Asks effectively The teacher checks Prompts to ask or the class only of a few of most 0% for understanding Determine responds in students students Understanding synchrony to ensure most students comprehend the lesson content. 4.2 Monitors 19% 31% 19% Moreover, the teacher During Does not Systematically Monitors some 31% Independent / monitor monitors most adjusts the pace students Group Work students students of the lesson to provide students with additional learning 22% 66% 12% opportunities. 4.3 Adjusts Adjusts, but Does not Substantially 0% teaching briefly and adjust adjusts superficially 55 APPENDICES - A5 Low Score & Medium Score High Score & Area Element Behaviors N/A Description & Description Description FEEDBACK: The 61% 25% 15% Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study teacher provides 5.1 Provides Does not Provides Provides Comments / provide com- general or specific & specific comments Prompts ments about superficial substantive 0% or prompts to to Clarify misunder- comments comments Misunderstand- standings or about misun- about misun- help identify ings comments are derstandings derstandings misunderstandings, simple understand successes, and guide 86% 10% 4% thought processes to 5.2 Provides Does not Provides Provides Comments provide general or specific and promote learning. / Prompts comments superficial substantive 0% to Identify about successes comments comments Successes or comments about successes about successes are simple Instruction CRITICAL THINKING: 15% The teacher builds Asks two or 7% 78% more but students’ critical 6.1 Asks Do not ask OR does not build Asks 3+ and at Open-ended least 1 builds 0% thinking skills by Questions asks one open- on student upon student ended question responses or 1 is encouraging them a follow-up to a responses to actively analyze response content. 34% 48% 18% 6.2 Does not Provides Provides Provides provide superficial substantial 0% Thinking thinking thinking thinking Tasks tasks tasks tasks 6.3 Students ask 54% 14% 32% Open-Ended Students do not Students ask Students Questions and/ ask, but perform and/or perform 0% neither ask nor or Perform superficial substantial perform Thinking Tasks thinking tasks thinking tasks AUTONOMY: 6% 7% The teacher provides 7.1 Provides 87% Explicitly Explicitly Does not provides provides with students with Students with explicitly with at least at least 1 Choices opportunities to provide choices 1 superficial substantive choice choice make choices & take Socioemotional Skills on meaningful roles 22% in the classroom. 29% 49% Provides 7.2 Provides Provides Students make use of Does not opportunities Opportunities to opportunities to provide to take on these opportunities Take on Roles take on limited opportunities meaningful roles by volunteering roles to take on roles & expressing their 38% 22% ideas & opinions Few students Most students 7.3 Students 40% volunteer by volunteer by throughout the Volunteer to Students do not expressing expressing lesson. Participate volunteer their ideas their ideas and taking and taking on roles on roles 56 APPENDICES - A5 Low Score & Medium Score High Score & Area Element Behaviors Description & Description Description PERSEVERANCE: The Teacher Practices in Indonesia: Results of the Teach Primary Classroom Observation Study teacher promotes 1% 87% 12% students’ efforts 8.1 Does not Sometimes Frequently Acknowledges acknowledges toward the goal of Students’ Efforts acknowledge acknowledges and identifies efforts efforts mastering new skills efforts or concepts, instead of focusing solely on results, intelligence, or natural abilities. In 8.2 Positive 3% 89% 8% addition, the teacher Attitude Toward Has a negative Has a neutral Has a positive Students’ has a positive attitude attitude attitude attitude Challenges toward challenges, framing failure & frustrations as useful parts of the learning 5% 94% Encourages 1% Socioemotional Skills process. The teacher Does not short or Encourages 8.3 Encourages also encourages encourage short long-term short and Goal setting or long-term goalsetting, or long-term students to set short- goalsetting discusses their goalsetting &/or long-term goals. importance SOCIAL & 82% 7% 11% COLLABORATIVE 9.1 Promotes Does not Promotes Promotes SKILLS: The teacher student promote superficial substantial collaboration collaboration student student encourages students’ among students collaboration collaboration collaboration with one another and promotes students’ 10% 87% 9.2 Promotes Briefly or 3% interpersonal skills. Does not Student superficially Promotes promote Students respond to Interpersonal promotes interpersonal interpersonal Skills interpersonal skills the teacher’s efforts skills skills by collaborating with one another in the 11% classroom, creating 11% 78% Students Students an environment free 9.3 Students Students do not consistently collaborate collaborate with collaborate or collaborate from physical or some and rarely one another display negative and display display negative emotional hostility. behaviors no negative behaviors behavior