Systems Approach for Better Education Results Making Great Strides Yet a Learning Crisis Remains in Tanzania SERVICE DELIVERY INDICATORS (SDI) SURVEY SABER SERVICE DELIVERY (SD) SURVEY ii Systems Approach for Better Education Results Making Great Strides Yet a Learning Crisis Remains in Tanzania RESULTS OF THE SDI AND SABER SERVICE DELIVERY SURVEYS Authors: Iva Trako, Ezequiel Molina and Salman Asim October 2019 iii Acknowledgements This report has been prepared by Iva Trako, Ezequiel Molina and Salman Asim under the guidance of Omar Arias. Additionally, Ravinder Casley Gera, Rohit Chhabra, Dmitry Chugunov, Renata Freitas Lemos, Eema Masood, Halsey Rogers, Cole Scanlon, Waly Wane and were part of the extended team. We also gratefully acknowledge all the helpful comments and suggestions from Safaa El Tayeb El-Kogali, Ciro Avitabile, Koen M. Geven, Shwetlena Sabarwal and Xiaoyan Liang. Data collection was undertaken by the Research on Poverty Alleviation (REPOA) under the leadership of Prof. Samwel Wangwe and Lucas Katera. The team would like to use the opportunity to thank the officials of the Ministry of Education as well as the respondents who participated in the survey without whom this study would not have been possible. We would also like to thank the World Bank’s Tanzania Country Office and especially Bella Bird (Country Director), Gayle Martin (former Program Lead for Tanzania), Samer Al-Samarrai and Cornelia Jesse (Tanzania Education Task Team Leaders) for valuable guidance and support. Finally, the team gratefully acknowledges the financial support from the U.K. Department for International Development (DFID) in addition to the resources from the World Bank. iv Table of Contents Acknowledgements ...................................................................................................................................................................... iv Executive Summary ..................................................................................................................................................................... 10 Introduction ................................................................................................................................................................................ 23 Methodology and Implementation .......................................................................................................................................................... 25 Framework - World Development Report 2018 ............................................................................................................................ 27 Background on SDI and SABER Service Delivery Surveys ............................................................................................................... 28 Box 0.1: The Link between SABER SD and SDI Approach ......................................................................................................................... 28 Box 0.2: The Service Delivery Indicators (SDI) Program ........................................................................................................................... 29 Chapter 1 : Is Tanzania’s Education System Aligned with Learning? .............................................................................................. 30 Box 1.1: Background on the SDI Student Assessment .............................................................................................................................. 32 Section I. Student Results: Language Assessment .................................................................................................................................... 32 Box 1.2: Sample of Language Questions .................................................................................................................................................. 33 Section II. Student Results: Mathematics Assessment ............................................................................................................................. 35 Box 1.3: Sample of Mathematics Questions ............................................................................................................................................ 35 Section III. Average Student Test Scores by Subject ................................................................................................................................. 37 Section IV. Disaggregation by Region, Ethnic Group and Gender ............................................................................................................ 38 Different Impacts on Different Students .......................................................................................................................................... 40 Box 1.4: Education Quality Improvement Tanzania (EQUIP-T)................................................................................................................ 42 Section VI. Looking Beyond the Mean ...................................................................................................................................................... 44 Difference in Learning Outcomes Between and Within Schools ...................................................................................................... 44 Results from the Learning Decomposition ....................................................................................................................................... 45 Box 1.5: Comparing Tanzania’s Students Internationally ......................................................................................................................... 47 Chapter 2 : Teachers .................................................................................................................................................................... 48 Section I. What do teachers do? Understanding teachers effective use of time ..................................................................................... 50 School Absence Rate, Classroom Absence Rate and Time Spent Teaching ..................................................................................... 50 Box 2.1 : Profile of Tanzania’s Teachers ................................................................................................................................................... 54 Section II. What do teachers know? Measuring teachers’ content knowledge in Language and Mathematics ..................................... 56 Teacher’s Content Knowledge in Language...................................................................................................................................... 56 Teacher’s Content Knowledge in Mathematics ................................................................................................................................ 57 Section III. How well do teachers teach? Teachers’ pedagogic content knowledge and teaching skills ................................................. 59 Pedagogical Knowledge .................................................................................................................................................................... 59 CLASS Overall Results................................................................................................................................................................................ 62 Box 2.2: Missing in Action in Tanzania ...................................................................................................................................................... 66 5 Box 2.3: Comparing Tanzania’s Teachers Internationally......................................................................................................................... 67 Chapter 3 : School Management .................................................................................................................................................. 71 Section I. School Governance and Accountability .................................................................................................................................... 72 Section II. Measuring Education Management Practices in Tanzania ...................................................................................................... 74 What management practices do Tanzanian principals follow?........................................................................................................ 74 Box 3.1: The Development-World Management Survey (D-WMS) .......................................................................................................... 75 Comparing management capacity between the education and manufacturing sectors ................................................................. 79 Box 3.2: Comparing Tanzania’s School Management Practices Internationally ...................................................................................... 81 Chapter 4 : Schools Inputs ........................................................................................................................................................... 84 Section I. Inputs for Learning – Classroom Level ...................................................................................................................................... 86 Section II. Basic Infrastructure – School Level .......................................................................................................................................... 89 Box 4.1: What does a typical school in Tanzania look like? ...................................................................................................................... 93 Box 4.2: Comparing Tanzania’s School Inputs Internationally ................................................................................................................. 95 Chapter 5 : Student Support ........................................................................................................................................................ 97 Box 5.1: Student’s Profile in Tanzania’s Primary School .......................................................................................................................... 98 Section I. Supportive Learning Environment in the School ...................................................................................................................... 99 Section ll. Student Dedication to Learning ............................................................................................................................................. 101 Box 5.2: Comparing Student Effort Indicators Internationally ............................................................................................................... 104 Chapter 6 : What Matters for Learning? ..................................................................................................................................... 105 Section I. What Matters for Learning?.................................................................................................................................................... 105 School factors.................................................................................................................................................................................. 105 Teacher factors ............................................................................................................................................................................... 105 Student factors................................................................................................................................................................................ 106 Section II. What factors are associated with improvement in scores? .................................................................................................. 112 Conclusions: What lessons can we draw?................................................................................................................................... 117 References ................................................................................................................................................................................ 118 Appendix................................................................................................................................................................................... 121 Appendix A: Tanzania SDI Sampling Methodology ................................................................................................................................ 121 A1. Sampling frame for the 2016 Tanzania SDI .............................................................................................................................. 121 A2. Sample size and sample allocation for the 2016 Tanzania SDI ................................................................................................ 122 A3. Sampling schools, teachers, and students................................................................................................................................ 123 A4. Weights for schools, teachers, and students ........................................................................................................................... 123 Appendix B: Additional Tables ................................................................................................................................................................ 124 6 List of Tables Table 0.1: Tanzania SDI Sample 2014 and 2016 ....................................................................................................................................... 26 Table 1.1: R-Squared- Variance decomposition of student’s learning scores .......................................................................................... 46 Table 2.1: Profile of the Average Tanzanian Teacher ............................................................................................................................... 54 Table 2.2: Comparison of teachers’ performance on Language by type of task ...................................................................................... 68 Table 2.3: Comparison of teachers’ performance on Mathematics by type of task ................................................................................ 69 Table 2.4: Comparison of teacher’s performance on pedagogical knowledge and skills in the classroom ............................................ 69 Table 3.1: Descriptive Statistics of School Management Practices in Tanzania ....................................................................................... 75 Table 4.1: Descriptive Statistics of Schools – Tanzania 2016 ................................................................................................................... 93 Table 4.2: Types of School in Tanzania ..................................................................................................................................................... 94 Table 5.1: Student Characteristics ........................................................................................................................................................... 98 Table 5.2: Student Effort Indicators Internationally ............................................................................................................................... 104 Table 6.1: Multi-level (Hierarchical Linear MLE) Model Estimation Results 2016 - Test Scores ............................................................ 107 Table 6.2: School Gain Estimation Results (OLS) - Test Scores ............................................................................................................... 113 Table A. 1: 2012 EMIS sample frame by stratum ................................................................................................................................... 121 Table A. 2: Teacher’s absence rate, average, standard errors and design effect SDI 2010 ................................................................... 122 Table A. 3: SDI sample allocation across regions .................................................................................................................................... 123 Table B. 1: Education SDI Survey Instrument ......................................................................................................................................... 124 Table B. 2: Students Learning Performance in Language – Tanzania 2014 & 2016 (Standard 4) .......................................................... 125 Table B. 3: Students Learning Performance in Mathematics – Tanzania 2014 & 2016 (Standard 4) .................................................... 126 Table B. 4: Students Average Test Scores – Tanzania 2014 & 2016 (Standard 4) .................................................................................. 127 Table B. 5: Comparing Tanzania’s Students Internationally – (Standard 4) ........................................................................................... 128 Table B. 6: Teacher’s Use of time – Tanzania 2014 & 2016.................................................................................................................... 129 Table B. 7: Teacher’s Content Knowledge in Language by Type of Task – Tanzania 2014 & 2016 ........................................................ 130 Table B. 8: Teacher’s Content Knowledge in Mathematics by Type of Task – Tanzania 2014 & 2016 .................................................. 131 Table B. 9: Teacher’s Pedagogical Knowledge and Teaching Skills in the Classroom – Tanzania 2016 ................................................. 132 Table B. 10: Teacher Practices by Urban/Rural and Teacher Gender – Tanzania 2016 ......................................................................... 133 Table B. 11: Comparing Tanzania’s Teachers Content Knowledge Internationally ................................................................................ 134 Table B. 12: Minimum Equipment Availability in the Classroom – Tanzania 2014 & 2016 ................................................................... 135 Table B. 13: Minimum Infrastructure Availability in the School – Tanzania 2014 & 2016 ..................................................................... 136 Table B. 14: Comparing Tanzania’s School Inputs Internationally ......................................................................................................... 137 List of Figures Figure 0.1: Student Learning Performance in Language and Mathematics by Student’s Gender – Standard 4...................................... 10 Figure 0.2: Student Learning Performance in Language by Medium of Instruction – Standard 4 ........................................................... 11 Figure 0.3 : Student’s Test Scores by Subject and Region – Standard 4................................................................................................... 12 Figure 0.4 : Comparing Average Student’s Test Scores Internationally by Subject – Standard 4 ............................................................ 13 Figure 0.5 : Average Student Test Scores by Ethnic Group (Mother Tongue) and Gender...................................................................... 14 Figure 0.6: Determinants of Learning ....................................................................................................................................................... 15 Figure 1.1: Student Learning Performance in Language – Standard 4 ..................................................................................................... 34 Figure 1.2: Student Learning Performance in Language by Medium of Instruction – Standard 4 ........................................................... 34 Figure 1.3: Students’ Learning Performance in Mathematics – Standard 4............................................................................................. 36 7 Figure 1.4: Students’ Test Scores – Standard 4 ........................................................................................................................................ 37 Figure 1.5: Distribution of Language and Mathematics Student Test Scores .......................................................................................... 38 Figure 1.6: Student’s Learning Performance in Language by Region and Gender ................................................................................... 38 Figure 1.7: Student’s Learning Performance in Mathematics by Urban/Rural and Gender .................................................................... 39 Figure 1.8: Average Student Test Scores by Ethnic Group (Mother Tongue) and Gender ...................................................................... 40 Figure 1.9: Trends in Overall Average Scores by Gender and Ethnic Group ............................................................................................ 41 Figure 1.10: Trends in Average Student Test Scores by Type of Region (Standard 4) ............................................................................. 42 Figure 1.11: Regional Changes in Student Learning Performance in Mathematics (Standard 4) ............................................................ 43 Figure 1.12: Differences in Learning Outcomes Between and Within Schools – Mathematics Score ..................................................... 44 Figure 1.13: Differences in Learning Outcomes Between and Within Schools – Language Score ........................................................... 45 Figure 1.14 : Comparing Average Student’s Test Scores Internationally by Subject – Standard 4 .......................................................... 47 Figure 2.1: Teacher Effort: What do Tanzanian teachers do? .................................................................................................................. 50 Figure 2.2: Trends in teacher school absence and classroom absence .................................................................................................... 51 Figure 2.3: Teacher school absence and classroom absence by presence of principal in the school ...................................................... 52 Figure 2.4: Trends in time spent teaching per day ................................................................................................................................... 53 Figure 2.5: Distribution of teaching years of experience by position at school ....................................................................................... 55 Figure 2.6: Position at school by gender................................................................................................................................................... 55 Figure 2.7: Trends in Teachers’ Content Knowledge in Language by Type of Task .................................................................................. 57 Figure 2.8: Trends in Teachers’ Content Knowledge in Mathematics by Type of Task ............................................................................ 58 Figure 2.9: Distribution of the teacher test scores in Language and Mathematics ................................................................................. 59 Figure 2.10: Teachers pedagogical knowledge ......................................................................................................................................... 60 Figure 2.11: Distribution of CLASS domain scores .................................................................................................................................... 62 Figure 2.12: Distribution of CLASS Dimension Scores .............................................................................................................................. 63 Figure 2.13 : Comparing Tanzania’s Teachers Use of Time ...................................................................................................................... 67 Figure 2.14 : Comparing Tanzania’s Teachers Content Knowledge Internationally................................................................................. 68 Figure 2.15 : Cross Country Comparison of CLASS scores ........................................................................................................................ 70 Figure 3.1 : Principal’s and Head Teacher’s School Absence Rate by Rural/Urban School ...................................................................... 72 Figure 3.3 : Frequency of supervision visits per school year .................................................................................................................... 73 Figure 3.2 : School Governance and Supervision Visits ............................................................................................................................ 73 Figure 3.4: Distribution of School Management Scores by Type of Practice ........................................................................................... 76 Figure 3.5 : Distribution of School Management Scores by Group of Practices ...................................................................................... 79 Figure 3.6 : Comparing management capacity across sectors in Tanzania – Education and Manufacturing .......................................... 80 Figure 3.7 : Average management score by country and relative to top-performing country ................................................................ 81 Figure 3.8: School management practices within countries, US density overlaid on all graphs.............................................................. 82 Figure 3.9: School management practices compared to manufacturing management practices within countries ................................ 83 Figure 4.1: Trends in Inputs for Learning - Minimum Equipment Availability.......................................................................................... 86 Figure 4.2: Minimum equipment availability in the classroom by urban/rural ....................................................................................... 88 Figure 4.3: Trends in Basic Infrastructure – Minimum Infrastructure Availability ................................................................................... 89 Figure 4.4: Minimum infrastructure availability in the school by urban/rural ......................................................................................... 91 Figure 4.5: Distribution of Student-Teacher Ratios by Urban/Rural – Tanzania 2016 ............................................................................. 93 Figure 4.6 : Comparing Tanzania’s School Inputs Internationally ............................................................................................................ 95 Figure 4.7 : Comparing Tanzania’s Student-Teacher Ratios Internationally – Standard 4 ....................................................................... 96 Figure 5.1 : Student’s Age Distribution for Standard 4 by Urban/Rural ................................................................................................... 98 Figure 5.2 : Student Characteristics – Standard 4..................................................................................................................................... 99 Figure 5.3 : Trends in Supportive Learning Environment in Tanzanian Classrooms .............................................................................. 100 Figure 5.4 : Supportive Learning Environment by Rural and Urban School ........................................................................................... 101 Figure 5.5 : Student Absence Rate and Students Off-Task by Urban/Rural School ............................................................................... 102 8 Figure 5.6: Student Engagement and Behavior Management using CLASS ........................................................................................... 103 List of Boxes Box 0.1: The Link between SABER SD and SDI Approach ......................................................................................................................... 28 Box 0.2: The Service Delivery Indicators (SDI) Program ........................................................................................................................... 29 Box 1.1: Background on the SDI Student Assessment .............................................................................................................................. 32 Box 1.2: Sample of Language Questions .................................................................................................................................................. 33 Box 1.3: Sample of Mathematics Questions ............................................................................................................................................ 35 Box 1.4: Education Quality Improvement Tanzania (EQUIP-T)................................................................................................................ 42 Box 1.5: Comparing Tanzania’s Students Internationally ......................................................................................................................... 47 Box 2.1 : Profile of Tanzania’s Teachers ................................................................................................................................................... 54 Box 2.2: Missing in Action in Tanzania ...................................................................................................................................................... 66 Box 2.3: Comparing Tanzania’s Teachers Internationally......................................................................................................................... 67 Box 3.1: The Development-World Management Survey (D-WMS) .......................................................................................................... 75 Box 3.2: Comparing Tanzania’s School Management Practices Internationally ...................................................................................... 81 Box 4.1: What does a typical school in Tanzania look like? ...................................................................................................................... 93 Box 4.2: Comparing Tanzania’s School Inputs Internationally ................................................................................................................. 95 Box 5.1: Student’s Profile in Tanzania’s Primary School .......................................................................................................................... 98 Box 5.2: Comparing Student Effort Indicators Internationally ............................................................................................................... 104 Unsurprisingly, students scored much higher on the Swahili test compared to the English test. Figure 1.2 shows the student learning performance by language of instruction. For Swahili students, close to nine out of ten students manage the simplest tasks, such as identifying a letter or recognizing a word compared to six out of ten students in English. Swahili test takers also performed better in more complex tasks such as reading a 50-word paragraph or answering factual comprehension questions about the paragraph they read. The picture is completely different for the English test, where one in three (33 percent) students were unable to identify a sample alphabet letter. In addition, when it comes to more complex tasks only one in four (24 percent) could read a 10-word sentence, 6 percent could fluently read a paragraph and 8 percent could answer a question about the meaning of the passage. 9 Executive Summary This report provides a snapshot of the basic education system in Tanzania using a combination of data collected from the Service Delivery Indicators (SDI) Surveys in 2014 and 2016, and data from SABER Service Delivery 2016 (i.e. Development-World Management Survey, Classroom Observation). Using data collected through direct observations, unannounced visits, and tests from primary schools in Tanzania, the report highlights strengthens and weaknesses of the education system and identifies specific bottlenecks that inhibit student learning. One of the main contributions of this report is to provide information on different domains which are crucial for learning but have not been examined previously due to lack of data such as quality of teaching practices and quality of school management. In addition, this report provides guidance to the Government on student, teacher and school level factors associated with learning outcomes, which are intended to help the Government to make informed choices on where to direct resources to further raise learning and reduce inequities in the basic education sector. What do Standard 4 Tanzania’s students know? Tanzania is confronted with a learning crisis in basic education. Even though Tanzania has made significant gains in access and equity in primary education, such as closing the gender gap in enrollment, learning is still low. Based on data from the 2016 Tanzania SDI survey, Standard 4 students have significant knowledge gaps. In the Language test, more than one-quarter of Standard 4 students could not manage the most basic tasks of recognizing a letter in the English/Swahili alphabet or reading a word. Student’s ability to identify a picture from a given word dropped to 26 percent with the trend continuing as less than one fourth of the class could read a sentence. Moreover, two-thirds could not read a paragraph in English or Swahili and three-quarters could not answer a reading comprehension exercise, which involved extracting relevant information from a passage to answer comprehension questions. In terms of gender, there is no evidence of a gender gap in Language (Figure 0.1). Unsurprisingly, students scored much higher on the Swahili test (45%) compared to the English test (8%) (Figure 0.2). Swahili test takers also performed better in more complex tasks such as reading a 50-word paragraph or answering factual comprehension questions about the paragraph they read. The picture is completely different for the English test, where one in three (33 percent) students were unable to identify a sample alphabet letter. In the Mathematics test, just 73 percent could add double-digit numbers, and only 50 percent could subtract double-digit numbers. Both single digit multiplication and division have not solidified yet as just less than half of the students gave correct answers. More difficult tasks such as triple digit multiplication, solving a word problem or completing a sequence seem to be a struggle for Standard 4 students as these tasks were correctly answered by an even smaller proportion of the class (only 15 percent). There is evidence of a gender gap in more difficult tasks such as double- digit division, triple digit multiplication, word problem, completing a sequence etc. Figure 0.1: Student Learning Performance in Language and Mathematics by Student’s Gender – Standard 4 LANGUAGE (English & Swahili) Read a letter 71.7% 72.0% Read a word 73.7% 71.2% Read a 30.8% 31.2% paragraph Reading 19.6% 18.2% comprehension 10 MATHEMATICS Single digit 87.3% 86.0% addition Double digit 75.0% 71.9% addition Double digit 51.5% 48.9%* subtraction Single digit 51.6% 46%*** multiplication Double digit 30.7% 24%*** division Triple digit 17.1% 13%*** multiplication Word problem 17.9% 12%*** Complete a 17.6% 14%** sequence Note: The table presents percentage of Standard 4 students in public schools able to perform various language and mathematics tasks by gender. Each specific test item represents the percentage of students that responded the question correctly. All student statistics are calculated using student-specific sampling weights. Results for student learning performance are based on around 4000 randomly sampled students. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1 Source: Tanzania SDI 2016 Figure 0.2: Student Learning Performance in Language by Medium of Instruction – Standard 4 100% 92% 90% 85% 80% 66% 64% 60% 60% 45% 40% 24% 20% 11% 10% 6% 8% 0% Read a letter Read a word Identify words Read a sentence Read a Reading paragraph Comprehension English Swahili Source: Tanzania SDI 2016 11 How have student’s learning outcomes changed in Tanzania’s primary schools? Learning outcomes have improved across a range of basic literacy and numeracy tasks between 2014 and 2016. Tanzania envisions becoming a middle-income country by 2025 with a high level of human development. 1 In order to accelerate economic growth, the country will need to invest to improve the quality of education. Analyzing trends in learning outcomes from 2014 to 2016, this report also finds evidence of improvements in learning outcomes in primary schools during this period: test scores in English, Math, and Swahili for Standard 4 students improved significantly over time. Nationwide, school average student scores improved across all subjects between 2014 and 2016 by an average of 11 percentage points. The largest increase was observed in English: students at the average school answered 52 percent of questions correctly on the SDI English test in 2016, up from 41 percent in 2014, an increase of more than one-fifth. The smallest improvement was in Mathematics, where school average scores increased by five points from 60 to 65 percent (Figure 0.3). Underpinning the improvements in overall test scores, the proportion of students who could correctly identify a particular letter or word rose from approximately 60 to 72 percent, those who could correctly read a paragraph from 25 to 30 percent and those who could answer a question about the meaning of a passage from 13 to 19 percent. In terms of numeracy, the share of students able to add double-digit numbers rose from 66 to 73 percent and those able to subtract double-digit numbers, from 44 to 50 percent. Improvement also extended to more difficult tasks: the share of students able to multiply triple-digit numbers rose from 12 to 15 percent, those able to single digit increased from 44 to 46 percent and those able to divide double digits, from 23 to 27 percent. Figure 0.3 : Student’s Test Scores by Subject and Region – Standard 4 100% 100% 91% 84% 80% 80% 74% 76% 71% 65% 65% 65% 60% 61% 60% 58% 60% 52% 53% 54% 60% 52% 41% 43% 40% 40% 20% 20% 0% 0% English Kiswahili Math NVR Overall EQUIP-T Rural Other Urban Dar es Salaam 2014 2016 2014 2016 Note: In each school, the SDI selected ten random Standard 4 students for testing; all were tested in Math and either English (approximately 70 percent of sampled pupils) or Swahili (approximately 30 percent) and Non-Verbal Reasoning. The figure presents average scores on the Language, Mathematics and Non-Verbal Reasoning tests of Standard 4 students in public schools. All student statistics are calculated using student-specific sampling weights. Results for student learning performance are based on around 4000 randomly sampled students. Source: Tanzania SDI 2014 and 2016 Tanzania’s students are among the top performers among the Sub Saharan African countries for which we have data (SDI countries). In Language, they significantly outperformed students in Mozambique, Uganda, Togo and Nigeria, but could not reach the results observed in Kenya. Students in Tanzania scored 62 percent on the Language test, which is more than 10 percentage points higher than the average SDI, but 13 points lower than Kenya. However, in Mathematics, Tanzania’s students in 2016 are the best performers among these countries. They scored almost 20 percentage points higher than the average SDI country on the Mathematics test (Figure 0.4). 1 The Tanzania Development Vision 2025, Ministry of Finance and Planning. It is envisioned that Tanzanians will have graduated from a least developed country to a middle-income country by the year 2025. The five main attributes of this human development goal are: high quality livelihood; peace, stability and unity, good governance; a well educated and learning society; and a competitive economy capable of producing sustainable growth and shared benefits. 12 Figure 0.4 : Comparing Average Student’s Test Scores Internationally by Subject – Standard 4 Language Student Test Score Mathematics Student Test Score Mozambique 2014 19 Mozambique 2014 25 Nigeria 2013** 31 Nigeria 2013** 32 Togo 2013 46 Uganda 2013 43 Uganda 2013 47 Togo 2013 45 Average SDI 50 Average SDI 47 Tanzania 2014 62 Kenya 2012 59 Tanzania 2016 71 Tanzania 2014 60 Kenya 2012 75 Tanzania 2016 65 0 20 40 60 80 0 10 20 30 40 50 60 70 Note: These figures present the percentage of Standard 4 students in public schools that responded correctly all the questions in the Language and Mathematics. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013, Kenya SDI 2012, ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. How does the learning performance vary by region, ethnic group and gender? Learning outcomes have improved all across Tanzania, with largest gains registered in disadvantaged targeted districts (EQUIP-T), followed by rural areas (Figure 0.3). 2 In EQUIP-T regions, the improvement in test scores are substantial, with overall scores increasing 18 percentage points from 43 to 61 percent, an increase of more than one-third. These regions also had the largest improvements in the share of students able to complete most basic tasks. Despite the considerable catching-up by rural schools, urban schools remain in the lead in terms of learning in both Mathematics and Language. Schools in urban areas obtained on average 74 percent in overall learning scores, versus 62 percent in rural schools. In rural areas, which had the next lowest performance in 2014, overall scores increased from 52 percent in 2014 to 60 percent in 2016. In urban areas outside of Dar es Salaam, the rate of increase was slower, but overall scores still rose from 65 percent in 2014 to 71 percent in 2016, only half the rate in rural schools. However, in Dar es Salaam, where overall performance was highest in 2014, overall scores improved only slightly, from 74 to 76 percent Looking at the disaggregation by gender and ethnic group (mother tongue), in general girls and non-Swahili speakers face persistent disadvantages, posing threats to the long-term equity of the system. In Language, boys tend to perform better than girls, but this difference in not statistically significant. However, in Mathematics boys significantly outperformed girls, especially in tasks with higher level of difficulty. Swahili is the language of instruction in Tanzanian primary schools, and the primary national lingua franca, but only around ten percent of Tanzanians speak Swahili as their primary home language or ‘mother tongue’ (Petzell, 2012). Both non-Swahili native speaking students and girls achieve scores consistently below Swahili speaking students and boys. 3 These barriers have not reduced as learning outcomes have risen. With half of the 2 Education Quality Improvement Programme in Tanzania (EQUIP-T) is a DFID funded program and implemented in cooperation with the Government in Tanzania in support of better learning outcomes at basic education level, especially girls. It aims to develop, implement and demonstrate the best approaches to strengthen the quality of education in seven regions of Tanzania. It began in 2014 in Dodoma, Kigoma, Shinyanga, Simiyu and Tabora; in 2015 the program expanded to Lindi and Mara. Currently, it operates in 9 regions of Tanzania with the objective of achieving comprehensive school improvements for improved learning outcomes. 3 Among factors undermining the performance of girls in school are: (1) more household demands on the time of girls compared to boys, leaving them with less time for schoolwork (see Blackden and Wodon 2006 on time use) (2) poor-quality conditions for managing hygiene and lack of school water and sanitation facilities for girls, leading them to miss more school days; (3) lower teacher expectations of girls, reducing the attention they receive from teachers; (4) a higher propensity for teacher to assign girls to chores like fetching water and cleaning classrooms, which reinforces gender stereotypes as well as taking time away from learning (UNICEF, 2003). 13 students being female and also with half of the students being non-Swahili native speakers, these disparities are likely to restrict the overall improvement in learning outcomes over time (Figure 0.5). Figure 0.5 : Average Student Test Scores by Ethnic Group (Mother Tongue) and Gender 100% 94% 100% 87% 90%93% 80% 80% 67% 67% 67%64% 66%63% 62% 59%56% 61% 60% 56% 60% 54% 53%50% 48% 51% 40% 40% 20% 20% 0% 0% English Swahili Mathematics Non-verbal Overall English Swahili Mathematics Non-verbal Overall reasoning reasoning Swahili Non-Swahili boys girls Note: In each school, the SDI selected ten random Standard 4 students for testing; all were tested in Math and either English (approximately 70 percent of sampled pupils) or Swahili (approximately 30 percent) and Non-Verbal Reasoning. The figure presents average scores on the Language, Mathematics and Non-Verbal Reasoning tests of Standard 4 students in public schools by mother tongue and gender. All student statistics are calculated using student-specific sampling weights. Results for student learning performance are based on around 4000 randomly sampled students. Source: Tanzania 2016 14 Figure 0.6: Determinants of Learning Source: Tanzania SDI/SABER SD 2016 School absence rate: 13.6% Student had breakfast: 33.3% Classroom absence rate: 41.9% Had breakfast with protein: 9% Language Test: 37.4% Student Engagement Classroom Absence Math Test: 64.7%Rate CLASS score: 4/7 Classroom Absence Rate 41% 41% CLASS Instructional Support: 3/7 Student absence rate: 16% CLASS Classroom Organization: 6/7 Students off-task: 3% CLASS Emotional Support: 3/7 Inefficient School Governance Classroom inputs: 75% and Accountability Basic Infrastructure: 41% Unstructured or Very Weak Management Classroom Practices Absence Rate Students with Classroom textbook: Absence 19% Rate 41% 41% D-WMS Management Score: 2/5 Informal practices & reactive approach Student-teacher ratio: 47:1 to managing the organization 15 Why is Tanzania’s Education System Not Aligned with Learning? The World Development Report 2018 provides a theoretical lens to examine why an education system may not be aligned with learning. In particular, ineffective education systems often lack one or more of the four key school-level determinants of improved learning: skilled and motivated teachers, effective school management, school inputs that affect teaching and learning, and prepared and supported students. Teachers While teachers in Tanzania have relatively low school absence rates, less than half were found in the classroom, where they spend only half of the time on learning activities. They also struggle to help student learn as they have relatively low content and pedagogical knowledge, as well as weak teaching skills. What do teachers do? Tanzanian teachers displayed patterns with relatively low school absence rate (13 percent), but high classroom absence rate with almost half of the teachers (42 percent) not found in classroom, either because they were absent from school or in the school but not in the classroom. While in the classroom, teachers spent only half of the time (56 percent) on learning activities. Factoring in teacher absence from school and class as well as the percentage of the lesson that the teacher devotes to non- teaching activities, students are taught, on average, 3 hours and 55 minutes per day out of 7 hours of scheduled teaching time. There are no major differences in learning time across urban/rural schools. Students in rural schools seem to receive slightly more time learning compared to their urban peers but this difference is not statistically significant. Between 2014 and 2016 Tanzania was able to reduce classroom absence from 47% to 42%. Similarly, the proportion of teachers found to be in class teaching increased from 48% to 51%, but the challenge of improving teacher’s presence in the classroom still remains. Comparing to other countries, Tanzanian teachers display a relatively high classroom absence rate, but Tanzanian students received more teaching than the average Sub-Saharan African country, where comparable data was collected. This includes Kenya, Mozambique, Togo, Senegal, Uganda, and 4 states in Nigeria. What do teachers know? The performance on the Language test was quite low, with few teachers mastering the content knowledge in Language (37 percent). 4 Teachers were asked to mark (or “grade”) mock student tests in Language and in Mathematics. In Language, the exercise assessed their ability to spell simple words (“traffic,” for example) and identify the correct grammatical option that completes a sentence such as “[Does, Where, How long] does it take to walk to this school?”. On average, language teachers correctly answered these questions 69 percent of the time. The test also included cloze passages, which consisted of a short story with certain words removed and the teacher had to fill the gaps in a meaningful way, which included “student” responses such as “[Where] do I have to go to the market?” (In this case, a correct answer could be either “Why or When.”). Teachers got 48 percent of the points in this exercise. They were also asked to correct spelling, grammar, syntax, and punctuation mistakes in a student’s letter that included segments such as “I went to tell you that my new school is better the oldone I have a lot of thing to tell you about my new school in Dar es Salaam.”. Teachers struggled with this task and got only 17 percent of the points. The performance on the Mathematics test was better than the one in Language, with more than half of teachers mastering the content knowledge in Mathematics (65 percent). In Mathematics, we tested if the teacher could accurately correct children’s work in such aspects of numeracy as manipulating numbers using whole number operations. Looking at specific tasks, Tanzanian teachers were much more at ease with 4 The objective of the teacher test was to examine the basic reading and writing skills of the language teachers and also the arithmetic skills of mathematics teachers that lower primary students need to have to progress further in their education. In Language, we assessed teacher’s content knowledge to correct students’s work in such aspects of literacy as reading comprehension, vocabulary, and formal correctness. Similarly, in Mathematics, we measured teacher’s ability to correct student’s work in various aspects of numeracy as manipulating numbers and using whole number operations. 16 simple operations such as double-digit addition or double-digit subtraction, than with slightly more complex computations such as interpreting a graph or a Venn diagram. For instance, one quarter of teachers could not interpret information in a Venn diagram, half could not solve an algebra exercise, and more than half could not interpret data from a graph. In terms of regional gap, urban teachers are statistically indistinguishable from rural teachers in terms of their subject knowledge in both Mathematics and Language. When comparing the 2016 to the 2014 SDI results, we find that Tanzanian teacher’s subject knowledge in Language have slightly deteriorated. However, teacher’s subject knowledge in Mathematics seems to have improved rapidly between 2014 and 2016. For instance, scores on exercises assessing skills in interpretation of graphs and Venn diagrams increased from 28 percent to 41 percent and from 49 percent to 69 percent, respectively. The share of teachers who could correctly solve a simple algebraic equation also increased from 51 percent to 55 percent. Comparing Tanzania’s teachers internationally, we find that they ranked almost last on Language among the Sub Saharan African countries, with only Mozambique performing lower. Nevertheless, in terms of content knowledge in Mathematics, Tanzanian teachers performed significantly better than the average teacher in Sub Sahara Africa and they were second only to Kenyans. How well do teachers teach? In general, Tanzanian teachers display weak pedagogical skills. Few teachers (less than 1 in 3) deploy the teaching practices identified in the literature as promoting learning in their lessons —structuring, planning, asking lower and higher order questions and giving feedback. To assess how well teachers teach we measure: (i) teachers’ pedagogical knowledge, (ii) teachers’ capacity to assess students and monitor their progress, and (iii) quality of teaching practices based on direct lesson observation. First, to measure pedagogical knowledge, we asked teachers to prepare a lesson plan by reading and extracting information from a factual text on a topic and to state what they would expect their students to learn from the lesson. While teachers struggled to read and understand the text (average score of 35 percent), they struggled even further to formulate what they wanted children to learn from the lesson based on their reading (average score of 17.6 percent). These results suggest that most Tanzanian teacher lack basic pedagogical skills. Second, to measure teachers’ ability to assess students’ learning and give feedback, teachers were asked to: (i) prepare lower- and higher-order questions, (ii) use a marking scheme to give feedback on strengths and weaknesses in students’ writing, and (iii) use a list of students’ grades to turn the raw scores into averages and comment on the students’ learning progression. As with general pedagogical knowledge, the results show that few teachers demonstrated an ability to assess student learning and respond to that assessment. Around 1 out of 3 teachers could formulate questions that checked basic understanding based on what they had read, and another 18 percent could formulate a question that asked students to apply what they had learned to other contexts. Just 1 out of 5 teachers could give feedback on strengths and weaknesses in students’ writing using a marking scheme. Furthermore, almost 1 out of 3 teachers (28 percent) could monitor and comment on the learning progression of students. Lastly, poor knowledge of general pedagogy was also mirrored in teaching behavior in the classroom. Inside the classroom, many teachers deployed some of the teaching practices identified in the literature as promoting learning, but few (less than 1 in 3) applied the full set of beneficial skills in their lessons. A subset of 112 Standard 4 classrooms were additionally observed using the Classroom Assessment Scoring System (CLASS). Tanzanian teachers scored relatively high in Classroom Organization (5.7/7), but they struggled between low and medium scores in Instructional (2.8/7) and Emotional support (2.8/7). Despite the poor skills in pedagogical knowledge, the 2016 SDI data suggest that Tanzanian teachers have slightly improved their teaching skills since 2014. For instance, the proportion of teachers who were able to read and extract information from a factual text increased from 26 percent to 35 percent and the proportion of those that could formulate what they would expect their students to learn also increased from 13 to 18 percent. Similarly, teacher’s ability to assess students improved in terms of checking for understanding, formulating questions to apply to other contexts and evaluating student’s progress. Comparing results internationally, we find that Tanzanian teachers fall behind the average teacher in SSA both in text comprehension and formulating aims and learning outcomes, but they overperformed the average teacher in SSA in terms of assessing students and teaching skills. With respect to their East African neighbors, they outperformed Ugandans and Mozambicans, but they underperformed Kenyans. 17 School Management Although Tanzanian principals or head teachers have relatively low absence rates, they run an establishment with practically no structured management practices or very weak school management practices. In addition, very few schools (1 out of 3) receive supervision visits only once or twice a year.5 Tanzania has the lowest management index among comparable countries of the World Management Survey (WMS) and D – World Management Survey (D-WMS). What characterizes school governance and accountability in Tanzanian primary schools? Even though school principals and head teachers displayed a pattern of relatively low absence rates, school governance and accountability mechanisms are very inefficient. In Tanzania, the principal absence rate is observed to be around 13.6 percent, with rural schools having significantly higher principal absence rate (14.3 percent) than urban schools (7.7 percent). Despite almost all Tanzanian schools (97 percent) have a school management committee or board of directors, the lack of autonomy and capacity prevents school management committees from improving service delivery. Ineffective school leadership can translate into lower accountability for teachers and an ineffective use of resources. One third of schools did not receive any supervision visit from an official quality assurance officer or a school inspector during the last academic year, which is very disconcerting in terms of performance monitoring. Urban schools received on average 2 supervision visits, while rural schools received only 1 supervision visit and this difference is statistically significant. Within the group of schools that received at least one supervision visit, only 68 percent received a written feedback. From all the recommendations received from the District Education Office or Ministry of Education, less than half of the feedback received (37 percent) was actually observed from the school staff in written feedback. Lastly, in terms of methods puts in place to recognize teacher performance (e.g. prize, award), only 60 percent of schools have at least one method to recognize teacher’s performance. What school management practices do Tanzanian principals follow? Tanzanian schools scored, on average, 1.8 in the five-point scale of the management index of the D-WMS, indicating an establishment with practically no structured management practices or very poor management practices implemented. Using the 2016 Tanzania D-WMS we investigated the adoption of 14 basic management practices, where the level of adoption is evaluated against a scoring grid that ranges from one (worst practice) to five (best practice). A high score indicates that a school adopts structured and well-defined strong management practices. The main measure of management practices (the management index) represents the average of the scores across all questions. When we analyze the distribution of the school management practices, we find that the management scores in Tanzania are shifted completely to the lower end of the distribution, with the vast majority of scores below 2 and no school scores above 3, suggesting that principals or head teachers in primary schools lack autonomy and authority in decision-making, have very weak management practices, almost no monitoring, very weak targets (e.g. only an annual school-level target) and extremely weak incentives (e.g. tenure based promotion, no financial or non-financial incentives and no action taken about underperforming teachers). Among the participating countries in the WMS and DWMS, Tanzania had the lowest management index and management capacity is substantially lower in schools than in manufacturing firms. 5 It is important to mention that the information provided by the SDI is very limited since this survey was not intended to capture information on school management, school governance or community involvement. For this reason, we use the 2016 Tanzania Development-World Management Survey (DWMS), which ran face-to-face interviews between enumerators and principal/head teachers in a random sample of 100 public schools. 18 School Inputs Significant gaps exist in the availability of inputs and basic infrastructure at the frontline of public schools in Tanzania. Tanzania has made significant strides in improving primary education enrollment in the past 15 years. In 2001, 4.8 million Tanzanian students were enrolled in primary school. 6 In 2016, there has been a two-fold increase in enrollment with more than 8 million students enrolled in 17,174 schools. 7 Despite the progress in access to education, this expansion has not kept pace with the availability of school inputs and infrastructure in Tanzania. What classroom inputs are available in Tanzanian schools? In terms of availability of teaching resources, three quarters (75 percent) of Tanzanian primary schools possess the minimum equipment required, which measures the availability of: (i) functioning blackboard and chalk and (ii) pens, pencils and exercise books (paper) in the Standard 4 classrooms. In particular, in more than 90 percent of schools, students have pen, pencil and exercise book, while 82 percent of the schools have a functioning blackboard. The main issue in classroom inputs is the lack of light for students to be able to read the blackboard. Schools in the EQUIP-T regions and rural schools have significantly less minimum equipment availability compared to urban schools. Most striking was textbook availability for students. Despite recent nationwide distribution of textbooks, only 1 out of 5 students (19 percent) had access to an English, Swahili or mathematics textbook in a typical Standard 4 classroom. Students in urban schools are significantly more likely to have a textbook in the classroom compared to their counterparts in rural school. Basic classroom furniture appears less of a constraint in Tanzania, since almost all schools (95 percent) have proper seats and desks for their students. Minimum equipment availability in Tanzania is comparable to schools in Kenya. In contrast, they fared much better than Nigeria (48 percent), Laos (47 percent), Afghanistan (36 percent) and Togo (24 percent). Tanzania did very poorly with textbook availability, only outperforming Uganda with the average Standard 4 Tanzanian student being more than three times more likely to use textbook in the classroom than her Ugandan peer. Analyzing trends in classroom inputs for learning from 2014 to 2016, we find that minimum equipment availability has increased, while availability of textbooks for students has decreased. On the one hand, the share of students with pens, pencils and exercise book has increased from 84 to 92 percent and similarly the proportion of schools with functioning blackboard has increased significantly from 74 to 83 percent. On the other hand, the proportion of students with a textbook declined, from 25 percent in 2014 to 18.8 percent in 2016, indicating that the system resources have not kept pace with enrollment increases. However, in urban schools outside of Dar es Salaam, where the share of students with a textbook was lowest in 2014, the share increased by four points from 18 percent of students to 22 percent, suggesting that textbooks are being relatively well targeted to the areas of greatest need. What basic infrastructure is available in Tanzanian schools? Tanzanian schools scored poorly on minimum infrastructure availability, which measures the availability of: (i) functioning toilets and (ii) classroom visibility. Only 41 percent of schools could meet the standard. The main constraint in infrastructure was the availability of functioning toilets given that fewer than half of schools (45 percent) had toilets meeting the standard (accessible, clean and private). The reason behind this result is generally due to the fact that in many of the schools, teachers do not have separate toilets and need to share toilets with students, especially in EQUIP-T regions and rural schools. Classroom visibility is less problematic compared to functioning toilets, but it is still an issue as students in 1 out of 5 schools struggled to read the blackboard from the back of the classroom. Drinking water seems to be a national challenge in Tanzania as half of public schools do not have access to clean drinking water, especially in rural areas. Working electricity is another input that almost all schools in Tanzania lack. Specifically, 95 percent of schools in Tanzania do not have available electricity in the classroom. Urban schools are more likely to have working electricity compared to rural schools. Most schools (88 percent) in Tanzania are accessible by road. The rest of the schools are accessible by foot path only. When compared to other countries, Tanzania is located in the medium level of the spectrum in terms 6 The World Bank 2016. The World Development Indicators. Available at: https://data.worldbank.org/indicator/SE.PRM.ENRL?locations=TZ 7 Tanzania in Figures 2016 19 minimum infrastructure availability. Tanzanian schools have worse infrastructure compared to schools in its East African Community (EAC) neighbors (i.e. Uganda and Kenya) and Laos, but better than Afghanistan, Mozambique, Togo and Nigeria. Analyzing trends in basic infrastructure for learning from 2014 to 2016, we find that there was no noticeable improvement in Tanzanian primary schools in terms of minimum infrastructure availability. Even though, minimum infrastructure availability has increased from 40 to 43 percent during this period, this increase is not statistically significant. Classroom visibility seems to have improved slightly from 76 to 88 percent. However, the most concerning result is that the presence of functioning toilets, which has remained in the same level of approximately 46 percent. When we disaggregate this result by urban/rural, we find that the gap has increased by almost 10 percentage points during this period (from 27 to 39 percent), making this a much more severe problem in rural schools. The results indicate that this gap is mostly driven by privacy of the toilet. Working electricity and access to drinking water still remain as main constraints, especially in rural areas. The observed student-teacher ratio averaged 47 students per teachers, slightly above the expected norm of 45:1. Staffing has not kept pace with growing student numbers, as the observed student-teacher ratio has increased from 43 in 2014 to 47 in 2016. Rising student-teacher-ratios pose risks to continued learning improvements, particularly as the Government is preparing for rapid expansion in enrolments in the wake of the Fee-Free Basic Education Policy. Tanzania’s primary level observed student-teacher ratio is one of the highest amongst the SDI countries and is also above the three SABER SD pilots (Afghanistan, Laos and Pakistan). Tanzania’s observed student-teacher ratio is higher than the neighboring countries: Mozambique and Kenya, but it is below Uganda’s student teacher-ratio, which is around 54:1. The deterioration of school staffing predates a recruitment freeze in 2017, which led to a reduction on the overall number of primary school teachers; it is therefore likely that future SDI rounds will reveal a continued deterioration in staffing. 20 Student Support Tanzanian students do not have the necessary support from their home and from the school to allow them to fully be prepared for learning. Despite the severe lack of an enabling environment, these students continue to show resilience and seem motivated about going to school, as observed by their attendance and low time off-task. A supportive environment can be instrumental in allowing students to focus on their responsibilities at school and be more dedicated towards learning. A stable, enabling environment at home, educated surroundings, teachers who motivate their students among others, all ensure that students are able to focus more on learning and less on the obstacles in the way of learning (World Bank, 2018). 8 Do Tanzanian students receive the necessary support from school? Tanzanian students appeared to have important socio-economic impediments. Two thirds of Tanzanian students did not have any breakfast before class and almost none of them had breakfast with proteins. When we analyze the results over time, we find that the proportion of students with breakfast has decreased from 37 percent in 2014 to 33 percent in 2016. Child nutrition is a challenge in Tanzania, where an estimated 2.7 million children under five are estimated to be suffering from malnutrition-related stunting (UNICEF, 2018). In response, the last years have seen a large increase in the provision of official and unofficial school feeding programs. In a range of studies from developed and developing countries, malnutrition and hunger are associated with significant negative impacts on learning, and students who miss breakfast are slower to respond to test questions and make more errors. Do Tanzanian students receive a supportive learning environment from at school? Tanzanian teachers show moderate support provided by teachers to students in the classroom. In particular, Tanzanian classrooms display an effective supportive learning environment in only less than one third of the classrooms (29 percent). Four out of five teachers called students by their names (especially female teachers), offered positive reinforcement (complimented students when answering questions correctly giving them positive feedback and/or encouragement), and corrected students mistakes. More than half (60 percent) smiled and played with students (especially male teachers) and visited them individually. In only 40 percent of classes the teacher invited students to write on the blackboard and gave them homework, but even fewer times (24 percent) did the teacher return corrected homework or take student’s work to correct. Lastly, around one third of teachers gave negative feedback that is scolding at a mistake and relatively very few teachers (6 percent) teachers hit or scolded students during the lesson. Are Tanzanian students engaged with learning in the classroom? While students in Tanzania exert high levels of effort as evidenced by their relatively low absence rate and low time off task (especially in urban schools), they only spend half of the time (46 percent) in school engaged in learning activities and they are just somewhat engaged in the classroom suggesting a moderate level in student dedication to learning. More specifically, Tanzanian schools present a student absence rate of around 16 percent, ranging from 8 percent in urban schools to 17 percent in rural schools. When present in class, only 3 percent of students were found not paying attention in class indicating a relatively low share of student’s off-task. In contrast, Tanzanian students spend about 3 hours 15 minutes of their time in school engaged in learning activities each day, which is equivalent to almost half of their time. In general, they behave well but they display a mix of student engagement with some students actively engaged, while others appeared distracted or disengaged. 8 It is important to point out that the Tanzania SDI/SABER SD could not cover the cost of a full survey of student household conditions and therefore the data for this section are sparser and the discussion more tentative. 21 What factors are associated with improvements in learning? Despite the improvement in the learning outcomes and the progress in terms of access to education during the past ten years, there is still a learning crisis in Tanzania. These poor learning outcomes reflect a somewhat weak provision in service delivery, especially in terms of (i) teaching practices, (ii) school management, and (iii) school inputs. First, the provision of education in many low-income countries, including Tanzania, is characterized by a combination of centralized, but typically weak, state control and low-capacity, locally governed institutions. It is easy to see how a vicious circle is created: today’s teachers have gone through an ineffective education system that does not adequately prepare them, through a teacher training system with low entry requirements that does not compensate for the flaws in the education system, or through no training at all, to be sent to a school where they struggle to teach the next generation of students. The institutional incentives for high teacher performance are largely missing, with both career progression and financial rewards delinked from performance. Second, the various state and local authorities provide limited technical support or supervision. While teachers have autonomy to choose what and how to teach, they do not receive support either in terms of materials and good practices or in terms of good professional development or coaching by the school principal or experienced teachers. Indeed, among the participating countries in the WMS and DWMS, Tanzania displayed the lowest management index, indicating an establishment with practically no structured management practices or very poor management practices implemented. Third, staffing has not kept pace with growing student numbers, as the observed student-teacher ratio has increased from 43 in 2014 to 47 in 2016 locating Tanzania below many comparable countries. Lastly, teaching and learning materials in school are limited. The proportion of students with a textbook declined, from 25 percent in 2014 to 18.8 percent in 2016, indicating that the system resources have not kept pace with enrollment increases. Given that the SDI employs the same sample of 400 schools in both 2014 and 2016, it is possible to examine which changes within particular schools were associated with improvements in student performance. For this purpose, we estimate the correlation of an increase or improvement in a range of school characteristics, as well as school average teacher and student characteristics, with the likelihood that a school achieved an increase in learning outcomes. Even though these results cannot be interpreted as causal, they provide some insight on the factors that may be correlated with learning outcomes. Improvements in test scores appear to be associated with improvements in teacher subject knowledge and pedagogical skills. In the analysis of the 2016 data, we found that schools with higher teacher content knowledge, and teachers who assign homework, are associated with higher learning outcomes. We find, too, that schools where these factors improved are associated with more learning gains. For instance, a ten percentage-point improvement in school average teacher scores in English knowledge is associated with a five percent increase in the likelihood that student English scores improved; for Math, a ten-point improvement in school average teacher content knowledge is associated with a two percent improvement in the likelihood of an increase in student scores. This result is consistent with previous research from low- and middle- income countries, which has found that students who are exposed to teachers with higher content knowledge learn substantially more (Metzler & Woessmann, 2012; Bietenbeck, et al., 2018; Bold, et al., 2018). Similarly, pedagogical skills, captured by the assignment and collection of homework, is also associated with higher learning outcomes. These findings suggest that investing in teacher motivation and knowledge are valuable priorities to improve learning outcomes. Schools where the share of teachers who assigned and collected homework increased by ten percent between 2014 and 2016 are associated with one percent increase in experiencing an improvement in overall Math scores. In sum, the 2014 and 2016 SDI surveys show a clear pattern of improving learning levels in Tanzania’s primary schools. However, despite these improvements in learning outcomes, the challenge of improving the quality of basic education is still immense. From the analysis of this study, the data seem to suggest that this can be achieved by making complementary interventions where teacher absence is reduced, teacher content knowledge and teaching practices are improved, and also where investments in school inputs (e.g. textbooks) and infrastructure are increased. In order to maintain these kinds of improvements in learning outcomes, the country needs to consolidate and accelerate these positive changes in the education system. 22 Introduction Tanzania is one of Africa’s fastest growing economies, with nearly 7 percent annual GDP growth since 2000. Yet widespread poverty persists - nearly half of Tanzania’s population is living on under $1.90 per day. 9 Income disparity is more pronounced in rural areas, where economic growth has been hardly perceptible. 10 Inclusive growth is further hindered by population growth and low gains in productivity for labor-intensive sectors like agriculture, which employs more than half of the population.11 Youth under age 15 make up 45 percent of Tanzania’s population of 57 million, and with an annual growth rate of 3 percent, the population is projected to reach 70 million by 2025. 12 With high rates of population growth, per capita income – US$900 in 2016 – is only growing slowly and while many families have recently escaped poverty they still remain vulnerable. Women and youth are key to Tanzania's continued development but are among the most marginalized citizens. Women's salaries, for example, average 63 percent lower than those paid to men, while female-owned businesses make 2.4 times less profit. 13 Tanzania's natural resources are an asset to the country, providing the basis for livelihoods—but unsustainable use of these resources threatens to perpetuate the cycle of poverty. Climate change, urban congestion, weak governance and rapid population growth also present obstacles to faster economic development. Education is a key component of the Government of Tanzania’s development agenda. The country has made significant gains in access and equity in primary education, with girl’s enrollment close to parity with boy’s at all primary education levels. 14 Between 2015/16 and 2017/18, the number of children enrolled in primary education increased from 8,639,202 to 10,111,671, and net enrollment rate rose from 84 percent to 91.1 percent. Despite these successes, many challenges persist related to out-of-school children, retention, completion, and transition to secondary education, as well as quality of education, actual learning outcomes, and the relevance of skills that graduates bring to the economy. For instance, about 1 in 4 girls (25 percent) age 6 and older have no formal education, compared with about 1 in 5 boys (19 percent). 15 Many children enrolled in school drop out before completing primary education, especially girls. 16 At higher levels of the education system, the situation is even worse: the net enrollment rate for lower secondary education is 33.4 percent, and for upper secondary education only 3.2 percent. 17 While primary school enrollment among girls and boys is nearly equivalent, only one in three girls who start secondary school will finish their lower secondary education. Causes of low secondary enrollment and retention among girls include: economic hardship; early marriage and/or teen pregnancy; and school related gender-based violence. Tanzania needs to improve its levels of human capital if it is to accelerate economic growth and achieve its goal of attaining middle income status by 2025. Primary schools in Tanzania have struggled in the past to impart language and numeracy skills needed. In 2014, according to the Service Delivery Indicators (SDI) Survey, only 61 percent of Standard 4 students could correctly identify a stated letter in the English/Swahili alphabet, and only 25 percent could read a paragraph in either English or Swahili. Only 64 percent could add double-digit numbers, and only 42 percent could subtract double-digit numbers (Service Delivery Indicators, 2014). The results placed Tanzania below comparator countries including Uganda, Cameroon and Rwanda in international learning comparisons (Bashir, et al., 2018). The poor learning outcomes reflected a crisis in service provision, in particular in teacher effort: 46 percent of teachers were found to be outside the classroom during allotted teaching time, and more than one-third of classrooms were ‘orphan classrooms,’ with students inside but no teacher present. These failures in primary school service provision pose a threat to the readiness of students for lower secondary. The aim of this report is to systematically analyze and document emerging trends in the evolution of students’ learning outcomes in Tanzania’s primary schools and in the quality of service delivery using a combination of data collected from the Service Delivery Indicators (SDI) Surveys in 9 World Bank. 2017. Data: Sub-Saharan Africa (from the Tanzania Overview). Available at http://data.worldbank.org/country/tanzania; Tanzania National Bureau of Statistics, 2017. 10 Tanzania’s Long-Term Perspective Plan recognizes the average growth rate did not have a significant impact on poverty reduction. 11 Women age 15-49 are most commonly employed in agriculture (56 percent) and unskilled manual labor (22 percent). Men are most commonly employed in agriculture (59 percent) and skilled manual labor (18 percent). Tanzania Demographic and Health Survey, 2015-16 12 Tanzania in Figures, 2017, NBS; projection figure from Tanzania’s Population Planning Commission, 2007. 13 World Bank, 2012 14 In Tanzania, the academic year begins in January and ends in November, and the official primary school entrance age is 7. The system is structured so that the primary school cycle lasts 7 years, lower secondary lasts 4 years, and upper secondary lasts 2 years. Tanzania has a total of 10,465,000 students enrolled in primary and secondary education. Of these students, about 8,639,000 (80%) are enrolled in primary education (Tanzania in Figures, 2016) 15 Tanzania Demographic and Health Survey, 2015-16 16 One in three students have completed primary school (32 percent), 7-8 percent of females and males have completed secondary education, and only 1-2 percent of females and males have completed beyond secondary education. The median number of years of schooling completed among females is 4.5 years and 5.1 years among males. (Tanzania Demographic and Heath Survey, 2015-16). 17 USAID (2018) “Tanzania Education Fact Sheet” 23 2010, 2014, and 2016 and data from SABER Service Delivery 2016 (i.e. Development-World Management Survey, classroom observation, and student assessment survey data). 18 Using data collected through direct observations, unannounced visits, and tests from primary schools in Tanzania, the report highlights strengthens and weaknesses of the education system. In addition, this report provides guidance to the Government on student, teacher and school level factors associated with learning outcomes and key observable factors associated with highest gains in test scores, which are intended to help the Government to make informed choices on where to direct resources to further raise learning and reduce inequities in the basic education sector. The SDI/SABER SD surveys collected a wide range of data on school conditions (inputs and infrastructure), teacher knowledge, teaching practices, and school management as well as carried out learning assessments in English, Math, and Swahili with Standard 4 students. Both the 2014 and 2016 SDI surveys employed the same questionnaires and indicators, and were carried out in the same nationally representative sample of 400 public primary schools, facilitating analysis of the change in service standards and learning outcomes. 19 The sample is also representative of key strata, including schools in Dar es Salaam; urban schools; rural schools; and schools in regions implementing Education Quality Improvement Program in Tanzania (EQUIP-T), a targeted reform program operating in disadvantaged regions. The SDI/SABER SD survey was implemented by Research in Poverty Alleviation (REPOA) in close coordination with the World Bank during the spring/summer of 2010, May and September of 2014, and July and December of 2016. The structure of this report follows closely the framework of the World Development Report (2018) in order to understand whether the Tanzanian education system is aligned with learning. The World Development Report (2018) builds on a vast body of literature on measuring the performance of schools and provides a solid structure for examining how well education systems are aligned with learning. Its starting point is a wealth of evidence showing that schooling is not the same as learning: in many systems, children are in school but learning very little. The World Development Report (2018) finds that struggling education systems often lack one or more of the following four key school-level determinants to improve learning: quality teaching, school management, availability of school inputs for learning, and prepared learners. These four determinants provide a snapshot of the learning environment and key resources which need to be in place for students to learn. As expenditure on teachers represents by far the largest share of education spending in developing countries, and as several recent studies convincingly demonstrate how changes in teacher behavior can improve learning achievement, a strong focus is placed on the knowledge, skills, and effort of teachers. This report shows that despite the improvement in the learning outcomes and the progress in terms of access to education during the past ten years, a large share of children that complete Standard 3 still lack basic reading, writing, and arithmetic skills—a state of affairs that is replicated in several low-income countries and that (UNESCO, 2013) dubbed the “global learning crisis.” This report is organized as follows. Chapter 1 analyzes the emerging trends in the evolution of student’s learning outcomes in Tanzania’s primary schools and explores variations by region, ethnic-group and gender. Chapter 2 describes different dimensions and trends of teacher quality, such as teacher absence, content knowledge, pedagogical skills and teaching practices. Chapter 3 provides descriptive evidence on school governance and school management quality using the Development-World Management Survey. Chapter 4 presents detailed information on school inputs and infrastructure, as well as emerging trends. Chapter 5 provides a more general description of the different types of support available to students and their engagement to learning. Chapter 6 analysis the correlation between service delivery indicators and learning outcomes, and also provides suggestive evidence on key observable factors associated with highest gains in test scores. Chapter 7 concludes by providing some clear lessons and priority areas for action. 18 The SDI survey was first piloted in Senegal and Tanzania in 2010 and lessons learned from these pilots led to a revised SDI being rolled out across Africa. SDI has followed a series of countries in Africa which have already implemented a full-fledged SDI (Kenya, Uganda, Nigeria, Togo, and Mozambique). Tanzania is, however, the first country to implement its third SDI (2010, 2014 and 2016) which allows for basic trend analysis. The SABER Service Delivery (SD) tool was developed in 2016 in the Global Engagement and Knowledge (GEAK) Unit of the Education Global Practice (GP) at the World Bank, as an initiative to uncover bottlenecks that inhibit student learning in low and middle-income countries and to better understand the quality of education service delivery in a country as well as gaps in policy implementation. Its main purpose is to provide a mechanism to assess the different determinants of learning through a diagnostic tool and also to uncover the extent to which policies translate into implementation and practice. 19 Data collected in the 2010 SDI is more limited and not always comparable to the 2014 and 2016 SDI Surveys. 24 Methodology and Implementation Even though this report uses data from all the SDI surveys collected in Tanzania (e.g. 2010, 2014 and 2016), it focuses mostly on the results from the 2016 Tanzania SDI. The sample of the 2016 Tanzania SDI is given in Table 0.1. Overall, 400 primary schools were visited, 2,145 Standard 3, 4, and 5 teachers were assessed on English, mathematics, and pedagogy and 3,642 teachers of all grades were followed for absence rate (not shown in Table 0.1). Also, although learning outcomes are not part of the indicators, 3,999 Standard 4 students have been assessed on language (English/Swahili), mathematics, and non-verbal reasoning (NVR). It is crucial that the indicators be correlated with student learning outcome because the SDI is geared towards capturing the drivers of learning outcomes at the school level. In each school, one Standard 4 English or mathematics class was observed. Up to 10 students were randomly chosen among Standard 4 students and a total of 3,999 students were assessed for literacy and numeracy skills. For comparison with the 2014 SDI and other SDI countries, 2,800 students were tested in English. In addition, 1,199 more students were tested in Swahili, which is the major medium for learning in Tanzania’s primary education system. Teachers also were assessed with 2,145 of them sitting through a 1 hour 10-minute assessment on their English, mathematics, and pedagogical skills. Finally, 3,642 teachers across grades were randomly chosen during the first visit and their whereabouts assessed in a second unannounced visit for estimation of teachers’ effort and the level of absence in schools and classrooms. The Tanzania SDI survey is representative of primary schools at the national level. It is also representative of Dar es Salaam, the main city, as well as other urban areas, and rural Tanzania. Because of a large DFID program called EQUIP-T being implemented in four regions (Dodoma, Kigoma, Tabora, and Shinyanga), it was also decided to make these four regions as a whole a stratum. Therefore, the SDI can report statistics on the EQUIP- T regions as well. The sampling strategy is fully explained in the Annex A of this report. It is noteworthy that each entity has its own weight. Weights for schools are therefore different from weights for students or teachers. For the latter weights even differ for the analysis of absenteeism or the knowledge content analysis. The difference in weights comes from the fact that for each unit of analysis a sample needs to be drawn. It is also noteworthy that the Tanzania Education SDI, unlike all other SDI countries, only reports on public schools. This is due to the near nonexistence of private schools at the primary level. Indeed, there are very few private schools in the sample frame and this is also confirmed by independent household surveys. In fact, according to the 2012 /13 National Panel Survey, only two percent of the children attending primary school are in the private sector. Under the SABER Service Delivery survey, over one hundred Tanzanian classrooms (112) were observed and recorded using the classroom observation tool CLASS (Classroom Assessment Scoring System) in order to measure teaching practices. In addition, the Development-World Management Survey was administered to the principals and head teachers through face-to-face interviews in 100 primary schools in order to collect information about school management practices. 25 Table 0.1: Tanzania SDI Sample 2014 and 2016 Schools Teachers Standard 4 Students (1) (2) (1) (2) (1) (2) 2016 Total Sample 400 % 2,145 % 3,999 % Stratum Dar es Salaam 47 11.8 312 14.5 470 11.8 Other Urban 58 14.5 393 18.3 580 14.5 Rural 220 55.0 1,063 49.6 2,199 55.0 EQUIP-T 75 18.8 377 17.6 750 18.8 Location All Rural 271 67.8 1,310 61.1 2,709 67.7 All Semi-urban 43 10.8 242 11.3 430 10.8 All Urban 86 21.5 593 27.6 860 21.5 2014 Total Sample 400 % 2,197 % 4,041 % Stratum Dar es Salaam 47 11.7 349 15.9 474 11.8 Other Urban 58 14.5 378 17.2 583 14.4 Rural 221 55.3 1,077 49 2,228 55.1 EQUIP-T 74 18.5 393 17.9 756 18.7 Location All Rural 271 67.8 1,427 65 2,901 71.8 All Semi-urban 43 10.8 … … … … All Urban 86 21.5 770 35 1,140 28.2 Source: Tanzania SDI 2014 and 2016 26 Framework - World Development Report 2018 The framework of the World Development Report 2018: Learning to Realize Education’s Why Learning Doesn’t Happen? (WDR 2018) Promise (WDR, 2018) provides a solid structure for examining if education systems are aligned with learning. One of the main premises of the WDR 2018 is that schooling (enrollment or attendance) is not the same as learning. The WDR identifies and examines the relationship between four crucial parts of an education system that directly affect student learning: teacher knowledge and motivation, school management, availability of school inputs and learner preparedness. One of the main messages of the WDR 2018 is that schools are failing learners – not just in Tanzania but around the world. Struggling education systems often lack one or more of these four key school-level determinants to improve learning. An absence of prepared learners, ineffective teaching, inputs that have nothing to do with learning, and an inability to align these, results in system where students do not learn. Many children do not arrive at school ready to learn – if they arrive at all. Malnutrition, low parental investments, and harsh environments associated with poverty, severely undermine childhood learning (Lupien, et al., 2000; McCoy, et al., 2016; Walker, et al., 2007). Moreover, many disadvantaged children do not attend school due to pervasive conflict, instability, and financial and cultural barriers. Breaking out of lower learning trajectories due to deprivation and lack of school participation has long-lasting effects, which further widen gaps in learning outcomes. Teachers are the most important determinant of student learning in both developed and developing countries, though they often lack the skills or motivation needed to be effective (Hanushek, 1992; Rockoff, 2004; Bau & Das, 2017). Despite the importance of teacher quality for student learning, most education systems, including Tanzania, do not attract strong candidates to the profession. Beyond that, weak candidates and poor education results in teachers’ lacking basic subject knowledge and pedagogical skills. This translates into poor use of instructional time at the school-level, and a system that is ill-equipped to support teachers. School inputs often fail to reach classrooms, or to affect learning when they do. Devoting enough resources to education is crucial, but often it’s how the resources are used that is just as important. Looking across systems and schools, similar levels of resources are often associated with vast differences in learning outcomes (Hanushek, 1995; Wolf, 2004; Mingat & Tan, 1998; Mingat & Tan, 1992). Moreover, increasing inputs in a given setting often has small effects on learning outcomes, which is due to the fact these inputs often fail to make it to where they are intended (Glewwe, et al., 2011; Hanushek, 1986; Kremer, 1995). Poor management and governance often undermine schooling quality. Although effective school leadership does not raise student learning directly, it does so indirectly by improving teaching quality and ensuring the effective use of resources (Robinson, et al., 2008; Waters, et al., 2003). Ineffective school leadership means school principals are not actively involved in helping teachers solve problems, do not provide instructional advice, and do not set goals that prioritize learning. This report builds upon the World Development Report 2018 framework in order to provide a diagnostic that assess the functionality and state of the Tanzanian education system (World Bank, 2018). 27 Background on SDI and SABER Service Delivery Surveys The SABER Service Delivery (SD) tool was developed in 2016 in the Global Engagement and Knowledge Unit of the Education Global Practice (GP) at the World Bank, as an initiative to uncover bottlenecks that inhibit student learning in low and middle-income countries and to better understand the quality of education service delivery in a country as well as gaps in policy implementation. This school survey is aligned to the latest education research on what matters for student learning and how best to measure it. Its main purpose is to provide a mechanism to assess these different determinants of learning through a diagnostic tool and also to uncover the extent to which policies translate into implementation and practice. In alignment with the World Development Report (WDR 2018), the SABER SD instrument examines the four key elements in an education system identified as the main determinants for student learning. This survey strategically collects information on the most important school inputs and processes that produce learning outcomes. The SABER SD survey builds upon and contributes to two current World Bank Group initiatives that produce comparative data and knowledge on education systems: SABER (Systems Approach for Better Education Results) and SDI (Service Delivery Indicators). One of the advantages of the previous surveys, such as SDI, is that we can use them for international comparison reasons. The SABER SD instrument collects data at the school level and asks questions related to the roles of all levels of government (including local and regional). The tool provides comprehensive data on teacher effort and ability; principal leadership; school governance, management, and finances; community participation; and student performance in Mathematics and Language and includes a classroom observation module. Box 0.1: The Link between SABER SD and SDI Approach The foundations of the SABER SD survey build upon two pre-existing World Bank Group initiatives that produce comparative data and knowledge on education systems: SABER (System Approach for Better Education Results) and SDI (Service Delivery Indicators). It also draws upon earlier surveys, namely, QSDS (Quality of Service Delivery Surveys) and PETs (Public Expenditure Tracking). On the one hand, the SABER SD tool builds on the evidence base and captures policy implementation measures from the core SABER domains. 20 On the other hand, the SABER SD tool adapts and extends the surveys that were developed and implemented under the SDI program, which provides a set of metrics for benchmarking service delivery performance in education and health in Africa. The overall objective of SDI is to gauge the quality of service delivery in primary education and basic health services. The SDI approach has been implemented in Kenya, Nigeria, Togo, Uganda, Mozambique, Senegal and Tanzania. 21 There are two main factors that distinguish the SABER SD tool from SDI: (i) it has expanded the measurement of service delivery in primary education outside Africa and into Asia through pilots in Afghanistan, Pakistan and Laos, and (ii) it has adapted and extended the SDI approach by including additional test items from TIMSS and PIRLS, different modality for test administration, different classroom observation modules, and additional questions on school management and student background. All these characteristics of the SABER SD survey make the instrument more appropriate for the Tanzanian context, while at the same time preserving some international comparability. However, it is important to acknowledge that we do not establish a particular logic behind the international comparisons used for the case of Tanzania. Basically, the reason behind the choice of countries used for international comparison in this report is based on data availability. 20 The core SABER domains include: Education Management Information Systems (EMIS), Education Resilience, School Autonomy and Accountability, School Finance, Student Assessment, and Teachers. 21 More information on the SDI survey instruments, data and reports, and more generally on the SDI initiative can be found at http://data.worldbank.org/sdi. There are also a series of published papers which have used the data from SDI surveys in order to analyze the learning process in primary schools in Africa, the lost human capital and also teacher’s effort, knowledge and skills (Bold, et al., 2018; Bold, et al., 2018) 28 Box 0.2: The Service Delivery Indicators (SDI) Program A significant share of public spending on education is transformed to produce good outcomes at schools. Understanding what takes place at these frontline service provision centers is the starting point in establishing where the relationship between public expenditure and outcomes is weak within the service delivery chain. Knowing whether spending is translating into inputs that teachers have to work with (e.g. textbooks in schools), or how much work effort is exerted by teachers (e.g. how likely are they to come to work), and their competency would reveal the weak links in the service delivery chain. Reliable and complete information on these measures is lacking, in general. To date, there is no robust, standardized set of indicators to measure the quality of services as experienced by the citizen in Africa. Existing indicators tend to be fragmented and focus either on final outcomes or inputs, rather than on the underlying systems that help generate the outcomes or make use of the inputs. In fact, no set of indicators is available for measuring constraints associated with service delivery and the behavior of frontline providers, both of which have a direct impact on the quality of services that citizens are able to access. Without consistent and accurate information on the quality of services, it is difficult for citizens or politicians (the principal) to assess how service providers (the agent) are performing and to take corrective action. The SDI provides a set of metrics to benchmark the performance of schools in Africa. The Indicators can be used to track progress within and across countries over time and aim to enhance active monitoring of service delivery to increase public accountability and good governance. Ultimately, the goal of this effort is to help policymakers, citizens, service providers, donors, and other stakeholders enhance the quality of services and improve development outcomes. The perspective adopted by the Indicators is that of citizens accessing a service. The Indicators can thus be viewed as a service delivery report card on education. However, instead of using citizens’ perceptions to assess performance, the Indicators assemble objective and quantitative information from a survey of frontline service delivery units, using modules from the Public Expenditure Tracking Survey (PETS), Quantitative Service Delivery Survey (QSDS), and Staff Absence Survey (SAS). The literature points to the importance of the functioning of schools and more generally, the quality of service delivery. The service delivery literature is, however, clear that, conditional on providers being appropriately skilled and exerting the necessary effort, increased resource flows for health can indeed have beneficial education outcomes. The SDI initiative is a partnership of the World Bank, the African Economic Research Consortium (AERC), and the African Development Bank to develop and institutionalize the collection of a set of indicators that would gauge the quality of service delivery within and across countries and over time. The ultimate goal is to sharply increase accountability for service delivery across Africa, by offering important advocacy tools for citizens, governments, and donors alike; to work toward the end goal of achieving rapid improvements in the responsiveness and effectiveness of service delivery. More information on the SDI survey instruments and data, and more generally on the SDI initiative can be found at: www.SDIndicators.org and www.worldbank.org/sdi, or by contacting sdi@worldbank.org. 29 Chapter 1 : Is Tanzania’s Education System Aligned with Learning? This chapter systematically analyzes and documents emerging trends in the evolution of student’s learning outcomes in Tanzania’s primary schools. This analysis presents the learning outcomes of approximately 4,000 Standard 4 students in public school based on data collected from two rounds of the Service Delivery Indicators (SDI) Survey in Tanzania, 2014 and 2016.22 Both the 2014 and 2016 Tanzania SDI surveys were carried out in the same nationally representative sample of 400 public schools across 22 regions in the country. 23 In each school, the SDI selected up to 10 random Standard 4 students for testing. The objective of the student’s assessment was to assess basic reading, writing and arithmetic skills for Standard 4 students. The test was designed as a one-on-one test with enumerator reading out instructions to students in their mother tongue. The student assessment also measured non-verbal reasoning skills which complement the student test scores in Language and Mathematics and can be used as a rough measure to control for innate student ability when comparing outcomes across different schools. Thus, the student assessment consisted of three parts: Language, Mathematics and Non-verbal reasoning. All students were tested in Mathematics, and either English (approximately 70% of sampled students) or Swahili (30% of sampled students). Given that Swahili is the main language of instruction in Tanzania’s primary schools, a set of students were administered exactly the same English test except that is was translated into Swahili. What do Standard 4 Tanzania’s students know? Tanzania is confronted to a learning crisis in basic education characterized by significant knowledge gaps for students in Standard 4. In the Language test, almost 3/4 of students managed the most basic tasks of recognizing a letter and reading a word. However, student’s scores dropped dramatically as the difficulty increased. Student’s ability to identify a picture from a given word dropped to 26 percent with the trend continuing as less than 1/4 of the class could read a sentence and answer reading comprehension questions. Unsurprisingly, students scored much higher on the Swahili test compared to the English test. In terms of arithmetic operations, students performed better on tasks involving only one-digit numbers (e.g. 87 percent for single digit addition and 81 percent for single digit subtraction). However, both single digit multiplication and division have not solidified yet as just less than half of the students gave correct answers. Although Tanzanian students correctly answered more than half of the Mathematics questions (65 percent), the test revealed that the majority of the Standard 4 students did not perform well at the Standard 3 level. For example, the complete 9x9 multiplication table and simple division are intended to be taught by Standard 3. However, only half of students could perform 6:3 or 7x8. More difficult tasks such as triple digit multiplication, solving a word problem or completing a sequence seem to be a great struggle for Standard 4 students as these tasks were correctly answered by an even smaller proportion of the class (around 15 percent). How have student’s learning outcomes changed in Tanzania’s primary schools? Learning outcomes have improved across a range of basic literacy and numeracy tasks between 2014 and 2016. The 2016 SDI shows a clear pattern of improving learning levels in Tanzania’s primary schools. Nationwide, school average student scores improved across all subjects by an average of 11 percentage points. The largest increase was observed in English: students at the average school answered 52 percent of questions correctly on the SDI English test in 2016, up from 41 percent in 2014, an increase of more than one-fifth. The smallest improvement was in Mathematics, where school average scores increased by five points from 60 to 65 percent. Underpinning the improvements in overall test scores, the proportion of students who could correctly identify a particular letter or word rose from approximately 60 to 72 percent, those who could correctly read a paragraph from 25 to 30 percent and those who could answer a question about the meaning of a passage from 13 to 19 percent. In terms of numeracy, the share of students able to add double-digit numbers rose from 66 to 73 percent and those able to subtract double-digit numbers, from 44 to 50 percent. Improvement also extended to more difficult tasks: the share of students able to multiply triple-digit numbers rose from 12 to 15 percent, those able to single digit increased from 44 to 46 percent and those able to divide double digits, from 23 to 27 percent. Tanzania’s students have not just improved in all subjects compared to 2014 but they also fall relatively well among the SDI countries. In Language, they significantly outperformed students in Mozambique, Uganda, Togo and Nigeria, but could not reach the results observed in Kenya. 22 Standard 4 or Grade 4 students are chosen in the SDI survey because: 1) there is no standardized national or international evaluation of this level; 2) the sample of students in school becomes more and more self‐selective in higher grades due to drop‐out rates; 3) there is growing evidence cognitive ability is most malleable at younger ages. It is therefore especially important to get a snapshot of student’s learning and the quality of teaching provided at younger ages. 23 The sample is also representative of key strata, including schools in Dar es Salaam; urban schools; rural schools; and schools in regions implementing the Education Quality Improvement Program in Tanzania (EQUIP-T), a targeted reform program operating in seven disadvantaged regions. 30 Students in Tanzania scored 63 percent on the Language test, which is more than 10 percentage points higher than the average SDI, but 12 points lower than Kenya. However, in Mathematics and Non-verbal Reasoning, Tanzania’s students in 2016 are the best performers among the SDI countries. They scored almost 20 percentage points higher than the average SDI country on the Mathematics test. How does the learning performance vary by region, ethnic group and gender? Lower-performing regions appear to be catching up with previously better-performing regions, and the EQUIP-T regions in particular have seen rapid increases in student test outcomes. In EQUIP-T regions, the improvement in test scores are substantial, with overall scores increasing 18 percentage points from 43 to 61 percent, an increase of more than one-third. These regions also had the largest improvements in the share of students able to complete most basic tasks. Despite the considerable catching-up by rural schools, urban schools remain in the lead in terms of learning in both Mathematics and Language. Schools in urban areas obtained on average 74 percent in overall learning scores, versus 62 percent in rural schools. In rural areas, which had the next lowest performance in 2014, overall scores increased from 52 percent in 2014 to 60 percent in 2016. In urban areas outside of Dar es Salaam, the rate of increase was slower, but overall scores still rose from 65 percent in 2014 to 71 percent in 2016, only half the rate in rural schools. However, in Dar es Salaam, where overall performance was highest in 2014, overall scores improved only slightly, from 74 to 76 percent Looking at the disaggregation by gender and ethnic group (mother tongue), in general girls and non-Swahili speakers face persistent disadvantages. In Language, boys tend to perform better than girls, but this difference in not statistically significant. However, in Mathematics boys significantly outperformed girls, especially in tasks with higher level of difficulty. Both non-Swahili native speaking students and girls achieve scores consistently below Swahili speaking students and boys. These barriers have not reduced as learning outcomes have risen. With half of the students being female and a majority non-Swahili native speaking, these disparities are likely to restrict the overall improvement in learning outcomes over time, as well as posing a threat to equity. What explains the most variation in the learning performance in Tanzania? Around 37 percent of the variation in learning performance is found across/between schools. Meaning, within the same district or province, there are schools where nearly every student has basic Standard 4 level skills and schools where nearly every student does not. Which is to say, there are good and bad schools in all the districts rather than good schools in certain districts and bad schools in others. Even though school-level variations explain the largest difference in learning outcomes, some differences in student scores are also explained within the same school. For instance, in some of Tanzania’s median-performing schools, test scores, measured as percentage of correct answers, vary from 20% to 95%. This Chapter is organized as follows. The first section assesses Tanzania’s student’s learning performance in Language, while the second section assess Tanzania’s students’ learning performance in Mathematics. The third section examines the trend in average student test scores by subject. The fourth section analyzes how the learning performance varies by region, urban/rural school, ethnic group and gender. In the fifth section, we characterize the learning performance beyond the mean, analyzing in particular the learning gap between and within schools. Lastly, we compare Tanzania’s students’ performance in learning outcomes to other students internationally. 31 Box 1.1: Background on the SDI Student Assessment It is instructive to think of the Service Delivery Indicators as measuring key inputs, with a focus on what teachers do and know, in an education production function. These inputs are actionable and they are collected using objective and observational methods at the school level. The outcome in such an education production function is student learning achievement. While learning outcomes capture both school-specific inputs (e.g., the quality and effort exerted by the teachers) and various child-specific factors (e.g., innate ability) and household-specific factors (e.g., the demand for education), and thus provide, at best, reduced form evidence on service provision, it is a still an important measure to identify gaps and to track progress in the sector. Moreover, while the Service Delivery Indicators measure inputs -- and learning outcomes are not part of the Indicators -- in the final instance we should be interested in inputs not in and of themselves, but only in as far as they deliver the outcomes we care about. Therefore, as part of the collection of the Service Delivery Indicators in each country, learning outcomes are measured for Grade four students. The objective of the student assessment was to measure basic reading, writing, and arithmetic skills. The test was designed by experts in international pedagogy and based on a review of primary curriculum materials from 13 African countries (For details on the design of the test, see (Johnson, et al., 2012) “Draft Final Report, Teaching Standards and Curriculum Review”). The student assessment also measured nonverbal reasoning skills on the basis of Raven’s matrices, a standard IQ measure that is designed to be valid across different cultures. This measure complements the student test scores in language and mathematics and can be used as a rough measure to control for innate student ability when comparing outcomes across different schools. Thus, the student assessment consisted of three parts: language, mathematics and non- verbal reasoning (NVR). The test, using material up to the grade three level was administered to grade four students. The reason for choosing students in grade four is threefold. First, there is scant information on achievement in lower grades. SACMEQ, for example, tests students in grade six. Uwezo is a recent initiative that aims to provide information on student’s learning irrespective of whether they are enrolled in school or not and tests all children under the age of 16 on grade two material. While this initiative has provided very interesting results, it is not possible to link student achievement to school level data, since the survey is done at the household level. Second, the sample of children in school becomes more and more self-selective as one goes higher up due to high drop-out rates. Finally, there is growing evidence that cognitive ability is most malleable at younger ages. It is therefore especially important to get a snapshot of student learning and the quality of teaching provided at younger ages. The test was designed as a one-on-one test with enumerators reading out instructions to students in their mother tongue. This was done to build up a differentiated picture of student’s cognitive skills; i.e. oral one-to-one testing allows us to evaluate whether a child can solve a mathematics problem even when her reading ability is so low that she would not be able to attempt the problem independently. The language test consisted of a number of different tasks ranging from testing knowledge of the alphabet, to word recognition, to a more challenging reading comprehension test. Altogether, the test included six tasks. The mathematics test also consisted of a number of different tasks ranging from identifying and sequencing numbers, to addition of one- to three-digit numbers, to one- and two-digit subtraction, to single digit multiplication and divisions. The mathematics test included six tasks and a total of 15 questions. The non-verbal reasoning section consisted of four questions. Section I. Student Results: Language Assessment The Language (English or Swahili) test consisted of six tasks that assess student’s ability to identify a specified letter, identify a specified word, name a simple noun from a picture, read a sentence, read a paragraph and carry out basic reading comprehension. Approximately, 70 percent of the sampled students were tested in Swahili, while 30 percent was tested in English. Box 1.1 contains a sample of some of the Language questions/items found in the survey. 32 Box 1.2: Sample of Language Questions Swahili Version English Version Hii ni nini? ________ What is this? ________ Swahili Version English Version chini Heshima hadithi under respect story ndizi Kijani salimia bananas green greet tunda Baba nje fruit father outside “Je, unaweza kunionyesha neon -salimia-, tafadhali?” “Can you show me the word ‘greet’, please?” Swahili Version English Version Bahati aliingia dukani. Aliketi pembeni mwa Furaha. Mbwa alikuwa Bahati walked into a shop. He sat down next to Furaha. A dog was amelala karibu na miguu ya Furaha. ‘Je mbwa wako anauma?’ sleeping next to Furaha’s feet. ‘Does your dog bite?’ asked Bahati. aliuliza Bahati. ‘Hapana,’ alijibu Furaha. Dakika chache baadae ‘No,’ Furaha replied. A few minutes later the dog woke up and bit mbwa aliamka na kuuma mguu wa Bahati. ‘Ah, mguu wangu!’ alilia Bahati’s leg. ‘Oh, my leg!’ cried Bahati. ‘You said your dog doesn’t Bahati. ‘Ulisema mbwa wako haumi.’ ‘Lakini huyo si mbwa wangu,’ bite.’ ‘But that’s not my dog,’ replied Furaha. alijibu Furaha. (a) Je Bahati na Furaha walikutana wapi? (a) Where did Bahati and Furaha meet? __________________________________________________ __________________________________________________ (b) Je ni mnyama gani alikuwa amelala pembeni ya Furaha? (b) What animal was sleeping next to Furaha? __________________________________________________ __________________________________________________ 33 Student learning performance in Language has improved across a range of basic literacy tasks. Figure 1.1 provides a complete breakdown of the Language test results in 2014 and 2016, along with the percentage of students who correctly answered the questions. The proportion of students who could correctly identify a particular letter or word rose from approximately 60 to 72 percent, those who could correctly read a paragraph from 25 to 30 percent and those who could answer a question about the meaning of a passage from 13 to 19 percent. Thus, the 2016 SDI shows a clear pattern of improving language levels in Tanzania’s primary schools. Figure 1.1: Student Learning Performance in Language – Standard 4 80% 72% 72% 59% 60% 60% 40% 31% 26% 25% 16% 17% 19% 20% 13% 13% 0% Read a letter Read a word Identify words Read a sentence Read a Reading paragraph Comprehension 2014 2016 Note: Each specific Language test item represents the percentage of student that responded the question correctly. Source: Tanzania SDI 2014 and 2016 Unsurprisingly, students scored much higher on the Swahili test compared to the English test. Figure 1.2 shows the student learning performance by language of instruction. For Swahili students, close to nine out of ten students manage the simplest tasks, such as identifying a letter or recognizing a word compared to six out of ten students in English. Swahili test takers also performed better in more complex tasks such as reading a 50-word paragraph or answering factual comprehension questions about the paragraph they read. The picture is completely different for the English test, where one in three (33 percent) students were unable to identify a sample alphabet letter. In addition, when it comes to more complex tasks only one in four (24 percent) could read a 10-word sentence, 6 percent could fluently read a paragraph and 8 percent could answer a question about the meaning of the passage. Figure 1.2: Student Learning Performance in Language by Medium of Instruction – Standard 4 100% 92% 90% 85% 80% 66% 64% 60% 60% 45% 40% 24% 20% 11% 10% 6% 8% 0% Read a letter Read a word Identify words Read a sentence Read a Reading paragraph Comprehension English Swahili Source: Tanzania SDI 2016 34 Section II. Student Results: Mathematics Assessment The Mathematics test consisted of six tasks and a total of 15 questions ranging from identifying and ordering numbers, to addition of one- to three-digit numbers, to one- and two-digit subtraction, to one-to-three-digit multiplication and divisions etc. Box 1.2 contains some of the Mathematics questions/items found in the 2016 SDI survey. Box 1.3: Sample of Mathematics Questions Swahili Version English Version Je, unaweza kunionyesha namba “23”, tafafhali? Can you show me number “23”, please? 55 3 23 55 3 23 4 12 6 4 12 6 34 21 9 34 21 9 Swahili Version English Version Tafadhali kutoa majibu sahihi kwa usawa wafuatayo: Please provide the correct answers to the following equations: 8+7 8+7 28 + 27 28 + 27 335+ 145 335+ 145 8-5 8-5 57-49 57-49 Swahili Version English Version Tafadhali kutoa majibu sahihi kwa usawa wafuatayo: Please provide the correct answers to the following equations: 7 × 8= 7 × 8= 37 × 40 = 37 × 40 = 214 × 104 = 214 × 104 = 6:3 6:3 75 : 5 75 : 5 Swahili Version English Version Sanduku ina machungwa 26. Ni machungwa ngapi yaliyomo A box contains 26 oranges. How many oranges are contained in 10 kwenye masanduku 10? boxes? Swahili Version English Version Nambari ya pili ni ipi? What is the next number? 48 → 24 → 12 → 6 → ____ 48 → 24 → 12 → 6 → ____ 35 Student performance in Mathematics has also improved across a range of basic numeracy tasks. Figure 1.3 provides a complete breakdown of the Mathematics test results in 2014 and 2016, along with the percentage of students who correctly answered the questions. The share of students able to add double-digit numbers rose from 66 to 73 percent and those able to subtract double-digit numbers, from 44 to 50 percent. Improvement also extended to more difficult tasks: the share of students able to multiply triple-digit numbers rose from 12 to 15 percent, those able to single digit increased from 44 to 46 percent and those able to divide double digits, from 23 to 27 percent. However, for two of the most difficult tasks in mathematics, solving a math story and completing a sequence, the learning outcomes in Math have worsen compared to 2014. Figure 1.3: Students’ Learning Performance in Mathematics – Standard 4 0% 20% 40% 60% 80% 100% Recognize numbers 93% 90% Order numbers 50% 56% Single digit addition 83% 87% Double digit addition 66% 73% Triple digit addition 66% 74% Single digit subtraction 78% 81% Double digit subtraction 44% 50% Single digit multiplication 43% 49% Double digit multiplication 15% 18% Triple digit multiplication 12% 15% Single digit division 44% 46% Double digit division 23% 27% Understanding of division 20% 20% Solving a math story 24% 15% Complete a sequence 21% 16% 2014 2016 Note: Each specific Mathematics test item represents the percentage of student that responded the question correctly. Source: Tanzania SDI 2016 36 Despite the improvement in the learning outcomes in Mathematics, there are still some significant knowledge gaps. In terms of arithmetic operations, students performed better on tasks involving only one-digit numbers (e.g. 87 percent for single digit addition and 81 percent for single digit subtraction). However, both single digit multiplication and division have not solidified yet as just less than half of the students gave correct answers. Although, Tanzanian students correctly answered more than half of the mathematics questions (65 percent), the test revealed that the majority of the Standard 4 students did not perform well at the Standard 3 level. For example, the complete 9x9 multiplication table and simple division are intended to be taught by Standard 3. However, only half of students could perform 6:3 or 7x8. In terms of performance, addition was followed by subtraction, then division as a distant third and, finally, multiplication. For operations involving two-digit numbers, three out of four students (73 percent) were able to solve double digit addition, one in two could solve subtraction, but this type of operation dropped down to 27 percent for division and only 18 percent for multiplication. More difficult tasks such as triple digit multiplication, solving a word problem or completing a sequence seem to be a great struggle for Standard 4 students as these tasks were correctly answered by an even smaller proportion of the class (around 15 percent). Section III. Average Student Test Scores by Subject Mathematics scores were slightly better than English scores, but below the Swahili performance. Students who sat for the Swahili test correctly answered 91 percent of the questions on the test compared to only 52 percent for those tested in English. The average score in Mathematics was 65 percent, which is still relatively low. The students were also tested on four non-verbal reasoning questions and received an average score of 58 percent on that part of the assessment. There is a wide variation in the distribution of student’s test score, especially in language. Indeed, the language score differences fully account for the differences in the average test scores (Figure 1.5). The SDI data demonstrates considerable improvements in learning outcomes between 2014 and 2016 (Figure 1.4). Nationwide, school average student scores improved across all subjects between 2014 and 2016 by an average of 11 percentage points. The largest increase was observed in English: students at the average school answered 52 percent of questions correctly on the SDI English test in 2016, up from 41 percent in 2014, an increase of more than one-fifth. The smallest improvement was in Mathematics, where school average scores increased by five points from 60 to 65 percent. Figure 1.4: Students’ Test Scores – Standard 4 100% 91% 84% 80% 65% 65% 60% 58% 54% Test Scores 60% 52% 53% 41% 40% 20% 0% English Swahili Mathematics Non-verbal Overall reasoning 2014 2016 Note: The SDI student learning assessments are based on a review of primary curricula from 13 African countries; see (Johnson, et al., 2012) for details. All sampled students completed tests in Math, Language (English or Swahili) and Non-Verbal Reasoning. Overall scores are calculated as share of maximum total possible score across all tests completed. Source: Tanzania SDI 2016 37 The increase in performance in Standard 4 is in line with observations at other stages of the primary cycle. The Early Grade Reading Assessment and Early Grade Mathematics Assessment, carried out in Standard 2, both identified similar rates of increase between 2014 and 2016. 24 The Primary Schools Leaving Examination, the main national primary school examination carried out in Standard 7, has also seen pass rates increase from 57 percent in 2014 to 68 percent in 2016. Figure 1.5: Distribution of Language and Mathematics Student Test Scores Source: Tanzania SDI 2016 Section IV. Disaggregation by Region, Ethnic Group and Gender In this section, we analyze the differential performance on learning outcomes by urban/rural schools, gender and ethnic group. Figure 1.6: Student’s Learning Performance in Language by Region and Gender 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 84% Read a letter 72% Read a letter 69% 72% 85% Read a word 74% Read a word 69% 71% 43% Identify words 26% Identify words 21% 26% Read a sentence 31% Read a sentence 17% 13% 17% Read a paragraph 36% Read a paragraph 31% 30% 31% Reading Comprehension 28% Reading Comprehension 20% 16% 18% urban rural boys girls Source: Tanzania SDI 2016 24 The national average score in oral reading for Standard 2 students in EGRA rose from 17.9 percent in 2014 to 23.6 percent in 2016, while the average for basic subtraction in EGMA rose from 5.5 points to 6.72 points. 38 Looking at the disaggregation between urban and rural schools, the general trend is in favor of urban schools in both Mathematics and Language (Figure 1.6 and Figure 1.7). In Language, urban school students showed significantly better results than rural school student in each one of tasks of the Language test. Similarly, students in rural schools had a lower performance in Mathematics compared to students in urban schools. For instance, students in urban schools are approximately 15 percentage points more likely to correctly answer a double digit addition exercise and almost 20 percentage points more likely to correctly answer a double digit subtraction. They also perform better in questions with a higher level of difficultly such as word problems. Figure 1.7: Student’s Learning Performance in Mathematics by Urban/Rural and Gender 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Recognize numbers 90% Recognize numbers 91% 90% 90% Order numbers 64% Order numbers 60% 54% 52% Single digit addition 92% Single digit addition 87% 85% 86% Double digit addition 85% Double digit addition 75% 70% 72% Triple digit addition 85% Triple digit addition 75% 70% 72% Single digit subtraction 87% Single digit subtraction 81% 79% 81% Double digit subtraction 64% Double digit subtraction 52% 46% 49% Single digit multiplication 56% Single digit multiplication 52% 47% 46% Double digit multiplication 31% Double digit multiplication 20% 14% 16% Triple digit multiplication 27% Triple digit multiplication 17% 12% 13% Single digit division 56% Single digit division 49% 43% 42% Double digit division 39% Double digit division 31% 24% 24% Understanding of division 24% Understanding of division 21% 19% 20% Solving a math story 19% Solving a math story 18% 14% 12% Complete a sequence 20% Complete a sequence 18% 15% 15% urban rural boys girls Source: Tanzania SDI 2016 Overall, urban schools achieve higher scores than rural schools. Despite the considerable catching-up by rural schools, urban schools remain in the lead in terms of learning. Schools in urban areas obtained on average 74 percent in overall learning scores, versus 62 percent in rural schools. 39 Looking at the disaggregation by gender and ethnic group (mother tongue), in general girls and non-Swahili speakers face persistent disadvantages (Figure 1.8). In Language, boys tend to perform better than girls, but this difference in not statistically significant. However, in Mathematics boys significantly outperformed girls, especially in tasks with higher level of difficulty. Swahili is the language of instruction in Tanzanian primary schools, and the primary national lingua franca, but only around ten percent of Tanzanians speak Swahili as their primary home language or ‘mother tongue’ (Petzell, 2012). Both non-Swahili native speaking students and girls achieve scores consistently below Swahili speaking students and boys. These barriers have not reduced as learning outcomes have risen. With half of the students being female and a majority non- Swahili native speaking, these disparities are likely to restrict the overall improvement in learning outcomes over time, as well as posing a threat to equity. Figure 1.8: Average Student Test Scores by Ethnic Group (Mother Tongue) and Gender 100% 94% 100% 87% 90%93% 80% 80% 67% 67% 67%64% 66%63% 62% 59%56% 61% 60% 56% 60% 54% 53%50% 48% 51% 40% 40% 20% 20% 0% 0% English Swahili Mathematics Non-verbal Overall English Swahili Mathematics Non-verbal Overall reasoning reasoning Swahili Non-Swahili boys girls Source: Tanzania SDI 2016 Different Impacts on Different Students The SDI data contains information on student characteristics for sampled students, as well as school conditions, enabling us to analyze which students gained most from the improvements in service delivery observed across the country. Learning outcomes improved for both girls and boys, but girls remain behind. While Tanzania has achieved near-universal enrollment of girls in primary school, female students have historically had lower learning outcomes than male students. Girls achieved an average overall score of 52 percent in the SDI learning assessments in 2014, versus 54 percent for boys. Both genders gained almost equally between 2014 and 2016, with girls’ average score increasing eleven points to 63 percent, and boys’ increasing twelve points to 66 percent (Figure 1.9). The finding suggests that girls are not being excluded from the gains in service delivery observed since 2014; however, more work is required to close the learning gap entirely. 25 The finding suggests a need for detailed observation and analysis of school conditions and teaching practices in order to identify the factors which disadvantage girls. The persistence of the gender gap, even in schools which improved their overall learning outcomes, may also suggest that factors outside school, such as reduced household investments in girls, have an impact on the readiness of these students to learn. Older students slightly reduced their learning gap with younger students. In 2014, students above the official age range for Standard 4 – above 10 years of age – achieved average overall learning outcomes seven percentage points lower than students ten years of age or younger. This gap had closed slightly by 2016, with older students achieving an increase in overall learning outcomes of 11 percentage points, versus nine points for younger students. 25 Girls obtain lower average outcomes in NVR as well as in learning assessments, suggesting that the differential in female students’ learning performance stems from inputs outside of rather than solely from differences in school. A likely explanation is that girls benefit from a lower level of investment by families than boys in early childhood, in terms of nutrition, learning support, and interaction, all factors associated with higher cognitive ability. 40 Students with Swahili as their mother tongue pulled further ahead of those with other languages. In 2014, Swahili-speaking students achieved consistently higher learning outcomes than those with other first languages, with average overall scores of 54 percent versus 51 percent for non- Swahili native speakers. In 2016, this gap has widened: Swahili speakers increased their average scores by 13 percentage points to 67 percent, while non-Swahili speakers increased their score by 10 percentage points to 61 percent (Figure 1.9). The learning gap between the two groups increased from 3.3 percent to 5.6 percent. With Swahili serving since 2015 as the language of instruction in Secondary schools as well, there is an urgent need to ensure non-native Swahili students are supported throughout the system. Figure 1.9: Trends in Overall Average Scores by Gender and Ethnic Group 80% 67% 66% 61% 63% 60% 54% 54% 52% 51% 40% 20% 0% Swahili Non-Swahili Boys Girls 2014 2016 Source: Tanzania SDI 2014 and 2016 Overall scores increased more rapidly in EQUIP-T regions (Education Quality Improvement Tanzania). 26 The results suggest particularly significant impacts from EQUIP-T regions. In EQUIP-T regions, the improvement in test scores are substantial, with overall scores increasing 18 percentage points from 43 to 61 percent, an increase of more than one-third. These regions also had the largest improvements in the share of students able to complete most basic tasks. For instance, the share of students who could identify a letter rose from 54 to 70 percent, while the share able to multiply single digits rose by almost half, from 33 to 49 percent. EQUIP-T regions, which saw the most rapid improvements in learning outcomes, had the lowest performance in 2014 (Figure 1.10). In rural areas, which had the next lowest performance in 2014, overall scores increased from 52 percent in 2014 to 60 percent in 2016. In urban areas outside of Dar es Salaam, the rate of increase was slower, but overall scores still rose from 65 percent in 2014 to 71 percent in 2016, only half the rate in rural schools. However, in Dar es Salaam, where overall performance was highest in 2014, overall scores improved only slightly, from 74 to 76 percent, and scores in Mathematics actually declined. This likely reflects increased overcrowding in the largest schools in Dar es Salaam in a context of rapid urbanization. The gap in overall average test scores between the EQUIP-T and Dar es Salaam strata – those with the lowest and highest learning scores in 2014 – has shrunk from a very high 30 percentage points in 2014 to 15 percentage points in 2016. 26Education Quality Improvement Tanzania (EQUIP-T) is a DFID funded program and implemented in cooperation with the Government in Tanzania in support of better learning outcomes at basic education level, especially girls. 41 Figure 1.10: Trends in Average Student Test Scores by Type of Region (Standard 4) 80% 74% 76% 71% 65% 61% 60% Overall student test score 60% 52% 43% 40% 20% 0% EQUIP-T Rural Othe Urban Dar es Salaam 2014 2016 Source: Tanzania SDI 2014 and 2016 Box 1.4: Education Quality Improvement Tanzania (EQUIP-T) Education Quality Improvement Tanzania (EQUIP-T) is a DFID funded program and implemented in cooperation with the Government in Tanzania in support of better learning outcomes at basic education level, especially girls. It aims to develop, implement and demonstrate the best approaches to strengthen the quality of education in seven regions of Tanzania. It began in 2014 in Dodoma, Kigoma, Shinyanga, Simiyu and Tabora; in 2015 the program expanded to Lindi and Mara. Currently, it operates in 9 regions of Tanzania with the objective of achieving comprehensive school improvements for improved learning outcomes. The program operates through a decentralized structure by engaging institutions at the Ministry level, Local Government Authorities (LGA), and communities. Program components include improving and strengthening: 1) Teacher professional development; 2) Leadership and management; 3) District management; 4) Community participation; and 5) Monitoring and Evaluation. To date, EQUIP-T has carried out a number of key interventions for quality improvement: capacity building and funding at the LGA level has trained 1,134 Ward Education Officers (WEOs) and equipped them with 1,010 motorbikes to conduct monitoring and support visits to schools; 8,900 head teachers and deputy head teachers were provided with school leadership training; 49,000 teachers have been provided with INSET teacher training on 3R’s[ii] and gender responsive pedagogy; 4,420 Parent-Teacher partnerships have been formed; and a new School Information System is being rolled out to support high-frequency monitoring of key school performance indicators via mobile phone, among others. The Midline Assessment (2016/17) found significant impacts from the program, including a 43% increase in the average reading fluency; a 32 percent increase in the number of schools developing Operational School Development Plans; and more schools were found to be utilizing gender-responsive and inclusive practices in the classroom. Ongoing challenges include delays in funds being received at the LGA level, as well as limited public financial management capacity within LGAs. _______________________________ [i] Lindi, Mara, Kigoma, Dodoma, Shinyanga, Simiyu and Tabora. The program is expanding to include Katavi and Singid [ii] 3Rs including reading, writing and numeracy 42 Analysis of the differential performance of regions provides further evidence of a process of ‘catching-up’, with the regions with the poorest performance in 2014 frequently registering the most rapid gains. In Mathematics, of the six regions with an increase in average test scores of nine percentage points or more, five were in the bottom ten regions in 2014. In Swahili, six regions attained an increase in average scores of more than twenty points; all were in the lowest seven regions in 2014. By contrast, of the seven highest-performing regions in 2014 in Swahili, four saw a reduction in average scores, and none achieved an increase of more than 1.1 percent. Figure 1.10 below shows the regional distribution of improvement in Mathematics scores. The largest improvements were in Tabora, Simiyu, Mara, Shinyanga and Kilimanjaro; of these, four out of five are EQUIP-T regions. Figure 1.11: Regional Changes in Student Learning Performance in Mathematics (Standard 4) 27 Source: Tanzania SDI 2014 and 2016 27 SDI sample includes 22 of Tanzania’s 26 regions 43 Section VI. Looking Beyond the Mean Difference in Learning Outcomes Between and Within Schools This section aims to analyze differences in Mathematics and Language test scores across/between schools and also within a same school. In other words, we want to compare the performance of students that go to different schools and the performance of students that go to the same school. To examine this, we first rank the schools by average performance in Language and Mathematics and then we plot the students’ overall test scores, school by school, for three different groups of the distribution: (i) the five worst performance schools (ii) the five median performance schools and (iii) the five best performance schools. These Figures also show the full distribution of students’ test scores grouped by the school they attend, which allows us to compare students inside a same school. The horizontal lines indicate students’ knowledge scores (% of total correct answers). Most of the variation in learning is found across schools. Figure 1.11 and Figure 1.12 plot each school’s Mathematics and Language score for the five bottom schools, five median schools and five top schools against a unique school identifier on bottom of the graph. At a first glance, we can clearly observe that some schools are performing fairly well and others are not. For instance, in the worst performance schools in Mathematics, students correctly answered only 31 percent of the questions in Mathematics, in the median performance schools, around 65 percent, while around 91 percent in top performance schools. This large variation across schools implies that in the same district, there are schools where every student can at least perform a double digit addition, and schools where the vast majority of children cannot. Figure 1.12: Differences in Learning Outcomes Between and Within Schools – Mathematics Score Note: The Figure plots each student’s Mathematics test score against a school identifier for the bottom 5 schools, the median 5 schools and the top 5 schools in Math performance in Standard 4. The figure shows the full distribution of students’ test scores (i.e. percentage of total correct answers) for each school grouped by the school they attended. Source: Tanzania SDI 2016 Even within the same schools (classrooms) taught by the same teachers, Tanzania’s students are at very different levels of learning. For instance, students in the median 5 performance schools scored around 65 percent on average in Mathematics, however the similarities seem to end there (Figure 1.11). Schools 272 and 463 have a denser distribution of scores which means that they have students who are performing at roughly similar levels and reasonably well, while schools 363 and 216 have a more varied distribution among their students which means that students in the same class and taught by the same teacher are very different in what they know. 28 Even though students in schools 363 and 216 are median 28 All five schools had a comparable number of students, ranging between 18 to 25, so the differences cannot be accounted by a difference in the strength of the tested students. 44 performers, they display a wide spectrum of learning in the same class, ranging from students who cannot perform at least half of the tasks on the test to those who are close to performing word problems. Figure 1.13: Differences in Learning Outcomes Between and Within Schools – Language Score Note: The Figure plots each student’s Language test score against a school identifier for the bottom 5 schools, the median 5 schools and the top 5 schools in Math performance in Standard 4. The figure shows the full distribution of students’ test scores (i.e. percentage of total correct answers) for each school grouped by the school they attended. Source: Tanzania SDI 2016 Similarly, students in the median performance schools in Language also present a large variation in student test scores within the same school (Figure 1.12). For instance, school 33 baffles – even though it is a median performing school in Language, at the bottom of the test score distribution, students end up performing as low as 10 percent in Language, while the highest performing students go up to an almost full-score. Given this level of variation within the same school, it is very difficult for teachers to respond to the needs of students who cannot even identify a picture from those who can solve a reading comprehension exercise. The same pattern can also be found in the bottom performing schools, where in the same school (e.g. school 214 in Figure 1.12) there is a relatively high proportion of students who can answer less than 40% of the questions correctly, but there are also students who can be compared to those in the top performance schools. This level of variation in the scoring patterns of students in Tanzania suggests that some of the differences in student scores are also explained within/inside the same school. Results from the Learning Decomposition This section aims to identify how much of the variability in learning/test score outcomes (i.e. percentage of correct answers in Mathematics and Language) is explained by difference across regions, districts, or schools. For this purpose, we regress the students’ test scores separately on region, district, and school level dummies to provide an estimate of the explanatory power of each one of these variables. The residual variation is assumed to be driven by differences across students and unexplained variation in the data such as measurement error, ability, etc. Most of the variation in learning is explained by the variation between schools. Large variation across schools implies that within the same district/region, there are schools were every student can at least master the basic Standard 4 level tasks, and schools where the vast majority of students cannot. 45 Table 1.1 reports the R-squared from the regressions where indicators of region, district, or school are the explanatory variables. The R-squared measures the percentage of the sample variation in student test scores that is explained by difference across regions, districts, or schools. Results from Table 1.1 indicate that school-level differences carry the most weight in explaining score variation in Mathematics and Language. More specifically, in the SDI sample, 37% of the variation in total student test scores is explained by differences across schools rather than across children in the same school. At the subject level, school effects explain around 33% of the variation in Mathematics test scores and 50% of the variation in Language test scores. On the other hand, the portion of learning variation attributable to regions and districts in Tanzania in relatively small. District variation explains around 20% of the variation in the overall student test scores, and region variation explains only 15% of the variation, which is much lower than the variation at the school level. Table 1.1: R-Squared- Variance decomposition of student’s learning scores Region District School R-squared R-squared R-squared Language score 0.13 0.18 0.31 English score 0.27 0.36 0.60 Swahili score 0.05 0.16 0.46 Mathematics score 0.08 0.12 0.33 Non-verbal ability score 0.04 0.05 0.21 Overall score 0.15 0.20 0.37 Note: This Table presents the R-squared from an OLS regression where indicators of region, district or school are the explanatory variables, respectively. Source: Tanzania SDI 2016 46 Box 1.5: Comparing Tanzania’s Students Internationally In terms of student’s learning outcomes, Tanzania’s students have not just improved in all subjects compared to 2014 but they also fall relatively well among the SDI countries. Figure 1.13 shows the average test score results by subject (Language, Mathematics, Non-Verbal Reasoning and Overall) along with the percentage of students who answered correctly for each SDI country. In Language, they significantly outperformed students in Mozambique, Uganda, Togo and Nigeria, but could not reach the results observed in Kenya. Students in Tanzania scored 63 percent on the Language test, which is more than 10 percentage points higher than the average SDI, but 12 points lower than Kenya. However, in Mathematics and Non-verbal Reasoning, Tanzania’s students in 2016 are the best performers among the SDI countries. They scored almost 20 percentage points higher than the average SDI country on the Mathematics test. Figure 1.14 : Comparing Average Student’s Test Scores Internationally by Subject – Standard 4 Overall Student Test Score Language Student Test Score Mozambique 2014 21 Mozambique 2014 19 Nigeria 2013** 32 Nigeria 2013** 31 Togo 2013 46 Togo 2013 46 Uganda 2013 49 Uganda 2013 47 Average SDI 50 Average SDI 50 Tanzania 2014 54 Tanzania 2014 62 Tanzania 2016 64 Tanzania 2016 71 Kenya 2012 72 Kenya 2012 75 0 20 40 60 80 0 20 40 60 80 Mathematics Student Test Score Non-verbal Reasoning Student Test Score Mozambique 2014 25 Mozambique 2014 44 Nigeria 2013** 32 Nigeria 2013** 50 Uganda 2013 43 Tanzania 2014 53 Togo 2013 45 Togo 2013 54 Average SDI 47 Average SDI 55 Kenya 2012 59 Uganda 2013 57 Tanzania 2014 60 Kenya 2012 58 Tanzania 2016 65 Tanzania 2016 58 0 10 20 30 40 50 60 70 0 10 20 30 40 50 60 70 Note: These figures present the percentage of Standard 4 students in public schools that responded correctly all the questions in the Language, Mathematics and Non-Verbal Reasoning tests. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013, Kenya SDI 2012, ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. Overall, Tanzania’s students are ranked second among the SDI countries only to Kenya (72 percent), who is the best performer with a difference of 8 percentage points. 47 Chapter 2 : Teachers This chapter analyzes different dimensions and trends of teacher quality in the context of Tanzania, which are considered crucial in order to understand the student learning outcomes and ultimately to guide policy design. A growing body of evidence, from both the teacher value-added literature and the experimental literature in development economics, shows that teacher quality is the most important determinant of student learning (World Bank, 2018). Using nationally representative data from two rounds of the Service Delivery Indicators (SDI) Survey in Tanzania, 2014 and 2016, which were collected through direct observations, unannounced visits, and tests from primary school teachers, we answer three basic questions: What do teachers do? What do teachers know? and How well do teachers teach? 29 In each school, the SDI Survey randomly selected up to 10 teachers to complete an assessment of subject knowledge and pedagogical skills; and in a second, unannounced visit, their presence or absence from the classroom was observed. More specifically, a sample of approximately 2,100 randomly selected teachers sat through a 1 hours 10 minutes assessment on English, Mathematics, and pedagogical skills. In essence, this test measured whether the teachers mastered their students’ curriculum and also assessed their ability to identify and suggest a correct answer. Moreover, around 3,650 teachers across grades were randomly chosen during the first visit and their whereabouts assessed in a second unannounced visit for estimation of teacher’s effort and absence rate in the schools and classroom. What do Tanzanian teachers do? Tanzanian teachers displayed patterns with relatively low school absence rate (13 percent), but high classroom absence rate with almost half of the teachers (42 percent) not found in classroom, either because they were absent from school or in the school but not in the classroom. While in the classroom, teachers spent only half of the time (56 percent) on learning activities. Factoring in teacher absence from school and class as well as the percentage of the lesson that the teacher devotes to non-teaching activities, students are taught, on average, 3 hours and 55 minutes per day out of 7 hours of scheduled teaching time. There are no major differences in learning time across urban/rural schools. Students in rural schools seem to receive slightly more time learning compared to their urban peers but this difference is not statistically significant. When comparing the 2016 SDI results to those of 2014, we find that Tanzanian teachers are less likely to be absent from class and they are also more likely to be in class teaching. Comparing to other countries, Tanzanian teachers display a relatively high classroom absence rate, but Tanzanian students received more teaching than the average Sub-Saharan African country, where comparable data was collected. This includes Kenya, Mozambique, Togo, Senegal, Uganda, and 4 states in Nigeria. What do Tanzanian teachers know? The performance on the Language test was quite low, with few teachers mastering the content knowledge in Language (37 percent). Teachers were asked to mark (or “grade”) mock student tests in Language and in Mathematics. In Language, the exercise assessed their ability to spell simple words (“traffic,” for example) and identify the correct grammatical option that completes a sentence such as “[Does, Where, How long] does it take to walk to this school?”. On average, language teachers correctly answered these questions 69 percent of the time. The test also included cloze passages, which consisted of a short story with certain words removed and the teacher had to fill the gaps in a meaningful way, which included “student” responses such as “[Where] do I have to go to the market?” (In this case, a correct answer could be either “Why or When.”). Teachers got 48 percent of the points in this exercise. They were also asked to correct spelling, grammar, syntax, and punctuation mistakes in a student’s letter that included segments such as “I went to tell you that my new school is better the oldone I have a lot of thing to tell you about my new school in Dar es Salaam.”. Teachers struggled with this task and got only 17 percent of the points. The performance on the Mathematics test was better than the one in Language, with more than half of teachers mastering the content knowledge in Mathematics (65 percent). In Mathematics, we tested if the teacher could accurately correct children’s work in such aspects of numeracy as manipulating numbers using whole number operations. Looking at specific tasks, Tanzanian teachers were much more at ease with simple operations such as double-digit addition or double-digit subtraction, than with slightly more complex computations such as interpreting a 29 The sample is also representative of key strata, including schools in Dar es Salaam; urban schools; rural schools; and schools in regions implementing the Education Quality Improvement Program in Tanzania (EQUIP-T), a targeted reform program operating in seven disadvantaged regions. 48 graph or a Venn diagram. For instance, one quarter of teachers could not interpret information in a Venn diagram, half could not solve an algebra exercise, and more than half could not interpret data from a graph. In terms of regional gap, urban teachers are statistically indistinguishable from rural teachers in terms of their subject knowledge in both Mathematics and Language. When comparing the 2016 SDI with the 2014 SDI results, we find that Tanzanian teacher’s subject knowledge in Language have slightly deteriorated. However, teacher’s subject knowledge in Mathematics seems to have improved rapidly between 2014 and 2016. For instance, scores on exercises assessing skills in interpretation of graphs and Venn diagrams increased from 28 percent to 41 percent and from 49 percent to 69 percent, respectively. The share of teachers who could correctly solve a simple algebraic equation also increased from 51 percent to 55 percent. Comparing Tanzania’s teachers internationally, we find that they ranked almost last on Language among the Sub Saharan African countries, with only Mozambique performing lower. Nevertheless, in terms of content knowledge in Mathematics, Tanzanian teachers performed significantly better than the average teacher in Sub Sahara Africa and they were second only to Kenyans. How well do Tanzanian teachers teach? In general, Tanzanian teachers display poor pedagogical skills. Few teachers (less than 1 in 3) deploy the teaching practices identified in the literature as promoting learning in their lessons —structuring, planning, asking lower and higher order questions and giving feedback. To assess how well teachers teach we measure: (i) teachers’ pedagogical knowledge, (ii) teachers’ capacity to assess students and monitor their progress, and (iii) teachers’ classroom practices based on direct lesson observation. First, to measure pedagogical knowledge, we asked teachers to prepare a lesson plan by reading and extracting information from a factual text on a topic and to state what they would expect their students to learn from the lesson. While teachers struggle to read and understand the text (average score of 35 percent), they struggled even further to formulate what they wanted children to learn from the lesson based on their reading (average score of 17.6 percent). These results suggest that most Tanzanian teacher lack basic pedagogical skills. Second, to measure teachers’ ability to assess students’ learning and give feedback, teachers were asked to: (i) prepare lower- and higher-order questions, (ii) use a marking scheme to give feedback on strengths and weaknesses in students’ writing, and (iii) use a list of students’ grades to turn the raw scores into averages and comment on the students’ learning progression. As with general pedagogical knowledge, the results show that few teachers demonstrated an ability to assess student learning and respond to that assessment. Around 1 out of 3 teachers could formulate questions that checked basic understanding based on what they had read, and another 18 percent could formulate a question that asked students to apply what they had learned to other contexts. Just 1 out of 5 teachers could give feedback on strengths and weaknesses in students’ writing using a marking scheme. Furthermore, almost 1 out of 3 teachers (28 percent) could monitor and comment on the learning progression of students. Lastly, poor knowledge of general pedagogy was also mirrored in teaching behavior in the classroom. Inside the classroom, many teachers deploy some of the teaching practices identified in the literature as promoting learning, but few apply the full set of beneficial skills—structuring, planning, asking questions, creating a positive environment and providing constructive feedback—in their lessons. A subset of 112 Standard 4 classrooms were additionally observed using the Classroom Assessment Scoring System (CLASS). Tanzanian teachers scored relatively high in Classroom Organization, but they struggled between low and medium in Instructional and Emotional support. Despite the poor skills in pedagogical knowledge, the 2016 SDI data suggest that Tanzanian teachers have slightly improved their teaching skills since 2014. For instance, the proportion of teachers who were able to read and extract information from a factual text increased from 26 percent to 35 percent and the proportion of those that could formulate what they would expect their students to learn also increased from 13 to 18 percent. Similarly, teacher’s ability to assess students improved in terms of checking for understanding, formulating questions to apply to other contexts and evaluating student’s progress. Comparing results internationally, we find that Tanzanian teachers fall behind the average teacher in SSA both in text comprehension and formulating aims and learning outcomes, but they overperformed the average teacher in SSA in terms of assessing students and teaching skills. With respect to their East African neighbors, they outperformed Ugandans and Mozambicans, but they underperformed Kenyans. 49 Section I. What do teachers do? Understanding teachers effective use of time To measure teacher effort, we use observational data on teacher absence and time spent teaching. Being present in the classroom is a necessary (though not sufficient) condition for teaching and learning to happen, and absence is therefore one measure of the effort that a teacher puts into teaching. A classroom with no teacher is an environment where no learning is taking place. High levels of teacher absence severely impair students’ ability to learn and result in significant losses of class time during the school year (Chaudhury, et al., 2006; Bruns & Luque, 2014; Lavy, 2015). A recent study in seven Sub Saharan African countries found that 44 percent of teachers were absent from class, either because they were absent from school, or were in the school but not in the classroom (Bold, et al., 2017). And absence matters: experimental research In India has found that reducing teacher absence by 21 percent increased students’ learning by 0.17 SD (Duflo, et al., 2012). To measure the time teachers spend teaching, an extended approach of that described in (Chaudhury, et al., 2006; Bold, et al., 2017) was employed. In each school, during a first announced visit, up to 10 teachers were randomly selected from the teacher roster. At least two teaching days after the initial survey, an unannounced visit was conducted, during which the enumerators were asked to identify whether the selected teachers were in the school, and if so, if they were in class teaching. Both assessments were based on directly observing the teachers and their whereabouts. There are three indicators designed to capture the effort teachers put into their job. These indicators are: (a) school absence rate, (b) classroom absence rate, and (c) time spent teaching per day. The rationale behind these indicators is that the low levels of accountability and weakened incentives observed in many countries especially in Sub-Saharan Africa have led to an upsurge of no-show for teachers. School Absence Rate, Classroom Absence Rate and Time Spent Teaching School absence rate is measured as the share of teachers who are absent from school at the time of an unannounced visit. As shown in Figure 2.1, teacher’s school absence rate is relatively low with approximately one out of eight (13 percent) not present at school at the time of the surprise visit. 30 Teachers in rural schools are equally likely to be absent from school compared to their urban colleagues (Figure 2.1). Figure 2.1: Teacher Effort: What do Tanzanian teachers do? 0.5% In classroom - teaching 13% In classroom - not teaching In school - not in the classroom 29% 51.5% In school - teaching outdoors 6% Absent from school Note: This figure shows the population average for teachers’ use of time for 400 public schools in Tanzania. It includes 3,642 teachers. To compute these numbers, we use a teacher weight to represent the average teacher in Tanzania. Absence is calculated based on an unannounced visit. Source: Tanzania SDI 2016 Classroom absence rate is measured as the share of teachers not in the classroom at the time of an unannounced visit. This indicator is constructed in the same way as the school absence rate indictor, with the exception that the enumerator now is the number of teacher who are either absent from school, or present at school but absent from the classroom. As shown in Figure 2.1, close to one out of three (29 percent) teachers found in school are not in the classroom teaching. This brings the teacher absenteeism from classroom to 42 percent nationally. This simply means that at any point in time, almost half of Tanzanian primary teachers are outside the classroom and are thus not teaching. Given that expenditure on teachers represents by far the largest share of education spending in developing countries including Tanzania, this very high absence from classroom clearly constitutes an important waste of time and resources with half of the time of teachers not utilized interacting with 30 The majority of the surprise visits took place during the morning with roughly 70 percent of the enumerator arriving before 12 a.m (the mode of arrival is between 9-10 a.m.) The surprise visit lasted 45 minutes on average. 50 their students. Interestingly, absence from classroom does not vary much across regions. In other words, teacher’s classroom absence rate in urban schools is not statistically different from those in rural schools (Figure 2.2). In terms of gender, we found that male and female teachers were equally likely to be absent (from school and the classroom). Where were the teachers at the time of the unannounced visit? Figure 2.1 provides an answer to this question by showing teacher’s whereabouts during the surprise visit. About half of teachers (51.5 percent) are teaching at the time of the unannounced visit, some of them (0.5 percent) outside of the classroom. Although classroom absence rate is at 42 percent, only 52 percent of teachers were actually teaching. Indeed, a non- negligible share (6 percent) of teachers was in class but attending to other matters than teaching. Figure 2.2: Trends in teacher school absence and classroom absence 80% 70% 68% 60% 53% 50% 50% 47% 46% 47% 42% 41% 42% 40% 36% 30% 23% 20% 20% 14%13% 15% 14%13% 12% 10% 0% School absence Classroom School absence Classroom School absence Classroom absence absence absence Overall Urban Rural 2010 2014 2016 Note: The figure reports the school and classroom absence rate for all teachers in Tanzanian public school by urban/rural schools using teacher weights to represent the average teacher in Tanzania. Teachers are marked as absent from school if during the second unannounced visit, they are not found anywhere on the school premises. Otherwise, they are marked as present. Teachers are marked as absent from class if during the second unannounced visit, they are absent from school or present at school but absent from the classroom. Otherwise, they are marked as present. Source: Tanzania SDI 2010, 2014 and 2016 What do these results imply for the amount of instruction time that students receive? Time spent teaching per day reflects the typical time that teachers spend teaching on an average day. This indicator combines data from the teacher roaster module (used to measure absence rate), the classroom observation module, and reported teaching hours. The teaching time is adjusted for the time teachers are absent from the classroom, on average, and for the time the teacher teaches while in classroom based on classroom observations. While inside the classroom distinction is made between teaching and non-teaching activities. The amount of time a teacher spends teaching in a school during a normal day was 3 hours and 55 minutes in Tanzania for the 2015/2016 school year. This means that teachers taught only about half of the scheduled time which is around 7 hours accounting for break times. We multiplied this number by the proportion of teachers absent from class. The idea is that if 10 teachers were supposed to teach 7 hours per day, but one of them was nowhere to be found in school, then the scheduled teaching time is reduced to 6 hours and 13 minutes. In this case, given that absence from class is 42 percent, the amount of time a teacher spends teaching is approximately 4 hours (7 hours x 0.58). Moreover, even when in the classroom, teachers may not necessarily be teaching. We carried out classroom observation as part of the survey, recording ten snapshots of what the teacher was doing, for a randomly selected fourth-grade Mathematics or Language class. The percentage of 51 the lesson lost to non-teaching activities was 44 percent. We then combine the absence-adjusted teaching time with the proportion of classroom time devoted to actual teaching activities to estimate instruction time as experienced by students. Students are taught, on average, 3 hours and 55 minutes per day, roughly 56 percent of the schedule teaching time (as shown in Figure 2.3). There are no major differences in learning time across urban/rural schools. Students in rural schools seem to receive slightly more time learning compared to their urban peers but this difference is not statistically significant. Above we documented that there was a high teacher classroom absence that is generally not approved, nor any notification is given to the school. One possible explanation of this pattern could be lack of accountability in the school. To explore this, we studied whether absence among teachers was related to principal’s absence. We found that when a principal was present at the school, the average school absence rate among teachers was 12 percent and the average classroom absence was 41 percent. However, when the director was absent, the average school absence rate for teachers doubled to 23 percent and the classroom absence increased to 47 percent (Figure 2.3). This implies that in schools where the principal was absent, teachers were almost twice as likely to be absent, indicating the importance of principal performance in a well-functioning school. Figure 2.3: Teacher school absence and classroom absence by presence of principal in the school 50% 47.0% 41.2% 40% 30% 22.9% 20% 12.0% 10% 0% School absence Classroom absence Principal absent from school Principal present in school Source: Tanzania SDI 2016 The improvements in learning outcomes observed in 2016 reflect a certain improvement in service provision, in particular in teacher effort. Teachers are less likely to be absent from class and they are more likely to be in class teaching. The 2016 data demonstrates significant improvements in teacher presence. For instance, the share of teachers found absent from the classroom declined from 53 percent in 2010 to 43 percent in 2016. The share of teachers actually found teaching increased from 49 percent in 2014 to 61 percent in 2016. In addition, the time spent teaching per day increased from around 3 hours to almost 4 hours (Figure 2.4). Similarly, the share of “orphan classrooms” (classrooms with students inside but no teacher present) declined from 38 percent in 2014 to 34 percent in 2016. This suggests a significant increase in teacher motivation. 31 In terms of regional differences, teacher effort does not vary much across regions. The proportion of teachers found teaching rose more slowly in Dar es Salaam than in EQUIP-T or other rural regions. And, in contrast to the rest of the country, the share of orphan classrooms in Dar es Salaam actually rose slightly, from 36.8 percent to 37.6 percent. Despite this impressive improvement, one must still keep in mind that Tanzanian students are still losing half of the teaching time owed to them by the education system. 31 This may in part reflect the impacts of the EPforR, in which teacher motivation is a central area of focus. EPforR supports teacher motivation through clearance of backlog of non-salary financial claims, and performance-based School Improvement Grants to provide an incentive for school performance. 52 Figure 2.4: Trends in time spent teaching per day 3h 55m 4h 3h 50m 2h 47m 2h 50m 2h 40m 2h 04m 2h 11m 1h 24m Overall Rural Urban 2010 2014 2016 Note: The figure represents teacher’s time spent teaching per day using teacher weights. Time spent teaching adjusts the length of the school day by the share of teachers who are present in the classroom, on average, and the time the teacher spends teaching while in the classroom. Source: Tanzania SDI 2010, 2014 and 2016 53 Box 2.1 : Profile of Tanzania’s Teachers Table 2.2 shows characteristics of the average Tanzanian teacher. On average, Teachers Educational Level Tanzanian teachers are 36 years old with the youngest being 22 and the oldest 2.4% 0.2% 59. Half of them are female and almost all of them work as contract teachers in 3.9% 5.3% Primary complete public schools and have completed a teacher training. Very few teachers work 5.0% in private schools. Secundary O Level Secundary A Level They have, on average, 11 years of education, with the vast majority having Diploma/certificate completed ordinary secondary level (83.2 percent) (i.e. four years after primary University/bachelor level), a small number of teachers having only completed primary school, 5 83.2% University/master percent completing advanced secondary level, another 5.3 percent having a diploma or certificate and only 3 percent have completed additional formal education (university degree). Source: Tanzania SDI 2016 Table 2.1: Profile of the Average Tanzanian Teacher Overall Urban Rural Regional Gap Female (%) 52.0 78.8 44.3 34*** Public (%) 99.1 98.1 99.6 1.5*** Private (%) 0.9 1 0.3 0.7*** Training (%) 99.3 98.5 99.7 1.2*** Salary delays (%) 4.2 2.6 5.2 2.6*** Age 36.2 40.3 33.4 6.9*** Years of education 11.3 11.6 11.1 0.5*** Years of experience (teachers) 10.8 14.4 9.3 5.1*** Years of experience (principals) 15.9 20.7 15.0 4.8*** Hours per day teaching 4h 15min 4h 4h 30min 30min Source: Tanzania SDI 2016 Tanzanian teachers have an average of 11 years of experience, but this masks the fact that many of them are new and some have more than 30 years of experience. On the other hand, principals have on average 5 more years of experience compared to teachers, but they do not differ in their level of education compared to teachers (Figure 2.5). Teachers teach around 4 hours a day, with 1 hour being the minimum and 9 hours the maximum. In Tanzania, 4 percent of teachers reported receiving their salary with delays. When analyzing teacher’s position by gender, we find that teachers are more likely to be female compared to deputy head teachers and principals. For instance, within the teacher position half of teachers are female but within the principal position only one in four principals are female (Figure 2.6). 54 There are many statistically significant differences among rural and urban teachers in Tanzania. Urban teachers are in general older, they have more years of education, more years of teaching experience and they are more likely to be female compared to rural teachers. Figure 2.5: Distribution of teaching years of experience by position at school Source: Tanzania SDI 2016 Figure 2.6: Position at school by gender 100% 80% 75% 58% 60% 52% 48% 42% 40% 24% 20% 0% Principal Deputy Head teacher Teachers Male Female Source: Tanzania SDI 2016 A 55 Section II. What do teachers know? Measuring teachers’ content knowledge in Language and Mathematics Even if teachers show up to school and spend the allocated time in the classroom engaging in teaching activities with their students, they need to have a fairly good command of the subject they teach as well as the required pedagogical skills to effectively pass the knowledge to their students. Research from low- and middle-income countries has found that students who are exposed to teachers with higher content knowledge learn substantially more (Metzler & Woessmann, 2012; Bietenbeck, et al., 2018; Bold, et al., 2018). In order to measure the subject content knowledge of primary school teachers, and specifically those teaching in the lower primary grades, all Language and Mathematics teachers teaching Standard 4 in the current year (or Grade 3 in the previous year) were assessed. The objective of the teacher test is to examine whether teachers have the basic reading, writing and arithmetic skills that lower primary students need to have to progress further with their education. On average, up to 10 teachers in each school were selected to complete an assessment of subject knowledge and pedagogical skills. In contrast to other approaches to assess teachers’ knowledge, where teachers take exams, teachers here were asked to mark (or “grade”) mock student tests in Language and in Mathematics. 32 This method of assessment has two potential advantages. First, it aims to assess teachers in a way that is consistent with their normal activities, namely marking student work. Second, by not testing teachers in the same way as students are tested, it recognizes teachers as professionals. In the analysis, we assess the Language knowledge of those teachers who teach Language, and the Mathematics knowledge of those teachers who teach Mathematics, although they might teach other subjects as well. Teacher’s Content Knowledge in Language What is the structure of the teacher’s Language Test? We start by assessing language tasks on the teacher test that covered the lower primary curriculum (first through third year of primary school) – specifically, spelling and simple grammar exercises. Possessing knowledge equivalent to the fourth-grade curriculum is not sufficient to teach language in lower primary. That is to say, teaching a student how to compose even a simple text requires knowledge that goes well beyond what is graded in the curriculum. We therefore measure teachers’ ability to correct children’s work in such aspects of literacy as reading comprehension, vocabulary, and formal correctness (grammar, spelling, syntax, and punctuation), all of which are competencies a teacher in lower primary would routinely be required to use. To this end, the language test contained (in addition to the spelling and grammar exercises) items involving sentences with blank spaces where students need to fill in words—so-called cloze passages—to assess vocabulary and reading comprehension, and a letter written to a friend describing the student’s school, which the teacher had to mark and correct. The performance on the Language test was quite low, with few teachers mastering the content knowledge in Language (37 percent). Figure 2.7 presents the average score on the teacher’s Language test, as well as detailed analysis of particular questions. The average score on the Language test is 37 percent indicating that less than half of Tanzanian teachers master the Standard 4 curriculum. In terms of regional gap, urban teachers performed slightly better than rural teachers in the Language test, but on average this difference is not statistically significant. First, more than a quarter of teachers could not correctly answer a grammar exercise that asked them to complete sentences with the correct conjunction, verb (active or passive voice and different tenses) or preposition. Four alternatives, including the correct one, were given for each sentence. For instance, teachers would complete a sentence such as “[___] [Does, Where, How long] does it take to walk to this school?”. Despite teachers scored on average 69 percent on the grammar assessment, there were still some serious gaps. For example, more than half of the teachers (53 percent) were not able to correct the sentence “If I were a doctor, I [___] (will, would, shall, am able to) work in a hospital.”, even though the correct alternative “would” was given. Second, most teachers struggled with those tasks that required at least some knowledge beyond the lower primary curriculum to mark. Less than half of the items in the cloze passage were marked correctly (48 percent). The exercise consisted of a short story with certain words removed and the teacher had to fill the gaps in a meaningful way, which included “student” responses such as “[Where] do I have to go to the market?” (In this case, a correct answer could be either “Why or When.”). While teachers were able to confirm that students had answered correctly, they struggled to correct wrong answers or complete sentences that the student had left blank. For example, 75 percent of teacher could not correct the sentence “I want not to go to school”. 32The subject test was designed by experts in international pedagogy and validated against 13 low income countries’ primary curricula and national teacher standards (Botswana, Ethiopia, Gambia, Kenya, Madagascar, Mauritius, Namibia, Nigeria, Rwanda, Seychelles, South Africa, Tanzania, and Uganda). See (Johnson, et al., 2012) for details. The test was validated against the Tanzanian primary curriculum, as well as the 12 other Sub-Saharan curricula. 56 Third, Tanzanian teachers presented their worst performance on the composition exercise with an average score of only 17 percent. In this exercise, teachers were asked to correct for grammar, punctuation, spelling, syntax and salutation mistakes in a child’s letter that included segments such as “I went to tell you that my new school is better the old one”. Overall, the text to be corrected consisted of 21 errors and the teachers, on average, caught 3 mistakes. Figure 2.7: Trends in Teachers’ Content Knowledge in Language by Type of Task 0% 10% 20% 30% 40% 50% 60% 70% 80% Language test 42% 37% Grammar task 73% 69% Cloze task 53% 48% Composition task 21% 17% 2014 2016 Note: The figure presents the average score on the Language test, and the percentage of teachers able to perform various language tasks (grammar, cloze and composition task), for teachers in public schools teaching Standard 4 or who taught Standard 3 in the previous year. All teacher statistics are calculated using teacher-specific sampling weights. Results for teacher content knowledge are based on observations from around 2100 randomly sampled teachers. Refer to Table B. 7 for more detail. Source: Tanzania SDI 2014 and 2016 Teacher’s content knowledge in Language has weakened over time. When comparing the 2016 SDI with the 2014 SDI results, we find that Tanzanian teacher’s skills in subject knowledge in Language have been slightly deteriorated. For instance, teacher’s test score was reduced from 42 to 37 percent. In addition, scores on grammar tasks and cloze tasks decreased from 73 to 69 percent and from 53 to 48 percent, respectively. The share of teachers who could correct a composition exercise also decreased from 21 to 17 percent. Teacher’s Content Knowledge in Mathematics What is the structure of the teacher’s Mathematics Test? In Mathematics, we assess teachers’ subject knowledge by checking whether they can accurately correct children’s work in such aspects of numeracy as manipulating numbers and using whole number operations (i.e. double digit addition, double digit subtraction, algebra, interpret a graph, word problem etc). In essence, this test measured whether the Mathematics teacher masters his or her students’ curriculum and also assessed their ability to identify and suggest a correct answer. The average score on the Mathematics test is 65 percent indicating that more than half of Tanzanian teachers master the Standard 4 curriculum. Looking at specific tasks in Mathematics, Tanzanian teachers were much more at ease with simple operations such as double-digit addition or double-digit subtraction, than with slightly more complex computations such as interpreting a graph or a Venn diagram. Figure 2.8 presents the average score on the teacher’s Mathematics test, as well as detailed analysis of particular questions. In particular, almost all teachers could add double digit numbers. However, 15 percent of the teachers could not subtract two-digit numbers; the same proportion could not multiply two digit numbers. Of course, we would expect a competent math teacher to have knowledge beyond that of his or her students; therefore, the Mathematics test also included questions one would only encounter in upper primary school. Many Mathematics teachers struggled with these tasks: one in four teachers could not interpret information in a Venn diagram, only half of teachers could solve an algebra exercise and only 41 percent could interpret data from a chart. Moreover, one in four teachers could not solve a simple math story (word problem), while half of them could not solve a difficult math story. 57 Figure 2.8: Trends in Teachers’ Content Knowledge in Mathematics by Type of Task 0% 20% 40% 60% 80% 100% Mathematics test 63% 65% Double digit addition 97% 95% Double digit subtraction 86% 85% Double digit multiplication 85% 83% Venn Diagram interpreation 49% 69% Algebra 51% 55% Graph interpretation 28% 41% Simple math story problem 76% Difficult math story problem 47% 2014 2016 Note: The figure presents the average score on the Mathematics test, and the percentage of teachers able to perform various Mathematics tasks for teachers in public schools teaching Standard 4 or who taught Standard 3 in the previous year. All teacher statistics are calculated using teacher-specific sampling weights. Results for teacher content knowledge are based on observations from around 2100 randomly sampled teachers. Refer to Table B. 8 for more detail. Source: Tanzania SDI 2014 and 2016 Teacher Mathematics subject knowledge has improved rapidly between 2014 and 2016. The SDI results overtime suggest significant improvements in many key skills between 2014 and 2016, particularly in Mathematics. Scores on exercises assessing skills in interpretation of graphs and Venn diagrams increased from 28 percent to 41 percent and from 49 percent to 69 percent respectively. The share of teachers who could correctly solve a simple algebraic equation increased from 51 percent to 55 percent. 33 The performance on the Mathematics test was better than the one in Language, with more than half of teachers mastering the content knowledge in Mathematics (65 percent). Figure 2.9 shows the distribution of teacher’s test scores in Language and Mathematics. There was a wide variation in the distribution of the teachers’ performance on the test, especially for Mathematics. Language test scores were consistently lower than teachers’ Mathematics test scores, with the largest share of teachers scoring below 40 percent. In terms of regional gap, urban teachers are statistically indistinguishable from rural teachers in terms of their subject knowledge in both Mathematics and Language. 33These improvements may in part be due to the impact of EPforR, LANES and EQUIP-T, all of which include activities relating to training of teachers in literacy, numeracy and pedagogy; more than 50,000 teachers have received training from EPforR alone. 58 Figure 2.9: Distribution of the teacher test scores in Language and Mathematics Section III. How well do teachers teach? Teachers’ pedagogic content knowledge and teaching skills Pedagogical Knowledge Good teaching is not just about subject content knowledge. It also requires that teachers know how to translate their subject knowledge into effective pedagogy and then apply this in the classroom. There is a broad agreement that for teaching to be effective, lessons must be well- designed and well-structured. Teachers must also know how to assess student capabilities and react appropriately, for example, by asking questions that require various types of responses and by giving feedback on those responses, commonly referred to as “knowledge of the context of learning” (Johnson, 2006; Danielson, 2011; Pianta, et al., 2007; Coe, et al., 2014; Ko & Sammons, 2014; Muijs, et al., 2014; Vieluf, et al., 2012). In a recent review, although not focused on Tanzania specifically, (Muijs, et al., 2014) identify a set of elements that categorize behavior in the classroom that are consistently associated with gains in student learning. 34 They are: i) structuring lessons, and in particular, introducing topics and learning outcomes at the start of the lesson and reviewing them at the end; ii) frequently checking for student understanding by asking questions and allowing time for students to review and practice what they learned, either individually or in groups; iii) varying the cognitive level of questions by mixing lower- and higher-order questions; and iv) creating a positive learning environment and providing substantive feedback to students by acknowledging correct answers in a positive fashion and correcting wrong answers, as skills and practices that are consistently associated with gains in student learning. To assess how well teachers teach, we measure (i) teachers’ pedagogical knowledge, (ii) teachers’ capacity to assess students and monitor their progress, and (iii) teachers’ classroom practices based on direct lesson observation. To measure general pedagogical knowledge, we asked teachers to prepare for a lesson with a specified topic by reading and extracting information from a factual text on that topic (general content knowledge) and to state (in 1-2 sentences) what they would expect their students to learn from the lesson. Both these tasks are consistent with professional tasks normally expected of primary teachers. To measure teachers’ ability to assess students’ learning and give feedback (which we shorten here to “assessing students”), teachers were asked to prepare questions that required students to recall what was learned (lower order) and questions that asked students to apply the material to 34 (Araujo, et al., 2016) show that teacher behaviors, as measured by the CLASS, are significantly associated with learning in math, language, and executive function. These findings support the conclusions from two recent reviews (Kremer, et al., 2013; Murnane & Ganimian, 2014), both of which argue that changing pedagogical practices is the key to improving learning outcomes in the developing world. (Kane & Staiger, 2012), from the MET project, also show that students randomly assigned to teachers with higher-quality interactions learn more in the United States. 59 new contexts (higher order) on the basis of a reading of a factual text. In a second task, teachers were asked to use a marking scheme to give feedback on strengths and weaknesses in students’ writing and to distinguish weak and strong learners. In a third task, teachers were provided with a list of students’ grades; they were then asked to turn the raw scores into averages and to comment on the learning progression of individuals and groups of students with the help of a bar chart. This section consisted of three tasks designed to capture all the skills teachers would routinely be asked to apply when teaching. Figure 2.10: Teachers pedagogical knowledge 0% 20% 40% 60% 80% A. Pedagogical Factual text comprehension 26% knowledge 35% Formuate aims and learning outcomes 13% 18% Formulate questions to check understanding 33% 37% B. Assessing students Formulate questions to apply to other contexts 15% 18% Assessing student's abilities 18% 13% Evaluating student's progress 22% 28% C. Skills and practices in the classroom Lesson appears planned to enumerator 75% Ask a mix of lower and higher order questions 58% Give reviews or collect homework 49% Introduce and summarize topic of the lesson 43% Creates a positive environment in the classroom 30% 2014 2016 Note: The figure presents scores on teacher’s pedagogical knowledge and teaching skills. Panel A reports on general pedagogical knowledge tasks for teachers in public schools in Standard 4 or who taught Standard 3 in the previous year. Panel B reports on specific pedagogical knowledge tasks in assessing students for teachers in government schools in Standard 4 or who taught Standard 3 in the previous year. Panel C presents teacher skills and practices in the classroom in public schools in grade 4. All teacher statistics are calculated using teacher-specific sampling weights. All scores are computed for teachers teaching either subject. Results for teacher pedagogical knowledge are based on observations from around 2100 randomly sampled teachers. Refer to Table B. 9 for more detail. Source: Tanzania SDI 2014 and 2016 To measure teachers’ classroom practices based on direct lesson observation, we measure whether the teacher had a lesson plan and whether this appeared planned to the enumerator. Secondly, we measure whether the teacher asked questions that required students to recall information, to demonstrate their understanding of what they had learned during the lesson, to apply information to new topics and to use their creativity and imagination. Thirdly, we measure whether the teacher either assigned homework to the class, gave reviews or collected homework 60 from the class. Fourthly, we measured whether the teacher introduced topics and learning outcomes at the start of the lesson and whether she reviewed them at the end. Lastly, we also measure whether the teacher called students by name, gave feedback of praise at least once and corrected a mistake at least once and did not scold at mistakes or hit, pinch, or slap any student. Most Tanzanian teachers lack basic pedagogical skills. As reported in Figure 2.10, Panel A, teachers struggle both to read and understand a factual text (average score of 35%), and translate this information into teaching; especially, they had a hard time formulating what they wanted children to learn from the lesson based on their own reading of the text (average score of 17.6% on this task) – ranging from 15% in urban schools to 18% in rural schools. As with general pedagogical knowledge, the results in Panel B show that few teachers demonstrated an ability to assess student learning and respond to that assessment. Around 1 out of 3 teachers could formulate questions that checked basic understanding based on what they had read — ranging from 29 percent in urban schools to 39 percent in rural schools, and another 18 percent could formulate a question that asked students to apply what they had learned to other contexts. Just 1 out of 5 teachers could give feedback on strengths and weaknesses in students’ writing using a marking scheme —ranging from 10 percent in urban schools to 16 percent in rural schools. Furthermore, almost 1 out of 3 teachers (28 percent) could monitor and comment on the learning progression of students. Although most teachers seem to lack basic abilities to assess students, the extent is more serious in urban schools compared to rural schools. Poor knowledge of general pedagogy was mirrored in teaching behavior in the classroom, as shown in Panel C. Less than half of the teachers explained the topic of the lesson at the start and summarized what was learned at the end, and only 1 in 4 lessons seemed unplanned to the observers. During their lessons, 60% of teachers asked questions that required students to recall information or to practice what was learned, but significantly fewer asked questions that required higher-order skills, encouraged students to apply what was learned to different contexts, or be creative. Overall, 33% of teachers mixed lower- and higher-order questions in their class. Around one third of the teachers consistently gave positive feedback and corrected mistakes in response to student answers without scolding students. In summary, general pedagogical knowledge, the ability to assess and respond to students’ learning, the institution of a monitoring system that measures what students know are all poor features across teachers in Tanzania. Inside the classroom, many teachers deploy some of the teaching practices identified in the literature as promoting learning, but few apply the full set of beneficial skills—structuring, planning, asking questions, creating a positive environment and providing constructive feedback—in their lessons. The low scores on the pedagogical skills combined with the performance on the curriculum content imply that teachers know little more than their students and that the little they know, they cannot teach adequately. Despite the poor skills in pedagogical knowledge, the 2016 SDI data suggest that Tanzanian teachers have slightly improved their teaching skills since 2014. For instance, the proportion of teachers who were able to read and extract information from a factual text increased from 26 percent to 35 percent and the proportion of those that could formulate what they would expect their students to learn also increased from 13 to 18 percent. Similarly, teacher’s ability to assess students improved in terms of checking for understanding, formulating questions to apply to other contexts and evaluating student’s progress (Figure 2.10). 61 CLASS Overall Results A subset of 112 Standard 4 classrooms were additionally observed using the Classroom Assessment Scoring System (CLASS), which is an instrument that measures the quality of teaching. More specifically, CLASS measures three broad domains of teaching practices: teachers’ emotional support, classroom organization, and instructional support. Each of these domains is broken down into sub-categories to create 11 constructs on which each classroom is assessed. The scores go from 1 to 7, with 7 being the highest. Rather than a checklist tool as the one with which we observed the classrooms reported above, we videotaped classrooms and then coded them by a certified CLASS coder, which allows us to look at the quality of the teacher-student interactions. Figure 2.11: Distribution of CLASS domain scores Instructional Support 6% 33% 38% 16% 6%1% Classroom Organization 1% 25% 68% 6% Emotional Support 6% 27% 36% 26% 5% 0% 20% 40% 60% 80% 100% 1 2 3 4 5 6 7 Source: Tanzania SDI 2016 Tanzanian teachers scored relatively high in Classroom Organization, but they scored between low and medium in Instructional and Emotional support (Figure 2.11). More than one-third of Tanzania’s classrooms struggled with low levels of Instructional Support and the remaining ranged within medium levels of efficiency. One in three Tanzanian teachers did not usually make use of a variety of instructional methods, displayed low levels of instructional dialogue and were not likely to deliver high quality feedback to their students. The area where teachers struggled the most was in providing students with opportunities to think and apply what was learned in a new problem. On average, teachers score 2.84 points out of the 7-points possible in this domain, which makes Instructional Support the lowest scoring domain. Similar to Instructional Support, one third of Tanzanian teachers displayed low levels of Emotional Support, while the remaining 70 percent showed medium levels of Emotional Support. Regard for Student Perspectives, which measures student autonomy and respect and integration of student ideas, was low in over 50 percent of all classrooms. Moreover, one third of the teachers did not show high levels of sensitivity towards students and neither did they consistently promote a positive climate in the classroom; the classrooms ranged from low to medium levels. On average, teachers score 2.89 points out of the 7-points possible in this domain. In terms of Classroom Management, Tanzanian classrooms appeared to be on the higher end of the scale. Three quarters (74 percent) of Tanzanian teachers displayed high levels of Classroom Management. There were very few instances of negative climate, and teachers were observed to perform well on behavior management. Productivity levels varied with a majority scoring in the mid to upper range of the scale. On average, teachers score 5.7 points out of the 7-points possible in this domain. Results on Classroom Management should be interpreted cautiously, as an analysis of the reliability of CLASS for the Tanzanian context showed that while Instructional Support and Emotional Support displayed 62 relatively high levels of internal consistency 35 (0.94 and 0.87 respectively), Classroom Organization showed very low levels (0.14) denoting that this domain is not measuring the same latent variable (good teaching) as the others. 36 This means that teachers that display good teaching practices, as measured by CLASS, do not score differently on Classroom Organization than teachers that display low teaching skills. The reason is that students do not misbehave in Tanzanian classrooms regardless of the quality of the teacher. We have observed the same pattern in a study of CLASS carried out in Afghanistan’s primary schools. Figure 2.12: Distribution of CLASS Dimension Scores I.Emotional Support Positive Climate Teacher Sensitivity Regard for Student Perspectives Behavior Management Management II.Classroom Negative Climate Productivity Instructional Learning Formats III.Instructional Support Content Understanding Quality of Feedback Analysis and Inquiry Instructional Dialogue 0% 20% 40% 60% 80% 100% 1 2 3 4 5 6 7 Source: Tanzania SDI 2016 35 We computed Cronbach’s Alpha .̅ ( = ) as our internal consistency estimator. His indicator estimates how closely related a set of items are as a group. We do �+(−1).̅ not report inter rater reliability as we only used one coder for all videos to guaranteed consistency in the application of the rubric. 36 The low reliability for Classroom Organization is in line with low correlations between this domain and the others. 63 Below we explain each construct and the results (See also Figure 2.12) Domain I: Emotional Support Dimension Tanzanian Classrooms Positive Climate: Reflects the emotional connection While one-fourth of the teachers do not display between the teacher and students and among affection and rarely provide positive comments, most students. of the teachers (76 percent) show some indication of genuine positive affection, and positive comments, but these interactions are brief. Teacher Sensitivity: Encompasses the teacher’s Three quarters of the teachers observed are sometimes awareness of and responsiveness to students’ responsive to students’ needs, although these academic and emotional needs. interactions are not consistent. The remaining one- third of teachers are unresponsive to students’ academic and social needs and fail to notice when students need help. Regard for Student Perspective: Captures the degree More than half of the teachers (55 percent) rigidly to which the teacher’s interactions with students and provide all the structure in the class and do not classroom activities place an emphasis on students’ encourage students’ ideas, nor create the opportunities interests, motivations, and points of view and for meaningful peer-peer interactions. This is the area encourage student responsibility and autonomy where Tanzanian teachers struggle the most in terms of Emotional Support. Domain II: Classroom Management Dimension Tanzanian Classrooms Behavior Management: Encompasses the teacher’s In around 80 percent of the classrooms, no student ability to provide clear behavioral expectations and use misbehavior is observed. effective methods to prevent and redirect misbehavior. Negative Climate: Reflects the overall level of In 95 percent of the classrooms there is no evidence of expressed negativity in the classroom. negativity or punitive control. Productivity: Considers how well the teacher manages Most teachers are on task and provide students with instructional time and routines and provides activities learning activities. However, as explained below, the for students so that they have the opportunity to be quality of the activities is often low and there is more involved in learning activities. time in transition than necessary. Domain III: Instructional Support Dimension Tanzanian Classrooms Instructional Learning Formats: Focuses on the way in Teachers discuss the learning objective but in general it which the teacher maximizes students’ interest, is not clear. There are some opportunities for students engagement, and ability to learn from lessons and to participate in activities, but both seem mildly activities. engaged. 64 Content Understanding: The depth of the lesson In nearly one-fourth of the classrooms, the material and content and the approaches used to help students the class discussion fail to effectively communicate the comprehend the framework, key ideas, and procedures essential attributes of concepts/procedures to in an academic discipline. students, and the focus is primarily on presenting discrete pieces of information. The broader ideas and/or students’ previous knowledge or potential misconceptions are not presented and/or clarified. Quality of Feedback: Assesses the degree to which the In 65 percent of classrooms, feedback is non-existent or teacher provides feedback that expands learning and mechanical. In several instances, upon students making understanding and encourages continued mistakes, the teacher either scolded them or asked participation. them to sit down without any attempt to scaffold student learning or help students deepen their understanding. In the rest of the cases, the teacher occasionally gave feedback to students but not to sufficient depth. Analysis and Inquiry: Assesses the degree to which Most of the teachers (67 percent) present the material students are engaged in higher level thinking skills without providing students with opportunities to apply through the application of knowledge and skills to novel those concepts in novel open-ended tasks. In the and/or open-ended problems. observed classrooms, most of the questions asked from students are either closed-ended or require a short yes/no answer. This is the area where Tanzanian teachers struggle the most in terms of Instructional Support. Instructional Dialogue: Content-focused discussion The class is dominated by teacher talk in 65 percent of among teacher and students that is cumulative, with classrooms. The rest of the teachers use some the teacher supporting students to chain ideas together facilitation strategy that encourages exchanges that in ways that lead to deeper understanding. build on one another; however, these instances are brief and inconsistent. Source: The explanations of the constructs come from the Upper Elementary CLASS manual and is an adaptation from (Coflan, et al., 2018). 65 Box 2.2: Missing in Action in Tanzania In Dar es Salam, Tanzania, students are left unattended with no learning activity for the first 20 minutes of class. When the teacher arrives, he asks the students to independently solve 36+19. During this time, the teacher sits at his desk. After 10 minutes, the teacher asks one student to solve the problem at the board. When the student cannot solve the problem, the teacher became impatient and asks another student to solve it. Source: Tanzania SDI 2016 (CLASS) 66 Box 2.3: Comparing Tanzania’s Teachers Internationally To put results in perspective, Tanzania displayed patterns with relatively low school absence rate (13 percent), but high classroom absence rate with almost half of the teachers not found in classroom at any point in time. Figure 2.13 compares Tanzania’s teachers with teachers from other countries in terms of their absence and effective use of time. Tanzania is doing better than its East African Community neighbors – Mozambique, Kenya and Uganda. However, the three West African countries (Nigeria, Senegal and Togo) have much lower classroom rates. Moreover, the difference between school absence rates and classroom absence rates was also narrower, meaning that when teachers went to school they tended to be in the classroom, although they were not necessarily teaching. Results from other countries out of Africa – Afghanistan, Pakistan and Lao PDR – indicate that Tanzania’s teacher school absence is among the lowest. Despite this, Tanzania shows a considerably higher classroom absence compared to Afghanistan, Pakistan and Laos. Figure 2.13 : Comparing Tanzania’s Teachers Use of Time Teacher School Absence Teacher Classroom Absence Afghanistan 2017 10 Afghanistan 2017 15 Tanzania 2016 13 Pakistan 2018 19 Pakistan 2018 14 Nigeria 2013** 23 Tanzania 2014 14 Lao PDR 2018 26 Kenya 2012 15 Senegal 2011 29 Lao PDR 2018 16 Togo 2013 39 Nigeria 2013** 17 Tanzania 2016 42 Senegal 2011 18 Kenya 2012 47 Uganda 2013 30 Tanzania 2014 47 Togo 2013 31 Mozambique 2014 56 Mozambique 2014 45 Uganda 2013 57 0 10 20 30 40 50 0 10 20 30 40 50 60 70 Bruns & Luque (2014) drawing on data from a large sample of classrooms Mozambique 2014 1h 41m 4h 17m in seven Latin American and the Caribbean countries, show that teachers only spend 52-85% of class time on academic activities (or time on task), Afghanistan 2017 2h 18m 3h 25m implying a loss of potential instructional time equivalent to one day of 2h 30m Kenya 2012 5h 31m instruction per week. Even though Tanzania is one of the countries with the highest amount of scheduled teaching time (almost 7 hours), teachers Tanzania 2014 2h 46m 5h 56m spend only half of class time on learning activities. Nevertheless, when Uganda 2013 2h 56m comparing Tanzania’s 2016 SDI results to itself in 2014, we find that 7h 13m Tanzanian teachers were less likely to be absent from class and they Nigeria 2013** 3h 1m 4h 44m were more likely to be in class teaching, which suggests certain Senegal 2011 3h 15m4h 36m improvement overtime. For instance, the share of teachers found Togo 2013 3h 15m 5h 28m absent from the classroom declined from 47 percent in 2014 to 43 percent in 2016 and the proportion of teachers found teaching Tanzania 2016 3h 55m 7h increased from 49 percent in 2014 to 61 percent in 2016. Lao PDR 2018 4h 08m 5h 18m Note: The figures report the absence rate for all teachers in Tanzanian public school, and the scheduled teaching time and actual teaching time for all public schools. All teacher Time spent teaching per day Scheduled teaching time statistics are calculated using teacher-specific sampling weights. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013, Kenya SDI 2012, Afghanistan SABER SD 2017, Laos SABER SD 2018 and Punjab SABER SD 2018. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. 67 Figure 2.14 compares Tanzania’s teachers with teachers from other countries in terms of their content knowledge in Language and Mathematics. Tanzanian teacher ranked almost last on Language among the Sub Saharan African countries, with only Mozambique performing lower. This is certainly due to the fact that they were tested in English, although they were supposed to have mastered that language. Table 2.2 provides information on the average scores of the different parts of the Language test, as well as more details. The composition task proved to be the hardest for all teachers, but this was also the area where the gap between Tanzanian teacher and their East African neighbors (Uganda and Kenya) was the largest in the Language test, except for Mozambique. Figure 2.14 : Comparing Tanzania’s Teachers Content Knowledge Internationally Language Teachers Test Score Mathematics Teachers Test Score Mozambique 2014 32 Togo 2013 33 Tanzania 2016 37 Mozambique 2014 37 Tanzania 2014 42 Nigeria 2013** 42 Nigeria 2013** 49 Tanzania 2014 63 Togo 2013 50 Uganda 2013 65 Uganda 2013 58 Tanzania 2016 65 Kenya 2012 65 Kenya 2012 81 0 20 40 60 80 100 0 20 40 60 80 100 Note: The figures report average teacher test scores in Language and Mathematics. All teacher statistics are calculated using teacher-specific sampling weights. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013 and Kenya SDI 2012. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. Table 2.2: Comparison of teachers’ performance on Language by type of task Language Average SSA (score out of 100) Tanzania 2016 Country Min Max Grammar task 69 79 58 [NIG] 92 [KEN] Cloze task 48 44 27 [TGO] 66 [KEN] Correct composition task 17 26 9 [MOZ] 50 [KEN] Number of teachers 2145 3770 Note: This table presents scores on Language tasks for teachers in public schools teaching grade 4 or who taught grade 3 in the previous year. Language knowledge is computed for teachers teaching language. All individual country statistics are calculated using country-specific sampling weights. The average for all countries, reported under the heading “All,” is taken by averaging over the country averages. The ISO 3-digit alphabetic codes of the countries with the lowest (Min) and highest (Max) score for each item are given in brackets. The figures from Sub Sahara Africa are taken from Bold et al 2017. In terms of content knowledge in Mathematics, Tanzanian teachers performed better than the average teacher in Sub Sahara Africa. They were second only to Kenyans. Tanzanian teachers obtained scores higher than counterparts in Togo, Mozambique and Nigeria and they were comparable to Ugandan teachers. Kenyan teachers are the only ones that performed considerably better than Tanzanian teachers in the Mathematics test, with almost 15 percentage points of difference. Table 2.3 provides information on the average scores of the different parts of the Mathematics test, as well as more details. When analyzing teacher’s performance by type of task, we find that Tanzanian teachers performed very similar to their Kenyan counterparts in task with lower level of difficultly, however they lost ground for the upper primary part of the test. For instance, Tanzanian teaches had more difficulty in interpreting data on a graph or Venn diagram. 68 Table 2.3: Comparison of teachers’ performance on Mathematics by type of task Mathematics Average (score out of 100) Tanzania 2016 SSA Teacher Min Max Can add double digits* (%) 82 91 75 [TGO] 98 [KEN] Can subtract double digits (%) 90 77 59 [NGA] 94 [TZA ‘14] Can multiply double digits (%) 78 68 44 [MOZ] 89 [SEN] Can solve simple math story problem (%) 67 55 17 [MOZ] 91 [SEN] Understands a Venn diagram (%) 59 41 19 [TGO] 70 [KEN] Can interpret data in a graph (%) 36 25 12 [TGO] 62 [KEN] Can solve algebra (%) 44 35 3 [MOZ] 74 [KEN] Can solve difficult math story problem (%) 47 15 7 [SEN] 22 [TZA ‘14] No. of teachers 2145 3957 Note: The table presents scores on Mathematics tasks for teachers in public schools teaching Grade 4 or who taught Grade 3 in the previous year. Mathematics knowledge is computed only for teachers teaching Mathematics. All individual country statistics are calculated using country-specific sampling weights. The average for all countries, reported under the heading “All,” is taken by averaging over the SSA country averages. The ISO 3-digit alphabetic codes of the countries with the lowest (Min) and highest (Max) score for each item are given in brackets. The figures from Sub Sahara Africa are taken from Bold et al 2017. Table 2.4 compares Tanzania’s teachers with teachers from other countries in terms of pedagogical knowledge and skills in the classroom. In terms of general pedagogical knowledge, Tanzanian teachers fall behind the average teacher in SSA both in text comprehension and formulating aims and learning outcomes. In terms of assessing students, teachers in Tanzania perform slightly better than the average teacher in SSA. More specifically, they outscored all others in evaluating student’s progress, but they lost ground on assessing student’s abilities. Table 2.4: Comparison of teacher’s performance on pedagogical knowledge and skills in the classroom Tanzania 2016 Average SSA Min Max Panel A: Pedagogical knowledge Factual text comprehension (0-100) 35 47 23 (MOZ) 78 (TZA) Formulate aims and learning outcomes (0-100) 18 23 11 (NGA) 41 (TZA) Panel B: Assessing students Formulate questions to check understanding (0-100) 37 23 5 (NGA) 55 (KEN) Formulate questions to apply to other contexts (0-100) 18 7 3 (NGA) 15 (TZA ‘14) Assessing students’ abilities 13 19 8 (NGA) 39 (KEN) Evaluating students’ progress 28 12 5 (NGA) 26 (KEN) Panel C: Skills and practices in the classroom Introduce and summarize topic of the lesson (%) 43 41 16 (MOZ) 62 (KEN) Lesson appears planned to enumerator (%) 75 - - - Ask a mix of lower and higher order questions (%) 58 31 14 (MOZ) 44 (UGA) Positive environment and feedback (%) 30 - - - Note: The figures from Sub Sahara Africa are taken from Bold et al 2017. 69 Lastly, in terms of teaching skills and practices in the classroom, Tanzanian teachers outperformed the average teachers in SSA. With respect to their EAC neighbors, they outperformed Ugandans and Mozambicans, but they underperformed Kenyans. For instance, they are one of the best at introducing and summarizing the topic of lesson and also at asking a mix of lower and higher order questions, except for Kenya. Furthermore, when using the classroom observation tool CLASS to compare countries in terms of teaching practices, we find that Tanzanian teachers fared at par with Chile and outperformed China and Afghanistan. However, teaching practices in Tanzania lost ground when compared to the U.S, Germany and Finland (Figure 2.15). More specifically, Tanzania scored low in Emotional and Instructional support, but they scored high in Classroom Management compared to the other countries. Figure 2.15 : Cross Country Comparison of CLASS scores CLASS score Afghanistan 3.33 China 3.73 Tanzania 3.83 Chile 3.84 U.S 4.2 Germany 4.21 Finland 4.87 1 2 3 4 5 6 7 70 Chapter 3 : School Management This chapter provides descriptive evidence on school governance and school management quality for primary schools in the context of Tanzania. Evidence suggests that effective school management and leadership, along with community monitoring, can play a crucial role in school performance only if they affect the interaction between teachers and students. Schools with better management have better learning outcomes (Bloom, et al., 2015). Effective leadership means having school principals who are actively involved in helping teachers solve problems, including by providing instructional advice. It also means having principals who set goals with teachers to prioritize and achieve high levels of learning (World Bank, 2018). Using nationally representative data from 2016 Tanzania Service Delivery Indicators (SDI) Survey and the 2016 Tanzania Development-World Management Survey we answer two basic questions: what characterizes school governance and accountability in Tanzanian primary schools? and what school management practices do Tanzanian principals follow? We also compare school management practices in Tanzania to other comparable countries and to those in another sector in the country (e.g. manufacturing sector). The 2016 Tanzania SDI survey was carried out in a nationally representative sample of 400 public schools across 22 regions. It is important to mention that the information provided by the SDI is very limited since this survey was not intended to capture information on school management, school governance or community involvement. For this reason, we use the 2016 Tanzania DWMS survey, which ran face-to-face interviews between enumerators and principal/head teachers in a random sample of 100 public schools and was carried out in partnership with the World Bank and the national government. What characterizes school governance and accountability in Tanzanian primary schools? Even though school principals and head teachers displayed a pattern of relatively low absence rates, school governance and accountability mechanisms are very inefficient. In Tanzania, the principal absence rate is observed to be around 13.6 percent, with rural schools having significantly higher principal absence rate (14.3 percent) than urban schools (7.7 percent). Despite almost all Tanzanian schools (97 percent) have a school management committee or board of directors, the lack of autonomy and capacity prevents school management committees from improving service delivery. Ineffective school leadership can translate into lower accountability for teachers and an ineffective use of resources. More than one-quarter of schools (30 percent) did not receive any supervision visits from an official quality assurance officer or a school inspector during the last academic year, which is very concerning in terms of performance monitoring. For those schools that had supervision visits, the vast majority received only 1-2 visits during an academic year. Lastly, in terms of methods puts in place to recognize teacher performance (e.g. prize, award), only 60 percent of schools have at least one method to recognize teacher’s performance. What school management practices do Tanzanian principals follow? Tanzanian schools scored, on average, 1.8 in the five-point scale of the management index of the D-WMS, indicating an establishment with practically no structured management practices or very poor management practices implemented. Using the 2016 Tanzania D-WMS we investigated the adoption of 14 basic management practices, where the level of adoption is evaluated against a scoring grid that ranges from one (worst practice) to five (best practice). A high score indicates that a school adopts structured and well-defined strong management practices. The main measure of management practices (the management index) represents the average of the scores across all questions. When we analyze the distribution of the school management practices, we find that the management scores in Tanzania are shifted completely to the lower end of the distribution, with the vast majority of scores below 2 and no school scores above 3, suggesting that principals or head teachers in primary schools lack autonomy and authority in decision-making, have very weak management practices, almost no monitoring, very weak targets (e.g. only an annual school-level target) and extremely weak incentives (e.g. tenure based promotion, no financial or non-financial incentives and no action taken about underperforming teachers). Among the participating countries in the WMS and DWMS, Tanzania had the lowest management index and management capacity is substantially lower in schools than in manufacturing firms. 71 Section I. School Governance and Accountability This section provides a short insight on school governance and accountability in the context of Tanzania using Tanzania SDI Survey 2016. It is important to mention that the information provided here is very limited due to the fact that this survey was not intended to capture information on school management, school governance or community involvement. Principal absence rate in Tanzania is relatively low at 13.6 percent, while absence rate for deputy head teachers is lower at 9.9 percent. There are two main school leadership positions in Tanzania: (i) principal and (ii) deputy head teacher or head teacher. 90 percent of schools are led by a principal and a deputy head teacher, 6 percent are led just by a principal, while 4 percent are led only by deputy head teachers or just head teachers. Principal absence rate is measured as the share of school principals (principal and deputy head teacher) who are absent from school at a time of an unannounced visit (second visit). Principals and head teachers who were "at school but not his/her shift" are classified as not absent from school. Figure 3.1 presents principal and head teacher absence rate by urban/rural schools. In Tanzania, the principal absence rate is observed to be around 13.6%, with rural schools having significantly higher principal absence rate (14.3 percent) than urban schools (7.7 percent). Head teacher’s absence rate is significantly lower compared to principal’s absence rate. Figure 3.1 : Principal’s and Head Teacher’s School Absence Rate by Rural/Urban School 20% 13.6% 14.3% 15% 9.9% 10.3% 10% 7.7% 7.6% 5% 0% All Urban Rural Principal Head Teacher Source: Tanzania SDI 2016 Even though almost all Tanzanian schools (97 percent) have a school management committee or board of directors, the lack of autonomy and capacity prevents school management committees from improving service delivery. It should be recognized that school boards effectiveness can contribute towards better school performance. Ineffective school leadership can translate into lower accountability for teachers and an ineffective use of resources. In the context of education this can be quite detrimental to the students since lack of effective school leadership results in a system that could be characterized by the absence of support for teachers to address problems, instructional advice, and goals for the prioritization of learning (Figure 3.2). On average, 70 percent of Tanzanian schools received at least one type of supervision visit from an official quality assurance officer or a school inspector during the last academic year (from the Ministry of Education or the District Education Office). This means that more than one-quarter of schools do not receive any supervision visits in primary schools, which is very disconcerting in terms of monitoring any school management practice. In general, the vast majority of primary schools receive only 1-2 supervision visits during an academic year. This number can range between 0 to 17 visits per year. One third of schools do not receive any supervision visit. Urban schools receive on average 2 supervision visits, while rural schools receive only 1 supervision visit and this difference is statistically significant. (Figure 3.3)Within the group of schools that received at least one supervision visit, only 68 percent received a written feedback. From all the recommendations received from the District Education Office or Ministry of Education, less than half of the feedback received (37 percent) was actually observed from the school staff in written feedback. 72 Figure 3.2 : Frequency of supervision visits per school year Source: Tanzania SDI 2016 Figure 3.3 : School Governance and Supervision Visits 0% 20% 40% 60% 80% 100% School Management Team 97% 3% Supervision Visits 70% 30% Writen Feedback from Visits 68% 32% Teacher Performance Recognition 60% 40% Yes No Source: Tanzania SDI 2016 Lastly, in terms of methods puts in place to recognize teacher performance (e.g. prize, award), only 60 percent of schools have at least one method to recognize teacher’s performance. The lack of policies to promote teacher motivation does not contribute to attracting the best teachers into the profession and neither does it motivate them to improve their instructional skills. 73 Section II. Measuring Education Management Practices in Tanzania Evidence suggests that effective school management and leadership, along with community monitoring, can play a crucial role in school performance only if they affect the interaction between teachers and students. Poor management and governance undermine the quality of education. Research using an extended version of the World Management Survey (WMS) to measure management practices in schools in numerous countries has found a strong positive relationship between school principal’s management quality and education outcomes (Bloom, et al., 2015; Di Liberto, et al., 2014). Though they are unable to establish causality, these studies point to school management as an important institutional characteristic that plays a role in learning. In fact, across the 8 countries included in the (Bloom, et al., 2015), a 1.0 standard deviation increase in an index of management capacity is associated with a 0.23-0.43 standard deviation increase in learning outcomes. Multiple rounds of international assessment data have also shown that decision-making school autonomy along with accountability have positive effects on learning (Woessmann, 2016). However, there is no consensus on the optimal allocation of responsibilities at the school, district, and national level (Adelman, et al., 2018; Bishop & Wößmann, 2004). Effective leadership means having school principals who are actively involved in helping teachers solve problems, including by providing instructional advice. It also means having principals who set goals with teachers to prioritize and achieve high levels of learning (World Bank, 2018). What management practices do Tanzanian principals follow? To measure management practices in the school, a random sample of 100 public schools in Tanzania were scored using the Development-World Management Survey (DWMS) in 2016 (Lemos & Scur, 2016). 37 The Tanzania DWMS 2016 survey was administered to principals and head teachers through face-to-face interviews by enumerators in each school and was carried out in partnership with the World Bank and the national government. 38 The D-WMS survey is designed to investigate the adoption of 24 basic management practices, where the level of adoption is evaluated against a scoring grid that ranges from one (worst practice) to five (best practice). 39 A high score indicates that a school adopts structured and well-defined strong management practices. The main measure of management practices (the management index) represents the average of the scores across all 24 questions. For the case of the 2016 Tanzania D-WMS, an abbreviated version of the D-WMS was used to assess the school management practices of the principal/head teacher. In particular, only 15 school management practices were included in the survey for which data exists for all schools. The management index is an average of the following 15 school management practices, namely: • Leadership practices: o Leadership Vision: school leaders broadly communicate a shared vision and purpose for the school that focuses on improving student learning and outcomes. o Clearly Defined Accountability for School Leaders: the school leaders (e.g. principal, head teacher etc.) are responsible for delivering school goals and have the autonomy and authority to take action when needed. • Operations management: o Standardization of Instructional Planning Processes: school uses meaningful processes that allow students to learn over time o Personalization of Instruction and Learning: school incorporates teaching methods that ensure all students master the learning objectives o Adopting Educational Best Practices: school incorporates and shares teaching best practices and student strategies across classroom accordingly • Performance monitoring: o Continuous Improvement: school implements processes towards continuous improvement and encourages lessons to be captured and documented o Performance Tracking: school performance is regularly tracked with useful metrics 37 The methodology for the data collection is described in Bloom, Lemos, Sadun and Van Reenen 2015 (WMS) and Lemos and Scur 2016 (D-WMS). 38 92 percent of the schools in the DWMS sample are rural schools. 39 According to Lemos and Scur (2016), the management indices can be interpreted as follows: A score from 1 to 2 refers to an establishment with practically no structured management practices or very weak management practices implemented; A score from 2 to 3 refers to an establishment with some informal practices implemented, but these practices consist mostly of a reactive approach to managing the organization; A score from 3 to 4 refers to an establishment that a good, formal management process in place (though not yet often or consistent enough) and these practices consist mostly a proactive approach to managing the organization; A score from 4 to 5 refers to well-defined strong practices in place which are often seen as best practices in the sector. 74 o Performance Review: school performance is reviewed with appropriate metrics. • Target Setting: o Balance of Goal Metrics: school covers sufficiently broad set of targets at the school, department and individual levels o Time Horizon of Goals: there is a rational approach to planning and setting targets. o Clarity and Comparability of Goals: school sets understandable targets and openly communicates and compares school, department and individual performance. • People Management: o Rewarding Top Performers: school implements a systematic approach to identifying good and bad performance, rewarding teachers proportionately o Removing Poor Performers: school deals with underperformers promptly o Promoting High Performers: school promotes employees based on job performance. o Participative Management: Box 3.1: The Development-World Management Survey (D-WMS) The World Management Survey (WMS) and its adaptation for middle and lower-income countries, the D-WMS, objectively measure the existence of effective practices in the areas of operations management and instructional planning, performance monitoring, target setting, human resources management, and leadership practices (Bloom, et al., 2015; Lemos & Scur, 2016). The instruments are adapted from a survey methodology described in (Bloom & Van Reenen, 2007) used in the manufacturing, retail and health care sectors, and focus on practices that are considered to be relevant across industries in addition to key education-specific practices that were developed in consultation with education practitioners, school leaders, and sector consultants (Bloom, et al., 2015). Data is collected through structured interviews with school principals by trained enumerators who score responses against a detailed scoring grid. By quantifying the quality of management practices, this approach enables consistent comparisons across schools and countries. Both the WMS and D-WMS are freely available instruments but require relatively skilled enumerators and rigorous training. Importantly, these instruments do not intend to measure the quality of a school’s principal or other leaders, but rather the existence and quality of the practices themselves, regardless of who specifically carries them out. Tanzanian schools scored, on average, 1.8 in the five-point scale of the management index of the D-WMS. An average management index below 2 indicates an establishment with practically no structured management practices or very poor management practices implemented. More specifically, Tanzanian schools score below two in each of the 5 broad management practices (leadership, operations, monitoring, target setting and people management) (Table 3.1). Table 3.1: Descriptive Statistics of School Management Practices in Tanzania Variable Number Mean Std. Dev. Min Max of schools Management Index 100 1.88 0.18 1.54 2.36 Leadership Practices Score 100 1.94 0.19 1.56 2.44 Operations Management Score 100 1.89 0.20 1.50 2.33 Performance Monitoring Score 100 1.96 0.23 1.56 2.50 Target Setting Score 100 1.82 0.27 1.28 2.56 People Management Score 100 1.78 0.21 1.39 2.33 Note: The school management score is a combination of 14 basic management practices for which data exist for all countries, each rated from 1 to 5. Source: Tanzania Development-World Management Survey, 2016 75 Figure 3.4: Distribution of School Management Scores by Type of Practice Source: Tanzania Development-World Management Survey, 2016 In Tanzania, the distribution of the management scores is shifted completely to the lower end. Figure 3.4 and Figure 3.5 show the distribution of the overall management scores, and also the scores for each one of the 15 management practices. The main result is that the vast majority of Tanzanian schools scores below 2 in each one of the broad management practices and no school scores above 3, indicating once again that principals or head teachers in Tanzanian schools lack autonomy and authority in decision-making, have very weak management practices, with very little monitoring, target setting, use of monetary and non-monetary incentives. In terms of leadership, this result suggest that the school vision is somewhat clear yet not well defined and it is not linked to student outcomes. For example, the principal says the vision is “to educate all students”. Effective leadership is clearly lacking in Tanzanian primary schools. Moreover, the school vision is not actively communicated to others involved in the school matters such as students, parents and other community members. Second, the school follows a standardized curriculum, but teachers individually prepare lesson plans and never check with other teachers to see if they are all teaching in the same manner across classes/grades. The principals randomly conduct rounds or classroom observations (e.g. only twice 76 or three times a year). 40 In addition, they might acknowledge new teaching practices and somewhat encourage teachers to learn new techniques but in an unstructured way (i.e. principal has conversations with teachers occasionally and ask them to come up with new ways to incorporate other teaching practices into the classroom). Third, there is almost no performance monitoring. Two main parameters (e.g. student attendance, teacher absenteeism, dropout rate etc.) in addition to student marks are tracked, but it does not show how well the school is doing overall. Fourth, there are very weak targets put in place (e.g. only an annual school level target). Lastly, Tanzanian schools are characterized by extremely weak incentives (e.g. tenure-based promotion, no financial or non-financial incentives and no action taken about underperforming teachers). Dimensions School Management Practices Domain I: Leadership Practices Leadership Vision: school leaders broadly 2.05/5 Tanzanian schools scored, on average, 2.05 in the five-point communicate a shared vision and purpose for the scale. Leadership vision is somewhat clear yet not well- school that focuses on improving student learning and defined, it is not linked to student outcomes and not actively outcomes. communicated to students, parents etc. Clearly Defined Accountability for School Leaders: the 1.84/5 Tanzanian schools scored, on average, 1.84 in the five-point school leaders (e.g. principal, head teacher etc.) are scale. There are technically school-level consequences for not responsible for delivering school goals and have the achieving goals, but these are only enforced sometimes. Only autonomy and authority to take action when needed. the principal has autonomy/authority to take action when the matter involves student outcomes. The principal has limited autonomy/authority to impact hiring and firing or budget tweaks to bring help in case it is needed to meet a goal but never exercises this autonomy/authority. Domain II: Operations Management Standardization of Instructional Planning Processes: 1.89/5 Tanzanian schools scored, on average, 1.89 in the five-point school uses meaningful processes that allow students scale. School follows a standardized curriculum and to learn over time textbooks, but teachers individually prepare lesson plans and never check with other teachers. Personalization of Instruction and Learning: school 1.84/5 Tanzanian schools scored, on average, 1.84 in the five-point incorporates teaching methods that ensure all students scale. There is an informal process which indirectly helps the master the learning objectives teacher identify individual student needs. The principal randomly conducts rounds or classroom observations. Adopting Educational Best Practices: school 1.93/5 Tanzanian schools scored, on average, 1.93 in the five-point incorporates and shares teaching best practices and scale. The principal somewhat acknowledges they could use student strategies across classroom accordingly new teaching practices but does not encourage teachers to learn new techniques. Domain III: Performance Monitoring Continuous Improvement: school implements 1.99/5 Tanzanian schools scored, on average, 1.99 in the five-point processes towards continuous improvement and scale. The principal is often informed about problems when encourages lessons to be captured and documented they are happening, but never documents the issues after the fact. There is only one staff group (e.g. only the principal) involved in solving the issue. 40(Strong, et al., 2011) experimental research shows that principals who are not trained in classroom observations do not know what to look for during an observation. As a result, rather than supporting teachers, they may end up providing counterproductive feedback. In the study, principals were shown videos of teachers with ranging levels of value- added, and were asked to categorize them “effective” or “ineffective” accordingly. The study revealed principals were below the 50% threshold in correctly identifying effective and ineffective teachers, which is akin to pure chance. 77 Performance Tracking: school performance is regularly 1.84/5 Tanzanian schools scored, on average, 1.84 in the five-point tracked with useful metrics scale. Two parameters (e.g. attendance, teacher absenteeism, dropout rates etc.) in addition to student marks are tracked, but it does not show how well the school is doing overall. Performance Review: school performance is reviewed 2.06/5 Tanzanian schools scored, on average, 2.06 in the five-point with appropriate metrics. scale. School reviews their set of parameters twice a year and only the principal and some senior teachers are involved in the review meetings. Domain IV: Target Setting Balance of Goal Metrics: school covers sufficiently 1.83/5 Tanzanian schools scored, on average, 1.83 in the five-point broad set of targets at the school, department and scale. There is a general sense that principals would like to individual levels improve two or more student outcome measures (i.e. enrollment, grades), but no numbers of how much. Time Horizon of Goals: there is a rational approach to 1.76/5 Tanzanian schools scored, on average, 1.76 in the five-point planning and setting targets. scale. The schools have mostly annual goals and few short- term goals. Clarity and Comparability of Goals: school sets 1.86/5 Tanzanian schools scored, on average, 1.86 in the five-point understandable targets and openly communicates and scale. The principals say some teachers should certainly be compares school, department and individual aware of their goals as they have informal conversations once performance. in a while but cannot say if they are all aware or not. Domain V: People Management Rewarding Top Performers: school implements a 1.63/5 Tanzanian schools scored, on average, 1.63 in the five-point systematic approach to identifying good and bad scale, which is the lowest management score. Good teachers performance, rewarding teachers proportionately are identified only on the observed academic results of students. Rewards are given to everyone regardless of performance. Removing Poor Performers: school deals with 1.91/5 Tanzanian schools scored, on average, 1.91 in the five-point underperformers promptly scale. Bad performance is identified on a range of student results and is addressed consistently, but with no much consequence. Promoting High Performers: school promotes 1.79/5 Tanzanian schools scored, on average, 1.79 in the five-point employees based on job performance. scale. Professional development opportunities exist for all teachers, such as additional training, but these come only from mandatory government or school board rules. Principal actively encourages teachers to attend these but does not keep track. Participative Management: 1.92/5 Tanzanian schools scored, on average, 1.92 in the five-point scale. 78 Figure 3.5 : Distribution of School Management Scores by Group of Practices Source: Tanzania Development-World Management Survey, 2016 Comparing management capacity between the education and manufacturing sectors In this subsection, we compare the management capacity between Tanzanian schools and manufacturing firms. For the manufacturing data, the methodology is described in (Bloom & Van Reenen, 2007) and updated data provided in (Bloom, et al., 2014). Samples have been randomly selected within countries and composed of firms between 50-5000 employees. In order to construct the management index for the manufacturing sector we use a set of 9 comparable practices across sectors. The management index is an average of these 9 practices, namely: • Performance monitoring: Continuous Improvement, Performance Tracking, Performance Review • Target Setting: Balance of Goal Metrics, Time Horizon of Goals, Clarity and Comparability of Goals • People Management: Rewarding Top Performers, Removing Poor Performers, Promoting High Performers Management capacity is significantly lower in Tanzanian schools compared to manufacturing plants, which have an average management index of 2.28 compared to just 1.6 for the education sector. However, the manufacturing management score in Tanzania is still very low. Figure 3.6 shows the management score distribution for schools and manufacturing plants in Tanzania using the 9-practice management index. Looking at the comparable set of practices between the education sector and the manufacturing sector, we find that the fraction of Tanzanian firms scoring above three is just 6 percent, compared to zero percent for schools. 79 Figure 3.6 : Comparing management capacity across sectors in Tanzania – Education and Manufacturing Source: Tanzania Development-World Management Survey, 2016 80 Box 3.2: Comparing Tanzania’s School Management Practices Internationally Among the participating countries in the WMS and DWMS, Tanzania has the lowest management index. Effective school management relies on capacity and autonomy for decision-making at the school level, which in many developing countries such as Tanzania are often missing. Figure 3.7 shows the average management scores across countries participating in the WMS and DWMS using a set of 14 practices for which data exit for all countries (the WMS contains 23 practices and the D-WMS contains 24 practices). 41 The World Management Survey ran telephone survey in schools offering education to 15-year old students (mostly secondary schools) in Brazil, Canada, Germany, India, Italy, Sweden, UK, and U.S. between 2009 and 2013. The Development-World Management Survey ran face-to-face surveys in Mexico and Tanzania and telephone surveys in Haiti between 2015 and 2016. 42 Samples have been randomly selected within countries and include both public and private schools with the exception of Mexico and Tanzania which only include public schools. Figure 3.7 : Average management score by country and relative to top-performing country Mean management score (14 point index) Relative management score 1 2 3 4 5 0 20 40 60 80 100 United Kingdom 2.9 United Kingdom 100 Sweden 2.8 Sweden 96 Canada 2.8 Canada 96 United States 2.8 United States 95 Germany 2.6 Germany 88 Mexico 2.2 Mexico 74 Brazil 2.1 Brazil 73 Italy 2.1 Italy 72 Haiti 1.9 Haiti 64 India 1.8 India 60 Tanzania 1.6 Tanzania 55 Note: The school management score is a combination of 14 basic management practices for which data exist for all countries, each rated from 1 to 5 (the WMS contains 23 practices and the D-WMS contains 24 practices). Schools with higher scores have more structured management practices. Source: (Bloom, et al., 2015) World Management Survey (secondary schools) for North America, Europe, Brazil and India. Development-WMS (primary schools) for Haiti, Mexico and Tanzania. The figure suggests that the adoption of modern managerial processes in schools is fairly limited: on an index of 1-5, the average management score across all countries is 2.32, which corresponds to a low level of adoption of many of managerial practices included in the questionnaires. However, there are significant differences across countries. The UK has the highest management score (2.9), closely followed by Sweden, Canada and the U.S (all on 2.8). Germany is slightly lower (2.5), while Italy is substantially lower (2.1) and comparable to developing countries. Among the developing countries, Mexico has the highest score (2.2), followed by Brazil (2.1), India (1.8) and Haiti (1.9). Tanzania falls last in school management practices with a score of only 1.6 suggesting that school principals are not actively involved in helping teachers solve problems, do not tend to provide instructional advice, and do not set goals that prioritize learning. 81 In Tanzania, the distribution of management score shifts completely to the left with no school scores above three, suggesting that Tanzanian schools have the weakest management practices among the participating countries. Figure 3.8 shows the distribution of the management score within each country with the smoothed (kernel) fit of the US for comparison. Overall, this figure indicates that schools vary significantly in management quality within countries. Across developed countries, lower average country-level management scores are associated with an increasing dispersion towards the left tail of the distribution. Every country except the UK has some schools scoring below 2. A score below 2 indicates very poor management practices and ineffective leadership (almost no monitoring, very weak targets and extremely weak incentives). We also observe that all developed countries have some schools scoring on average above three, which would correspond to medium to widespread adoption of the management practices. The fraction of schools scoring above 3 ranges from 46 percent in the UK to zero percent in Tanzania. Tanzania performs very poorly compared to both the developed and developing countries. Figure 3.8: School management practices within countries, US density overlaid on all graphs Brazil Canada Germany Haiti 1 1.5 2 .5 0 India Italy Mexico Sweden 1 1.5 2 .5 0 1 2 3 4 5 Tanzania UK US 1 1.5 2 .5 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Management Score Note: 14-practice index. World Management Survey (secondary schools) for North America, Europe, Brazil, India. Development-WMS (primary schools) for Haiti, Mexico, Tanzania. 41 An alternative would be to extrapolate the missing data for Tanzania and include 23 questions in the index composition, however, the average management practices scores for the full index as well as the restricted index are highly correlated. The authors chose not to extrapolate and maintain to a clean, comparable dataset showing that there is substantial variation in management practices across countries. 42 These D-WMS surveys were carried out in partnership with World Bank teams and national governments. The Haiti survey was conducted in partnership with Melissa Adelman and Juan Baron, the Mexico survey was conducted in partnership with Rafael de Hoyos, the Tanzania survey was conducted in partnership with Ezequiel Molina and Deon Filmer. 82 Lastly, the management capacity in Tanzania is substantially lower in schools than in manufacturing firms. Looking at a comparable set of practices across other sectors, the school management distribution shifts to the left when compared to the manufacturing distribution within countries. In general, school management capacity tends to be lowest in those countries with the lowest income levels, and management capacity is substantially lower in schools than in manufacturing. Figure 3.9 shows the management distribution across countries for schools and manufacturing plants, suggesting that the pattern mentioned above is also observed in Tanzania. This figure uses a set of 9 comparable practices across sectors for which data exists for all countries, except Haiti (for which no manufacturing data has been collected). 43 Figure 3.9: School management practices compared to manufacturing management practices within countries Brazil Canada Germany Haiti 1.5 1 .5 0 India Italy Mexico Sweden Kernel Density 1.5 1 .5 0 1 2 3 4 5 Tanzania UK US 1.5 1 .5 0 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 Management Score Manufacturing Education Note: Comparable 9-practice index. World Management Survey (secondary schools) for North America, Europe, Brazil, India. Development-WMS (primary schools) for Haiti, Mexico, Tanzania. 43 The full manufacturing survey contains 18 practices. 83 Chapter 4 : Schools Inputs This chapter provides information on school inputs and infrastructure in the context of Tanzania, as well as emerging trends. Inputs and infrastructure are essential to learning, but they are not a sufficient condition for learning. While having certain inputs and infrastructure does not necessarily lead to more learning , having them is necessary as teaching aids for the teacher and for the creation of an effective learning environment. Every education system invests heavily in infrastructure and inputs, but the key is to ensure that these investments reach communities and improve the teaching and learning process in the classroom (World Bank, 2018). Using nationally representative data from two rounds of the Service Delivery Indicators (SDI) Survey in Tanzania, 2014 and 2016, we answer two basic questions: What classroom inputs are available in Tanzanian schools? and What basic infrastructure is available in Tanzanian schools? 44 We also analyze how Tanzanian schools compare internationally and how school inputs have evolved over time. Both SDI surveys were carried out in the same nationally representative sample of 400 public schools across 22 regions. In each round of the SDI survey, school level data was collected through visual inspections (direct observations) of a Standard 4 classroom and the school premises in each primary school. The Standard 4 classroom was randomly selected and observed for available classroom inputs by the field enumerators. What classroom inputs are available in Tanzanian schools? In terms of availability of teaching resources, three quarters (75 percent) of Tanzanian primary schools possess the minimum equipment required, which measures the availability of: (i) functioning blackboard and chalk and (ii) pens, pencils and exercise books (paper) in 4th grade classrooms. In particular, in more than 90 percent of schools, students have pen, pencil and exercise book, while 82 percent of the schools have a functioning blackboard. The main issue in classroom inputs seems to be the lack of light for students to be able to read the blackboard. Schools in the EQUIP-T regions and rural schools have significantly less minimum equipment availability compared to urban schools. Textbook availability is quite concerning in Tanzania. Despite recent nationwide distribution of textbooks, only 1 out of 5 students (19 percent) had access to an English, Swahili or mathematics textbook in a typical Standard 4 classroom. Students in urban schools are significantly more likely to have a textbook in the classroom compared to their counterparts in rural school. Basic classroom furniture appears less of a constraint in Tanzania, since almost all schools (95 percent) have proper seats and desks for their students. Minimum equipment availability in Tanzania is comparable to schools in Kenya. In contrast, they fared much better than Nigeria (48 percent), Laos (47 percent), Afghanistan (36 percent) and Togo (24 percent). Tanzania did very poorly with textbook availability, only outperforming Uganda with the average Standard 4 Tanzanian student being more than three times more likely to use textbook in the classroom than her Ugandan peer. . Analyzing trends in classroom inputs for learning from 2014 to 2016, we find that minimum equipment availability has increased, while availability of textbooks for students has decreased. On the one hand, the share of students with pens, pencils and exercise book has increased from 84 to 92 percent and similarly the proportion of schools with functioning blackboard has increased significantly from 74 to 83 percent. On the other hand, the proportion of students with a textbook declined, from 25 percent in 2014 to 18.8 percent in 2016, indicating that the system resources have not kept pace with enrollment increases. However, in urban schools outside of Dar es Salaam, where the share of students with a textbook was lowest in 2014, the share increased by four points from 18 percent of students to 22 percent, suggesting that textbooks are being relatively well targeted to the areas of greatest need. What basic infrastructure is available in Tanzanian schools? Tanzanian schools score poorly on minimum infrastructure availability, which measures the availability of: (i) functioning toilets and (ii) classroom visibility. Only 41 percent of schools could meet the standard. Tanzanian schools had severe problems regarding functioning toilets. The main constraint in infrastructure was the availability of functioning toilets given that fewer than half of schools (45 percent) had toilets meeting the standard (accessible, clean and private). The reason behind this result is generally due to the fact that in many of the schools, teachers do not have separate toilets and need to share toilets with students, especially in EQUIP-T regions and rural schools. Classroom visibility is less problematic compared to functioning toilets, but it is still an issue as students in 1 out of 5 schools struggled to read the blackboard from the back of the 44 The sample is also representative of key strata, including schools in Dar es Salaam; urban schools; rural schools; and schools in regions implementing the Education Quality Improvement Program in Tanzania (EQUIP-T), a targeted reform program operating in seven disadvantaged regions. 84 classroom. Drinking water seems to be a national challenge in Tanzania as half of public schools do not have access to clean drinking water, especially in rural areas. Working electricity is another input that almost all schools in Tanzania lack. Specifically, 95 percent of schools in Tanzania do not have available electricity in the classroom. Urban schools are more likely to have working electricity compared to rural schools. Most schools (88 percent) in Tanzania are accessible by road. The rest of the schools are accessible by foot path only. Tanzania is located in the medium level of the spectrum in terms minimum infrastructure availability. Tanzanian schools have worse infrastructure compared to schools in its East African Community (EAC) neighbors (i.e. Uganda and Kenya) and Laos, but better than Afghanistan, Mozambique, Togo and Nigeria. Analyzing trends in basic infrastructure for learning from 2014 to 2016, we find that there was no noticeable improvement in Tanzanian primary schools in terms of minimum infrastructure availability. Even though, minimum infrastructure availability has increased from 40 to 43 percent during this period, this increase is not statistically significant. Classroom visibility seems to have improved slightly from 76 to 88 percent. However, the most notable result is that the presence of functioning toilets, which has remained in the same level of approximately 46 percent. When we disaggregate this result by urban/rural, we find that the gap has increased by almost 10 percentage points during this period (from 27 to 39 percent), making this a much more severe problem in rural schools. The results indicate that this gap is mostly driven by privacy of the toilet. Working electricity and access to drinking water still remain as main constraints, especially in rural areas. The observed student-teacher ratio averaged 47 students per teachers, slightly above the expected norm of 45:1. Staffing has not kept pace with growing student numbers, as the observed student-teacher ratio has increased from 43 in 2014 to 47 in 2016; Tanzania’s primary level observed student-teacher ratio is one of the highest amongst the SDI countries and is also above the three SABER SD pilots (Afghanistan, Laos and Pakistan). Tanzania’s observed student-teacher ratio is higher than the neighboring countries: Mozambique and Kenya, but it is below Uganda’s student teacher-ratio, which is around 54:1. The deterioration of school staffing predates a recruitment freeze in 2017, which led to a reduction on the overall number of primary school teachers; it is therefore likely that future SDI rounds will reveal a continued deterioration in staffing. 85 Section I. Inputs for Learning – Classroom Level Inputs and infrastructure are essential to learning, but they work only when they serve the relationship between teaching and learning. Every education system invests heavily in infrastructure and inputs, but the key is to ensure that these investments actually reach communities and improve the teaching and learning process in the classroom. Although a consistent relationship has not been established between learning and specific school resources as measured by learning materials, student-teacher ratios, teacher experience and qualifications, and spending per student, a lack of critical inputs is likely to negatively influence learning outcomes (Harbison & Hanushek, 1992; Glewwe, et al., 1995; Tan, et al., 1997). For several reasons, however, school input shortages explain only a small part of the learning crisis. Part of the reason is that inputs often fail to make it to the front lines. This section documents and analyzes trends in key classroom inputs necessary for learning at the classroom level in Tanzania. To measure basic inputs, we use the following 4 indicators: (i) functioning blackboard and chalk, (ii) pens, pencils, and exercise books in Standard 4 classrooms, (iii) students with textbooks, and (iv) basic classroom furniture (tables/desks and chairs). Figure 4.1: Trends in Inputs for Learning - Minimum Equipment Availability 120% 96% 98% 96% 97% 99% 98% 98% 100% 96% 94% 92% 88% 84% 83% 80% 74% 74% 75% 61% 60% 40% 25% 19% 20% 0% 2014 2016 Minimum equipment availability Studens with exercise book (paper) Classroom with board Students with pen/pencil Classroom with chalk Students with proper seats and desks Students with pen/pencil and exercise book Classroom with contrast to read the board Functioning blackboard Students with textbook Note: This figure reports statistics for minimum equipment availability in the classroom, share of students with textbook and other specific classroom input indicators. Minimum equipment availability is a binary indicator capturing the availability of: (i) functioning blackboard and chalk and (ii) pens, pencils and exercise books (paper) in 4th grade classrooms. In one randomly selected 4th grade classroom in the school the enumerator assessed if there was a functioning blackboard by looking at whether text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. We considered that the classroom met the minimum requirement of pens, pencils, and exercise books (paper) if both the share of students with pen or pencils and the share of students with exercise books were above 90 percent. All school-level statistics are calculated using school-specific sampling weights. Results for classroom inputs are based on observations from around 400 randomly sampled schools. Refer to Table B. 12 for more detail. Source: Tanzania SDI 2014 and 2016 The importance of having a functional blackboard and chalk originates in a large number of studies that speak to the idea that students learn 86 better by having information presented through multiple modalities, especially through visual means (Mayer, 2003). In particular, blackboards are the most iconic tool for presenting information visually in a classroom setting. Studies have shown that regardless of the type of board-blackboard, whiteboard, interactive whiteboard, or PowerPoint-there are no significant differences on learning outcomes (Shallcross & Harrison, 2007). The use of functioning blackboards to elaborate visually on the material has been linked to higher levels of active listening, which in turn facilitate better learning outcomes (Meo, et al., 2013). Pens, pencils, textbooks, and exercise books also provide students with the opportunity to engage with the material in a way that enhances learning. For instance, since the 1990s, the importance of textbooks in improving the quality of education has been highlighted in the literature (Braslavsky & Halil, 2006). In Ghana, a study based on a program that supported basic education showed that improvements in Math and English test scores were largely due to the increased availability of textbooks (White, 2004). A cross-country analysis based on data from regional assessments in 22 countries in Sub-Saharan Africa showed that pedagogical resources are effective for improving learning. In the same study, it was estimated that providing a textbook to every student in a classroom increased literacy scores by 5-20% (Fehrler, et al., 2009). At the same time, other papers have shown that provision of textbooks does not lead to an increase in learning outcomes (Glewwe, et al., 2009; Sabarwal, et al., 2014). Though research that connects classroom furniture to learning is fairly limited, it is reasonable to assume that lack of basic furniture can hinder learning by keeping students from fully taking advantage of the lessons. If students do not have access to a table/desk in which to place their notebooks to write on, they will not be able to take notes, do practice problems, or engage with the material directly. Similarly, if they do not have chairs to sit on, being in an uncomfortable state may hinder their ability to pay attention and follow the lesson. For this reason, having chairs or somewhere to sit on and tables/desks that each student has access to (can be shared) are considered part of the basic inputs that need to be present in a classroom setting. Methodological Note Minimum equipment availability is a binary indicator capturing the availability of: (i) functioning blackboard and chalk and (ii) pens, pencils and exercise books (paper) in 4th grade classrooms. In one randomly selected 4th grade classroom in the school the enumerator assessed if there was a functioning blackboard by looking at whether text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. We considered that the classroom met the minimum requirement of pens, pencils, and exercise books (paper) if both the share of students with pen or pencils and the share of students with exercise books were above 90 percent. Share of students with textbooks reflects the typical ratio in student to textbooks in a 4th grade classroom. It is measured as the number of students with the relevant textbooks (language or mathematics conditional on which randomly selected class is observed) in one randomly selected 4th grade class and divided by the number of students in that classroom. In terms of availability of teaching resources, three quarters (75 percent) of Tanzanian primary schools possess the minimum equipment required. In particular, in more than 90 percent of schools, students have pen, pencil and exercise book, while 82 percent of the schools have a functioning blackboard. As shown in Figure 4.1, all sub-indicators of pens, pencils exercise book, board, chalk are close to 100 percent except for the lack of light for students to be able to read the blackboard. In more than one out of ten schools the Standard 4 classroom was judged by the enumerator as not having enough contrast to allow reading from a distance. 45 Schools in the EQUIP-T regions and rural schools in general have significantly less minimum equipment availability compared to urban schools, especially in terms of sufficient contrast to read the board and functioning blackboard (Figure 4.2). 45 Although lack of teaching equipment does not appear to be a binding constraint for providing high-quality teaching in most Tanzanian primary schools, lack of light is a concern that needs to be addressed. As shown by Mott et al. (2012) lighting quality in the classroom may significantly impact learning process and outcomes. 87 Textbook availability is quite concerning in Tanzania. Roughly, only 1 out of 5 students (19 percent) had access to an English, Swahili or mathematics textbook in a typical Standard 4 classroom (Figure 4.2). This indicator measures the number of language and mathematic textbooks depending on the subject that the students use in the classroom divided by the number of students present in a randomly observed Standard 4 classroom. Students in urban schools are significantly more likely to have a textbook in the classroom compared to their counterparts in rural school. In particular, 30 percent of students in urban schools have a textbook compared to just half of these in rural schools (17 percent). These results coincide with a large distribution of textbooks which took place recently in Tanzania and for which schools in the SDI survey acknowledge being beneficiaries. Basic classroom furniture appears less of a constraint in Tanzania, since almost all schools (95 percent) have proper seats and desks for their students. The survey results indicate that urban schools do not differ from rural schools in terms of basic classroom furniture (Figure 4.2). Figure 4.2: Minimum equipment availability in the classroom by urban/rural 0% 20% 40% 60% 80% 100% Minimum equipment availability 87% 74% Students with pen/pencil 97% 98% Studens with exercise book (paper) 99% 99% A Students with pen/pencil and exercise book 91% 92% Classroom with board 99% 98% Classroom with chalk 99% 95% B Classroom with contrast to read the board 97% 87% Functioning blackboard 96% 81% Students with textbook 32% 17% C Students with proper seats and desks 91% 94% D urban rural Source: Tanzania SDI 2016 Analyzing trends in classroom inputs for learning from 2014 to 2016, we find that minimum equipment availability has increased, while availability of textbooks for students has decreased. On the one hand, the share of students with pens, pencils and exercise book has increased from 84 to 92 percent and similarly the proportion of schools with functioning blackboard has increased significantly from 74 to 83 percent. On the other hand, the proportion of students with a textbook declined, from 25 percent in 2014 to 18.8 percent in 2016, indicating that the system resources have not kept pace with enrollment increases. However, in urban schools outside of Dar es Salaam, where the share of students with a textbook was lowest in 2014, the share increased by four points from 18 percent of students to 22 percent, suggesting that textbooks are being relatively well targeted to the areas of greatest need. 88 Section II. Basic Infrastructure – School Level Basic infrastructure availability has also been linked to learning outcomes, especially when used in combination with improved teaching. Previous literature has found that better school infrastructure measured through school construction can increase access to education and years of schooling (Duflo, 2001). Building a school where there was none gives the opportunity to children in that area to have access to education, though it may not lead to more learning if other factors like teaching are not satisfactory. Another example is availability of clean drinking water and adequate sanitation. Provision of safe drinking water and hygiene at schools have been linked to lower student school absenteeism and higher learning due to significant reductions in the diarrheal disease burden impacting students across the developing world (O'Reilly, et al., 2008; Barde & Walkiewicz, 2014). Additionally, inadequate access to sanitation facilities in the schools often impedes educational attainment, particularly for girls who are dropping out at higher rates and report this as one of the primary drivers (Adukia, 2017; Fentiman, et al., 1999; Burgers, 2000). In addition to those, other inputs and aspects of infrastructure either play an important role in the learning process or are generally thought to be things that stakeholders see as necessary in a school setting. This section documents and analyzes trends in basic infrastructure necessary for learning at the school level in Tanzania. To measure basic infrastructure, we use the following 4 indicators: (i) drinking water, (ii) functioning toilets, (iii) classroom visibility, (iv) working electricity, (v) road access, and (iv) crowding of classes (student-teacher ratio). Figure 4.3: Trends in Basic Infrastructure – Minimum Infrastructure Availability 120% 97% 97% 98% 97% 100% 92% 88% 84% 80% 80% 76% 60% 56% 55% 53% 47% 48% 46% 43% 40% 40% 20% 5% 4% 0% 2014 2016 Minimum infrastructure availability Toilet available Toilet accesible Road access Toilet clean Classroom visibility Toilet private Drinking water Functioning toilet Working electricity Corner Library Note: This figure reports statistics for minimum infrastructure availability in the school and other specific school input indicators. Minimum infrastructure availability is a binary indicator capturing the availability of: (i) functioning toilets and (ii) classroom visibility. Functioning toilets is defined as whether toilets were functioning, accessible, clean and private (enclosed and with gender separation) as verified by an enumerator. To verify classroom visibility, we randomly selected one 4th grade classroom in which the enumerator placed a printout on the board and checked whether it was possible to read the printout from the back of the classroom. All school-level statistics are calculated using school-specific sampling 89 weights. Results for classroom inputs are based on observations from around 400 randomly sampled schools. Road access, working electricity and corner library are not available for 2014. Refer to Table B. 13 for more detail. Source: Tanzania SDI 2014 and 2016 Drinking water has been highlighted as an important input in schools. As outlined earlier, provision of safe drinking water at schools has been linked to lower student school absenteeism and higher learning due to significant reductions in the diarrheal disease burden impacting students across the developing world (O'Reilly, et al., 2008; Barde & Walkiewicz, 2014). Studies have also shown that drinking water can improve students’ readiness to learn by increasing the level of cognitive functioning (Edmonds, 2009; D'Anci, et al., 2006; Benton & Burgess, 2009). For drinking water to be considered safe, it must come from an improved source; that is piped water supply, protected well/supply, truck/cart, rainwater, or packaged bottled water. The importance of sanitation not only concerns the availability of drinking water, but also of functioning toilets. Hygiene at schools has been linked to lower student school absenteeism and higher learning due to significant reductions in the diarrheal disease burden impacting students across the developing world. Ensuring that there are functioning toilets to guarantee health standards to prevent children from getting ill at school plays an important role in lowering student absenteeism and improving student performance. Classroom visibility is crucial to enhance the effectiveness of other teaching aids. If there is no classroom visibility, students will not be able to effectively see/use their exercise books, or see the blackboard, or observe the demonstrations that the teacher offers. Therefore, while there are no direct links in the literature between classroom visibility and learning, its relationship to the other teaching aids makes it a necessary condition for an effective learning environment. Lastly, the overcrowding of classes is often cited as one of the problems impeding learning throughout the developing world. There are various reasons to this, but the main one being that it hinders the ability of the teacher to effectively teach and provide the level of individual attention required to effectively facilitate learning. In many African countries, class sizes average over 50 students per teacher. In some extreme cases, the class sizes average up to 120 students as is the case of Malawi (UNESCO Database 2015). Most studies, though there are some that indicate otherwise, have shown that smaller class sizes are associated with higher learning (Urquiola, 2006; Chingos, 2013; Jepsen & Rivkin, 2009; Angrist & Lavy, 1999). However, as with the other inputs and infrastructure items, class size does not lead to more learning by itself. Rather, it is linked to other factors and thus enhances their efficacy as is the case with effective teaching. Methodological Note Minimum infrastructure availability is a binary indicator capturing the availability of: (i) functioning toilets and (ii) classroom visibility. Functioning toilets is defined as whether toilets were functioning, accessible, clean and private (enclosed and with gender separation) as verified by an enumerator. To verify classroom visibility, we randomly selected one 4th grade classroom in which the enumerator placed a printout on the board and checked whether it was possible to read the printout from the back of the classroom. Observed student-teacher ratio reflects the typical ratio of students to teachers in a 4th grade classroom. It is measured as the number of students in one randomly selected 4th grade class at the school. Overall, Tanzanian schools score poorly on minimum infrastructure availability with only 41 percent of them meeting the standard. Tanzanian schools have severe problems regarding functioning toilets. The main constraint in infrastructure is the availability of functioning toilets given that less than half of schools (45 percent) have at least one toilet which is accessible, clean and private. Indeed, as shown in Figure 4.3, there is near universal access to toilets in Tanzania’s primary schools and almost all of them are accessible and clean, except for privacy of the toilet. The reason behind this result is generally due to the fact that in many of the schools teachers do not have separate toilets and need to share toilets with students, especially in rural areas. As a result, fewer than three out of five (55 percent) of the schools are considered as having private toilet. When we disaggregate the results by urban/rural school, it is especially in rural areas as well as EQUIP-T regions that this phenomenon is widespread. More specifically, only 42 percent of rural school have a functioning toilet. This number doubles in urban schools, where 81 percent of schools have a functioning toilet. This statistically significant difference in functioning toilet between rural and urban schools is mostly driven by availability of toilet privacy (Figure 4.4). 90 Classroom visibility is less problematic compared to functioning toilets, but it is still an issue as students in 1 out of 5 schools struggled to read the blackboard from the back of the classroom. The SDI results indicate that urban schools are indistinguishable from rural schools in terms of classroom visibility (Figure 4.4). 46 Drinking water seems to be a national challenge in Tanzania as half of public schools do not have access to clean drinking water, especially in rural areas. A school is classified as having a clean drinking water source if it has a piped water connection, a public tap, a tube-well, a protected well or spring. The main source of drinking water in most Tanzanian schools is piped water (33 percent), followed by protected dug well (21 percent), tube well or borehole (20 percent), public tap or standpipe (20 percent) and protected spring (5 percent). One quarter of schools (26 percent) do not have any source of water available to students. Rural schools are significantly worse off in terms of their access to drinking water compared to urban schools. Approximately, only 44 percent of the schools have drinking water available to students in rural areas compared to 83 percent in urban areas (Figure 4.4). Working electricity is another input that almost all schools in Tanzania lack. Specifically, 95 percent of schools in Tanzania do not have available electricity in the classroom. Urban schools are more likely to have working electricity compared to rural schools (Figure 4.4). In particular, 18 percent of urban schools have working electricity compared to only 3 percent for rural schools. Figure 4.4: Minimum infrastructure availability in the school by urban/rural 0% 20% 40% 60% 80% 100% Minimum infrastructure availability 64% 37% Classroom visibility 81% 80% A Toilet available 100% 97% Toilet accesible 99% 97% Toilet clean 90% 83% B Toilet private 87% 52% Functioning toilet 81% 42% Road access 100% 87% Drinking water 83% 44% C Working electricity 18% 3% urban rural Source: Tanzania SDI 2016 46 Classroom with contrast to read the board measures whether the blackboard has sufficient contrast for reading what is written on, while classroom visibility measures whether there is sufficient light for reading a text from the back of the classroom. 91 Most schools (88 percent) in Tanzania are accessible by road. The rest of the schools are accessible by foot path only. More specifically, most schools in the country are accessible by packed dirt road (43 percent), followed by gravel road (34 percent), tarmac road (11 percent). Accessibility by road has reached universal coverage for urban schools, but 13 percent of schools in rural areas still lack this input. Analyzing trends in basic infrastructure for learning from 2014 to 2016, we find that there was no noticeable improvement in Tanzanian primary schools in terms of minimum infrastructure availability. Even though, minimum infrastructure availability has increased from 40 to 43 percent during this period, this increase is not statistically significant. Classroom visibility seems to have improved slightly from 76 to 88 percent. However, the most concerning result is that the presence of functioning toilets, which is quite severe in Tanzania, has remained in the same level of approximately 46 percent. When we disaggregate this result by urban/rural, we find that although urban schools already seem to have had better functioning toilets, the issue now is that the gap has increased by almost 10 percentage points during this period (from 27 to 39 percent), making this a much more severe problem in rural schools. Moreover, the results indicate that this gap is mostly driven by privacy of the toilet. Working electricity and access to drinking water still remain as main constraints, especially in rural areas. 92 Box 4.1: What does a typical school in Tanzania look like? Table 4.1 reports the mean, the median, the 10th percentile and the 90th percentile for a number of indicators related to school size and student- teacher ratios. The typical school in the Tanzania SDI 2016 survey sample has 445 students, 12 teachers, and 8 classrooms. Some schools have as many as 3000 to 5000 students, especially in urban areas. Schools in Dar es Salaam are under increasing pressure, particularly in the largest schools. There is evidence of increasing concentration of students in Dar es Salaam into the largest schools: the ten percent of schools with the highest enrollment in 2014 expanded very rapidly between 2014 and 2016, from a size of at least 3,040 students per school to at least 3,402, while the smallest ten percent actually saw enrollment decline. Table 4.1: Descriptive Statistics of Schools – Tanzania 2016 10th 90th Mean Median percentile percentile Number of Students 521 445 215 864 Number of Teachers 12 9 5 21 Number of Classrooms 8 7 6 12 Student - Teacher Ratio 50 46 27 79 Observed Student - Teacher Ratio 47 40 22 82 Source: Tanzania SDI 2016 In Tanzania, the median student-teacher ratio is 46:1 and the observed average student-teacher ratio is 40:1, which is below Tanzania’s recommended benchmark of 45:1. The administrative student-teacher ratio is calculated using the enrollment rates at the classroom level, while the observed student-teacher ratio is the one assessed using the classroom observation tool. Nevertheless, the student-teacher ratio is a real problem in schools in the 90th percentile. The administrative student-teacher ratio for the 10th percentile of schools is 27:1 compared to as high as 79:1 for the 90th percentile. This number can even be as high as 120:1, especially for rural schools, which suggests that some classrooms are very overcrowded. The observed student-teacher ratio is on average similar to the administrative student-teacher ratio, but these indicators differ significantly by urban/rural school. In particular, the observed student-teacher ratio is significantly higher in urban schools, which suggest that in reality there are many out of school students in rural areas. These students are generally enrolled in school, but they do not attend. Schools in Dar es Salaam have significantly more students than the typical Standard 4 classroom in the nation. Interestingly, it is rural schools which respect the norm on class size and this may be due to the recent acceleration of urbanization in Tanzania. Figure 4.5: Distribution of Student-Teacher Ratios by Urban/Rural – Tanzania 2016 Source: Tanzania SDI 2016 93 Figure 4.5 presents the distribution of administrative student-teacher ratios and observed student-teacher ratios across urban and rural schools using a smooth histogram. The Y-axis represents the density of schools for a given student-teacher ratio, while the X-axis represents the administrative student-teacher ratio/observed student-teacher ratio. Basically, half of Tanzanian primary schools have an observed student teacher ratio that is above the norm. The typical urban school has a lower administrative student-teacher ratio (38:1) compared to the typical rural school (50:1). While the typical urban school has a higher observed student-teacher ratio (66:1) compared to the typical rural school (44:1). This means that students register and enroll in rural areas, but they are not actually attending or learning. The difference in observed student-teacher ratio between the top 10% and the bottom 10% in rural schools is smaller (20 students per teacher compared to 80 students per teacher) relative to that in urban schools (35 students per teacher in the top 10% and 107 students per teacher in the bottom 10%). In addition, not only do urban schools have a higher average observed student-teacher ratio, but there are also many more urban schools with very high student-teacher ratios. Table 4.2 shows the percentage of students attending school by type of ownership and gender. Almost all schools in Tanzania are public schools with just a very small fraction being private or private with a religious affiliation. In terms of gender mix, almost all schools in Tanzania are co- educational schools with a mixed-gender instruction in the classroom. Table 4.2: Types of School in Tanzania Overall Urban Rural Public School 98.9 95.6 99.3 Private School 0.1 1.2 0.0 Private School (religious) 1.0 3.1 0.7 Co-educational School 98.5 99.8 98.3 Source: Tanzania SDI 2016 94 Box 4.2: Comparing Tanzania’s School Inputs Internationally Bearing in mind that basic infrastructure is low in the comparative countries, Tanzania is located in the medium level of the spectrum in terms of minimum equipment availability and also minimum infrastructure availability. Figure 4.6 shows national averages of three school input indicators that allow us to compare Tanzania to other countries. Minimum equipment availability in Tanzania is comparable to schools in Kenya. Schools in neighboring countries such as Uganda and Mozambique, and Pakistan had slightly better levels of minimum equipment availability (80 percent, 77 percent, and 82 percent respectively) compared to 75 percent for Tanzania. As shown previously, the main constraint was sufficient contrast for students to be able to read the blackboard. In contrast, Tanzanian schools fared much better than Nigeria (48 percent), Laos (47 percent), Afghanistan (36 percent) and Togo (24 percent). Figure 4.6 : Comparing Tanzania’s School Inputs Internationally Minimum Equipment Availability Minimum Infrastructure Availability Togo 2013 24 Nigeria 2013** 13 Afghanistan 2017 36 Togo 2013 14 Laos PDR 2018 47 Mozambique 2014 29 Nigeria 2013** 48 Afghanistan 2017 35 Tanzania 2014 61 Tanzania 2014 40 Kenya 2012 74 Tanzania 2016 43 Tanzania 2016 75 Mozambique 2014 77 Uganda 2013 57 Uganda 2013 80 Laos PDR 2018 58 Pakistan 2018 82 Kenya 2012 60 0 20 40 60 80 100 0 20 40 60 80 100 Share of Students with Texbook Uganda 2013 6 Tanzania 2016 19 Tanzania 2014 25 Nigeria 2013** 34 Kenya 2012 45 Mozambique 2014 68 Togo 2013 76 Laos PDR 2018 84 Afghanistan 2017 86 Pakistan 2018 96 0 20 40 60 80 100 Note: This figure presents indicators on school inputs and infrastructure across SDI and SABER SD survey countries. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013, Kenya SDI 2012, Afghanistan SABER SD 2017, Laos SABER SD 2018 and Pakistan (Punjab) SABER SD 2018. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. 95 In terms of minimum infrastructure availability, Tanzanian schools are doing worse than Uganda (57 percent), Laos (58 percent) and Kenya (60 percent), but better than Afghanistan (35 percent), Mozambique (29 percent), Togo (14 percent) and Nigeria (13 percent). In this case the gap between Tanzania and its EAC neighbors (Uganda and Kenya) was much wider, at almost 20 percentage points. The main constraint in Tanzania’s minimum infrastructure availability is the lack of functioning toilets given that less than half of schools have an accessible, clean and private toilet. Tanzania did very poorly with textbook availability, only outperforming Uganda with the average Standard 4 Tanzanian student being more than three times more likely to use textbook in the classroom than her Ugandan peer. Students in Nigeria, Kenya, Togo, Mozambique, Laos, Afghanistan and Pakistan were, however, more likely to use a textbook in class compared to Tanzanian students. As highlightened by Figure 4.7 above, Tanzania’s primary level observed student-teacher ratio is one of the highest amongst the SDI countries and is also above the three SABER SD pilots (Afghanistan, Lao PDR and Pakistan). Tanzania’s observed student-teacher ratio is higher than the neighboring countries: Mozambique and Kenya, but it is below Uganda’s student teacher-ratio, which is around 54:1. The deterioration of school staffing predates a recruitment freeze in 2017, which led to a reduction on the overall number of primary school teachers; it is therefore likely that future SDI rounds will reveal a continued deterioration in staffing. Staffing has not kept pace with growing student numbers, as the observed student-teacher ratio has increased from 2014 to 2016. Despite the overall gains in student learning outcomes, the 2016 Tanzania SDI data presents a negative trend in observed student-teacher ratio, which could pose a threat to continued improvement. Basically, the increase in school size has not been met with concomitant improvement in teacher numbers. The typical school had the same number of teachers in 2016 as in 2014 (12 teachers) despite the increase in enrollment. The average observed student-teacher ratio rose from 43 in 2014 to 47 in 2016; the average class size in Standard 4 also rose from 55 in 2014 to 61 in 2016, a similar rate of increase. Figure 4.7 : Comparing Tanzania’s Student-Teacher Ratios Internationally – Standard 4 Observed Student-Teacher Ratio Laos PDR 2018 21 Mozambique 2014 21 Nigeria 2013** 22 Togo 2013 31 Senegal 2011 34 Pakistan 2018 35 Kenya 2012 39 Afghanistan 2017 43 Tanzania 2014 43 Tanzania 2016 47 Uganda 2013 54 0 10 20 30 40 50 60 Source: This figure presents indicators on observed student-teacher ratio across SDI and SABER SD survey countries. All individual country statistics are calculated using country- specific sampling weights. Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013, Kenya SDI 2012, Afghanistan SABER SD 2017, Laos SABER SD 2018 and Pakistan (Punjab) SABER SD 2018. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. 96 Chapter 5 : Student Support This chapter provides a general description of the different types of support available to students and their engagement to learning in the context of Tanzania. Having a supportive home and school environment can be instrumental in allowing students to focus on their responsibilities at school and be more dedicated towards learning (WDR 2018). A supportive environment at home, educated surroundings, teachers who motivate their students, all ensure that students are able to focus more on learning and less on the obstacles in the way of learning. One of the main driving causes of low learning is the fact that many children arrive at school unprepared to learn. Factors like malnutrition, illness, and low levels of early childhood stimulation associated with poverty undermine early childhood learning (Lupien, et al., 2000; McCoy, et al., 2016; Walker, et al., 2007). Using nationally representative data from two rounds of the Service Delivery Indicators (SDI) Survey in Tanzania, 2014 and 2016, we answer three basic questions: Do Tanzanian students receive the necessary support from home? Do Tanzanian students receive a supportive learning environment at school? and Are Tanzanian students engaged with learning in the classroom? 47 Both SDI surveys were carried out in the same nationally representative sample of 400 public schools across 22 regions. In each round of the SDI survey, data was collected through visual inspections (direct observations) of a Standard 4 classroom. The Standard 4 classroom was randomly selected and observed during a Language (English or Swahili) or mathematics lesson. For the case of this chapter, it is important to point out that the SDI surveys were not designed to collect extensive data on student’s household conditions, therefore the data for this section is sparser and the discussion more tentative. Do Tanzanian students receive the necessary support from home? Tanzanian students appeared to have serious socio-economic impediments. Two thirds of Tanzanian students did not have any breakfast before class and almost none of them had breakfast with proteins. When we analyze the results over time, we find that the proportion of students with breakfast has decreased from 2014 to 2016. Child nutrition is a challenge in Tanzania, where an estimated 2.7 million children under five are estimated to be suffering from malnutrition-related stunting (UNICEF, 2018). In response, the last years have seen a large increase in the provision of official and unofficial school feeding programs. In a range of studies from developed and developing countries, malnutrition and hunger are associated with significant negative impacts on learning, and students who miss breakfast are slower to respond to test questions and make more errors. Do Tanzanian students receive a supportive learning environment at school? Tanzanian teachers show moderate support provided by teachers to students in the classroom. In particular, Tanzanian classrooms display an effective supportive learning environment in only less than one third of the classrooms (29 percent). Four out of five teachers called students by their names (especially female teachers), offered positive reinforcement (complimented students when answering questions correctly giving them positive feedback and/or encouragement), and corrected students mistakes. More than half (60 percent) smiled and played with students (especially male teachers) and visited them individually. In only 40 percent of classes the teacher invited students to write on the blackboard and gave them homework, but even fewer times (24 percent) did the teacher return corrected homework or take student’s work to correct. Lastly, around one third of teachers gave negative feedback that is scolding at a mistake and relatively very few teachers (6 percent) teachers hit or scolded students during the lesson. Are Tanzanian students engaged with learning in the classroom? While students in Tanzania exert high levels of effort as evidenced by their relatively low absence rate and low time off task (especially in urban school), they only spend half of the time (46 percent) in school engaged in learning activities and they are just somewhat engaged in the classroom suggesting a moderate level in student dedication to learning. More specifically, Tanzanian schools present a student absence rate of around 16 percent, ranging from 8 percent in urban schools to 17 percent in rural schools. When present in class, only 3 percent of students were found not paying attention in class indicating a relatively low share of student’s off-task. In contrast, Tanzanian students spend about 3 hours 15 minutes of their time in school engaged in learning activities each day, which is equivalent to almost half of their time. In general, they behave well but they display a mix of student engagement with some students actively engaged, while others appeared distracted or disengaged. 47 The sample is also representative of key strata, including schools in Dar es Salaam; urban schools; rural schools; and schools in regions implementing the Education Quality Improvement Program in Tanzania (EQUIP-T), a targeted reform program operating in seven disadvantaged regions. 97 Box 5.1: Student’s Profile in Tanzania’s Primary School Table 5.1 presents descriptive statistics of Tanzanian student characteristics. An average Standard 4 student in Tanzania is between 11 and 12 years old, ranging from 8 to 20. Students in rural schools are slightly older than those in urban schools, but this difference is not statistically significant. However, the share of overaged students (i.e. students above the official age range for Standard 4 – above 10 years of age) amounts to almost 70 percent, with rural school being 30 percentage points more likely to have overaged students compared to urban schools (See Figure 5.1). The primary source of overage students is late enrollment given that Tanzania does not currently practice grade repetition. Table 5.1: Student Characteristics Overall Urban Rural Regional Gap Age 11.3 10.5 11.5 1.0 Male students (%) 49.4 51.5 48.8 2.7 Female students (%) 50.6 48.5 51.2 2.7 Swahili students (%) 51.2 86.3 41.6 44.6*** Overaged students (%) 67.2 43.6 73.7 30.0*** Had breakfast (%) 33.3 46.2 29.8 16.3*** Had breakfast with protein (%) 5.9 6.4 5.7 0.7 Source: Tanzania SDI 2016 On a national scale, primary schools have a very even division between male and female students, with a male-female ratio of 50,6 – 49.4. The average student is more likely to speak Swahili at home, followed by Kisukuma, Kihaya, Kihehe etc. In particular, half of Tanzanian students speak Swahili. In contrast, less than 1 percent of Tanzanian students speak English at home, which partially explains the low performance of Tanzanian students in the English test. Figure 5.1 : Student’s Age Distribution for Standard 4 by Urban/Rural urban rural 50% 40% 30% 20% 10% 0% 8 9 10 11 12 13 14 15 16 17 18 19 20 Student's age distribution - Standard 4 Source: Tanzania SDI 2016 98 The SDI data also revealed that Tanzanian students have serious socio-economic impediments. As a measure of student socio-economic status, the SDI asked sampled students whether they ate breakfast on the day of testing and what ingredients were included in this breakfast. Two thirds of Tanzanian students did not have any breakfast before class and almost all of them (94 percent) had breakfast with no proteins. When we analyze the results over time, we find that the proportion of students with breakfast has decreased from 2014 to 2016. Child nutrition is a challenge in Tanzania, where an estimated 2.7 million children under five are estimated to be suffering from malnutrition-related stunting (UNICEF, 2018). In response, the last years have seen a large increase in the provision of official and unofficial school feeding programs. In a range of studies from developed and developing countries, malnutrition and hunger are associated with significant negative impacts on learning, and students who miss breakfast are slower to respond to test questions and make more errors. Figure 5.2 : Student Characteristics – Standard 4 80% 67% 60% 60% 51% 48% 40% 37% 33% 20% 9% 6% 0% Kiswahili Overaged Had breakfast Had breakfast with protein 2014 2016 Source: Tanzania SDI 2014 and 2016 Section I. Supportive Learning Environment in the School The objective of the classroom observation was to have a better understanding of the dynamics in a typical Standard 4 class. Information was collected on the classroom environment and how the teacher carried out her teaching activity (that is, how she behaved with students, whether she asked questions, provided feedback, visited individually the students, called them by name, and so on). In order to measure whether Tanzanian students have a supportive learning environment in the classroom we use the following indicators: (i) students write on the blackboard (ii) the teacher visits the students individually (iii) the teacher calls students by their name (iv) the teacher provides positive reinforcement (v) the teacher corrects students mistakes (vi) the teacher provides negative reinforcement, and (vii) the teacher hits or scolds students. Given this, we define supportive learning environment as a binary indicator that takes the value of one if the teacher calls students by their name, gave feedback of praise at least once, corrected a mistake at least once and did not scold at mistakes or hit, pinch, or slap any student. Figure 5.3 presents the average classroom practices in terms of supportive environment observed by enumerators during the classroom observation. We also analyze whether the supportive environment in the classroom varies by urban/rural school. 48 48 Table B. 10 also presents the average classroom practices observed by enumerators during the classroom observation disaggregated by urban/rural school and teacher gender. 99 Figure 5.3 : Trends in Supportive Learning Environment in Tanzanian Classrooms 0% 20% 40% 60% 80% 100% Supportive learning environment 29% Correct students mistakes 80% 81% Call students by their name 78% 80% Suportive Learning Environment Give positive feedback and/or encouragement 70% 79% Visit students individually 54% 60% Students write on the black board 43% 40% Give negative feedback that is scolding at a mistake 30% 33% Hit, pinch or slap students 3% 6% 2014 2016 Source: Tanzania SDI 2014 and 2016 Tanzanian teachers show moderate support provided by teachers to students in the classroom. In particular, Tanzanian classrooms display an effective supportive learning environment in only less than one third of the classrooms (29 percent). When looking at the interaction with students, four out of five teachers called students by their names (especially female teachers), offered positive reinforcement (complimented students when answering questions correctly giving them positive feedback and/or encouragement), and corrected students mistakes. More than half (60 percent) smiled and played with students (especially male teachers) and visited them individually. Another interesting fact was that while most teachers introduced the lesson, less than half of them (45 percent) summarized the lesson at the end of the class. In only 40 percent of classes the teacher invited students to write on the blackboard and gave them homework, but even fewer times (24 percent) did the teacher return corrected homework or take student’s work to correct. Only one out of 10 teachers used the local language during the lesson. Lastly, around one third of teachers gave negative feedback that is scolding at a mistake and relatively very few teachers (6 percent) teachers hit or scolded students during the lesson. Teachers in urban schools do not differ significantly from teachers in rural schools in terms of creating a supportive learning environment. The only statistically significant difference we found is that urban teachers are more likely to correct a mistake compared to their rural peers (Figure 5.2). Female teachers are more likely than their male counterparts to create a supportive environment to their students, especially by calling students by their name. In contrast, male teachers are more likely to create a supportive learning environment by smiling and using jokes during class. For instance, 85 percent of female teachers called students by their name compared to 76 percent for male teachers. However, female teachers do not differ much from male teachers in terms of correcting student’s mistakes, giving them positive or negative feedback, visiting them individually or mistreating students (hit, pinch or slap students). (Figure 5.4). 100 Figure 5.4 : Supportive Learning Environment by Rural and Urban School 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% Supportive learning environment 30% 29% Correct students mistakes 80% 73% Call students by their name 82% Suportive Learning Environment 79% Give positive feedback and/or encouragement 79% 79% Visit students individually 62% 59% Students write on the black board 41% 40% Give negative feedback that is scolding at a mistake 38% 36% Hit, pinch or slap students 3% 7% urban rural Source: Tanzania SDI 2016 Section ll. Student Dedication to Learning Another important indicator of student preparedness is their dedication/effort and engagement to learning. We measure 5 student engagement indicators using CLASS classroom observation instrument: (i) student absence rate, (ii) share of students off-task, (iii) time spent receiving instruction per day, (iv) student engagement, and (iv) behavior management. First, student absence rate is measured, using the classroom observation data, as the percentage of the number of students absent from class out of all the students registered in the class. Second, by measuring every five minutes the number of students that were not paying attention to the lesson, we created a picture of the ratio of students who were not paying attention in a given class. In other words, the indicator students off-task is measured as the percentage of students who did not pay attention during class. Third, time spent learning per day is measured as the time spent by an average student receiving instruction from a teacher. Time spent learning per day adjusts the length of the school day for the share of teachers that are present in the classroom and teaching (time on task), on average, for the share of students present in school, on average, and for the time that students are on-task while the teacher teaches. Fourth, the CLASS student engagement construct measures the extent to which students are participating in the learning exercises taking place in the class. It distinguishes between disengagement, passive engagement, and active engagement. Lastly, the CLASS Behavior Management scale measures any student misbehavior in the class. Tanzanian schools present a student absence rate of around 16 percent, ranging from 8 percent in urban schools to 17 percent in rural schools. While officially there were around 50 students enrolled per class, only an average of 42 students effectively attended. Students in rural schools are significantly more likely compared to their urban peers to be absent from school (Figure 5.4). The overall number for Tanzania in terms of students who were not paying attention in class, therefore considered “off-task”, was only about 3 percent. Students in rural schools are significantly more likely to be found off-task compared to those in urban schools (Figure 5.4). Teaching is hard, especially with students in Standard 4. If teachers do not motivate students to pay attention, the activities or practices the teachers are 101 doing will not affect student’s understanding of the material, nor will it help increase their knowledge. The Stallings method involves measuring classroom activity 10 times for the duration of a lesson in a randomly selected Grade 4 lecture. Observers mark the general activity of both the teacher and the student in the snapshots, and the size of the group of students engaged in that activity. The percentage of time students were engaged in off-task activities out of the total allotted time of the class is used to generate the Student off-task rate. We use this to arrive at an estimate of students’ attentiveness in class. Figure 5.5 : Student Absence Rate and Students Off-Task by Urban/Rural School 20% 17.0% 16.2% 15% 10% 8.0% 5% 3.0% 3.1% 1.7% 0% Overall Urban Rural Overall Urban Rural Students off-task Student absence rate Note: This figure shows student absence rate and share of students off task. Student Absence Rate is measured using the classroom observation data as the percentage of the number of students absent from the class out of all the students registered in the class. Share of students off-task is an indicator that measures the percentage of time students do not pay attention. It is computed as the average percentage of students who are off-task, which is measured by 10 snapshots through the entire duration of the lesson following the Stallings method. Collected from a randomly selected grade 4 classroom of students from 400 schools in Tanzania. School level weights are used to compute these numbers. Source: Tanzania SDI 2016 Tanzanian students spend about 3 hours 15 minutes of their time in school engaged in learning activities each day, which is equivalent to almost half of the time (46 percent). The average scheduled teaching day, excluding break times, averaged 7 hours in Tanzania. Teachers were in the classroom teaching 56 percent of the time, which means that 44 percent of the time, teachers were off-task. However, from the student’s side, we know that students were present 85 percent of the time and, when in class, they were on task –paying attention to the teacher – 97 percent of the time. Therefore, the student learning time was 46 percent (0.56*0.85*0.97). This means that out of a possible 190 school days, students in Tanzania received about 87 effective teaching days. Tanzanian students scored at a moderately (medium level) in terms of student engagement, meaning that in most Tanzanian classrooms (82 percent) there is a mix of student engagement with some students actively engaged, but others appeared distracted or disengaged (CLASS method). In the lower end of the spectrum, 10 percent of students appeared to be distracted or disengaged and in the upper end of the spectrum, only 8 percent of students seemed to be actively engaged in classroom discussion and activities (Figure 5.5). Tanzanian students scored on the high end of the behavioral engagement scale, meaning that very little misbehavior was observed in the classroom (CLASS method). This CLASS construct shows that most Tanzanian teachers (82 percent) state clear rules to their students and use effective strategies to redirect misbehavior. This result suggests that most Tanzanian students do not misbehave in the classroom, which points to again to little time being lost in activities that are not related to learning. Note that this may not be reflective of teacher skills but rather cultural norms of students. In sum, while students in Tanzania exert high levels of effort as evidenced by their relatively low absence rate and low time off task (especially in urban school), they only spend half of the time (46 percent) in school engaged in learning activities and they are just somewhat engaged in the classroom suggesting a moderate level in student dedication to learning. 102 Figure 5.6: Student Engagement and Behavior Management using CLASS 50% 50% 48% 40% 40% 35% 34% 31% 30% 30% 20% 16% 20% 10% 11% 10% 8% 10% 6% 1% 0% 0% Student Engagement Behavior Management 1 2 3 4 5 6 7 1 2 3 4 5 6 7 Note: This figure displays the average score of the CLASS Student Engagement domain and Behavior Management, on a scale of 1 to 7 with 7 being the highest. In the Tanzania SDI, 112 randomly selected Grade 4 classrooms were observed using the CLASS instrument. Source: Tanzania SDI 2016 103 Box 5.2: Comparing Student Effort Indicators Internationally Tanzanian students display a relatively low absence rate (16 percent) compared to other comparable countries. Analyzing the SDI data over time, we also find that student absence rate has decreased from 26 percent in 2014 to 16 percent in 2016, which represents a decrease of more than one third. Tanzanian students had a higher student absence rate compared to Lao PDR (9 percent), but lower than that in Mozambique (56 percent) and Afghanistan (23 percent). This student absence rate is measured by observing the number of students absent from a Grade 4 classroom, out of the total number of students registered in the class. Tanzanian students outperformed their Mozambican peers in terms of paying attention in class given that only about 3 percent of students were found off-task compared to 10 percent in Mozambique. Tanzanian students underperformed slightly to Afghan students (2 percent of students off task). However, it should be noted that the off-task rate is calculated using the Stallings method in Afghanistan, whereas in Mozambique and Tanzania it is calculated using the SDI CLASS instrument. Both instruments require observers to mark students as off-task or uninvolved, but while Stallings measures the classroom activity 10 times during the entirety of the lesson (every 3-4 minutes in Afghanistan), the SDI tools require observers to mark activity every 5 minutes during the lesson (which lasts around 45 minutes on average in Mozambique). Table 5.2: Student Effort Indicators Internationally Student absence rate Students off-task 0 10 20 30 40 50 60 0 2 4 6 8 10 12 Mozambique 2014 56 Mozambique 2014 10 Tanzania 2014 26 Tanzania 2014 5 Afghanistan 2017 23 Tanzania 2016 3 Tanzania 2016 16 Laos PDR 2018 9 Afghanistan 2017 2 Note: This figure presents the student absence rate and students off-task rate compared to Mozambique, Afghanistan and Laos. All individual country statistics are calculated using country-specific sampling weights. The figures for Mozambique are taken from the Mozambique 2014 SDI Educational Technical Report. Figures for Laos PDR and taken from the Delivery of Education Services in Lao PDR report. Figures from Afghanistan are taken from the “The Learning Crisis in Afghanistan – SABER SD Report 2017” 104 Chapter 6 : What Matters for Learning? In this Chapter we present analysis of the factors associated with higher learning outcomes in the 2016 SDI data. School, teacher, and student characteristics can interact in complex ways. For example, urban schools may obtain higher learning outcomes than rural areas not because of better teaching, but because of different student populations. To more clearly isolate the factors which correlate with learning, we conduct a regression analysis on a wide range of school, teacher, and student characteristics to capture the most significant correlates of learning; Table 6.1 presents the results for the overall test score, English, Swahili, Mathematics and Non-verbal reasoning outcomes. Even though these results cannot be interpreted as causal, they provide some insight on the factors that may have an impact on learning outcomes. 49 Section I. What Matters for Learning? School factors Urban schools achieve higher scores than rural schools. 50 Despite the considerable catching-up by rural schools, urban schools remain in the lead in terms of learning. Schools in urban areas obtained an average 75 percent in overall learning scores, versus 60.9 percent in rural schools. Even while controlling for amenities such as piped water, road access, and for the number of students, urban schools still achieve higher learning outcomes than rural schools. 51 Large schools achieve higher scores than small schools. Similarly, despite the catching-up of smaller schools with the learning outcomes of larger schools, large schools retain an advantage. The largest one-fifth of schools obtained average overall scores of 70 percent in 2016, versus 60 percent in the smallest one-fifth of schools. Controlling for urban/rural status, physical amenities, and other factors, large schools still achieve significantly higher learning outcomes. Schools with more physical facilities achieve higher learning outcomes. The SDI includes two minimum indicators for school physical inputs. A school has minimum infrastructure availability if it has classrooms with adequate illumination and clean, functioning toilets; schools above this minimum standard achieved average overall scores 6.7 percent higher than those below the standard. A school has minimum classroom equipment availability if a randomly selected Standard 4 classroom has a functioning blackboard and chalk, and at least 90 percent of students had both pen or pencil and an exercise book. Schools meeting the standard obtain average overall scores 8.7 percentage points above those which do not. These differences are reinforced in the regression analysis. We assign an infrastructure score to each school based on the availability of water, electricity, and functional toilets; schools with a higher score achieved significantly higher learning overall outcomes, as did schools achieving the minimum classroom equipment standard. Teacher factors Schools with more teachers obtain significantly higher learning outcomes. Tanzania’s policies establish a target for primary school average student-teacher ratio of 40 students per teacher. Schools with a student-teacher ratio below the target achieved higher overall learning outcomes in 2014, obtaining average overall scores of 60 points versus 51 in schools with a student-teacher ratio above 40. Even controlling for school infrastructure, size, and other characteristics, schools with lower student-teacher ratio achieve significantly higher learning outcomes than those with higher student-teacher ratio. The finding suggests that addressing student-teacher ratio disparities is a key area of potential for Government to obtain further increases in learning. More experienced teachers are associated with lower learning outcomes. The average years of teachers’ experience at a school is significant and slightly negative: the more experienced a school’s teachers, the lower the school’s learning outcomes in 2016. The finding is consistent with those from other countries (Andrabi, et al., 2009; Aslam & Kingdon, 2011; Hanushek & Rivkin, 2006; Barrera-Osorio, et al., 2016). Barrera-Osorio et al. 49 We conduct multi-level model regression analysis of a range of school, teacher and student characteristics on learning outcomes in Math, English, Swahili and non-verbal reasoning (NVR). We employ strata fixed effects to compare schools within Dar es Salaam; other urban areas; rural areas; and EQUIP-T regions with other schools in the same stratum. 50 Urban dummy employed in OLS regressions. 51 Impacts on student total overall scores reported unless subjects specified. 105 (2016) speculate that teachers do no actually build skills or convert their skills into effective teaching, presumably due to poor governance and accountability in the Government school system. Teacher knowledge and effort are associated with higher learning outcomes. Given the extensive improvements in teacher knowledge observed in the 2016 SDI data, we would expect to observe a relationship between knowledge and learning outcomes. Indeed, in the case of English, we find that in simple comparisons, the top fifth of schools in terms of overall teacher English knowledge achieved student English test outcomes an average 11 points higher than those in the bottom fifth. In the regression analysis, we find that teachers’ subject knowledge in English was also positively and significantly correlated with student performance in English. 52 In addition, one measure of teacher effort – whether teachers assigned and collected homework – is positively and significantly correlated with student learning outcomes. Female teachers appear to be concentrating in schools with more able students. The share of female teachers at a school is significantly and positively correlated with student learning. This might appear evidence that Tanzania’s female teachers are more effective than the males; however, analysis of another student test indicator, non-verbal reasoning, suggests an alternative explanation. In addition to learning assessments in Math, English, and Swahili, the SDI includes a test of student non-verbal reasoning (NVR) ability based on Raven’s Progressive Matrices, widely accepted as a test of reasoning ability. The majority of factors which have a positive impact on student learning, such as student-teacher ratio and assignment of homework, have no significant relationship to NVR scores, providing confidence that the measure correctly identifies non-verbal reasoning ability rather than cognitive achievement. However, the share of female teachers at a school is highly positively correlated with NVR scores, suggesting that the higher learning outcomes at schools with a large share of female teachers reflect not improved teaching effectiveness, but larger share of students with non-verbal reasoning ability. Evidence from Malawi suggests that female teachers are particularly able to exercise choice over teacher placements owing to rules allowing reassignment on the basis of marriage (Asim, et al., 2017). Teachers’ average number of years of formal education are also positively correlated with NVR, suggesting that teachers with more years of schooling sort in schools with better students. Student factors Female and overage students underperform male and younger students. Controlling for age, teacher characteristics, and other factors, female students still underperform substantially in terms of learning. The finding suggests that urgent attention is needed to the drivers of unequal performance by girls. In addition, overage students – those 11 years of age or above – obtain lower learning outcomes by a similar degree. Students who ate breakfast on the day of testing did not achieve higher learning outcomes. As a measure of student socioeconomic status, the SDI asks sampled students whether they ate breakfast on the day of testing. Child nutrition is a serious challenge in Tanzania, where an estimated 2.7 million children under five are estimated to be suffering from malnutrition-related stunting (UNICEF, 2018). In response, the last years have seen a large increase in the provision of official and unofficial school feeding programs. In a range of studies from developed and developing countries, malnutrition and hunger are associated with significant negative impacts on learning, and students who miss breakfast are slower to respond to test questions and make more errors (Levin, 2011). Surprisingly, the 2016 SDI data suggests that students who had eaten breakfast on the day of the visit achieved slightly lower learning outcomes than those who had not, in Math and in overall learning score. We speculate that school feeding program is insufficient to address nutritional disadvantages and poor early childhood investments that children faced in early years of development 52 However, the relationship in other subjects appears less robust: the top fifth of schools in terms of Swahili teacher knowledge achieved only slightly better average student Swahili scores than the bottom fifth (1.6 points), while for Math the schools with the highest teacher knowledge achieved slightly lower scores than the bottom fifth. Similarly, in regression analysis, teacher Math knowledge is not associated with higher student learning outcomes. Further research is required to identify the differential dynamics of teacher knowledge and student performance across subjects. 106 Table 6.1: Multi-level (Hierarchical Linear MLE) Model Estimation Results 2016 - Test Scores Panel A: Multilevel Regression Analysis: Total (1) (2) (3) (4) (5) (6) COMBINED Score in 3 tests out of total max BIVARIATE SCHOOL TEACHER STUDENT COMBINED (STRATA) 13.53*** 6.71*** 5.53** School is in an urban area (1.05) (2.20) (2.57) 7.93*** 3.27* 4.06** 4.52** Infrastructure index (0-3) (0.98) (1.72) (1.86) (1.86) 8.47*** 7.12*** 5.87*** 5.60*** Minimum equipment available (1.07) (1.78) (1.94) (1.95) 3.75*** 0.68 0.13 0.31 Road (gravel or tarmac) (0.89) (1.51) (1.67) (1.69) -1.94*** -1.62*** -1.12** -1.13** Student-Teacher Ratio (0.20) (0.39) (0.45) (0.46) 0.05*** 0.05*** 0.02 0.03 Number of students/(10) (0.01) (0.01) (0.02) (0.02) 1.97*** 1.86*** 0.52 0.52 Share of female teachers (x10) (0.16) (0.31) (0.37) (0.38) 2.25*** 1.88** 1.20 1.28 Average years of formal education (0.44) (0.82) (0.79) (0.80) 0.11 -0.10 -0.25* -0.24* Teaching experience (years) (0.07) (0.14) (0.14) (0.14) -7.59*** -7.37** -5.25 -5.32 (mean)Teacher class presence (1.96) (3.56) (3.44) (3.47) -0.15 4.51 3.06 3.16 Share of teachers using textbook during lessons (1.58) (2.92) (2.80) (2.81) 3.16*** 3.47* 4.10** 3.94** Share of teachers that assign/collect homework (1.14) (2.09) (1.99) (2.00) 0.95 0.45 0.44 1.13 (mean) Teacher asked questions (1.50) (2.75) (2.61) (2.62) 3.65*** 2.73 2.62 2.66 (mean) Positive feedback (1.08) (2.02) (1.95) (1.96) -7.49*** -7.32** -7.19** -6.58** (mean) Teacher never hit a student (1.81) (3.45) (3.30) (3.30) -0.12*** -0.08 -0.08 -0.08 (mean) Average teacher Math knowledge (0.03) (0.05) (0.05) (0.05) 0.21*** 0.20* 0.12 0.14 (mean) Average Teacher English Knowledge (0.06) (0.11) (0.10) (0.10) Swahili spoken at home 6.80*** 3.78*** 1.69 1.72 (0.87) (1.03) (1.13) (1.14) Overaged -6.24*** -3.51*** -2.51** -2.53** (0.93) (0.93) (1.01) (1.01) Girl -2.10** -2.19*** -2.35*** -2.38*** (0.88) (0.80) (0.87) (0.87) Student had breakfast 0.78 -0.18 -0.03 -0.11 (0.93) (0.96) (1.05) (1.05) 56.47*** 35.97*** 65.18*** 48.86*** 50.16*** Constant (3.42) (11.18) (1.29) (11.48) (12.54) level1 Observations 3889 3429 3999 3329 3329 level2 Observations 388 343 400 333 333 student level variance 567.32 576.55 565.46 573.80 573.81 school level variance 147.30 157.00 188.95 134.32 135.27 ICC 0.21 0.21 0.25 0.19 0.19 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 107 Panel B. Multilevel Regression Analysis: Math (1) (2) (3) (4) (5) (6) Average student score in Math test BIVARIATE SCHOOL TEACHER STUDENT COMBINED COMBINED (STRATA) School is in an urban area 8.43*** 6.97*** 4.81** (0.80) (1.77) (1.97) Infrastructure index (0-3) 3.35*** 0.63 0.76 1.36 (0.75) (1.39) (1.46) (1.46) Minimum equipment available 4.26*** 3.76*** 3.77** 3.91*** (0.81) (1.43) (1.49) (1.50) Road (gravel or tarmac) 1.33** -0.52 -1.66 -2.14 (0.67) (1.21) (1.28) (1.31) Student-Teacher Ratio -0.90*** -0.50 -0.33 -0.29 (0.15) (0.31) (0.34) (0.35) Number of students (/10) 0.02*** 0.01 0.00 0.00 (0.01) (0.01) (0.01) (0.01) Share of female teachers (x10) 1.09*** 0.92*** 0.35 0.47 (0.12) (0.24) (0.29) (0.29) Average years of formal education 1.92*** 1.64*** 1.07* 1.07* (0.33) (0.63) (0.62) (0.63) Teaching experience (years) 0.14** 0.02 -0.02 -0.03 (0.05) (0.10) (0.10) (0.10) (mean) Teacher class presence -1.04 -0.45 1.03 0.89 (1.48) (2.73) (2.69) (2.70) Share of teachers using textbook -3.69*** -1.23 -1.95 -1.86 (1.19) (2.26) (2.21) (2.22) Share of teachers that assign/collect homework 2.31*** 3.56** 3.64** 3.68** (0.86) (1.59) (1.55) (1.55) (mean) Teacher asked questions 1.31 1.17 1.49 1.56 (1.13) (2.08) (2.02) (2.03) (mean) Positive feedback 1.10 0.93 1.28 1.01 (0.81) (1.55) (1.53) (1.53) (mean) Teacher never hit a student -1.05 -1.98 -2.43 -1.80 (1.37) (2.43) (2.38) (2.38) (mean) Average Teacher Math Knowledge Score -0.02 0.00 0.01 0.03 (0.02) (0.04) (0.04) (0.04) Swahili spoken at home 4.67*** 3.24*** 2.25*** 2.07** (0.66) (0.78) (0.83) (0.83) Overaged (>=11) -1.89*** -0.97 -0.28 -0.32 (0.71) (0.70) (0.74) (0.74) Girl -2.98*** -3.01*** -3.05*** -3.10*** (0.66) (0.60) (0.63) (0.63) Student had breakfast -0.41 -0.70 -1.09 -1.16 (0.70) (0.72) (0.76) (0.76) Constant 61.90*** 41.83*** 65.90*** 48.90*** 50.86*** (2.75) (8.30) (0.98) (8.57) (9.29) level1 Observations 3889 3709 3999 3599 3599 level2 Observations 388 371 400 360 360 student level variance 322.86 325.61 320.21 322.81 322.83 school level variance 100.03 102.26 110.71 94.21 93.86 ICC 0.24 0.24 0.26 0.23 0.23 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 108 Panel C. Multilevel Regression Analysis: English (1) (2) (3) (4) (5) (6) Average student score in English test BIVARIATE SCHOOL TEACHER STUDENT COMBINED COMBINED (STRATA) School is in an urban area 17.87*** 8.02*** 6.23* (1.27) (3.05) (3.53) Infrastructure index (0-3) 10.89*** 4.64* 5.04* 5.70** (1.20) (2.39) (2.58) (2.59) Minimum equipment available 11.87*** 9.92*** 8.68*** 8.49*** (1.30) (2.46) (2.68) (2.69) Road (gravel or tarmac) 5.03*** 0.92 -0.17 -0.01 (1.09) (2.09) (2.31) (2.34) Student-Teacher Ratio -2.74*** -2.37*** -1.72*** -1.71*** (0.25) (0.53) (0.63) (0.64) Number of students/(10) 0.07*** 0.07*** 0.04* 0.04* (0.01) (0.02) (0.02) (0.02) Share of female teachers (x10) 2.55*** 2.41*** 0.48 0.48 (0.19) (0.43) (0.51) (0.52) Average years of formal education 2.62*** 2.10* 1.33 1.34 (0.55) (1.14) (1.11) (1.12) Teaching experience (years) 0.12 -0.03 -0.21 -0.21 (0.09) (0.19) (0.19) (0.19) (mean) Teacher class presence -12.83*** -13.20*** -10.48** -10.84** (2.40) (4.97) (4.82) (4.86) Share of teachers using textbook during lessons 0.24 5.40 2.94 3.00 (1.94) (4.04) (3.88) (3.89) Share of teachers that assign/collect homework 5.02*** 5.83** 6.84** 6.68** (1.40) (2.89) (2.77) (2.78) (mean) Teacher asked questions 2.10 0.52 0.67 1.53 (1.84) (3.82) (3.65) (3.66) (mean) Positive feedback 4.70*** 1.81 1.66 1.69 (1.32) (2.78) (2.69) (2.70) (mean) Teacher never hit a student -12.99*** -13.47*** -13.22*** -12.47*** (2.22) (4.81) (4.63) (4.61) (mean) Average Teacher English Knowledge 0.30*** 0.33** 0.23* 0.25* (0.07) (0.14) (0.14) (0.14) Swahili spoken at home 7.24*** 3.73*** 2.13* 2.15* (1.07) (1.08) (1.15) (1.15) Overaged -9.22*** -5.26*** -4.52*** -4.54*** (1.14) (0.92) (0.97) (0.98) Girl -3.24*** -3.57*** -3.37*** -3.39*** (1.08) (0.80) (0.84) (0.84) Student had breakfast 2.17* 0.84 1.43 1.38 (1.14) (0.96) (1.03) (1.03) Constant 42.85*** 17.91 55.35*** 35.44** 39.30** (4.74) (15.17) (1.50) (15.60) (17.06) level1 Observations 2723 2443 2800 2373 2373 level2 Observations 388 349 400 339 339 student level variance 372.02 363.56 365.24 356.26 356.26 school level variance 338.56 367.92 423.71 327.64 328.55 ICC 0.48 0.50 0.54 0.48 0.48 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 109 Panel D. Multilevel Regression Analysis: Swahili (1) (2) (3) (4) (5) (6) Average student score in Swahili test BIVARIATE SCHOOL TEACHER STUDENT COMBINED COMBINED (STRATA) School is in an urban area 7.37*** 3.85* 2.33 (1.47) (2.15) (2.37) Infrastructure index (0-3) 4.41*** 1.97 1.17 1.20 (1.37) (1.69) (1.78) (1.80) Minimum equipment available 3.63*** 2.94* 2.51 2.38 (1.49) (1.74) (1.80) (1.80) Road (gravel or tarmac) 2.40** 0.76 0.21 0.42 (1.24) (1.47) (1.57) (1.59) Student-Teacher Ratio -0.95*** -0.75** -0.61 -0.67 (0.28) (0.38) (0.42) (0.43) Number of students/(10) 0.03*** 0.02* 0.01 0.02 (0.01) (0.01) (0.01) (0.02) Share of female teachers (x10) 1.25*** 1.22*** 0.63* 0.63* (0.22) (0.28) (0.35) (0.36) Average years of formal education 1.63 1.26* 1.00 1.11 (0.62) (0.76) (0.76) (0.76) Teaching experience (years) 0.08 -0.08 -0.14 -0.12 (0.10) (0.12) (0.13) (0.13) (mean) Teacher class presence -0.53 0.35 1.03 1.16 (2.74) (3.33) (3.31) (3.32) Share of teachers using textbook during lessons 1.08 1.75 1.24 1.22 (2.20) (2.70) (2.68) (2.68) Share of teachers that assign/collect homework -0.18 0.08 -0.01 -0.08 (1.59) (1.93) (1.90) (1.90) (mean) Teacher asked questions -1.20 -2.10 -2.08 -1.81 (2.09) (2.53) (2.48) (2.49) (mean) Positive feedback 3.13** 2.99 2.64 2.73 (1.50) (1.86) (1.86) (1.86) (mean) Teacher never hit a student 0.09 -0.20 0.17 0.32 (2.54) (2.99) (2.95) (2.95) Swahili spoken at home 6.61*** 6.18*** 3.68** 3.87** (1.21) (1.35) (1.51) (1.54) Overaged -1.40 0.99 2.52* 2.52* (1.30) (1.33) (1.42) (1.43) Girl 2.72** 3.00** 3.10** 3.13** (1.22) (1.20) (1.25) (1.25) Student had breakfast -0.50 -1.14 -1.68 -1.71 (1.31) (1.35) (1.43) (1.44) Constant 87.20*** 69.25*** 86.23*** 69.33*** 68.20*** (3.35) (9.65) (1.66) (10.09) (11.06) level1 Observations 1166 1142 1199 1109 1109 level2 Observations 388 381 400 370 370 student level variance 356.24 372.76 356.52 370.54 370.65 school level variance 76.32 78.53 77.65 69.57 69.54 ICC 0.18 0.17 0.18 0.16 0.16 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 110 Panel E. Multilevel Regression Analysis: Non-verbal reasoning (1) (2) (3) (4) (5) (6) Average student score in NVR BIVARIATE SCHOOL TEACHER STUDENT COMBINED COMBINED (STRATA) School is in an urban area 5.78*** 3.88** 0.71 (0.89) (1.63) (1.75) Infrastructure index (0-3) 2.34*** 0.68 0.37 0.68 (0.83) (1.27) (1.31) (1.32) Minimum equipment available 2.55*** 2.62** 2.33* 2.36* (0.90) (1.31) (1.32) (1.32) Road (gravel or tarmac) 1.66** 0.65 0.30 0.30 (0.75) (1.11) (1.15) (1.17) Student-Teacher Ratio -0.14 0.01 0.26 0.25 (0.17) (0.29) (0.31) (0.32) Number of students/(10) 0.03*** 0.02** 0.01 0.00 (0.01) (0.01) (0.01) (0.01) Share of female teachers (x10) 1.00*** 0.91*** 0.74*** 0.77*** (0.13) (0.21) (0.26) (0.26) Average years of formal education 1.89*** 1.30** 1.15** 1.12** (0.37) (0.55) (0.56) (0.56) Teaching experience (years) -0.01 -0.16* -0.17* -0.16* (0.06) (0.09) (0.09) (0.09) (mean) Teacher class presence 2.08 2.80 2.57 2.32 (1.65) (2.41) (2.44) (2.45) Share of teachers using textbook during lessons -2.38* -1.28 -1.90 -2.07 (1.32) (1.95) (1.96) (1.96) Share of teachers that assign/collect homework -0.41 0.15 0.28 0.27 (0.96) (1.40) (1.40) (1.40) (mean) Teacher asked questions -2.84** -3.34* -3.15* -2.93 (1.26) (1.83) (1.83) (1.83) (mean) Positive feedback -0.16 0.76 0.77 0.79 (0.91) (1.35) (1.36) (1.37) (mean) Teacher never hit a student 1.80 1.29 1.35 1.50 (1.53) (2.16) (2.16) (2.16) Swahili spoken at home 2.44*** 1.26 -0.36 -0.31 (0.74) (0.87) (0.93) (0.94) Overaged -2.61*** -2.59*** -1.84** -1.89** (0.79) (0.82) (0.86) (0.86) Girl -3.26*** -3.30*** -3.28*** -3.28*** (0.74) (0.72) (0.74) (0.74) Student had breakfast 1.52** 0.64 0.48 0.46 (0.78) (0.84) (0.86) (0.87) Constant 52.36*** 40.79*** 60.78*** 42.35*** 44.33*** (2.53) (6.99) (1.06) (7.39) (8.09) level1 Observations 3889 3809 3999 3699 3699 level2 Observations 388 381 400 370 370 student level variance 473.16 472.47 470.32 469.57 469.55 school level variance 64.22 59.64 69.83 57.94 57.47 ICC 0.12 0.11 0.13 0.11 0.11 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 111 Section II. What factors are associated with improvement in scores? The changes in learning outcomes observed in the student population as a whole provide only a rough guide to the dynamics of change in individual schools. Because the SDI employs the same sample of 400 schools in both 2014 and 2016, it is possible to examine which changes within particular schools were associated with improvements in student performance. We estimate the impact of an increase or improvement in a range of school characteristics, as well as school average teacher and student characteristics, on the likelihood that a school achieved an increase in learning outcomes. 53 Table 6.2 presents the regression results associated with improvement in scores. The results indicate areas where focused intervention to raise standards is likely to have positive impacts on learning. Again it should be noted that even though these results cannot be interpreted as causal, they provide some insight on the factors that may have an impact on learning outcomes Improvements in teacher content knowledge matter. In the analysis of the 2016 data, we found that schools with higher teacher content knowledge, and teachers who assign homework, are associated with higher learning outcomes. We find, too, that schools where these factors improved were more likely to obtain an increase in learning. A ten percentage-point improvement in school average teacher scores in English knowledge was associated with a five percent increase in the likelihood that student English scores improved; for Math, a ten point improvement in school average teacher content knowledge was associated with a two percent improvement in the likelihood of an increase in student scores. The findings suggest that improvement in the subject knowledge of teachers played a key role in the improvement in student learning observed between 2014 and 2016; and that efforts to further increase teachers’ knowledge may help drive further gains in learning. Improvements in teacher pedagogical skills matter. Similarly, the finding from the 2016 analysis that students whose teachers assign and collect homework achieve higher scores also holds when we focus on improvement in outcomes. Schools where the share of teachers who assigned and collected homework increased by ten percent between 2014 and 2016 were one percent more likely to experience an improvement in overall Math scores. This suggests that encouraging teachers to set and mark homework may be a low-cost way to improve student performance. 54 Improvements in physical inputs by themselves do not seem to drive improved outcomes. As we noted in the previous sections, although schools with higher levels of physical inputs, such as infrastructure and classroom equipment have higher learning outcomes, schools with lower levels ‘caught up’ substantially between 2014 and 2016. This suggests that the improvements observed throughout the system in learning did not stem primarily from physical input investments. The regression analysis confirms that, within a single school, improvements in physical inputs did not appear to be associated with improvement in learning outcomes. Schools which increased their level of infrastructure availability between 2014 and 2016 were not significantly more likely to gain in average student test scores in any subject. Similarly, schools which achieved the minimum standard for classroom equipment availability between 2014 and 2016 were not significantly more likely to experience improvements in student scores in any subject and, surprisingly, were significantly less likely to obtain improvements in Math scores. Schools which gained additional classrooms were also not significantly more likely to improve scores. The findings suggest that improvements in school physical conditions may not be an effective way to drive improvements in learning. 53 In addition to estimating the impact on an increase in key characteristics on the likelihood of increased student average scores, we also conduct additional estimations on a sub- sample basis for schools with increasing scores and decreasing scores. Findings are consistent across all estimations; results available on request. 54 Surprisingly, however, both the 2016 and delta analyses find that a key indicator of teacher effort – teacher presence in the classroom – is not significantly associated with learning outcomes. The finding is surprising in the light of a similar analysis carried out on SDI data from seven countries, including the previous Tanzanian surveys in 2010 and 2014, by Bold et al (2017). Controlling for a range of student, school and teacher factors and employing district fixed effects, they find that teacher presence in the classroom is strongly and positively associated with student performance. The most likely explanation is that the classroom presence indicator is subject to measurement error or survey bias. 112 Table 6.2: School Gain Estimation Results (OLS) - Test Scores Panel A. Multivariate Regression Model: Math (1) (2) (3) (4) School Improved in Math Score (0/1) SCHOOL TEACHER STUDENT FULL MODEL (Strata F.E.) Number of classrooms 0.008 0.008 (0.008) (0.010) Infrastructure index -0.023 -0.024 (0.046) (0.046) Minimum equipment available -0.021 -0.053 (0.042) (0.045) Student-Teacher Ratio -0.005 -0.011 (0.013) (0.015) Number of students (divided by 10) -0.006 -0.004 (0.007) (0.008) Share of female teachers (x10) -0.001 -0.001 (0.002) (0.001) Average years of formal education 0.013 0.010 (0.021) (0.020) Teaching experience (years) -0.009*** -0.006 (0.003) (0.004) (mean) Teacher class presence (x100) -0.000 0.001 (0.001) (0.001) Share of teachers using textbook during lessons (x100) 0.000 0.000 (0.001) (0.001) Share of teachers that assign/collect homework(x10) 0.014*** 0.009* (0.005) (0.005) (mean) Teacher asked questions(x100) 0.000 0.000 (0.001) (0.001) (mean) Positive feedback(x100) -0.001 -0.000 (0.000) (0.001) (mean) Teacher never hits student(x100) 0.000 -0.000 (0.001) (0.001) (mean) Average teacher Math knowledge(x10) 0.025** 0.019* (0.011) (0.011) Share of overaged students (x10) -0.011 -0.009 (0.010) (0.011) Share of female students (x10) 0.000 -0.006 (0.009) (0.010) Share of students with Swahili spoken at home (x10) -0.003 0.002 (0.007) (0.008) Share of students had breakfast (x10) -0.023*** -0.023*** (0.007) (0.008) Constant 0.671*** 0.668*** 0.677*** 0.677*** (0.025) (0.026) (0.024) (0.028) Observations 392 343 390 342 R-squared 0.0066 0.0629 0.0307 0.1421 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 113 Panel B. Multivariate Regression Model: English (1) (2) (3) (4) School Improved in English Score (0/1) SCHOOL TEACHER STUDENT FULL MODEL (Strata F.E.) Number of classrooms 0.003 0.001 (0.008) (0.009) Infrastructure index -0.090** -0.063 (0.045) (0.054) Minimum equipment available 0.047 -0.028 (0.042) (0.049) Student-Teacher Ratio -0.023* -0.013 (0.014) (0.017) Number of students (divided by 10) 0.005 0.008 (0.010) (0.009) Share of female teachers (x10) -0.002 -0.002 (0.002) (0.002) Average years of formal education 0.021 0.008 (0.022) (0.022) Teaching experience (years) -0.007** -0.005 (0.003) (0.004) (mean) Teacher class presence (x100) -0.002*** -0.001 (0.001) (0.001) Share of teachers using textbook during lessons (x100) 0.000 0.000 (0.001) (0.001) Share of teachers that assign/collect homework(x10) 0.005 0.002 (0.005) (0.005) (mean) Teacher asked questions(x100) 0.000 0.000 (0.001) (0.001) (mean) Positive feedback(x100) 0.000 0.000 (0.000) (0.000) (mean) Teacher never hits student(x100) 0.000 0.000 (0.001) (0.001) (mean) Average teacher English knowledge(x10) 0.062*** 0.053*** (0.019) (0.020) Share of overaged students (x10) -0.003 0.014 (0.010) (0.011) Share of female students (x10) 0.001 0.003 (0.009) (0.010) Share of students with Swahili spoken at home (x10) -0.010 -0.004 (0.007) (0.008) Share of students had breakfast (x10) -0.027*** -0.030*** (0.006) (0.007) Constant 0.688*** 0.746*** 0.698*** 0.740*** (0.024) (0.032) (0.024) (0.034) Observations 399 316 397 316 R-squared 0.0213 0.0684 0.0508 0.1587 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 114 Panel C. Multivariate Regression Model: Swahili (1) (2) (3) (4) School Improved in Swahili Score (0/1) SCHOOL TEACHER STUDENT FULL MODEL (Strata F.E.) Number of classrooms 0.010 0.008 (0.009) (0.010) Infrastructure index 0.035 0.022 (0.047) (0.047) Minimum equipment available 0.051 0.051 (0.042) (0.043) Student-Teacher Ratio -0.004 -0.012 (0.013) (0.016) Number of students (divided by 10) 0.006 0.008 (0.008) (0.007) Share of female teachers (x10) -0.002 -0.002 (0.002) (0.002) Average years of formal education -0.003 -0.009 (0.020) (0.020) Teaching experience (years) -0.003 -0.002 (0.003) (0.004) (mean) Teacher class presence (x100) -0.000 0.000 (0.001) (0.001) Share of teachers using textbook during lessons (x100) -0.000 -0.000 (0.001) (0.001) Share of teachers that assign/collect homework (x10) 0.001 -0.005 (0.005) (0.005) (mean) Teacher asked questions (x100) -0.000 -0.001 (0.001) (0.001) (mean) Positive feedback (x100) -0.001 -0.000 (0.000) (0.000) (mean) Teacher never hits student (x100) -0.000 -0.000 (0.001) (0.001) Share of overaged students (x10) -0.009 -0.005 (0.010) (0.010) Share of female students (x10) 0.006 0.008 (0.010) (0.011) Share of students with Swahili spoken at home (x10) -0.006 -0.003 (0.008) (0.008) Share of students had breakfast (x10) -0.011 -0.004 (0.007) (0.008) Constant 0.640*** 0.661*** 0.653*** 0.661*** (0.025) (0.026) (0.025) (0.028) Observations 380 356 378 355 R-squared 0.0130 0.0158 0.0111 0.1003 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 115 Panel D. Multivariate Regression Model: Non-Verbal Reasoning (1) (2) (3) (4) School Improved in Non-verbal Reasoning Score (0/1) SCHOOL TEACHER STUDENT FULL MODEL (Strata F.E.) Number of classrooms -0.000 0.002 (0.009) (0.009) Infrastructure index 0.045 0.044 (0.050) (0.051) Minimum equipment available -0.008 -0.030 (0.046) (0.050) Student-Teacher Ratio -0.014 -0.012 (0.015) (0.016) Number of students (divided by 10) 0.013** 0.009 (0.007) (0.007) Share of female teachers (x10) -0.001 -0.001 (0.002) (0.002) Average years of formal education 0.018 0.010 (0.021) (0.022) Teaching experience (years) -0.002 -0.000 (0.004) (0.004) (mean) Teacher class presence (x100) 0.002** 0.002*** (0.001) (0.001) Share of teachers using textbook during lessons (x100) -0.000 -0.000 (0.001) (0.001) Share of teachers that assign/collect homework (x10) 0.002 -0.003 (0.005) (0.005) (mean) Teacher asked questions (x100) -0.001 -0.001 (0.001) (0.001) (mean) Positive feedback (x100) -0.000 0.000 (0.001) (0.001) (mean) Teacher never hits student (x100) -0.001 -0.001 (0.001) (0.001) Share of overaged students (x10) 0.007 0.009 (0.010) (0.011) Share of female students (x10) 0.002 0.005 (0.011) (0.011) Share of students with Swahili spoken at home (x10) -0.006 -0.005 (0.008) (0.009) Share of students had breakfast (x10) -0.017** -0.015* (0.007) (0.008) Constant 0.574*** 0.545*** 0.571*** 0.547*** (0.027) (0.028) (0.027) (0.030) Observations 369 343 367 342 R-squared 0.0095 0.0284 0.0201 0.0883 Standard errors in parenthesis; *p<0.1, **p<0.05, ***p<0.01 116 Conclusions: What lessons can we draw? The preceding analysis provides a range of findings regarding services in Tanzania’s primary schools, many of which run counter to prevailing wisdom or evidence from other developing countries. Given that the SDI/SABER SD remain new instruments, care must be taken not to over- interpret the findings. However, some clear lessons do emerge: Learning outcomes have improved. The 2016 SDI shows a clear pattern of improving learning levels in Tanzania’s primary schools. Lower- performing regions appear to be catching up with previously better-performing regions, and the EQUIP-T regions in particular have seen rapid increases in student test outcomes. The findings confirm those of EGRA, EGMA and PSLE among others. The students tested by SDI in 2016 will enter Form I in 2018/19; in order to achieve the maximum impact from the gains in primary school learning, careful targeting of resources will be required to maximize the efficiency and efficacy of lower secondary education, particularly in disadvantaged regions. Teaching knowledge and effort matter for learning. The SDI 2016 data shows clear increases in teacher subject knowledge. In both analyses of the 2016 learning outcomes, and of improvements in learning between 2014 and 2016, increased teacher subject knowledge is associated with higher learning outcomes. Teacher effort, captured by the assignment and collection of homework, is also associated with higher learning outcomes. The findings suggest that investing in teacher motivation and knowledge are valuable priorities to improve learning outcomes. This is in line with the emphasis of SEQUIP, which emphasizes capacity building for teachers in science and Math teaching and the use of ICT in the classroom. Physical school inputs by themselves do not appear to be correlated with improvement in learning. In regression analyses of both the 2016 data and the improvement in school performance over time, the availability of facilities and physical inputs in school – toilets, piped water, road access, and additional classrooms – are not connected with higher or improving learning outcomes. The evidence suggests that while investment in physical infrastructure is likely to be important to allow both primary and secondary education in Tanzania to meet the needs of increasing enrollment, enhancements to school buildings and facilities are unlikely to directly drive improvements in learning. School management practices in Tanzanian primary schools are very weak. In the D-WMS survey, we found that Tanzanian schools have an establishment with practically no structured management practices or very poor management practices implemented. As in the case of teacher policies, there is a need to strengthen recruitment, support, and monitoring of principals. On recruitment, candidates should be screened to check for teaching and management skills. On principal support, there is a need to help principals become better at using data to guide instruction, observing teachers, and providing them with useful feedback to improve their practice. This involves redefining principals’ role and strengthening both pre-service and in-service training. Once the system is strengthened to support principals, a monitoring system is needed to assess which principals are good performers and which need more help. Late enrollment impairs children’s chances in school. Overage students attain significantly lower learning outcomes in our analysis. As Tanzania does not currently practice grade repetition, the primary source of overage students is late enrollment; efforts to ensure that all students enroll at the correct age are likely to have benefits for overall learning outcomes. In addition, the lower performance of older students, which is in line with findings from other countries such as Malawi, which do practice repetition, suggests that the introduction of repetition in Tanzania would be unlikely to raise learning outcomes. Non-Swahili speakers and girls face persistent disadvantages. Both non-Swahili native speaking students and female students achieve scores consistently below male and Swahili-speaking students. These barriers have not reduced as learning outcomes have risen. With half of all students female and a majority non-Swahili native speaking, these disparities are likely to restrict the overall improvement in learning outcomes over time, as well as posing a threat to equity. Attention is needed to school and teaching practices, both in primary schools at secondary level, to ensure that these students, as well as overage students, are not excluded from school cultures or denied their full learning potential. 117 References Adelman, M., Lemos, R., Vargas, M. J. & Nayar, R., 2018. Managing for learning: (in)coherence in education systems in Latin America. The World Bank. Adukia, A., 2017. Sanitation and Education. American Economic Journal: Applied Economics, 9(2), pp. 23-59. Andrabi, T., Das, J., Khwaja, A. & and Zajonc, T., 2009. Do value-added estimates add value? Accounting for learning dynamics, Washington D.C.: The World Bank. Angrist, J. & Lavy, V., 1999. Using Maimonides' rule to estimate the effect of class size on scholastic achievement. The Quarterly Journal of Economics, 114(2), pp. 533-575.. Araujo, M. C., Carneiro, P., Cruz-Aguayo, Y. & Schady, N., 2016. Teacher Quality and Learning Outcomes in Kindergarten. Quarterly Journal of Economics, 131(3), pp. 1415-1453.. Asim, S., Chimombo, J., Chugunov, D. & Gera, R., 2017. Moving teachers to Malawi's remote communities: a data-driven approach to teacher deployment., World Bank Policy Research Working Paper 8253, Washington, D.C.: World Bank Group. Aslam, M. & Kingdon, G., 2011. What can teachers do to raise pupil achievement?. Economics of Education Review, 30(3), pp. 559-574.. Barde, J. & Walkiewicz, J., 2014. Access to piped water and human capital formation: Evidence from Brazilian primary schools (No. 28).. Volume Discussion Paper Series, University of Freiburg, Department of International Economic Policy.. Barrera-Osorio, F., de Galbert, P., Habyarimana, J. & and Sabarwal, S., 2016. Impact of public-private partnerships on private school performance: evidence from a randomized controlled trial in Uganda. , Washington D.C.: The World Bank. Bashir, S., Lockheed, M., Ninan, E. & Tan, J., 2018. Facing forward: Schooling for learning in Africa., Washington D.C.: World Bank Publications. Bau, N. & Das, J., 2017. The misallocation of pay and productivity in the public sector: Evidence from the labor market for teachers. The World Bank. Benton, D. & Burgess, N., 2009. The effect of the consumption of water on the memory and attention of children.. Appetite, 53(1), pp. 143-146.. Bietenbeck, J., Piopiunik, M. & Wiederhold, S., 2018. Africa’s Skill Tragedy Does Teachers’ Lack of Knowledge Lead to Low Student Performance?. Journal of Human Resources,, 53(3), pp. 553-578.. Bishop, J. & Wößmann, L., 2004. Institutional effects in a simple model of educational production. Education Economics, 12(1), pp. 17-38. Bloom, N. et al., 2014. The New Empirical Economics of Management.. Journal of European Economic Association. Bloom, N., Lemos, R., Sadun, R. & Van Reenen, J., 2015. Does management matter in schools?. The Economic Journal, 125(584), pp. 647-674. Bloom, N. & Van Reenen, J., 2007. Measuring and explaining management practices across firms and countries. The Quarterly Journal of Economics, 122(4), pp. 1351-1408.. Bold, T. et al., 2017. What do teachers know and do? does it matter? Evidence from primary schools in Africa.. The World Bank. Bold, T., Filmer, D., Molina, E. & Svensson, J., 2018. The Lost Human Capital: Teacher Knowledge and Student Achievement in Africa. Working Paper, IIES.. Braslavsky, C. & Halil, K., 2006. Textbooks and quality learning for all: Some lessons learned from international experiences. Vol 355 ed. Paris: UNESCO Publishing. Bruns, B. & Luque, J., 2014. Great Teachers: How to Raise Student Learning in Latin America and the Caribbean, Washington, DC: The World Bank.. Burgers, L., 2000. Background and rationale for school sanitation and hygiene education., New York: International Rescue Committee (IRC).. Chaudhury, N. H. J. K. M. M. K. a. R. F. et al., 2006. Missing in action: teacher and health worker absence in developing countries.. Journal of Economic Perspectives, 20(1), pp. 91-116.. Chingos, M., 2013. Class size and student outcomes: Research and policy implications. Journal of Policy Analysis and Management, 32(2), pp. 411- 438.. Coe, R., Aloisi, C., Higgins, S. & Major, L., 2014. What makes great teaching? Review of the underpinning research.. London: The Sutton Trust. Coflan, A. M., Ragatz, A. B., Hasan, A. & Pan, Y., 2018. Understanding effective teaching practices in Chinese classrooms: evidence from a pilot study of primary and junior secondary schools in Guangdong, China (English)., Washington, D.C. : Policy Research working paper; no. WPS 8396. World Bank Group. D'Anci, K., Constant, F. & Rosenberg, I., 2006. Hydration and cognitive function in children. Nutrition reviews, 64(10), pp. 457-464. Danielson, C., 2011. Enhancing professional practice: A framework for teaching. 2nd edition. Association for Supervision and Curriculum Development ed. s.l.:ASCD. Di Liberto, A., Schivardi, F. & Sulis, G., 2014. Managerial Practices and Students' Performance.. Mimeo, p. LUISS. Duflo, E., 2001. Schooling and labor market consequences of school construction in Indonesia: Evidence from an unusual policy experiment. American Economic Review, 91(4), pp. 795-813. Duflo, E., Hanna, R. & Ryan, S., 2012. Incentives work: Getting teachers to come to school.. American Economic Review,, 102(4), pp. 1241-78.. Edmonds, C. a. J. B., 2009. 2009. Does having a drink help you think? 6–7-Year-old children show improvements in cognitive performance from baseline to test after having a drink of water. Appetite, 53(3), pp. 469-472. 118 Fehrler, S., Michaelowa, K. & Wechtler, A., 2009. The effectiveness of inputs in primary education: insights from recent student surveys for Sub- Saharan Africa.. The Journal of Development Studies, 45(9), pp. 1545-1578.. Fentiman, A., Hall, A. & Bundy, D., 1999. 1999. School enrolment patterns in rural Ghana: a comparative study of the impact of location, gender, age and health on children's access to basic schooling. .. Comparative Education, 35(3), pp. 331-349. Glewwe, P., Grosh, M., Jacoby, H. & Lockheed, M., 1995. An eclectic approach to estimating the determinants of achievement in Jamaican primary education, s.l.: World Bank Economic Review. Glewwe, P., Hanushek, E., Humpage, S. & Ravina, R., 2011. School resources and educational outcomes in developing countries: A review of the literature from 1990 to 2010, (No. w17554): National Bureau of Economic Research.. Glewwe, P., Kremer, M. & Moulin, S., 2009. Many children left behind? Textbooks and test scores in Kenya.. American Economic Journal: Applied Economics, 1(1), pp. 112-35.. Hanushek, E. A., 1986. The economics of schooling: Production and efficiency in public schools.. Journal of Economic Literature,, 24(3), pp. 1141- 1177.. Hanushek, E. A., 1992. The trade-off between child quantity and quality. Journal of Political Economy, 10(1), pp. 84-117. Hanushek, E. A., 1995. Interpreting Recent Research on Schooling in Developing Countries. World Bank Research Observer, 10(2), pp. 227-246. Hanushek, E. & Rivkin, S., 2006. Teacher quality. Handbook of the Economics of Education, Volume 2, pp. 1051-1078.. Harbison, R. & Hanushek, E., 1992. Educational Performance of the Poor: Lessons from Rural Northeast, Washington D.C.: The World Bank. Jepsen, S. & Rivkin, S., 2009. Class size reduction and student achievement the potential tradeoff between teacher quality and class size. Journal of Human Resources, 44(1), pp. 223-250.. Johnson, D., 2006. Investing in Teacher Effectiveness to Improve Educational Quality in Developing Countries: does in-service education for primary mathematics teachers in Sri Lanka make a difference to teaching and learning?. Research in Comparative and International Education, , 1(1), pp. 73-87.. Johnson, D., Cunninghan, A. & Dowling, R., 2012. Teaching Standards and Curriculum Review, Washington D.C.: The World Bank Group. Kane, T. & Staiger, D., 2012. Gathering Feedback for Teaching: Combining High-Quality Observations with Student Surveys and Achievement Gains., s.l.: Research Paper MET Project. Bill & Melinda Gates Foundation.. Ko, J. & Sammons, P., 2014. Effective Teaching: A Review of Research and Evidence.. 60 Queens Road, Reading, RG1 4BS, England.: CfBT Education Trust. Kremer, M., 1995. Research on Schooling: What We Know and What We Don't A Comment on Hanushek. The World Bank Research Observer, pp. 247-254. Kremer, M., Brannen, C. & Glennerster, R., 2013. The challenge of education and learning in the developing world. Science,. Science, 340(6130), pp. 297-300.. Lavy, V., 2015. Do differences in schools' instruction time explain international achievement gaps? Evidence from developed and developing countries.. The Economic Journal, 125(588), pp. F397-F424.. Lemos, R. & Scur, D., 2016. Developing management: An expanded evaluation tool for developing countries.. RISE Working Paper 16/007.. Levin, M., 2011. Breakfast for Learning: Scientific research on the link between children’s nutrition and academic performance, Available at: http://frac.org/wp-content/uploads/breakfastforlearning-1.pdf. Last accessed: 3rd December 2018: Food Research and Action Center. Lupien, S., King, S., Meaney, M. & McEwen, B., 2000. Child’s stress hormone levels correlate with mother’s socioeconomic status and depressive state. Biological psychiatry, 48(10), pp. 976-980. Martin, G. H. & Pimhidzai, O., 2013. Education and Health Services in Kenya: Data for Results and Accountability. Service Delivery Indicators, Washington D.C.: World Bank. https://openknowledge.worldbank.org/handle/10986/16288 License: CC BY 3.0 IGO. Mayer, R., 2003. The promise of multimedia learning: using the same instructional design methods across different media. Learning and instruction, 13(2), pp. 125-139. McCoy, D. et al., 2016. Early childhood developmental status in low-and middle-income countries: national, regional, and global prevalence estimates using predictive modeling. PLoS Medicine, 13(6), p. e1002034. Meo, S. et al., 2013. Comparison of the impact of powerpoint and chalkboard in undergraduate medical teaching: an evidence based study. J Coll Physicians Surg Pak, 23(1), pp. 47-50. Metzler, J. & Woessmann, L., 2012. The impact of teacher subject knowledge on student achievement: Evidence from within-teacher within-student variation. Journal of Development Economics, Volume 99, p. 486–496. Mingat, A. & Tan, J. P., 1992. Education in Asia: A comparative study of cost and financing. The World Bank. Mingat, A. & Tan, J. P., 1998. The mechanics of progress in education: Evidence from cross-country data. The Word Bank. Muijs, D. et al., 2014. State of the art–teacher effectiveness and professional learning. School effectiveness and school improvement, 25(2), pp. 231- 256.. Murnane, R. & Ganimian, A., 2014. Improving educational outcomes in developing countries: Lessons from rigorous evaluations., s.l.: Cambridge, MA: National Bureau of Economic Research. O'Reilly, C. et al., 2008. The impact of a school-based safe water and hygiene programme on knowledge and practices of students and their parents: Nyanza. Epidemiology & Infection, 136(1), pp. 80-91. 119 Petzell, M., 2012. The linguistic situation in Tanzania. Moderna språk. Pianta, R., La Paro, K. & Hamre, B., 2007. Classroom Assessment Scoring System (CLASS): Manual. Baltimore: Paul H. Brookes Publishing Co.. Robinson, V., Lloyd, C. & Rowe, K., 2008. The impact of leadership on student outcomes: An analysis of the differential effects of leadership types.. Educational Administration Quarterly, 44(5), pp. 635-674.. Rockmore, C., Shearer, J. & Papakosta, E., 2016. Togo - Service delivery indicators: Education 2013 (English), Washington D.C.: World Bank Group. http://documents.worldbank.org/curated/en/662691504016909364/Education-2013. Rockoff, J., 2004. The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), pp. 247- 252. Sabarwal, S., Evans , D. & Marshak, A., 2014. The permanent input hypothesis: the case of textbooks and (no) student learning in Sierra Leone., Washington D.C.: The World Bank. Service Delivery Indicators, 2014. Education Service Delivery in Tanzania, Washington D.C.: The World Bank. Shallcross, D. & Harrison, T., 2007. Lectures: electronic presentations versus chalk and talk–a chemist’s view. Chemistry Education Research and Practice, 8(1), pp. 73-79. Strong, M., Gargani, J. & Hacifazlioğlu, O., 2011. Do we know a successful teacher when we see one? Experiments in the identification of effective teachers. Journal of Teacher Education, 62(4), pp. 367-382.. Tan, J. P., Lane, J. & Paul Coustere, P., 1997. Putting inputs to work in elementary schools: What can be done in the Philippines?, s.l.: Economic Development and Cultural Change. UNESCO, 2013. The Global Learning Crisis: Why Every Child Deserves a Quality Education, Paris: UNESCO . UNICEF, 2018. Tanzania Nutrition, s.l.: Available at: https://www.unicef.org/tanzania/nutrition.html. Last accessed: 3rd December 2018.. Urquiola, M., 2006. Identifying class-size effects in developing countries: Evidence from rural Bolivia.. Review of Economics and Statistics., 88(1), pp. 171-177. Vieluf, S., Kaplan, D., Klieme, E. & Bayer, S., 2012. Teaching Practices and Pedagogical Innovations: Evidence from TALIS.. Paris: OECD Publishing. Walker, S. et al., 2007. Child development: risk factors for adverse outcomes in developing countries. The Lancet, 369(9556), pp. 145-157. Wane, W. & Martin, G., 2013. Education and health services in Uganda: data for results and accountability (English). Service delivery indicators, Washington D.C.: The World Bank. http://documents.worldbank.org/curated/en/680381468174903963/Education-and-health-services-in- Uganda-data-for-results-and-accountability. Waters, T., Marzano, R. & McNulty, B., 2003. Balanced Leadership: What 30 Years of Research Tells Us about the Effect of Leadership on Student Achievement. s.l.:Working Paper. Woessmann, L., 2016. The Importance of School Systems: Evidence from International Differences in Student Achievement. Journal of Economic Perspectives, 30(3), pp. 3-32.. Wolf, A., 2004. Education and Economic Performance: Simplistic Theories and Their Policy Consequences. Oxford Review of Economic Policy, 20(2), pp. 315-333. World Bank, 2014. Education and health services in Nigeria: data for results and accountability (English). Service delivery indicators, Washington D.C.: World Bank. World Bank, 2015. Mozambique - Service delivery indicators: Mozambique service delivery indicators (English). Mozambique service delivery indicators brief; education, Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/287341468181503193/Mozambique-service-delivery-indicators. World Bank, 2018. The Learning Crisis in Afghanistan (English), Washington, D.C.:: World Bank Group. http://documents.worldbank.org/curated/en/588881536147087211/The-Learning-Crisis-in-Afghanistan. World Bank, 2018. World Development Report 2018: Learning to Realize Education’s Promise. World Bank Group, Washington D.C.: https://doi.org/10.1596/978-1-4648-1096-1. 120 Appendix Appendix A: Tanzania SDI Sampling Methodology The overall objective of the SDI is to produce accurate and representative indicators at the national, urban, and rural levels. In some countries, like Tanzania, it may be required that the indicators be representative at a sub-national level (for example, region or province). The main units of analysis are facilities (schools and health centers) as well as providers (teachers and health workers). In the case of education, the SDI also aims to produce accurate information on standard four students’ performance on Swahili, English, and mathematics. A1. Sampling frame for the 2016 Tanzania SDI The sampling frame for the 2014 and 2016 Tanzania Education SDI was based on the 2012 EMIS data provided by the Ministry of Education and Vocational Training (MoEVT). The original sample frame contained 15,331 schools with identifier variables, such as region and council. This was close to the 15,362 schools contained in the PLSE 2012 school ranking database. The final sample frame was purged of the six schools that had no standard two students, which left the frame with 15,325 primary schools overall. The frame contained only information on standard two’s enrollment, whereas SDI was focused on standard four. A separate list of schools, which was used for the distribution of the so-called ‘radar books’, was also obtained. This list had fewer schools (14,120), but enrollment numbers for all grades. That information was used to estimate standard four enrollment. The number of students enrolled in standard four was estimated at 1.2 out of a total primary student body of 8.1 million. With 6,025 schools (almost 40 percent) with missing location information (i.e., urban/rural), the sample frame had an important challenge to offer. The issue was dealt with by a two-step procedure. First, all the schools located in a municipal council (MC) or a town council (TC) were considered urban, whereas those in a district council (DC) were tallied in the rural column. This eliminated 1,447 schools, leaving us with 4,578 schools (30 percent) with unknown locations. For the second step, because there was no other variable that could provide information on the location, the remaining schools were randomly split between urban and rural, with 80 percent of the schools considered rural. During the data collection, the head teachers were asked whether their school was urban or rural. This new information was used for post-stratification adjustment. Although the SDI is usually representative at the national, urban, and rural levels, in Tanzania it was requested that the survey be also representative of the traditional strata in household surveys, which were (a) Dar es Salaam, (b) other urban areas, and (c) rural areas. Because of a large DFID education program, it was agreed to regroup the regions in which the EQUIP-T operates as a single stratum called (d) EQUIP-T. Table A. 1 shows the overall sample frame with the number of administrative units, such as councils, the number of standard two students (our final variable used for weights), and the total number of primary students within each stratum. Table A. 1: 2012 EMIS sample frame by stratum # Council # Schools # S2 Students # Total Students Dar es Salaam 3 352 69,841 436,952 EQUIP-T 33 3,149 297,576 1,861,751 Other Urban 26 2,097 189,538 1,185,823 Rural 98 9,927 736,937 4,610,562 Total 160 15,325 1,293,892 8,095,088 Source: Author’s calculations using MoEVT 2012 EMIS database The stratification variables provided the domains (strata) and reporting levels (the analysis tables followed these levels) of the survey. The stratification also depended on the most important indicators to be measured in the survey (absence rates and performance levels). Finally, it was advisable to order the clusters within each stratum by variables that were correlated with key survey indicators for further implicit stratification when systematic selection was used. 121 A multi-stage clustered sampling strategy was adopted for the 2014 Tanzania SDI. The first stage cluster selection was carried out independently within each explicit stratum. The primary cluster considered was the council, which was, therefore, the primary sampling unit (PSU). At the second stage, schools were selected and, at the third stage, teachers and standard four students. 55 It was decided that within each stratum, except Dar es Salaam, 25 councils would be chosen with probability proportional to size (number of standard two students). Note that this implied, that at this stage, a standard two (and by extension standard four) student in each stratum had an equal probability for her council to be selected. A2. Sample size and sample allocation for the 2016 Tanzania SDI The optimal sample size of any survey depends on the precision required for the main estimates and resource constraints. The precision of survey estimates depends on the sampling and non-sampling errors. Whereas the sampling error can be measured within a survey, this is not the case for the non-sampling error. The sampling error is smaller the larger the sample, but the non- sampling error grows with the size of the survey. It is, thus, highly advisable to carry out a survey of reasonable sample size that can be managed with effective quality controls to help contain the non- sampling error. To gauge the precision of the estimate, a previous similar survey or a survey measuring the same indicator is very useful. For Tanzania, a pilot 180-school SDI survey was carried out in 2010. The pilot SDI collected almost identical data to the present survey, therefore, providing us with a very strong advantage for a good measure of design effect and standard errors as basis for the current survey sampling strategy. The design effect is critical for determining the optimal sample size. It is the ratio of the variance of an estimate based on the actual multi-stage sample design and the same variance, if the sample was a simple random one of the same size. The design effect is a measure of the relative efficiency of the sample design. Table A. 2: Teacher’s absence rate, average, standard errors and design effect SDI 2010 Design Sample Sample Percent Std. [95% Conf. Interval] Effect Size Size School absence rate Rural 19 1.6 15.9 22.3 2.04 135 1,278 Urban 43 2.9 37.1 48.5 1.94 45 490 Tanzania 27 1.8 23.0 30.3 3.05 180 1,768 Classroom absence rate Rural 51 2.3 46.4 55.5 1.60 135 1,278 Urban 59 3.2 52.1 65.0 1.57 45 490 Tanzania 53 1.9 49.6 57.1 2.58 180 1,768 Source: Author’s calculations using 2010 SDI data. Table A. 2 provides information on teachers’ school and classroom absence rates in the 2010 SDI survey, which were estimated at 27 percent and 53 percent, respectively. It also varied a great deal across the urban and rural strata used in the 2010 survey. The design effect for teachers’ absence rate was around 3.0 and 2.6, which indicates a more or less efficient sampling strategy (it is, indeed, not uncommon to have design effect above 3.0 for cluster sampling). The standard errors were, however, relatively large, especially for urban areas, as shown by the wide confidence intervals. The 2014 SDI aimed at a national standard error around 1.2 percent for absence rates. Using the 2010 SDI as our basis, it was possible to estimate the necessary sample size, for any given standard error. 55 The selection of teachers and standard four students was done once the enumerator was at the school premises. For the purpose of sampling schools, the number of standard two students was used as the weight variable with the (reasonable) assumption that the ratio between standard two and standard four students was constant. 122 Because the design effect for the 2010 SDI was already at 3.0 for school absence, and the current SDI planned for two more strata, it was expected to keep the design effect around the same level, keeping the last item on the right-hand side of the above equation at 1. It was, then, easy to compute the necessary sample size given the objective of a 1.2 standard error. For that standard error, the estimated sample size was 427 schools. Tolerating a slightly higher standard error of 1.3 percent for school absence rate, the sample size came down to 364 schools. It was decided that 400 schools would strike the right balance between the budget and the desired precision. After determining the sample size, the sample allocation across strata needed to be decided. Because the number of strata in the 2014 SDI was larger than in the previous survey, the information from 2010 for allocating the 400 schools across the four strata was not used. There are several allocation mechanisms possible for efficient sampling. For the Tanzania 2014 SDI, an adjusted- proportional allocation was used, whereby the share of schools in the stratum was similar to the share of students in the stratum compared to the overall population. Adjustments were then made if, for instance, in a given strata the number of schools allocated was too small due to the small student population in the stratum. The final sample allocation is given in Table A. 3. Table A. 3: SDI sample allocation across regions # # S2 Sample Schools Students allocation Dar es Salaam 352 69,841 47 EQUIP-T 3,149 297,576 74 Other Urban 2,097 189,538 58 Rural 9,927 736,937 221 Total 15,325 1,293,892 400 A3. Sampling schools, teachers, and students Now that the total sample size and its allocation across strata had been decided, the sampling of the actual schools that were included in the final sample and, within each school, the assessment of students and teachers remained. This was done using a two-stage sampling method. First, in each stratum schools were chosen within the selected councils. Once at a selected school, the enumerator selected teachers and students depending on the structure of the classrooms. The schools were chosen using probability proportional to size (PPS), where size was the number of standard two students as provided by the 2012 EMIS database. As for the selection of the cluster, the use of PPS implied that each standard four students within a stratum had an equal probability for her school to be selected. Finally, within each school, up to 10 Standard four students and 10 teachers were selected. Students were randomly selected among the standard four student body, whereas for teachers, there were two different procedures for measuring absence rate and assessing knowledge. For absence rate, 10 teachers were randomly selected from the teachers’ roster and the whereabouts of those teachers was ascertained in a return surprise visit. For the knowledge assessment, however, all teachers who were currently teaching in primary four or taught primary three the previous school year were included in the sample. Then a random number of teachers in upper grades were included to top up the sample. These procedures implied that students across strata, as well as teachers across strata and within a school (for the knowledge assessment) did not all have the same probability of selection. It was, therefore, warranted to compute weights for reporting the survey results. A4. Weights for schools, teachers, and students To be representative of the population of interest, sample estimates from the 2016 Tanzania SDI had to be properly weighted, using a sampling weight, or expansion factor. Note that different weights needed to be applied depending on the relevant level for the variable, which could be the school, teacher, or student. The basic weight for each entity was equal to the inverse of its probability of selection, which was computed by multiplying the probabilities of selection at each sampling stage. All the weights were computed and included in the dataset. 123 Appendix B: Additional Tables Table B. 1: Education SDI Survey Instrument Module Description Module 1: School Information Administered to the head of the school to collect information about school type, facilities, school governance, student numbers, and school hours. Includes direct observations of school infrastructure by enumerators. Module 2a: Teacher Absence and Information Administered to head teacher and individual teachers to obtain a list of all school teachers, to measure teacher absence and to collect information about teacher characteristics. Module 2b: Teacher Absence and Information Unannounced visit to the school to assess absence rate. Module 3: School Finances Administered to the head teacher to collect information about school finances. Module 4: Classroom Observation An observation module to assess teaching activities and classroom conditions. Module 5: Student Assessment A test of students to have a measure of student learning outcomes in mathematics and language in grade four. Module 6: Teacher Assessment A test of teachers covering mathematics and language subject knowledge and teaching skills. 124 Table B. 2: Students Learning Performance in Language – Tanzania 2014 & 2016 (Standard 4) Overall Urban Rural Regional Boys Girls Gender Gap Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 Read a letter 71.8 83.7 68.6 15.1*** 71.7 72.0 -0.3 Read a word 72.4 85.1 68.9 16.1*** 73.7 71.2 2.50* Identify words 26.0 43.1 21.3 21.8*** 26.4 25.7 0.7 Read a sentence 17.1 30.8 13.3 17.5*** 17.3 16.9 0.4 Read a paragraph 31.0 35.6 29.8 5.8*** 30.8 31.2 -0.5 Reading Comprehension 18.9 28.0 16.4 11.6*** 19.6 18.2 1.3 Tanzania 2014 Read a letter 58.9 - - - - - - Read a word 59.7 - - - - - - Identify words 12.7 - - - - - - Read a sentence 15.8 - - - - - - Read a paragraph 25.0 - - - - - - Reading Comprehension 13.0 - - - - - - Note: The table presents percentage of Standard 4 students in public schools able to perform various language tasks. Each specific language test item represents the percentage of students that responded the question correctly. All student statistics are calculated using student-specific sampling weights. Results for student learning performance are based on around 4000 randomly sampled students. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 125 Table B. 3: Students Learning Performance in Mathematics – Tanzania 2014 & 2016 (Standard 4) Overall Urban Rural Regional Boys Girls Gender Gap Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 Recognize numbers 90.3 89.8 90.4 0.7 90.6 90.0 0.7 Order numbers 55.8 64.2 53.6 10.6*** 60.0 51.8 8.20*** Single digit addition 86.6 92.2 85.1 7.1*** 87.3 86.0 1.3 Double digit addition 73.4 85.1 70.2 14.8*** 75.0 71.9 3.1* Triple digit addition 73.6 85.2 70.4 14.8*** 75.2 72.0 3.2* Single digit subtraction 81.2 87.4 79.5 7.9*** 80.9 81.5 0.6 Double digit subtraction 50.2 64.0 46.4 17.5*** 51.5 48.9 2.6* Single digit multiplication 48.8 55.7 46.9 8.8*** 51.6 46.0 5.6*** Double digit multiplication 17.7 31.4 14.0 17.4*** 19.6 15.9 3.7*** Triple digit multiplication 15.0 26.7 11.8 14.9*** 17.1 13.0 4.1*** Single digit division 45.8 55.9 43.0 12.9*** 49.4 42.2 7.2*** Double digit division 27.5 38.7 24.4 14.3*** 30.7 24.3 6.3*** Understanding of division 20.3 23.6 19.4 4.2*** 21.1 19.5 1.5 Word problem 14.9 19.2 13.7 5.4*** 17.9 12.0 5.9*** Complete a sequence 16.2 20.1 15.1 4.9*** 17.6 14.9 2.7** Tanzania 2014 Recognize numbers 95.6 98.3 94.8 3.5 95.5 95.7 0.2 Order numbers 45.1 52.4 42.8 9.6 48 42.3 5.7 Single digit addition 79.7 87.6 77.2 10.4 80.4 78.9 1.5 Double digit addition 61.1 72.5 57.6 14.9 63.5 59 4.5 Triple digit addition 60.3 73.9 56 17.9 63.2 57.5 5.7 Single digit subtraction 74.1 84.9 70.8 14.1 76.4 72 4.4 Double digit subtraction 39.3 55 34.5 20.5 41.3 37.5 3.8 Single digit multiplication 37.6 41.4 36.5 4.9 40.4 35 5.4 Double digit multiplication 12.1 20.6 9.5 11.1 14.8 9.6 5.2 Triple digit multiplication 9.2 16.6 7 9.6 10.6 8 2.6 Single digit division 37.9 43.5 36.1 7.4 40 36 4 Double digit division 18.1 24.6 16.1 8.5 20.4 15.9 4.5 Understanding of division 19.6 17.1 20.4 3.3 21.3 18 3.3 Word problem 8.9 13.6 7.4 6.2 10.4 7.4 3 Complete a sequence 14.1 16.6 13.3 3.3 15.2 13 2.2 Note: The table presents percentage of Standard 4 students in public schools able to perform various mathematics tasks. Each specific mathematics test item represents the percentage of students that responded the question correctly. All student statistics are calculated using student-specific sampling weights. Results for student learning performance are based on around 4000 randomly sampled students. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 126 Table B. 4: Students Average Test Scores – Tanzania 2014 & 2016 (Standard 4) Overall Urban Rural Regional Boys Girls Gender Gap Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 Test score English 52.2 66.2 48.3 17.8*** 53.8 50.6 3.2 Test score Swahili 91.2 97.0 89.6 7.3*** 89.9 92.6 2.7** Test score Mathematics 65.1 71.8 63.3 8.4*** 66.7 63.7 2.9*** Test score Non-verbal reasoning 58.2 62.7 56.9 5.7*** 59.8 56.6 3.2*** Test score Overall 64.5 73.6 62.0 11.5*** 65.7 63.3 2.4*** Tanzania 2014 Test score English 40.6 - - - - - - Test score Swahili 84.2 - - - - - - Test score Mathematics 60 - - - - - - Test score Non-verbal reasoning 53.2 Test score Overall 54.4 - - - - - - Note: The table presents average scores on the Language, Mathematics and Non-Verbal Reasoning tests of Standard 4 students in public schools. All student statistics are calculated using student-specific sampling weights. Results for student learning performance are based on around 4000 randomly sampled students. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 127 Table B. 5: Comparing Tanzania’s Students Internationally – (Standard 4) Tanzania Tanzania Mozambique Average Uganda Togo Nigeria Kenya 2016 2014 2014 SDI 2013 2013 2013** 2012 Overall Score 64 54 21 50 49 46 32 72 Language Score 63 62 19 50 47 46 31 75 Read a letter 72 59 38 79 86 78 58 96 Read a word 72 60 21 57 66 65 30 80 Identify words 26 13 Read a sentence 17 16 13 47 53 26 26 82 Read a paragraph 31 25 7 19 10 17 12 33 Reading Comprehension 19 13 5 16 0.8 18 16 45 Mathematics Score 65 60 25 47 43 45 32 59 Single digit addition 87 80 48 77 83 77 57 92 Double digit addition 73 61 18 60 56 65 36 84 Single digit subtraction 81 74 28 70 76 65 50 87 Double digit subtraction 50 39 5 34 27 22 22 62 Double digit multiplication 49 38 4 29 24 11 22 51 Triple digit multiplication 18 12 0.1 6 2 6 4 8 Single digit division 46 38 9 38 37 36 21 60 Double digit division 27 18 3 19 13 12 12 36 Non-verbal reasoning score 58 53 44 55 57 54 50 58 Number of Students 3999 4041 1731 N/A 3831 1518 6644 2953 Note: The table presents average scores on the Language, Mathematics and Non-Verbal Reasoning test, and the percentage of students able to perform various tasks per subject. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013 and Kenya SDI 2012. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. 128 Table B. 6: Teacher’s Use of time – Tanzania 2014 & 2016 Overall Urban Rural Regional Male Female Gender Gap Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 School absence 13.6 11.6 13.4 1.8 13.9 12.1 1.8 Classroom absence 41.9 41.3 42 0.7 42.5 41.3 1.2 Teacher found teaching 59.7 58.3 60.2 1.9 60.3 59.3 1 Time on task 56.7 56.4 56.8 0.4 - - - Schedule teaching time 7h 7h 7h - - - - Time spent per day teaching 3h 55min 3h 50min 4h - - - - Tanzania 2014 School absence 14.4 14.6 14.3 0.3 13.5 15.3 1.8 Classroom absence 46.6 45.8 47 1.2 46.6 46.7 0.1 Teacher found teaching 49 - - - - - - Time on task 47 46.1 47.1 - - - - Schedule teaching time 6h 6h 6h - - - - Time spent per day teaching 2h 47min 2h 40min 2h 50min - - - - Note: The table reports the absence rate for all teachers in Tanzanian public school, teachers found teaching, time on task, the scheduled teaching time and actual teaching time for all public schools. All teacher statistics are calculated using teacher-specific sampling weights. Teachers are marked as absent from school if during the second unannounced visit, they are not found anywhere on the school premises. Otherwise, they are marked as present. Teachers are marked as absent from class if during the second unannounced visit, they are absent from school or present at school but absent from the classroom. Otherwise, they are marked as present. The scheduled teaching time is the length of the school day minus break time. Time spent teaching adjusts the length of the school day by the share of teachers who are present in the classroom, on average, and the time the teacher spends teaching while in the classroom. Results for teacher effort are based on observations from around 3,600 randomly sampled teachers. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 129 Table B. 7: Teacher’s Content Knowledge in Language by Type of Task – Tanzania 2014 & 2016 Overall Urban Rural Regional Male Female Gender Gap Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 Language test 37.4 38.9 36.8 2.1 37.3 37.5 0.2 Grammar task 68.9 70.7 68.1 2.6* 69.4 68.3 1.1 Cloze task 48.4 51.5 47.2 4.3*** 48.4 48.5 0.1 Composition task 16.6 17.2 16.4 0.8** 16.2 17.1 0.9** Tanzania 2014 Language test 41.9 45.9 40.8 5.1 Grammar task 73 77.8 71.7 6.1 Cloze task 53 59.2 51.3 7.9 Composition task 21.3 23.8 20.6 3.2 Note: The table presents scores on Language tasks, and the percentage of teachers able to perform various language tasks (grammar, cloze and composition task), for teachers in public schools teaching Standard 4 or who taught grade 3 in the previous year. All teacher statistics are calculated using teacher-specific sampling weights. Results for teacher content knowledge are based on observations from around 2100 randomly sampled teachers. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 130 Table B. 8: Teacher’s Content Knowledge in Mathematics by Type of Task – Tanzania 2014 & 2016 Overall Urban Rural Regional Male Female Gender Gap Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 Mathematics test 65 66.1 64.3 1.8 65.9 62.9 3 Double digit addition 95 93.9 95.4 1.5 95.3 94.5 0.8 Double digit subtraction 85.2 82.1 86.4 4.3* 86.9 83.2 3.7* Double digit multiplication 82.5 81.2 83 1.8 83.4 81.3 2.1 Simple math story problem 76.2 77.2 75.9 1.3 76.5 75.8 0.7 Venn Diagram interpretation 68.9 67.6 69.5 1.9 69.2 68.6 0.6 Graph interpretation 40.5 36.3 42.3 6** 42.9 37.3 5.6** Algebra 55.2 55.4 55.1 0.3 57.2 52.6 4.6* Difficult math story problem 47.4 50.5 46.1 4.4* 47.9 46.8 1.1 Tanzania 2014 Mathematics test 63 65.4 62.5 2.9 Double digit addition 97.1 97.6 96.8 0.8 Double digit subtraction 85.8 85.5 85.8 0.3 Double digit multiplication 85.3 85.3 84.9 0.4 Simple math story problem - - - - Venn Diagram interpretation 48.9 51.2 48.3 2.9 Graph interpretation 27.5 25.5 28 2.5 Algebra 51.2 52.3 50.2 2.1 Difficult math story problem - - - - Note: The table presents scores on mathematics tasks, and the percentage of teachers able to perform various mathematics tasks (e.g. double-digit addition, double digit subtraction, double digit multiplication, simple math story -word problem-, Venn diagram, graph interpretation, algebra and difficult math story), for teachers in public schools teaching Standard 4 or who taught grade 3 in the previous year. All teacher statistics are calculated using teacher-specific sampling weights. Results for teacher content knowledge are based on observations from around 2100 randomly sampled teachers. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 131 Table B. 9: Teacher’s Pedagogical Knowledge and Teaching Skills in the Classroom – Tanzania 2016 Overall Urban Rural Regional Gap (1) (2) (3) (4) Tanzania 2016 Panel A: Pedagogical Knowledge Factual text comprehension 35.5 35.2 35.5 0.3 Formulate aims and learning outcomes 17.6 15.4 18.3 2.9*** Panel B: Assessing Students Formulate questions to check understanding 36.6 28.9 39.4 10.5*** Formulate questions to apply to other contexts 18.0 17.3 20.6 3.3*** Assessing student's abilities 13.3 9.9 16 6.1*** Evaluating student's progress 27.8 27.8 27.7 0.1 Panel C: Skills and Practices in the Classroom Introduce and summarize topic of the lesson 43.3 46.5 42.3 4.2 Lesson appears planned to enumerator 74.8 81.3 72.9 8.4 Ask a mix of lower and higher order questions 57.8 58.1 57.6 0.5 Creates a positive environment in the classroom 30.3 31.3 29.9 1.4 Gives reviews or collects homework 48.7 33.7 52.8 19.1** Note: The table presents scores on teacher’s pedagogical knowledge and teaching skills. Panel A reports on general pedagogical knowledge tasks for teachers in public schools in Standard 4 or who taught Standard 3 in the previous year. Panel B reports on specific pedagogical knowledge tasks in assessing students for teachers in government schools in Standard 4 or who taught Standard 3 in the previous year. Panel C presents teacher skills and practices in the classroom in public schools in grade 4. All teacher statistics are calculated using teacher-specific sampling weights. All scores are computed for teachers teaching either subject. Results for teacher content knowledge are based on observations from around 2100 randomly sampled teachers. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2016 132 Table B. 10: Teacher Practices by Urban/Rural and Teacher Gender – Tanzania 2016 Overall Urban Rural Regional Male Female Gender Gap Teacher Teacher Gap (1) (2) (3) (4) (5) (6) (7) Tanzania 2016 Teacher uses textbook 88.8 83.7 90.2 6.5* 91.9 84.9 6.9* Teacher writes on blackboard 99.0 100.0 98.7 1.3 99.6 98.3 1.3 Students write on blackboard 40.3 40.7 40.1 0.6 40.5 39.9 0.6 Teacher visits students individually 59.8 61.6 59.2 2.4 56.4 64.2 7.8 Teacher calls students by their name 80.0 82.6 79.3 3.3 76.2 85.0 8.7* Teacher smiles/jokes during class 55.8 61.6 54.1 7.5 59.9 50.3 9.6* Teacher hits/scolds students 6.3 3.5 7.0 3.5 5.7 6.9 1.2 Teachers asks to apply new info 82.3 80.2 82.8 2.6 84.6 79.2 5.4 Teacher tests creativity 75.3 82.6 73.2 9.3* 75.8 74.6 1.2 Teacher gives positive feedback 79.0 79.1 79.0 0.1 79.3 78.6 0.7 Teacher gives corrective feedback 80.5 73.3 82.5 9.2* 83.3 76.9 6.4 Teacher introduces lesson 89.5 96.5 87.6 8.9* 88.5 90.8 2.2 Teacher summarizes lesson 44.5 46.5 43.9 2.6 46.7 41.6 5.1 Teacher gives homework 42.8 29.1 46.5 17.4*** 45.4 39.3 6.1 Teacher corrects homework 24.1 16.3 26.2 9.9 25.6 22.1 3.5 Teacher uses local language 10.8 20.9 8.0 12.9*** 11.0 10.4 0.6 Note: The table presents scores on teaching practices in the classroom in public schools in Standard 4 by urban/rural school and teacher gender. All teacher statistics are calculated using teacher-specific sampling weights. All scores are computed for teachers teaching either subject. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2016 133 Table B. 11: Comparing Tanzania’s Teachers Content Knowledge Internationally Tanzania Tanzania Mozambique Uganda Togo Nigeria Kenya 2016 2014 2014 2013 2013 2013** 2012 Language test 37.4 41.9 32.3 58 50 49 65 Grammar task 68.9 73 79 90 74 64 93 Cloze task 48.4 53 31.5 62 30 38 69 Composition task 16.6 21.3 9.3 43 26 24 51 Mathematics test 65 63 37 65 33 42 81 Double digit addition 95 97.1 82 97 75 89 97 Double digit subtraction 85.2 85.8 60 83 65 70 88 Double digit multiplication 82.5 85.3 44 76 51 61 87 Simple math story problem 76.2 - 17 - - - - Venn Diagram interpretation 68.9 48.9 20 72 22 36 73 Graph interpretation 40.5 27.5 12 32 14 20 67 Algebra 55.2 51.2 3 55 9 15 72 Difficult math story problem 47.4 - - - - - - Note: The table presents scores on Language tasks, and the percentage of teachers able to perform various math tasks, for teachers in public schools teaching grade 4 or who taught grade 3 in the previous year. Language knowledge is computed for teachers teaching language, and mathematics knowledge is computed for teachers teaching mathematics. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013 and Kenya SDI 2012. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. 134 Table B. 12: Minimum Equipment Availability in the Classroom – Tanzania 2014 & 2016 Overall Urban Rural Regional Gap (1) (2) (3) (4) Tanzania 2016 Minimum equipment availability 75.0 87.2 73.7 13.5** Students with pen/pencil 98.1 97.1 98.2 1.1 Students with exercise book 98.8 99.0 98.8 0.2 Students with pen/pencil and exercise book 91.7 90.8 91.8 1.0 Classroom with board 98.4 98.6 98.4 0.3 Classroom with chalk 95.5 99.1 95.1 4.0 Classroom with contrast to read the board 87.6 97.0 86.6 10.4* Functioning blackboard 82.7 95.7 81.3 14.3** Students with textbook 18.8 31.5 17.4 14.1*** Students with proper seats and desks 93.5 91.4 93.7 2.4 Number of students without a proper seat and desk 1.2 3.2 1.0 2.2 Tanzania 2014 Minimum equipment availability 61.4 80.4 58.3 22.1 Students with pen/pencil 95.8 96.7 95.7 1 Students with exercise book 96.3 97.2 96.2 1 Students with pen/pencil and exercise book 84 - - - Classroom with board 98.4 97.5 98.5 1 Classroom with chalk 97 96.8 97.1 0.3 Classroom with contrast to read the board 73.9 89 71.4 17.6 Functioning blackboard 74 - - - Students with textbook 25.3 16.7 26.7 10 Students with proper seats and desks - - - - Number of students without a proper seat and desk - - - - Note: This table reports statistics for minimum equipment availability in the classroom, share of students with textbook and other specific classroom input indicators. Minimum equipment availability is a binary indicator capturing the availability of: (i) functioning blackboard and chalk and (ii) pens, pencils and exercise books (paper) in 4th grade classrooms. In one randomly selected 4th grade classroom in the school the enumerator assessed if there was a functioning blackboard by looking at whether text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. We considered that the classroom met the minimum requirement of pens, pencils, and exercise books (paper) if both the share of students with pen or pencils and the share of students with exercise books were above 90 percent. All school-level statistics are calculated using school-specific sampling weights. Results for classroom inputs are based on observations from around 400 randomly sampled schools. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 135 Table B. 13: Minimum Infrastructure Availability in the School – Tanzania 2014 & 2016 Overall Urban Rural Regional Gap (1) (2) (3) (4) Tanzania 2016 Minimum infrastructure availability 43.0 63.9 37.2 26.7*** Classroom visibility 80.4 80.7 80.4 0.3 Toilet available 97.7 100.0 97.4 2.6 Toilet separation by gender 96.5 98.5 96.3 2.2 Toilet clean 84.2 90.2 83.1 7.1 Toilet private 55.2 86.6 51.7 34.9*** Toilet accessible 97.4 99.2 97.0 2.2 Functioning toilet 46.0 80.9 41.8 39.1*** Permanent classrooms 88.8 93.2 88.3 4.9 Semi-permanent classrooms 10.3 6.1 10.8 4.7 Temporary classrooms 0.7 0.8 0.7 0.0 Working electricity 4.9 18.0 3.4 14.5*** Drinking water 47.7 82.8 43.8 38.9*** Road access 88.0 100.0 86.7 13.3*** Corner Library 3.7 6.8 3.3 3.5*** Tanzania 2014 Minimum infrastructure availability 40.4 62.3 36.9 25.4 Classroom visibility 75.8 71.8 73.5 1.7 Toilet available 97.1 99.8 96.6 3.2 Toilet separation by gender - Toilet clean 92 95.9 91.4 4.5 Toilet private 56.2 80.6 52.3 28.3 Toilet accessible 96.6 93.1 97.2 4.1 Functioning toilet 47 70.1 42.3 27.8 Permanent classrooms - - - - Semi-permanent classrooms - - - - Temporary classrooms - - - - Working electricity - - - - Drinking water 0.53 - - - Road access - - - - Corner Library - - - - Note: This table reports statistics for minimum infrastructure availability in the school and other specific school input indicators. Minimum infrastructure availability is a binary indicator capturing the availability of: (i) functioning toilets and (ii) classroom visibility. Functioning toilets is defined as whether toilets were functioning, accessible, clean and private (enclosed and with gender separation) as verified by an enumerator. To verify classroom visibility, we randomly selected one 4th grade classroom in which the enumerator placed a printout on the board and checked whether it was possible to read the printout from the back of the classroom. All school-level statistics are calculated using school-specific sampling weights. Results for classroom inputs are based on observations from around 400 randomly sampled schools. Levels of significance: *** p < 0.01; ** p < 0.05; * p < 0.1. Source: Tanzania SDI 2014 and 2016 136 Table B. 14: Comparing Tanzania’s School Inputs Internationally Laos Tanzania Tanzania Mozambique Uganda Togo Nigeria Kenya Afghanistan PDR Pakistan 2016 2014 2014 2013 2013 2013** 2012 2017 2018 2018 Minimum equipment availability 75 61 77 80 24 48 74 36 47 82 Minimum infrastructure availability 43 40 29 57 14 13 60 35 58 - Students with textbook 19 25 68 6 76 34 45 86 84 96 Observed student-teacher ratio 46 43 21 54 31 22 39 43 21 35 Note: The table presents indicators on school inputs and infrastructure across SDI and SABER SD survey countries. All individual country statistics are calculated using country-specific sampling weights. Source: Tanzania SDI 2014 and 2016, Mozambique SDI 2014, Uganda SDI 2013, Togo SDI 2013, Nigeria SDI 2013, Kenya SDI 2012, Afghanistan SABER SD 2017, Laos SABER SD 2018 and Pakistan (Punjab) SABER SD 2018. ** Values for Nigeria 2013 are the weighted average of the four states surveyed, namely Anambra, Bauchi, Ekiti, and Niger. 137 138