96500 Why Do Students Learn So Little? Seeking Answers Inside Haiti’s Classrooms1 Melissa Adelman, Juan D. Baron, Moussa Blimpo, David K. Evans, Atabanam Simbou, and Noah Yarrow May 2015 1. Motivation The Haitian education system made substantial improvements in access over the last decade, such that today the majority of Haiti’s children are in school. The majority of preschool age children, and 90% of children of official primary school age (6-11), are enrolled in school. Primary net enrollment has increased in recent years, from about 50-60% in the early 2000s to 70-80% in 2012 (World Bank 2014). This represents important progress for Haiti, where 58.5% of the population lives in poverty. However, enrollment is still far from universal, and outcomes, in terms of learning and attainment, are very weak. Despite improvements, the primary education system is highly inefficient: children start primary school 2 years late on average, and fewer than 60% will reach the last grade of the cycle. 2 While the official age for beginning primary school is 6, the average child enters first grade for the first time at 7.8 years old, after having spent 2 or more years in some form of preschool. This distortion grows over time, as about 10% of children repeat and 2-6% drop out of each grade of primary. These repetition rates are higher than the averages in the rest of Latin America and the Caribbean (LAC) and Sub-Saharan Africa. Using a simulated cohort approach, these rates imply that only about 58% of children in first grade will arrive at sixth grade, and only 29% will ever reach the final year of upper secondary. For those who do eventually complete primary, the average ages by grade suggest that it takes 7 to 8 years on average to complete 6 years of primary school. Most importantly, many students, particularly in poor communities, seem to learn little. Assessments administered in early grades in selected schools have found that fundamental skills are acquired very slowly or not at all, particularly in schools in poor communities. For example, assessments conducted in schools located in disadvantaged areas of the Artibonite and Nippes departments found that the average third grader could only read 23 words per minute, well below the estimated speed of 35- 1 The authors would like to thank Violeta Arancibia, Barbara Bruns, Peter Holland, Raju Singh, Samira Halabi, Ludovic Signarbieux, and Jeff Ramin for discussions and comments, and the Strategic Impact Evaluation Fund at the World Bank for financing the data collection. 2 The data used for all the calculations mentioned in this paragraph comes from the 2012 Demographic and Health Survey and the World Development Indicators, and details of these estimates can be found in World Bank 2014. 1 60 words per minute required for comprehension of a basic text (Research Triangle Institute 2010; USAID 2012). What drives these weak learning outcomes? Once students are enrolled, many inputs must come together to produce learning: schools must be open, students and teachers must be present, students must be prepared to learn, and teachers must have the skills and motivation to teach effectively. In Haiti, several of these inputs appear to be lacking. Teachers may often shirk their responsibilities, working multiple jobs to make ends meet, while students may miss school for many reasons, including responding to teachers’ absences. Evidence from a sample of developing countries found that primary school teacher absenteeism averaged 19% (Chaudhury et al 2006). At the same time, instructional quality and the provision of learning materials are generally believed to be very limited in Haiti (MENFP – Ministère d’Education Nationale et Formation Profesionnelle 2013). For example, in French language and math assessments of primary school teachers in the Central Plateau, where the questions were drawn from teacher training institute exams, only 10% (French) and 22% (math) of teachers were able to answer at least half of the questions correctly (MENFP and World Vision 2013). Overall, according to the 2013- 14 School Census, fewer than 50% of primary school teachers self-report having at least completed a secondary degree. Teacher absenteeism is not the problem. Given the absenteeism rates observed in other countries, the Ministry of Education (Ministère de l’Education nationale et de la Formation professionnelle, or MENFP) with support from the World Bank designed an intervention to increase monitoring of teacher attendance, called Initiative de Renforcement des Services Scolaires – IRSS. School directors were provided with cell phones equipped with cameras and connected to the internet, and trained to take daily pictures of teachers, which were uploaded to a central database for Ministry inspectors to verify. An impact evaluation of the IRSS intervention in a random sample of 200 primary schools (100 treatment and 100 control) in the Nord and Nord Est departments found that teacher absenteeism follows an unusual pattern and is actually very low on average. At any given unannounced school visit, teacher absenteeism was under 5% in over 70% of schools, but reached almost 30% in the remainder of schools. The group of schools with high absenteeism varied across unannounced visits. Nord Est is very rural and is one of the three poorest departments of the ten in the country, while Nord is home to the country’s second largest city and is more prosperous (World Bank 2014). While these are two particular departments in the country, there is no reason to believe that the results observed would differ systematically from what occurs more broadly across Haiti. These surprising findings are explained in part by the somewhat unique set of incentives Haitian teachers face. Over 80% of primary schools are non-public: teachers working there are essentially at- will employees, and potentially have better accountability mechanisms than those working in public schools. Even in public schools, where job security is higher, assurances of being paid are not, which may also motivate teachers to come to work in the hopes of eventually receiving payment. When it does occur, absenteeism appears to be a systemic issue – the majority of a school’s staff will stop showing up because payments have not been received from the Government for so long that the school can no longer function. Further discussion and evidence can be found in Adelman et al (2015). Given that teachers are in classrooms, could what teachers do in the classroom help explain the weak learning outcomes of students? A minimum amount of class time is a necessary, but far from sufficient, input into student learning – without enough time actually spent in class, the benefits of 2 improving other inputs to education may not be fully realized (Gillies and Quijada 2008; UNESCO 2008). Moreover, how teachers use their class time helps determine the productivity of that time and how much students learn (Bruns and Luque 2015). To obtain measures of how much class time students in Haiti actually get, and of average teacher classroom practices, a sub-sample of 97 schools from the impact evaluation was revisited.3 At each school, classroom observations were conducted using the Stallings Classroom Snapshot instrument, and questions about the school calendar and daily schedule asked. The results provide a representative picture of class time and teacher classroom practice in the Nord and Nord Est departments, and while not representative of Haiti as a whole, do provide a starting point for better understanding the major constraint to achieving a high-quality education for all children: the quality of teacher instruction. Section 2 describes the sample of schools and the Stallings instrument; Sections 3 and 4 present the main results of the classroom observations on teacher time use and pedagogical practices; Section 5 provides estimates of overall class time that students receive; and Section 6 concludes. 2. Classroom observations: the sample and the instrument MENFP partnered with the World Bank and two organizations (Institut de Formation du Sud – IFOS and FHI360) to collect data in a sub-sample of 97 schools that had taken part in the original IRSS project. The sample is a stratified random sample, where the strata are defined based on schools belonging to the treatment vs control groups, having low vs high absenteeism, and having high and low reading test scores conditional on measured school characteristics.4 Between November 10 and December 5, 2014, each school received an unannounced one-day visit from data collectors to conduct the classroom observations and ask the director about the school’s calendar and hours of operation. The schools are generally small, with an average class size observed of 23 students and one class per grade. 5 An average of 3.8 classes were observed at each school. Table 1 describes the sample. The observations were conducted using a French translation of the Stallings Classroom Snapshot instrument that has recently been used in six Latin American and Caribbean countries (Bruns and Luque 2015). The instrument was originally developed by Jane Stallings in the 1970s to measure the efficiency and quality of primary teachers in the United States (Stallings 1977; Stallings and Mohlman 1988). The Stallings instrument generates quantitative data on four central aspects of teacher classroom practice: use of class time; use of materials; core pedagogical practices; and ability to engage students. A more detailed description is in Box 1, and the full instruments used in Haiti are in Appendix 1. 3 The sample was originally 100 schools, but the team was unable to visit 3 due to ongoing political unrest in local communities. 4 These strata were formed primarily in order to investigate whether teacher time use was affected by the intervention, or if teacher time use correlates with absenteeism. No effects were found. 5 On average, public schools are larger than non-public, and across schools boys and girls each make up about half of students. 3 Table 1: Sample Panel A: Number of schools observed by department and sector Department Public Non-public Total Nord 15 53 68 Nord Est 6 23 29 Total 21 76 97 Panel B: Number of classrooms observed by grade 1st grade 2nd grade 3rd grade 4th grade 5th grade 6th grade Total Number of classes 18 78 101 72 89 11 369 observed The instrument generates robust, quantitative, and comparable data, making it useful in developing country contexts. Research has shown that inter-rater reliability of 0.8 and above can be achieved with limited training, and because the instrument is curriculum-neutral, results are directly comparable across different school and geographic contexts (Abadzi 2007; DeStefano et al 2010; Bruns and Luque 2015). In addition, results can be compared to good practice benchmarks identified by Stallings, Knight, and Markham in 2014 based on research in the United States (table 2). At the same time, the instrument has important limitations that must be kept in mind when interpreting the results.6 First, teachers are aware of the observer’s presence and purpose in the classroom, raising the potential for substantial Hawthorne effects. Results could therefore be interpreted as capturing teachers exerting near-maximum effort. Second, the details of pedagogical practice and content of what is being taught are not measured by the instrument. While this facilitates comparability, it also means that the instrument does not capture important elements of teacher classroom practice, such as teacher-student interactions, which are captured by some more complex instruments. These two limitations may be particularly relevant in Haiti, and will be discussed in the next section. A third limitation is that since observations were conducted only once per classroom, they are not reliable measures of any individual teachers’ classroom practice, so all results in the next section are presented at 6 An additional limitation of the Stallings method is the lack of detailed information on student-teacher interactions. Instructional interactions, and in particular those that are intentional, focused, direct, and use feedback loops, are important because they provide students with the opportunity to practice existing abilities and enhance cognitive development (Pianta et al 2002). Other instruments focused on these interactions, such as the Classroom Assessment Scoring System (CLASS), could complement the valuable information collected through the Stallings. However, given the complexity of administering such instruments, they may be most appropriate for use as part of teacher training exercises, rather than large-scale data collection. 4 the school level or in more aggregated form. Finally, the selective assignment of students to teachers may also affect teachers’ practices and students’ learning, for example if particularly gifted students are clustered together. In Haiti, schools are generally small and most schools in the sample have only one class and teacher per grade, reducing the importance of this effect. Table 2: Stallings good practice benchmarks for teacher time use Activity Benchmark from US research Instruction 85% or more Active instruction 50% or more Lecture/demonstration Reading aloud Discussion/debate/question & answer Practice and drill Passive instruction 35% or less Individual seat work Copying Classroom management 15% or less Off-task 0% Sources: Stallings, Knight, and Markham 2014; Bruns and Luque 2015 3. Use of class time and student engagement Haitian teachers use more class time for instruction than other countries in LAC, but less than good practice benchmarks. Based on several decades of research in the United States, Stallings, Knight, and Markham (2014) find that teachers in high-performing school districts spend an average of 85% of class time on instruction, 15% on classroom management, and 0% off-task (see table 2). Across Latin America and the Caribbean, teachers only spend an average of 52-65% of class time on instruction, compared to 76% in the Haitian sample (figure 1). This extra time on instruction primarily comes from teachers in Haiti spending substantially less time on classroom management activities (grading assignments, disciplining students, etc.). Time off-task is slightly lower in Haiti as well, at 7% compared to 9-14% across LAC. However, while average time on instruction for Haiti is better than for neighboring countries, it is still well below the good practice benchmark of 85%, meaning that about 10% of instructional time, the equivalent of half of a school day, is lost. In addition, as observed in figure 2, there is substantial variation across schools in time spent on instruction. The shape of this distribution mirrors that observed in LAC neighbors (Bruns and Luque 2015). 5 Box 1: The Stallings Classroom Snapshot instrument (from Bruns and Luque 2015) The Stallings method uses a standardized coding grid to register the activities and materials being used by a teacher and students over the course of a single class. Ten separate observations or “snapshots” are made at regular intervals over the course of each class period. For example, if a class is 50 minutes long, observations are made every 5 minutes. Each observation takes 15 seconds. During those 15 seconds, the observer scans the room in a 360 degree circle, starting with the teacher, and codes for key aspects of classroom dynamics in detail: (1) how the teacher is using the class time within three broad categories (instruction, classroom management, or other activities considered off-task); (2) if the time is being used for instruction, which pedagogical practices are being used; (3) if the time is being used for instruction, which learning materials are being used; and (4) what share of students are visibly engaged in the activity being led by the teacher and/or engaged in off-task behaviors (such as social interaction or obviously not paying attention to the activity at hand). The categorization of activities is listed below: Use of time Specific practice Academic activities Lecture/demonstration Reading aloud Discussion/debate/question & answer Practice and drill Individual seat work Copying Classroom management Verbal instruction Discipline Classroom organization Classroom organization alone Teacher off-task Teacher absent from classroom Teacher in social interaction with students Teacher uninvolved or in social interaction w/ others The coding grid has a matrix format in which different activities are listed along the vertical axis and materials used along the horizontal axis. Within each activity, there are two lines: the top line captures who in the classroom the teacher is engaging with – whether it is the entire class, a large group of students (6+), a small group of students, or only one student. The bottom line registers what different students are doing, in the event that the entire class is not engaged in the same activity as the teacher. INSTANTANE D’OBSERVATION D’UNE CLASSE MATERIEL AIDES PAS DE TABLEAU DIDACTIQUES LIVRE CAHIER TIC COLLABORATION MATERIEL NOIR /MANIPULAT ACTIVITE IVES 1. LECTURE A VOIX 1 P G C M 1 P G C 1 P G C 1 P G C 1 P G C 1 P G C P G C HAUTE E 1 P G 1 P G 1 P G 1 P G 1 P G 1 P G P G COCHER SI LA LECTURE EST A VOIX HAUTE ET A L’UNISON La rangée M: La rangée E: indique les 1, P, G, C: indiquent un individu, un indique les activités activités qui impliquent groupe petit ou grand et une classe qui impliquent le l’élève & et pas Le maître entière respectivement maître / la maîtresse / la maîtresse le maître / la maîtresse 6 Figure 1: Haitian teachers spend more time on instruction than LAC neighbors Teacher's average use of class time 90% 0.85 0.76 80% 70% 0.62 0.64 0.64 0.65 0.61 60% 0.52 50% 0.39 40% 0.28 0.25 0.24 0.27 0.25 30% 0.17 0.14 0.15 20% 0.09 0.11 0.12 0.10 0.09 0.07 10% 0 0% Haiti (Nord Mexico D.F. Jamaica Peru Honduras Brazil Colombia Stallings and Nord good Est) practice indicator Academic activities Classroom management Off-task Note: Samples from each country are not statistically representative of all schools in the country. See Bruns and Luque for details on the samples in countries other than Haiti Sources: Data from countries other than Haiti comes from Bruns and Luque 2015 Most non-instructional time is devoted to classroom management, but an important share is spent off-task. On average, teachers spend nearly 10% of class time on management activities without involving any students, and 5% out of the classroom (figure 3). During these times, while some students are usually engaged in an assignment such as copying off of the blackboard, at least half of the time a large group of students (6 or more) is interacting socially. Given the scarcity of available class time (discussed in Section 4), this represents a wasted opportunity. 7 Figure 2: Average time on instruction varies substantially across schools Across school variation 4 3 Density 2 1 0 0 .2 .4 .6 .8 1 Time on academic activities kernel = epanechnikov, bandwidth = 0.0324 Figure 3: Most non-instructional class time goes to classroom management Use of non-academic class time 20% 15% 10% 5% 0% Classroom management Off-task Management alone Class management Discipline Teacher out of the classroom Social interaction w/ others Social interaction w/ students What drives greater use of class time for instruction in Haiti? As discussed in Section 1, over 80% of primary schools are non-public, including a mix of non-profits, for-profits, religious schools, and others. In the non-public sector, teachers are contracted directly by the schools where they work and are not civil servants. This suggests that their incentives to demonstrate effort may be higher, as they are effectively at-will employees who could be let go at any time. At the same time, teachers in public schools have greater job security, but interviews find that most are not paid regularly, and any interactions with 8 MENFP officials are seen as an opportunity to demonstrate that they are working, in hopes that the Ministry will send payments (Adelman et al 2015). Figure 4 shows that indeed, time use in the classroom is quite similar across the public and non-public schools in the sample. In both cases, Hawthorne effects may be more important in Haiti than in other LAC countries, as teachers across schools may have seen the observer as someone who could affect either their continued employment or their chances of receiving a paycheck. Despite relatively high use of class time for instruction, many students are not engaged, and therefore missing out on what the teacher is teaching. Figure 5 shows that Haitian teachers fail to engage at least some students in what they are doing as or more often than in other LAC countries. Observers noted that unengaged students were often staring off into space or had their heads down, possibly sleeping, while others were playing games or passing notes to each other. Teachers largely ignored these students and did not try to discipline or engage them. Figure 4 Teacher's use of class time similar across public and non-public schools 90% 0.77 0.76 80% 70% 60% 50% 40% 30% 0.17 0.17 20% 0.05 0.07 10% 0% Public Non-public Academic activities Classroom management Off-task 9 Figure 5 Share of total class time with unengaged students 70% 60% 50% 40% 30% 20% 10% 0% Haiti Brazil Peru Mexico DF Jamaica Honduras Colombia Share of class time w/ some students not engaged Share of class time w/ large group (6+) not engaged Note: The share of class time w/ large group not engaged is a subset of the class time w/ (at least) some students not engaged. Sources: Data from countries other than Haiti comes from Bruns and Luque (2015). Suggestive results show that instructional time correlates with learning only when the entire class is engaged. Across LAC, there is a positive correlation between teachers using more class time for instruction and students’ test scores (Bruns and Luque 2015). In Haiti, early grade reading assessments (EGRA) were administered to a sample of 3rd and 5th grade students at the end of the 2013-14 school year as part of the IRSS impact evaluation endline. In figure 6a, average total EGRA scores in Kreyol are correlated with the share of class time spent on instruction for teachers who taught in both 2013-14 and 2014-15.7 There is essentially no relationship between total instructional time and test scores. This lack of relationship may be due in part to the lag between the testing (May-June 2014) and the observations (November-December 2014), during which time teachers could have changed their practices, and the fact that a different group of students took the test versus were in the classes observed. However, when we look at the correlation of EGRA score with the share of class time with the entire class engaged, there is a clear positive relationship (figure 6b). The results suggest that teachers who are able to effectively engage all students are having an impact on learning, but it is important to note that the highest average scores observed of 80-90 points are still very low. The test is designed to test skills that should be acquired in the early grades (1-3), and a perfect performance would equal 321 points. As discussed in the next section, most teachers are using pedagogical practices with limited effectiveness, which likely helps explain these results. 7 Students were given the choice to take the EGRA in French or Haitian Kreyol – 88% chose Kreyol. 10 Figure 6a: Overall time on instruction doesn’t strongly correlate with learning outcomes in 3rd and 5th grade classrooms 90 80 70 60 50 .2 .4 .6 .8 1 Share of class time on instruction Figure 6b: Instructional time with entire class engaged is positively correlated with learning outcomes in 3rd and 5th grade classrooms 90 80 70 60 50 0 .2 .4 .6 .8 Share of class time on instruction w/ whole class engaged Note: graphs display the simple sum of EGRA subcomponent averages for 3rd and 5th grade students who chose to take the test in Haitian Kreyol (88% of students), in classrooms where the teacher had taught in both school years. Correlation coefficient in Figure 5a = 0.029 and in Figure 5b = 0.083. 11 4. Pedagogical practices Teachers primarily utilize “active” instructional methods, but rely too much on repetition and fail to truly engage students. Figure 7 shows that lecture/demonstration, discussion/debate/Q&A, reading aloud, and practice/memorization account for a combined total of 54% of the 76% of class time spent on instruction. While these methods have been shown to correlate more strongly with learning than passive instructional methods (copying and individual seat work), they have their limitations in Haiti. Teachers spend the majority of time classified as lecture/demonstration lecturing to students and doing little to check their comprehension of what is being taught. Discussion/debate/Q&A primarily consists of teachers asking students to complete their sentences in unison or to respond in unison to questions with memorized answers. Reading aloud and practice/memorization were also primarily observed in unison. Across these activities, observers noted that teachers rarely paid attention to the fact that many students answered incorrectly or did not participate. The predominance of these practices does not vary across grades. How effective are these methods? Research shows, for example, that students’ reading skills improve when they are actively engaged in reading, writing, talking, discussing, researching, and experimenting around a specific concept (Guthrie et al 2004). These types of activities were not noted by observers in Haiti, as response and repetition in unison was the main form of interaction. Repetition does foster cognitive skills such as memory, but focusing mainly on repetition does not allow the education process to build more complex cognitive abilities such as comparing, analyzing, ordering, and evaluating (Taylor et al 2003). In addition, research shows that when lecture and demonstration are the primary form of instruction, there is an inverse relationship with achievement in reading (Taylor et al 2002; Taylor and Peterson, 2006). If students are sitting and listening for the majority of the school day, they rarely have the opportunity to actually read, write, and talk about what they have read. The content of instruction is also weak. While observers were not education practitioners and not asked to comment on the content of what was being taught, they did record the subjects taught and the language of instruction. One troubling result is that while the vast majority of students learn Haitian Kreyol as their first language at home, many schools do not teach students how to read or write in Kreyol, choosing to focus exclusively on French – only about 9% of classes observed were Kreyol subject classes, while French subject classes accounted for about 30%. As shown in table 3, even in 1st grade, only 12% of classes observed had Kreyol as the subject matter, compared to 34% with French as the subject matter, which seems to contradict official MENFP policy that reading and writing should be introduced exclusively in Kreyol in 1st grade, with French added in 2nd grade. At the same time, teachers and students appear to lack enough mastery to be able to conduct French classes primarily in French – even among 4th to 6th grade classes with French reading and writing as the subject, only about 50% of these classes are taught primarily in French. In the remaining classes, the teacher used either a mix of Kreyol and French, or exclusively Kreyol. These results suggest that students may not be obtaining a mastery of either language, and may cast doubts on the quality of teacher training. 12 Table 3: Subject matter by grade French Kreyol Math Science Other 1st grade 34% 12% 30% 10% 14% 2nd grade 26% 14% 35% 11% 15% 3rd grade 32% 7% 41% 5% 14% 4th grade 19% 13% 49% 12% 7% 5th grade 32% 6% 43% 8% 11% 6th grade 35% 0% 51% 10% 5% Figure 7 Breakdown of academic activities 20% 18% 16% 14% 12% 10% 8% 6% 4% 2% 0% Teachers rely almost exclusively on traditional materials, and in many cases students have no materials, also limiting the effectiveness of instructional time. Figure 8 shows that for about 95% of instructional time, teachers are using the blackboard, a notebook, textbook, or no material at all. This heavy reliance on traditional materials is not surprising, as schools in Haiti are sparsely equipped with learning aids. The 2013-14 school census showed that only 17% of all primary schools in the country have a library, and in this sample, observers recorded that in over 70% of classrooms there were no pictures, posters, or any kind of learning materials displayed in the classroom. At the same time, while students spend over 20% of instructional time doing individual seat work or copying from the board, observers noted that all students had the materials needed (such as paper and a writing implement) less than 40% of the time, meaning that many students were essentially left out of these activities. 13 Figure 8 Teacher's use of materials during academic activity 60% 50% 40% 30% 20% 10% 0% Blackboard Book or notebook No material Other didactic ICT materials 5. Getting children and teachers into the classroom While teacher absenteeism is not a widespread problem in Haiti, time in the classroom is. As mentioned in Section 1, a minimum amount of class time is a necessary input into student learning (Gillies and Quijada 2008; UNESCO 2008). In order to estimate the class time students actually receive, observers asked the school director several questions related to the school’s calendar and daily schedule. At a maximum, following the official academic calendar and average hours of class per day, primary students would receive an average of 824 hours of instruction per year. This would provide a minimum considered necessary for learning, and be in line with regional averages as shown in table 4 (UNESCO 2008). However, at every level, students lose out on time due to various systemic reasons. Table 4: Median hours of instructional time in 4th grade by region Region Median yearly instructional hours in grade 4 Sub-Saharan Africa 888 East Asia & Pacific 821 South & West Asia 734 Latin America & Caribbean 796 North America & Western Europe 808 Central & Eastern Europe 645 Sources: Gillies and Quijada 2008 MENFP’s official academic calendar contained 183 school days in 2013-14, but many days are lost due to recurring shocks. While comprehensive data on unplanned school closings is not available, extreme weather events and civil unrest are common occurrences that can cause schools to close. For example, during the actual data collection period, heavy rains led MENFP to announce that all schools 14 should close for an entire week, and as mentioned earlier, three schools could not be reached for several weeks due to civil unrest. Demonstrations and political violence, teacher strikes (often in protest over not being paid), and other sporadic but common shocks can shut down schools for days or weeks at a time. Among the 97 schools in the sample, directors reported that their school was unexpectedly closed an average of 1.2 days the previous month (October 2014). This would add up to a loss of at least 5% of school days over the course of the academic year. When schools are open, they provide an average of 4.5 hours of class time per day, but much of that time is lost for at least some students. Since most schools in Haiti and in the sample are non- public, they set their own hours of operation. The majority start classes between 7:00 and 8:00am, and end classes between 12:00 and 1:00pm, with a 30 minute break during the day. In many countries, the school day is similarly structured, but that is in order to allow a second afternoon shift. In Haiti, only 8% of all primary schools reported having a second shift in the 2013-14 school census, suggesting that infrastructure is not a constraint to longer school days. Within the 4.5 hours of class time, only 76% is used for instruction as discussed in Section 3, and only 29% of class time, or 1.3 hours per day, is used for instruction with the entire class engaged. Moreover, in many schools, a substantial amount of time may be taken up by children waiting to be fed as part of school feeding programs, which are often provided during class hours. However, data on this aspect was not collected. These losses mean that even the most committed students are not getting adequate instructional time, and the most disengaged students could be getting very little instruction, even if they come to school every day. Adding up the losses of school days due to shocks, and the loss of class time spent on management or off-task, students who are fully engaged and present every day would only receive an average of 594 hours of instruction per year, while students who were present every day but disengaged while in class could get as little as 230 hours of instruction per year. These estimates mean that students who are regularly attending school are not getting nearly enough instructional time to build the skills they should be acquiring. 6. Conclusions Taken together, the results from the Stallings classroom observations provide several valuable insights into what is happening inside Haitian classrooms. Teachers in Haiti spend more time on instruction than their LAC counterparts, perhaps because of greater incentives to exert effort. While instructional time with the entire class engaged is positively correlated with learning, indications of the quality of instruction suggest that it is relatively low. Pedagogical practices rely heavily on lecture and unison response from the class, and teachers often do not appear to concern themselves with ensuring the understanding of individual students. This is particularly ineffective in early primary, when each child needs to build the foundational skills that will make attending school productive in later grades. Given that at least basic motivation to teach appears to be present for most teachers, equipping them with the skills and tools to do it effectively – through targeted and practical in-service training, scripted lessons, and other means – is a high priority. 15 The findings in this study align well with Hanushek and Woessmann (2012), who analyze international student assessments in LAC and conclude that the great effort to increase access in the countries of the region has not translated into better learning outcomes, which would foster skills and improve productivity. In other words, children go to school, but they are not getting as much as out of it as they potentially could. It is imperative that schools throughout LAC, and in Haiti in particular where resources are extremely scarce, deliver a service that has real human capital-building value for children. In this study, we have presented evidence that teachers do spend a significant amount on their time on instruction, but that instruction is not effective and students are learning very little. Why? Our first hypothesis is the issue of language of instruction in the early grades. Information collected during the class observations suggests that children are not being taught to develop reading and writing skills as they should, either in French or in Kreyol. New insights into neurolinguistics could shed light on the optimal strategy for introducing two languages in contexts where most children, but in particular the most disadvantaged, have only limited exposure to one of the languages. Global research finds that, unsurprisingly, children have a very difficult time learning to read in a language with which they have limited oral familiarity, and that teaching reading and writing in two languages without ensuring mastery of at least one can lead to failure in both (Nakamura and de Hoop 2014). A second hypothesis is that instruction is relatively ineffective due to the outdated methodologies that are used. Teachers without appropriate training (that combines content with pedagogical expertise) on their own are unlikely to move away from traditional instruction techniques that rely heavily on lecturing, memorization, and unison response. Studies on child learning and cognitive development show that memory has an important role, but it should be limited in favor of the development of other cognitive skills. Teachers should try to develop the capacity of their students to comprehend what they are teaching and should use strategies such as “real” question and answer, and stimulate student engagement and participation by asking them to compare, analyze, revise, assess, and apply what is being taught. Many of these strategies do not appear to be commonly used in most Haitian classrooms. But how to improve instruction in Haitian classrooms? In the long-run, improved teacher training that enables mastery of content, provides classroom practice and feedback, and that aligns with standards based on a revised curriculum could contribute to improved instruction and student learning. Moving away from instruction practices centred on student repetition and which only passively engage children would also improve learning. New instruction methods in Haiti should demand students to ask questions, compare, analyse, solve problems, and discuss, among others activities. This should become the norm rather than the exception. In the short-term, given the weaknesses of existing teacher training programs, scripted instruction methods in which teachers are guided through each class, albeit with adequate support and coaching, could be a way to foster student learning in the classroom. Two of these approaches, the All Children Reading Project (Tout Timoun Ap Li, ToTAL) supported by USAID and the M’Ap Li Nèt Ale method supported by the World Bank, are examples of promising interventions designed to support teachers in improving their instruction practices in Haitian schools. In the medium term, all these should be complemented with other measures to attract, groom, develop, and support Haitian teachers to deliver better education to students. 16 References Abadzi, Helen. 2007. “Absenteeism and Beyond: Instructional Time Loss and Consequences.” World Bank Policy Research Working Paper 4376. World Bank: Washington, D.C. Adelman, Melissa, Moussa Blimpo, David Evans, Atabanam Simbou, and Noah Yarrow. 2015. “Enhanced Teacher Monitoring for Student Results in Haiti.” Working Paper. World Bank: Washington, D.C. Bruns, Barbara and Javier Luque. 2015. Great Teachers: How to Raise Student Learning in Latin America and the Caribbean. World Bank: Washington, D.C. Chaudhury, Nazmul, Jeffrey Hammer, Michael Kremer, Karthik Muralidharan, and F. Halsey Rogers. 2006. “Missing in Action: Teacher and Health Worker Absence in Developing Countries.” Journal of Economic Perspectives 20(1): 91-116. DeStefano, J., E. Adelman, and A. Moore. 2010. Using Opportunity to Learn and Early Grade Reading Fluency to Measure School Effectiveness in Nepal. EQUIP2 Working Paper. USAID: Washington, D.C. Gillies, John and Jessica Jester Quijada. 2008. Opportunity to Learn: A High Impact Strategy for Improving Educational Outcomes in Developing Countries. EQUIP2 Working Paper. USAID: Washington, D.C. Guthrie, J.T., A. Wigfield, P. Barbosa, K.C. Perencevich, A. Taboada, and M.H. Davis. 2004. “Increasing Reading Comprehension and Engagement through Concept Oriented Reading Instruction.” Journal of Educational Psychology (96): 403-423. Hanushek, E.A., and L. Woessmann. 2012. “ Schooling, Educational Achievement, and the Latin American Growth Puzzle.” Journal of Development Economics 99(2): 497-512. Ministere de l’Education Nationale et de la Formation Profesionnelle (MENFP). 2013. Programme D’Interventions Prioritaires en Education (PIPE) : 2013-2016. MENFP : Port au Prince, Haiti. Ministere de l’Education Nationale et de la Formation Profesionnelle (MENFP) and World Vision. 2013. Rapport du Pre-test de Mathematique and Rapport du Pre-test de la Communication Francaise. World Vision Haiti : Port au Prince, Haiti. Nakamura, Pooja and Thomas de Hoop. 2014. Facilitating Reading Acquisition in Multilingual Environments in India: Final Report. American Institutes for Research: Washington, D.C. Pianta, R. C., K.M. La Paro, C. Payne, M.J. Cox, and R. Bradley. 2002. “The Relation of Kindergarten Classroom Environment to Teacher, Family, and School Characteristics and Child Outcomes.” The Elementary School Journal (102): 225–238. Research Triangle Institute. 2010. Haiti : Early Grade Reading Assessment ; Rapport pour le Ministere de l’Education et la Banque Mondiale. RTI : Research Triangle Park, NC. Stallings, J. 1977. Learning to Look: A Handbook on Classroom Observation and Teaching Models. Wadsworth Publishing: Belmont, CA. 17 Stallings, J. and G. Mohlman. 1988. “Classroom Observation Techniques.” In Educational Research, Methodology, and Measurement: An International Handbook. Edited by J. Keeves. Pergamon: Oxford. Stallings, J. S. Knight, and D. Markham. 2014. “Using the Stallings Observation System to Investigate Time on Task in Four Countries.” World Bank: Washington, D.C. https://openknowledge.worldbank.com/handle/10986/20687 Taylor, B.M., D.S. Peterson, P.D. Pearson, and M.C. Rodriguez. 2002. “Looking Inside Classrooms: Reflecting on the “how” as well as the “what” in Effective Reading Instruction.” The Reading Teacher (56): 70-79. Taylor, B.M., P.D. Pearson, D.S. Peterson, and M.C. Rodriguez. 2003. “Reading Growth in High-Poverty Classrooms: The Influence of Teacher Practices that Encourage Cognitive Engagement in Literacy Learning.” The Elementary School Journal (104): 3-28. Taylor, B.M. and D.S. Peterson. 2006. The Impact of the School Change Framework in Twenty-three Minnesota REA Schools. Minneapolis: University of Minnesota, Minnesota Center for Reading Research. UNESCO. 2008. Education for All by 2015: Will We Make It? EFA Global Monitoring Report. UNESCO: Paris, France. USAID. The Early Grade Reading Assessment in Haiti. 2012. USAID: Washington, D.C. World Bank. Haiti Poverty Assessment. 2014. World Bank: Washington, D.C. 18