from EVIDENCE to POLICY Learning what works for better programs and policies January 2021 RWANDA: Can performance pay for teachers improve students’ learning? Performance pay for teachers generates debate. Proponents argue Whether this happens in practice is an empirical question. To that many school systems have low levels of accountability and inform the Rwandan government about its own incentive struc- advocate incentivizing teachers by linking their pay to either their ture for teachers, researchers designed a two-year experiment in EDUCATION own efforts or their students’ learning. Critics, however, raise con- partnership with Rwandan Education Board and the Ministry of cerns that performance pay attracts people to the teaching work- Education, with support from the World Bank’s Strategic Impact force who are “in it for the money” and could diminish the in- Evaluation Fund. The evaluation was set up to separately measure trinsic motivation to teach among teachers already in classrooms. the impact of performance pay on the composition of teachers attracted to the teaching force and its impact on their effort in the classroom once hired. After two years, researchers found that offering performance- based bonuses for the top 20 percent of teachers did not attract teachers with lower teacher skills, compared to offers of fixed wage contracts. Once in school, offers of performance-based bonuses increased teachers’ presence in the classroom and improved their pedagogical practices. The performance pay also helped them elic- it higher test-score performance from their students. Following this work, the government requested the research team to propose options for improving teacher recruitment, motivation, and reten- tion to support the implementation of a new national system of teacher hiring and deployment. This collaboration is ongoing and also extends to the government’s investments in a comprehensive assessment for basic education. These policies may provide key infrastructure for implementation of performance pay at scale. Source: Dominic Chavez/World Bank This policy note is based on, “Recruitment, Effort, and Retention Effects of Performance Contracts for Civil Servants: Experimental Evidence from Rwandan Primary Schools” Clare Leaver, Owen Ozier, Pieter Serneels, Andrew Zeitlin, World Bank Policy Research Working Paper (2020) Context Rwanda has made huge strides in expanding access to education, ning, and observed pedagogy) and students’ test scores. Students’ achieving 97 percent net primary school enrollment by 2015. test scores at baseline were divided into percentile-based brackets, Learning, however, remains a problem, with more than 30 per- and teachers’ incentives were based on endline percentiles within cent of students dropping out before they reach sixth grade and these brackets. The use of bracket-based percentiles yields two im- 85 percent repeating a grade at least once. portant benefits with respect to equity and sustainability: because While several performance-pay programs already exist in every baseline ability bracket is competing separately, there should Rwanda’s public sector, the government has expressed interest in be equal incentives for teachers to improve the performance of reforming the incentive structure for teachers to make it more children of different levels of initial ability; and because the incen- evidence-based. Under the existing imihigo system, public sector employees in other sectors receive financial rewards of up to 5 per- cent of their salary based on subjective performance evaluations. In public schools, a substantial share of teachers’ existing salaries is made up of bonuses that are discretionary in theory, but in prac- tice, all teachers receive a fixed bonus amount. A key challenge put to the researchers by the Rwanda Education Board was to design a performance award scheme that would not only reward student learning, through test-score outcomes, but would include measures of teachers’ inputs into the classroom, which they could better control themselves, and which would reinforce professional norms. The result was a “4Ps” contract, emphasizing preparation, presence, pedagogy, and pupils’ learning. To test the impacts of paying the bonuses according to teacher Source: Dominic Chavez/ performance, the study took place during and after the recruit- tives are based on percentiles rather than specific score thresholds, World Bank ment for civil service teaching jobs for upper primary school in exams used at baseline and endline or the following year need not six districts in 2016, covering over 600 hiring lines, or more than have exactly the same questions, making teaching-to-the-test less 60 percent of the country’s planned recruitment in that year. likely in a longer-run implementation. Within-district rather than The bonuses tested in the evaluation were set to RWF 100,000, within-school competition can also help to ensure that teachers equivalent to about 15 percent of teachers’ annual salary, and would not directly compete with their peers. The fixed-wage were to accrue to the top 20 percent of upper-primary teachers contract instead provided a top-up of RWF 20,000 to all upper- within a district. Teacher performance was a composite measure primary teachers in the school. Both types of contracts had the of teacher inputs (teacher presence in the classroom, lesson plan- same total costs to the government. Evaluation To separately identify the effects of a compositional change in the Researchers first randomly assigned 18 labor markets––defined as teaching force attracted to contracts with performance-based pay application pools for a specific subject in specific districts–– to and the incentivizing effects of performance pay when teachers are performance pay or fixed-wage contracts. In one set of markets, already in classrooms, the study used a two-stage randomization. through the advertised contracts, potential applicants applied to positions where recruits to new primary posts would receive a con- ally receive fixed-wage contracts; likewise, there were teachers who tract with the performance-based bonus for the 2016 and 2017 thought they would be receiving fixed-wage contracts when they school years. In the other set of markets, potential applicants were were recruited but who then were hired with a performance-based told that recruits to new primary posts would receive the fixed- contract. To mitigate potential disappointment, all new recruits wage contract for the 2016 and 2017 school years. Since teach- were offered an RWF 80,000 retention bonus if they remained in ers rarely go to teach in another district, comparing the recruits their post until the end of the year. Comparing these two sets of attracted to these two different types of contracts that had been teachers – those who actually received a contract with the perfor- randomly assigned to these labor markets would show whether mance-based bonus and those who actually received a fixed-wage performance pay attracted a different type of teacher to the teach- contract - would show how performance pay might affect teacher ing workforce. effort on the job. In the second stage of the randomization, 164 schools hiring Researchers surveyed applicants, hired teachers, and head these new recruits were assigned to either the performance-based teachers. They also made unannounced visits to the schools to or fixed-wage contracts. All upper-primary teachers within each measure teacher presence, their lesson planning, and their peda- school, including the new recruits, received the contract type that gogy in classrooms. was assigned to the school, regardless of what their contract as- They measured students’ academic achievement three times signment in the first stage was. This meant that there were some – at baseline, midline, and endline – using an oral exam based teachers who thought they would receive performance-based con- on the national curriculum that covered five core subjects (Kin- tracts at the time of recruitment but then learned they would actu- yarwanda, English, Mathematics, Sciences, and Social Studies). Findings Teachers recruited under performance contracts ex- under performance-based pay scored 0.11 standard deviations erted at least as much effort in the classroom as those higher per year on average across the two years, and 0.16 stan- recruited under fixed-wage contracts. dard deviations higher in the second year of the study. Effective- ly, this means that in the second year, the performance-linked Individuals recruited under the performance contract were more bonuses moved a student from the median (50th percentile) money-oriented in a game that tested their intrinsic motivation up to the 56th percentile of students. The net effect of being compared to individuals recruited under the fixed-wage contract. recruited and then working under performance contract was However, this difference observed in a lab-in-field experiment 0.20 standard deviations of learning gain among students in did not spill over to their performance in the classroom. Recruits the second year. from the labor markets where performance-based contracts had been advertised performed no worse than the fixed-wage recruits The improvement in learning outcomes likely followed in terms of their presence, lesson planning, or classroom con- from the improvement in teacher presence and their duct. The type of advertised contract also did not affect student conduct in the classroom. learning. Teacher presence was 8 percentage points higher among those The experience of teaching under performance-based who experienced the performance contract compared to those pay improved student learning. who experienced the fixed-wage contract (a relatively large im- pact given that baseline teacher presence was close to 90 per- Compared to students assigned to teachers who experienced cent.) Teachers who received performance contracts also scored fixed wage contracts, students assigned to teachers working higher on a scale that summed up their classroom practices. Performance pay did not appear to stress out Overall, the performance-based bonus appears to be teachers; teachers with these contracts were no more cost-effective, as it was designed to be budget-neu- likely to quit during the two years of the study than tral for the government. teachers working under fixed-wage contracts. The performance-based salaries were designed to be the same The retention rate was identical across the two experimental cost to government as the fixed-wage contracts. It is worth not- groups at around 80 percent and when asked about their in- ing, however, that measuring performance could require some tentions to leave the following year, the two groups were also additional costs, depending on how the government measures statistically indistinguishable. Moreover, among the 20 percent performance at scale. At minimum the government can use stu- of teachers who did decide to leave, there was no difference dent test scores. Other aspects of performance—teacher pres- across the groups assigned to different contracts in terms of ence, preparation, and pedagogy—could potentially be mea- their observed skills in the subjects they were teaching. sured by head teachers or district staff at modest cost. Conclusion Ensuring children around the world are not only in school less motivated teachers into the profession or increasing teach- but also learning is challenging. When framed as a way to er turnover. Further research will be needed in other contexts motivate teachers and improve student learning, performance- to validate these findings and examine longer-term impacts, based pay often generates debate. This evaluation in Rwanda but policymakers should be encouraged by these results as they among public upper primary schools demonstrates that a well- show that implementing appropriate performance pay struc- designed incentive structure can improve teacher effort in the tures can be a cost-effective way to improve the performance classroom and benefit student achievement without attracting of teachers. The Strategic Impact Evaluation Fund, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help allevi- ate poverty. The goal is to collect and build empirical evidence that can help governments and development organizations design and implement the most appropriate and effective policies for better educational, health, and job opportunities for people in low and middle income countries. For more information about who we are and what we do, go to: http://www.worldbank.org/sief. The Evidence to Policy note series is produced by SIEF with generous support from the British government’s Foreign, Commonwealth and Devel- opment Office and the London-based Children’s Investment Fund Foundation (CIFF). THE WORLD BANK, STRATEGIC IMPACT EVALUATION FUND 1818 H STREET, NW, WASHINGTON, DC 20433