RESULTS-BASED FINANCING RBF EDUCATION EVIDENCE BANGLADESH AND NEPAL Can Understanding How Middle Managers Make Decisions Help Design Effective Results-Based Financing Mechanisms in Education? MARCH 2020 REACH funded research to investigate how principals and district education officials (DEOs) from Nepal and Bangladesh understand their roles, set priorities, and make decisions. The Results in Education for All Children (REACH) Trust Fund supports and disseminates research on the impact of results-based financing on learning outcomes. The EVIDENCE series highlights REACH grants around the world to provide empirical evidence and operational lessons helpful in the design and implementation of successful performance-based programs. The quality of school management However, there is little information is of central importance to the on how “middle managers” make The quality of school quality of teaching and learning, decisions and how the context management is of central importance to the quality accounting for around one-quarter and prevailing norms they work of teaching and learning. of the total impact that schools within affect their attitudes towards have on student learning according ‘results-based management’. To to some studies.1 School principals better understand decision making and district education officials processes and how they are aligned (DEOs) are some of the most with results-based financing important actors in the education principles, the Results for Education School principals and management system. They act as for All Children (REACH) Trust Fund DEOs are some of the most important actors in the “middle managers” who provide the funded research to investigate how education management link between policymaking at the principals and DEOs from Nepal and system. central level and implementation at Bangladesh understand their roles, the school level. set priorities, and make decisions. This note was adapted from Asaduzzaman. T.M., Ramachandran, Deepika, & Sabarwal, Shwetlena. 2019. “Managing the Middle: Decision-Making within Education Bureaucracies.” 2 RFB EDUCATION | EVIDENCE The findings suggest that middle they exhibit preferences that may Fourth, these agents are divided in managers are aligned with principles hinder the application of RBF. terms of prioritizing equity in inputs of results-based financing in First, they prioritize the demands vs. equity in outcomes. These results three ways: (a) they prioritize of vocal parents over the needs of suggest potential entry points in how objective data for decision-making; disadvantaged students. Second, to build the capacity, mindsets, and (b) they prioritize student learning they show low ownership over effectiveness of middle-management as their main goal, and (c) they the learning outcomes of lagging for increased effectiveness, favor results based accountability students. Third, they appear unwilling especially around RBF. for teachers. At the same time, to sanction low-performing teachers.   CONTEXT role of linking the organization’s internal networks with external sources of infomation. Given these Middle managers in the education characteristics and their importance system—school principals and in determining education outcomes, DEOs—exert considerable influence it is important to understand their on how schools perform.2 Not only attitudes and beliefs, the way in Nepal & Bangladesh do they play an important role in which they make decisions, and how education service delivery, they have they respond to incentives. several characteristics that affect their decision-making role. First, they hold a position of power, but given the tightly circumscribed systems they operate within it can be hard to determine how much autonomy and discretion they truly exercise.3 Second, partly because of the first point, this layer can also be hard to influence or incentivize. Third, based on the management and governance structures, they operate with little day to day supervision. Fourth, being middle managers, they simultaneously face policy and information demands from above; and resource, information, and personnel demands from below. This gives them both hard-to- quantify challenges and influence.4 Finally, they have been identified as ‘boundary spanners’—actors within a system who perform the BANGLADESH AND NEPAL 3 WHY WAS THE roles and that monetary incentives can crowd out this intrinsic motivation There is little reliable INTERVENTION to do good.8/9/10 However, little of this empirical data on research has focused on education, district and other CHOSEN? which is technically complex and mid-level government   particularly liable to be subject to There is a growing amount of empirical research on school political influence.11 The research officials. This evidence outlined in this evidence note helps principals and their impact on student to fill this gap by using an innovative note helps to fill outcomes.5/6/7 However, there is little and low-cost approach to study the the gap by using an reliable empirical data on district and attitudes, beliefs, decision-making, other mid-level government officials. and motivation of both principals and innovative and low- Studies of these kinds of officials DEOs. The findings reveal how these cost approach to study have been conducted in India and officials filter and process available Ghana among other countries. Their information and how they are likely to the attitudes, beliefs, findings have shown that middle respond to incentives such as those decision-making, and managers are motivated mainly by a range of non-monetary incentives associated with RBF. It also sheds light on how they might respond to motivation of those including increased autonomy and the other initiatives targeted at them, individuals. chance to be promoted to top-level such as capacity building programs. HOW DID THE INTERVENTION WORK? To understand the decision-making process of middle managers, a game was developed that could be played on a mobile phone and that used hypothetical real-life scenarios to reveal respondents’ attitudes. The game was administered to primary school principals and district education officials in Bangladesh and Nepal. The game presented each principal and official with a series of scenarios and asked them to choose one option each time. For example, the study was interested in whether principals were more 4 RFB EDUCATION | EVIDENCE concerned about investing money accurately than if they were asked in a way that seemed equitable or in The game more direct questions. a way that maximized the gains to was designed Imagine that you have to avoid In addition to the scenarios, the student learning. The respondents common biases TK80,000 to spend on were asked to imagine that they contained in game also contained more traditional one classroom. Would had TK80,000 (US$940) to spend you choose: traditional multiple-choice questions, as well as surveys asking respondents to the extent to on one classroom. They could  lassroom A: C larger class with good choose either a larger class that had exam results which they agreed with a series of achieved good examination results  Classroom B: statements such as, “Schools deserve small class with poor or a smaller class with worse results. exam results more resources if the students come If the principal chose to invest in from poor households.” the larger class, this suggested that answers that they think are likely to The questions and scenarios their priority was to equalize inputs be popular or well received by the measured the attitudes and beliefs per student, indicating a desire to interviewer (social desirability bias). of principals and DEOs about a wide increase equity, whereas if they Respondents may also be biased range of topics including school chose the smaller class with lower towards giving answers in which management, the allocation of school scores, this suggested that their they take credit for successes but resources, teacher accountability, priority was increasing learning. blame others for failures (self-serving student learning, and their perception The game was designed to avoid bias) or that justify actions that they about leadership. All the scenarios common biases contained in have taken (rationalization bias). and questions were administered traditional surveys, which, although These biases are hard to prevent in the local language used in each widely used, are not always a reliable and can distort evidence gathered country. The game and questions way to collect information on from surveys on people’s opinions. were administered in Bangladesh people’s opinions and preferences. The idea behind the game was that and Nepal in 2018. Altogether, 234 For example, when people self- respondents’ choices would reveal school principals and 220 meso-level respond, they tend to provide their preferences and attitudes more administrators took part in the game. BANGLADESH AND NEPAL 5 WHAT WERE Figure 1: Support Test-Based Accountability for Teachers (Bangladesh) THE RESULTS? How are decisions being made at the 91% 91% 91% 92% 91% Affirmative Responses (%) 83% 83% meso and school level? How likely are 75% 70% 71% these decisions to follow principles 59% 61% of RBF? The study answered these questions in two parts. First, it identified factors that may facilitate the application of RBF principles at the meso- and school levels. Second, Teacher promotion or The main factor used Support for teacher School principals the study identified factors that may transfer should partly to assess teachers’ bonuses if students should be able to be dependent on my performance should be perform well on official discipline teachers constrain the use of RBF. student’s performance students’ test scores exams on tests Factors that facilitate School principals DEOs Pooled the application of RBF Principals and DEOs were supportive of results-based accountability Figure 2: School Principals and DEOs Rely on Objective Data for teachers (Figure 1). A majority (Bangladesh and Nepal) of respondents from Bangladesh “You have to select the best teacher/principal of the year. Which information would you rely on?” supported taking student test scores into account when evaluating, 87% 13% Pooled promoting, and rewarding teachers. Around 71 percent felt that student 95% 5% DEOs test scores should be taken into account when assessing teacher 79% 22% School principals performance, 61 percent said they should be considered when choosing Responses (%) teachers for promotion, and 91 percent supported providing teachers with Student test scores Your impression of the teacher during personal interactions bonuses if their students performed well on official examinations. Principals and DEOs valued of development often resist using middle managers in education value objective data over opinion when objective measures to evaluate objective data. When asked how making decisions (Figure 2). For performance for a range of reasons they would choose the best school RBF to be effective, results must be including personal incentives, or teacher of the year, 87 percent of evaluated using factual evidence political influence, and a lack of principals and DEOs in Bangladesh rather than subjective opinion familiarity with the principles of and Nepal claimed that they would before assigning any rewards. evidence-based policymaking and rely on student test scores rather However, private and public sector scientific methods.12/13 However, the than their personal opinions. institutions in countries at all levels findings of this study suggest that 6 RFB EDUCATION | EVIDENCE Middle managers were able to as primarily administrative. When other, typically less well-off students. distinguish between higher-order asked to choose the priority tasks The findings suggest that this could results (like foundational skills of a good principal or DEO from a be an area of concern as 75 percent for all) and lower-order results list, only 15 percent of respondents of DEOs and 59 percent of school (like completion of curriculum). chose more leadership tasks than principals said that there was little Respondents were asked to choose administrative tasks. The most cited a school could do to help students one of two teachers to receive a priority task was “keep good records,” learn if their parents did not seek reward. The first teacher’s students and only 10 percent of respondents feedback on student performance. could recognize all the letters of the thought that “provide teachers with It may be that principals and DEOs alphabet but had completed only goals” was a high priority. are responding to parental demands 60 percent of the curriculum, while rather than to objective measures of the second teacher’s students had Factors that constrain school needs. More than 90 percent completed the curriculum, but only the application of RBF of both principals and DEOs said that one in five could recognize the entire a school deserved more resources alphabet. Ninety-one percent of The respondents were focused on children with vocal parents (Figure 3). if parents were demanding better the respondents said that the first education, while only 52 percent of More than 80 percent of DEOs and teacher should be chosen to receive school principals in Bangladesh principals and 38 percent of DEOs the reward, which suggests that said that they would listen directly said that a school deserved more principals and DEOs prioritize skills to a parent’s complaint about a test resources if learning levels were low. development over finishing report rather than referring them to In response to a hypothetical scenario, the curriculum. the teacher. However, this can have 54 percent of DEOs and 58 percent of Principals and DEOs saw a negative impact if they focus on principals chose to spend resources themselves as leaders with the the students of the most vocal and on a school with a higher pass rate responsibility to ensure that engaged parents to the exclusion of where parents frequently complained students learn. The researchers asked respondents in Bangladesh to choose terms from a list that best Figure 3: School Principals and DEOs Disclaim Responsibility for Improving Student Learning (Bangladesh) described themselves. One-third (35 percent) of them chose words “There is little a school can do to help students learn if…” that related to qualities of leadership, 83% 75% 76% 74% such as ‘“committed,” “resilient,” 71% Responses (%) and “visionary” as opposed to less 59% leader-related terms like “well-liked” or “good writer.” Furthermore, they valued student learning and inclusion over other, less pedagogical parts of their jobs. Around 93 percent of respondents saw their main goal Parents do not seek Parents do not have the Parents have too many feedback from the teacher necessary education to personal/financial problems as maximizing student learning on student performance help their child be more to be concerned with their as opposed to being a leader and successful at school child’s performance at school inspiration for the community. However, principals and district School principals DEOs officers also saw their leadership BANGLADESH AND NEPAL 7 students and lower test scores or on Figure 4: School Principals and DEOs Tolerate Absenteeism Under Certain Circumstances (Bangladesh) one with more students and higher scores. If they put a high priority on “I think it is acceptable for teachers to be absent as long as…” ensuring equitable outcomes, they would select the first class even though it had fewer students. In Responses (%) 58% 50% fact, 54 percent chose the class with 46% 42% more students and better outcomes. 33% 34% However, when respondents were asked what advice they would give to another principal or DEO who was in the same situation, 65 percent They are doing They leave my They completed the something useful for students with work to assigned curriculum said the reverse, that they should the community do in their absense invest in the smaller class with School principals DEOs the lower scores. This suggests inconsistencies in the reasoning that principals and district officials use to make decisions. over one with a lower pass rate but no getting full marks, which suggests complaints from parents. that they put a higher priority on In general, principals and DEOs maximizing the achievement of a gave similar answers, but in a few Principals and DEOs prioritized few high-performing students than key areas they disagreed. DEOs students with the most potential on ensuring broad and inclusive were approximately five times more and felt they had limited ability to learning. likely than principals to choose help all students learn. A majority of test scores as the way to evaluate respondents in Bangladesh said that Principals and DEOs were reluctant the best teacher of the year rather there was little a school could do to to hold teachers to high standards. than their own opinion. Principals help a student learn if their parents Only 21 percent of respondents in in Bangladesh were 83 percent less were not educated, if the parents Nepal and Bangladesh claimed that likely than DEOs to choose to invest had too many financial or personal they would give teachers a formal in a classroom with fewer students problems, or if the student was not warning if they had high absentee and a lower pass rate over one well prepared in previous grades. rates. More than 50 percent of with more students and a higher Responding to another hypothetical DEOs in Bangladesh believed that pass rate, suggesting that they reward scenario, respondents in absenteeism was acceptable if the put a higher priority on equalizing Bangladesh and Nepal were asked students were given work to do while inputs than on equalizing outcomes. to choose between two teachers to the teacher is away or if the assigned Principals were also much more receive an award. In one teacher’s curriculum had been completed. likely than DEOs to support giving class, more students passed the extra resources to classrooms exam but no students received full Principals and DEOs had with inadequate infrastructure over marks, while in the other, fewer inconsistent views on equalizing those with low learning outcomes students passed but one in ten inputs and outcomes. The (93 percent of principals versus received full marks. A majority of respondents in Nepal and 52 percent of DEOs). the respondents (53 percent) chose Bangladesh were asked whether to reward the teacher with fewer they would prefer to spend extra students passing but a small number money on a classroom with fewer CONCLUSIONS The study points to potential entry points for the use of results-based study also points to issues that could impede a broader use for results- The study findings highlight the financing to strengthen incentives based financing. For example, the importance of directly targeting and improve education outcomes. study finds that school principals are middle-managers as important The findings of the study highlight unwilling to sanction low-performing agents of change to promote areas where decision-making by these teachers or support students in need principles of results-based financing managers are already aligned with of remediation because they recognize at the front lines of education principles of results-based financing that these actions fall outside of service delivery. Important including the use of objective their direct control. Overall the study decisions about how schools information, and a focus on higher- provides useful insights into how function are made by meso-level order results. These findings point to decisions are made by these agents and school-level managers. As areas where RBF mechanisms would that can provide a useful starting point such, their actions can have big be able to strengthen incentives for to consider how RBF can improve implications for the quality of better information and improved decision-making and ultimately education service delivery. learning outcomes. However, the improve education outcomes. 1 Leithwood, K., K. Louis, S. Anderson, and K. Wahlstrom (2004). “How Leadership Influences Student Learning: Review of Research.” New York: Wallace Foundation. 2 Bruns, B., D. Filmer, and H. Patrinos (2011). Making Schools Work: New Evidence on Accountability Reforms. Washington, DC: World Bank Group. 3 Patrinos, H.A., F. Barrera-Osorio, and T. Fasih (2009). Decentralized decision-making in schools: The theory and evidence on school-based management. Washington, DC: World Bank Group. 4 Guth, W.D. and I.C. MacMillan (1986). “Strategy implementation versus middle management self-interest.” Strategic Management Journal, 7(4), 313-327. 5 Witziers, B., R.J. Bosker, and M.L. Krüger (2003). “Educational Leadership and Student Achievement: The Elusive Search for an Association.” Educational Administration Quarterly 39(3):398-425. 6 Robinson, V., C. Lloyd, and K. Rowe (2008). “The Impact of Leadership on Student Outcomes: An Analysis of the Differential Effects of Leadership Types.” Educational Administration Quarterly 44(5):635-674. 7 Bloom, N., R. Lemos, R. Sadun, and J. Van Reenen (2015). “Does Management Matter in Schools?” The Economic Journal 125:647–674. 8 Bertrand, M., R. Burgess, A. Chawla, and G. Xu (2017). The Glittering Prizes: Career Incentives and Bureaucrat Performance. University of Chicago. 9 Finan, F., B. Olken, and R. Pande, R., (2015). “The Personnel Economics of the State.” NBER Working Papers 21825, National Bureau of Economic Research, Inc. 10 Rasul, I., D. Rogger, and M.J. Williams (2018). Management and Bureaucratic Effectiveness: Evidence from the Ghanaian Civil Service. Washington, DC: World Bank Group. 11 World Bank (2018b). World Development Report 2018: Learning to Realize Education’s Promise. Washington, DC: World Bank Group. 12 Rousseau, D. (2006). “Is There Such a Thing as ‘Evidence-Based Management’?” Academy of Management Review 31(2): 256-269. 13 Dopson, S. and L. Fitzgerald (2005). Knowledge to Action?: Evidence-Based Health Care in Context. Oxford: Oxford University Press. PHOTO CREDITS: Cover: “Education in Nepal: Primary wing of Adarsha Saula Yubak Higher Secondary School” by GPE/NayanTara Gurung Kakshapati, license: CC BY-NC-ND 2.0 Page 2: “Alice Albright visits Cox’s Bazar, Bangladesh. September 2019” by GPE/Stephan Bachenheimer, license: CC BY-NC-ND 2.0 Page 3: “Visit to Azimpur Government Primary School. Dhaka, Bangladesh” by GPE/Chantal Rigaud, license: CC BY-NC-ND 2.0 Page 4: “Alice Albright visits Cox’s Bazar, Bangladesh. September 2019” by GPE/Stephan Bachenheimer, license: CC BY-NC-ND 2.0 RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) worldbank.org/reach REACH is funded by the Government of Norway through NORAD, the Government of the United States of America through USAID, and the Government of Germany reach@worldbank.org through the Federal Ministry for Economic Cooperation and Development.