RESULTS-BASED FINANCING RBF EDUCATION EVIDENCE NEPAL Can Incentives Drive Publishers to Produce Quality Reading Materials and Schools to Buy and Use Them? FEBRUARY 2022 The REACH Trust Fund supported a study to evaluate how the use of incentives impacted the production, procurement, and utilization of supplementary reading materials in lower primary grades Photo courtesy of World Vision in Nepal. The Results in Education for All Children (REACH) Trust Fund supports and disseminates research on the impact of results-based financing on learning outcomes. The EVIDENCE series highlights REACH grants around the world to provide empirical evidence and operational lessons helpful in the design and implementation of successful performance-based programs. Books are one of the most important are not always used properly or Solely distributing books does not tools for student learning. Evidence regularly in the classroom. necessarily mean that they are being used, or are provided in the from a wide range of countries language needed. Results-based financing (RBF) demonstrates that to improve literacy can help governments overcome and learning outcomes, students these obstacles and ensure that must have access to, and regularly all students have access to quality use, quality learning materials. books in an appropriate language. Unfortunately, many education Such incentives, which link rewards systems struggle to provide quality to the achievement of specific books in adequate quantities to targets, can motivate stakeholders at schools because of inflated costs, all stages of the supply chain—from RBF incentives provided along the book chain process inefficiencies in the supply chain, or book development and production keep book production ample, a lack of published content in the to procurement and distribution— appropriate, and accessible. appropriate language. Even when to improve their performance. schools have access to books, they Incentives can also be used to This note is adapted from the “Knowledge Improvement through Access to Books (KITAB) Project Endline Evaluation Report” commissioned by World Vision Australia and World Vision Nepal. 2 RFB EDUCATION | EVIDENCE encourage the utilization of books in WHAT ARE SUPPLEMENTARY READING MATERIALS? schools and at home. Supplementary reading materials are books or other reading The REACH Trust Fund partnered materials that complement the core textbooks for a curriculum with with World Vision International to additional information, alternative learning approaches, alternative create a digital, one-stop marketplace languages, or knowledge of relevant subjects not directly covered to facilitate the publishing and by the school curriculum. Teachers can use these reading materials procurement of books. The project for supplementary instruction and students can use them in provided financial incentives to structured or unstructured study. publishers to encourage them to Evidence from a range of countries produce supplementary reading suggests that supplementary materials (SRM) for lower primary reading materials can significantly students and to deliver orders increase learning compared to on time. It provided nonfinancial reliance on core textbooks alone. incentives to schools to encourage But in low-income countries, such them to buy these reading materials materials are rarely supplied to through the digital marketplace and schools in adequate quantities. Photo courtesy of World Vision to use them in the classroom. CONTEXT According to the government’s standards for primary schools, change the behavior of publishers and their end-customers (schools). known as the Prioritized Minimum The ultimate goal was to strengthen Nepal, a lower-middle income Enabling Conditions (PMEC), Nepal’s book supply chain and get country in South Asia, has achieved schools are required to have a “book quality books written in appropriate near-universal access to education1 corner” that is stocked with high- languages to children for use as but its 20,000-plus primary schools quality, appropriate textbooks and supplementary reading material in struggle to provide effective learning supplementary reading materials the classroom. to students. A 2018 assessment2 which are available for teachers and found that 60 percent of Grade 2 students to use. In actuality, very few students could not read a single schools have their own book corner. word of a short text in the Nepali Nepal In the Morang and Sunsari districts language. Children who start school in Southern Nepal’s Terai region at age four in Nepal can expect to where the REACH-funded activity complete 12.3 years of school by took place, only 15 percent of public their 18th birthday. Adjusting for primary schools had a book corner, 60% what they actually learn during that and none of the books included in time, however, their true expected these corners were peer-reviewed years of schooling is only 7.2 years.3 educational materials.4 of Grade 2 students One potential driver of Nepal’s The intervention decided to explore could not read a single poor learning outcomes is the country’s shortage of textbooks and whether results-based financing, word of a short text in supplementary reading materials. in the form of both financial and the Nepali language. nonfinancial incentives, could NEPAL 3 Photo courtesy of World Vision WHY WAS THIS INTERVENTION CHOSEN? The shortage of books in Nepal’s government,5 most report that the schools in 38 districts across Nepal schools reflects problems in the amount is inadequate for purchasing and worked with teachers, parents, book supply chain. Publishers the number of books they need. and the community to support early generally prioritize private schools The intervention sought to address grade reading. More generally, the in the provision of supplementary these challenges by establishing researchers set out to contribute to reading materials given that most an online marketplace to serve as a the global evidence base on the use private schools are based in urban convenient, single location to review of incentives to increase children’s areas that have easier procurement and order books. The project offered access to quality supplementary processes and a larger guarantee incentives to both publishers and reading materials. of payment. In addition, few schools to use the marketplace, publishers operate outlets outside with the goal of improving both the of the capital city Kathmandu. As supply of quality books and the a result, public schools in other process for purchasing them. In Public schools parts of the country traditionally particular, the researchers set out traditionally have had have had no reliable, centralized to compare the effectiveness and location where they can procure sustainability of this approach with no reliable, centralized suitable books, instead making do an alternative approach taken by location where they with whatever materials they can the National Early Grade Reading find at local shops. Lack of funds Programme (NEGRP), supported can procure suitable is another issue. Although schools by the U.S. Agency for International books. receive some funding for books as Development (USAID). The USAID part of their annual grant from the program provided free books to 4 RFB EDUCATION | EVIDENCE HOW DID THE each listed book written in a minority language and endorsed by the CDC. and stock book corners, the project awarded free books for every book INTERVENTION A smaller bonus of US$20 was paid for each listed book that was peer- that the schools purchased through the online marketplace at the start of WORK? reviewed but not CDC-endorsed. To the 2018/19 school year. Category incentivize timely publication and A schools received nine free copies The intervention, known as distribution of books, the intervention of various books, while Category B Knowledge Improvement through offered publishers a bonus equivalent schools received four free copies. Access to Books (KITAB), had two to 5 percent of the total value of each To motivate schools to maintain key parts: (i) the online marketplace book order that was delivered to the the stock of their book corners, for book selling and ordering, and Palika on schedule. the intervention offered smaller (ii) incentives at multiple stages incentives to Category A schools along the book chain aimed at The intervention was implemented to make a second round of book promoting the creation, procurement, in two districts. Both were selected purchases later in the school year, distribution, and utilization of quality as they had generally low student books. The marketplace, called KITAB literacy levels and a large population Bazaar, provided a single location that spoke minority languages. At where publishers could list SRM the same time, these areas had Book Supply Chain books prepared especially for lower comparatively strong Internet grades; schools could then visit the connectivity, which was needed Publishers online marketplace to place orders at to implement the online portal incentivized the beginning of the school year and aspect of the intervention. In total, forecasting to create endorsed to purchase additional books later on 898 government primary schools in the year, if needed. Publishers were in the two districts were eligible ? books in multiple expected to deliver orders on time to to use the online portal, or digital languages local municipalities, known as Palika, marketplace, and receive incentives. development where head teachers could collect The study began with a baseline and production their books. study in November 2018. The digital marketplace and incentives scheme The intervention offered two types were operational through summer of incentives: financial incentives of 2019, and an evaluation was KITAB Bazaar for publishers and nonfinancial provided for procurement conducted in April 2021. incentives for schools. Publishers procurement would receive US$100 for each book For schools, incentives primarily they listed on the marketplace that took the form of free books. Schools was written in the Nepali language were divided into two categories supply chain management and and endorsed by the Curriculum based on need: Category A consisted distribution Development Centre (CDC), the of “vulnerable” schools—very small government body which officially schools6 and those with a high Publishers endorses school materials. To number of Dalit or minority-language expected to deliver encourage increased production students—while Category B schools orders on time of high-quality books in minority included all other community schools use for added languages, publishers would get a from the project implementation incentive larger bonus totaling US$200 for area. To incentivize schools to create NEPAL 5 Photo courtesy of World Vision issuing one bonus book for each but ineligible for the incentives. Incentives book purchased. (Category B schools The researchers had intended to were not eligible for this incentive.) construct a pseudo-control group Publishers using administrative data from Schools received other incentives for Nepal’s Education Management Received US$100 for each endorsed book (Nepali language) listed on the utilizing the book corners. Officials Information System (EMIS), but the marketplace from sub-education districts made system was not fully operational two observational visits to schools at the local level at the time of Received US$200 for each endorsed during the school year to verify the intervention. As a result, the book (minority language) listed on the marketplace usage. Schools found to be using researchers were not able to access the book corners during both visits information on key indicators for the Schools: Category A received a mobile library bag (i.e. planned control group, such as the Vulnerable schools received nine a pocket chart) or a rack for book share of schools that had and were free books for every book purchased storage.7 Schools found to be using book corners. Instead, the (round 1), plus one free book for using books during one of the visits researchers used a mixed-method every book (round 2). received a teachers’ kit, including approach, employing interviews teaching and learning materials such with key stakeholders including Schools: Category B as a dictionary, scissors, and ruler. publishers, Palika education heads, Other participating schools and head teachers of schools to The study had some limitations. received four free books for every identify whether, and how, the book purchased at the start of the The evaluation did not include marketplace and the incentives school year. a formal control group, either of improved book development, schools without access to the publication, procurement, 898 schools participated from online marketplace or of schools distribution, and utilization. 2 districts with access to the marketplace 6 RFB EDUCATION | EVIDENCE WHAT WERE THE where free books were distributed through the NEGRP program. Under suggesting an increasing rate of use over time. Only one in 10 schools RESULTS? the USAID-supported program, schools received an average 59 was found to be not using its book corner on either visit. Schools also The school incentives were books per grade. bought more books during the year to effective in increasing the number maintain their book corners. The vast of books in book corners and in The incentives also appear to have majority (90 percent) of Category schools generally. At endline, 828 successfully motivated schools to A schools, which were eligible schools (92 percent of the total) utilize and maintain book corners. to receive a second incentive for had a book corner, compared to 138 Some 232 schools (29 percent) purchasing additional books midway schools (15 percent) at baseline. A were found to be actively using their through the year, did so. Category B total of 133,008 books were ordered book corner during both visits by the schools were not eligible to receive and delivered, with an average of Palika education head and received a the incentive for a second round of 161 books distributed per school mobile library bag or a book storage purchases, but one-third nonetheless and an average of 54 books per rack, while 70 percent were found to purchased additional books using grade for grades 1–3, benefiting an be “fully utilizing”8 the corner during their own funds. estimated 53,198 students. Despite at least one visit and received a that this results-oriented approach teachers’ kit.9 Strikingly, the share The publisher incentives were required schools to invest some of of schools utilizing the corners was successful in generating more their own funds in books, the rate of higher during the second visit (543 peer-reviewed books for the book receipt was similar to that for schools, or 67 percent) than the online marketplace. Publishers the neighboring district of Siraha, first (308 schools, or 38 percent), uploaded 136 peer-reviewed titles Photo courtesy of World Vision NEPAL 7 to the digital portal, even though incentives were less financially WHAT WERE could not overcome publishers’ belief that CDC should not be approving valuable than for CDC-endorsed books and the bonuses mostly went THE LESSONS supplementary reading materials. “CDC is not the appropriate institution to paying reviewers. Given that the official CDC approval process was LEARNED? to endorse creative or literary pieces of work,” commented one publisher. very lengthy, reviewers included Even nonfinancial incentives can CDC and education ministry be very effective for improving Technical support may be needed to officials, as well as professors and book access and supply, especially help users of digital book-ordering journalists, and publishers said they when schools lack funding to buy systems take full advantage of felt the review process improved books on their own. Despite the lack incentives. Although the intervention the quality of the books. of a control group, the researchers included training for head teachers on concluded based on their findings how to use the online marketplace, The publisher incentives also and interviews that the incentives a number reported problems using succeeded in increasing publication played a key role in encouraging the system.11 Because the portal of minority-language books, but the schools to purchase books and only allowed procurement during two absence of incentives for schools to establish book corners. “Books are specific time periods each school purchase such books meant few of expensive and we have no budget. year, head teachers did not have an those books were ordered. Nineteen Without the incentive we cannot opportunity to place multiple orders percent of the uploaded titles were purchase new books. It motivates us and become more familiar with the in minority languages. However, to order new books,” commented one system. In addition, some of the only one percent of portal orders head teacher. trained headteachers transferred to consisted of such books, possibly schools outside the districts in the because school incentives did not Incentives tend to be most effective study and were replaced by untrained provide specific incentives aimed at when supported by a clear policy teachers. Other teachers, meanwhile, motivating schools to order these framework. As noted above, reported challenges related to particular titles. establishment of book corners was Internet access. World Vision among the five minimum conditions International provided extensive However, the publisher incentives required of primary schools under the support to individual users to help aimed at increasing the number of government’s PMEC standards. The them overcome these barriers, and CDC-approved books did not work. information dissemination activities any future scale-up of the intervention In fact, publishers did not add any conducted under the intervention to would likely require support to be books endorsed by the CDC to the inform schools about the incentives provided in a more systematic way. online marketplace. Researchers also raised awareness about these conducted interviews with the standards.10 publishers to determine why. The publishers said that while the CDC Incentives may not be effective in Incentives tend to be has authority to review textbooks, overcoming resistance to specific they did not believe the agency reforms or initiatives. Although most effective when had the authority or expertise to publishers responded well to the supported by a clear incentives linked to publication of review literary works. In addition, the process for obtaining CDC peer-reviewed books, the (larger) policy framework. endorsement can be costly and incentives aimed at encouraging time-consuming. publication of CDC-endorsed books The online CONCLUSION marketplace facilitated access to quality supplemental reading marketplace The REACH Trust Fund partnered materials and improved the book facilitated access to with World Vision International to test supply chain in the two districts. incentives across the supplementary Several head teachers noted that the quality supplemental reading material book chain in Nepal. marketplace and an assured delivery reading materials Incentives for publishers increased system for ordered books made it the number of peer-reviewed SRM easier to buy books especially for and improved the books published and made available schools in remote areas. At the book supply chain in through an online marketplace, same time, head teachers said in but failed to overcome resistance interviews that the book incentive (for the two districts. among publishers to having such purchasing books through the portal) material approved by a government made a big difference for schools, curriculum authority. especially in Category A, which otherwise wouldn’t have adequate A majority of stakeholders, including budget to buy books to stock and school head teachers, Palika leaders, maintain a book corner. and publishers agreed that the online 1 Ninety-six percent of primary school-age children in Nepal were enrolled in 2019 (UNESCO Institute for Statistics). 2 USAID Early Grade Reading Assessment. Available at: https://earlygradereadingbarometer.org/nepal-2018/snapshots. ((“(accessed July 1, 2021).”)) 3 World Bank 2020. Human Capital Index. 4 According to the baseline survey for this study conducted in October 2018. 5 Known as the Per Child Fund. 6 Those with fewer than 65 students in grade 1–3 7 The intervention initially intended to deliver mobile library bags to all schools achieving full utilization. However, following feedback from schools in Sunsari District, researchers agreed to provide a book rack instead to schools in that district. 8 “Fully utilizing” means: (i) the books were in the classroom easily accessible to the students; (ii) there was a register maintained if the students were burrowing the books to take home; (iii) teachers were making use of those books to teach the students in the classroom. 9 A total of 801 and 806 of the 828 schools that received books (97 percent) were visited by Palika education heads at the first and second visit respectively, and were eligible for utilization awards. We are a little confused on which is the right number: the previous statement or this one? “A total of 828 schools received books and were eligible for rewards for using their book corners. To confirm use, Palika education heads observed 801 schools during the first round of visits and 806 schools during the second round.” 10 Seventy-nine percent of participating head teachers at endline said the project had raised their awareness of the PMEC requirement 11 At the midterm review conducted in November–December 2019, only 30 percent of head teachers had been able to place an order independently without seeking help from colleagues or the project team. RESULTS IN EDUCATION FOR ALL CHILDREN (REACH) worldbank.org/reach REACH is funded by the Government of Norway through NORAD, the Government of the United States of America through USAID, and the Government of Germany reach@worldbank.org through the Federal Ministry for Economic Cooperation and Development.