67286 Social Safety Nets An Evaluation of World Bank Support, 2000–2010 ©2011 The Independent Evaluation Group The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.worldbank.org/ieg/ E-mail: ieg@worldbank.org All rights reserved 1 2 3 4 5 14 13 12 11 This volume, except for the “Management Response� and The Chairman’s Summary,� is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. The findings, interpretations, and conclu- sions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. This volume does not support any general inferences beyond the scope of the evaluation, including any inferences about the World Bank Group’s past, current, or prospective overall perfor- mance. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank Group or IEG concerning the legal status of any territory or the endorsement or acceptance of such boun- daries. Rights and Permissions The material in this publication is copyrighted. IEG encourages dissemination of its work and permits this doc- ument to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. How to cite this report: IEG (Independent Evaluation Group). 2011. Social Safety Nets: An Evaluation of World Bank Support, 2000–2010. Washington, DC: Independent Evaluation Group, the World Bank Group. Cover: Children in Ethiopia, beneficiaries of the Productive Safety Net Program supported by the World Bank as well as a consortium of several other donors. The program provides work opportunities to create local infra- structure and cash transfers to food-insecure households. Photo courtesy of Jennie I. Litvack. ISBN-13: 978-1-60244-183-5 ISBN-10: 1-60244-183-9 Printed on Recycled Paper Independent Evaluation Group Strategy, Communication, and Learning E-mail: ieg@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 Contents Evaluation Managers � Vinod Thomas Director-General, Evaluation ABBREVIATIONS ...................................................................................................... V  � Cheryl Gray Director, Independent Evlaution Group, Public Sector ACKNOWLEDGMENTS ........................................................................................... VII  � Monika Huppi Manager, Independent FOREWORD.............................................................................................................. IX  Evaluation Group, Public Sector EXECUTIVE SUMMARY ............................................................................................ X  � Jennie Litvack Task Manager, Indpendent Evaluation Group, Public MANAGEMENT RESPONSE ................................................................................. XVII  Sector CHAIRPERSON’S SUMMARY: COMMITTEE ON DEVELOPMENT EFFECTIVENESS (CODE) ....................................................................................XXXI  1. INTRODUCTION ..................................................................................................... 1  Social Safety Nets and the Functions They Serve ..................................................... 1  Bank Engagement in Social Safety Nets ................................................................... 3  Objectives, Scope, and Methodology of the Evaluation ............................................. 4  Organization of the Report ......................................................................................... 6  2. ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS ........................ 9  The SSN Lending Portfolio ......................................................................................... 9  Commitments ..................................................................................................................10  Trust Funds..................................................................................................................... 14  Operational Objectives ................................................................................................... 15  SSN Instruments............................................................................................................. 16  Sector Boards and Lending Instruments ........................................................................ 17  Safety Nets in Fragile Settings ....................................................................................... 18  Countercyclicality of World Bank Support ...................................................................... 18  Analytic and Advisory Activities ................................................................................ 20  Economic and Sector Work and Nonlending Technical Assistance ............................... 20  Spreading Global Knowledge about SSNs ..................................................................... 21  Impact Evaluations of Bank-Supported SSNs ................................................................ 23  3. EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS...................... 25  Effectiveness of the SSN Portfolio ........................................................................... 25  Aligning Choice of SSN Instruments, Design, and Context ..................................... 27  Public Works Programs .................................................................................................. 27  Conditional Cash Transfers ............................................................................................ 30  SSN Instruments in Theory and Practice: The Reality of Second-Best Options ............ 31  i CONTENTS Results Frameworks ................................................................................................ 33  Development Impact ................................................................................................ 35  Long-Term Impacts ........................................................................................................ 36  Weighing the Costs and Trade-offs of Different Policy Options ..................................... 38  Risks to Sustainable Outcomes ............................................................................... 39  Fiscal Risks..................................................................................................................... 40  Political Risks ................................................................................................................. 41  Institutional Risks ............................................................................................................ 43  4. BEYOND PROJECT SUPPORT: THE BANK’S ASSISTANCE TO COUNTRIES TO ESTABLISH SOUND SSNS ............................................................................... 45  Focus and Effectiveness of Bank Support for Country SSNs .................................. 45  Reducing Chronic Poverty and Inequality ...................................................................... 47  Developing Human Capital ............................................................................................. 48  Protecting against Idiosyncratic Shocks ......................................................................... 49  Protecting against Systemic Risks ................................................................................. 50  Cushioning Effects of Economic Reforms ...................................................................... 53  The Bank’s Support for Country SSNs Systems and Institutions: The Building Blocks of SSNs.................................................................................................................... 55  Targeting......................................................................................................................... 57  Payment Systems ........................................................................................................... 58  Monitoring and Evaluation .............................................................................................. 59  Support to SSNs within Broader Social Protection Systems ................................... 59  5. UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS ...................... 63  Political Economy Issues in the Design, Implementation, and Reform of Programs and Systems of SSNs .............................................................................................. 63  The Bank’s Use of Political Economy Knowledge .......................................................... 66  Nature of Bank Engagement ................................................................................... 68  Sustained Bank Engagement ......................................................................................... 68  Strategic Engagement and Links to Broader Country Dialogue ..................................... 72  Links to the Growth Agenda ........................................................................................... 73  Internal Coordination ...................................................................................................... 75  External Coordination ..................................................................................................... 76  6. MAJOR FINDINGS AND RECOMMENDATIONS ................................................ 79  Boxes Box 1. International Conferences and the Impact on Country Policies ..................... 22  Box 2. The Latin America and the Caribbean CCT Learning Circle .......................... 22  ii CONTENTS Box 3. Indirect Effects of SSNs .................................................................................. 37  Box 4. Unemployment Insurance for Rural Areas: The Mahatma Gandhi National Rural Employment Scheme in India ........................................................................... 50  Box 5. Pakistan—Learning from Experience and Developing SSN Systems to Address Shocks .........................................................................................................52  Box 6. Difficult SSN Reforms in South Asia ............................................................... 64  Box 7. Indonesia—SSNs from Supporting Macro Reforms to Permanent System .... 67  Box 8. Brazil—Sustained Engagement and Building SSN Institutions through a Range of Inputs ..................................................................................................................... 69  Box 9. Moldova—Combining Early-Period AAA and Technical Assistance with a Multidonor Lending Program in Preparation for Government Reform ........................ 70  Box 10. Links between SSNs and Growth ................................................................. 74  Box 11. Improving Results Frameworks for SSNs: Key Questions to Ask of Each Program .....................................................................................................................82  Tables Table 1. Share of Projects Supporting SSNs in LICs and MICs Containing Specific SSN Instruments ........................................................................................................17  Table 2. Projects with Substantial or High Efficacy Ratings of SSN Objectives, by SSN Program Type .................................................................................................... 27   Table 3. Average Impacts of Bank-Supported SSN Programs .................................. 36  Figures Figure 1. Trends in World Bank Commitments and Project Approvals ($ millions), by Fiscal Year of Approval ......................................................................10  Figure 2. Projects Supporting SSNs by Region ......................................................... 11  Figure 3. SSNs Commitments by Region ($ millions) ................................................ 11  Figure 4. Top 10 SSN Borrowers ............................................................................... 12  Figure 5. SSN Operations and Lending as a Percentage of Total World Bank Operations and Lending, by Country Income Level ................................................... 13  Figure 6. Trends in RETF Expenditures ($ millions) .................................................. 15  Figure 7. SSN Functions Served by Projects Supporting SSNs ................................ 16  Figure 8. Trends in Projects Supporting SSNs, by Lending Instrument ..................... 18  Figure 9. Safety Nets Commitments and GDP Growth in Turkey .............................. 19  Figure 10. Safety Nets Commitments and GDP Growth in Colombia ........................ 20  Figure 11. Trends in SSN ESW and Nonlending Technical Assistance Activities ..................................................................................................................... 21  Figure 12. Impact Evaluations of Bank-supported SSN Programs, by Region .......... 23  Figure 13. Percentage of Projects with a Performance Rating of Moderately Satisfactory or Higher, by Country Income Level ....................................................... 26  Figure 14. Project Objectives and Performance Indicators Coded as Outcome Driven by Loan Type .................................................................................. 34  Figure 15. Average Ratings of the Relevance and Bank Objectives of SSN Functions ........................................................................................................... 46  Figure 16. Trends in SSN Functions .......................................................................... 46  iii CONTENTS Figure 17. Rankings of the Bank’s Application of the Social Risk Management Framework, by Period ............................................................................................... 60  Appendixes APPENDIX A HISTORY OF SOCIAL SAFETY NETS AT THE WORLD BANK ..... 85  APPENDIX B METHODOLOGY ............................................................................... 99  APPENDIX C SUPPLEMENTAL TABLES AND BOXES ...................................... 149  APPENDIX D SUPPLEMENTAL FIGURES ........................................................... 157  BIBLIOGRAPHY..................................................................................................... 161  ENDNOTES ............................................................................................................ 161  iv Abbreviations AAA Analytic and advisory activities KPI Key performance indicator BISP Benazir Income Support Program LIC Low-income country CCT Conditional Cash Transfer MIC Middle-income country CDD Community-driven development M&E Monitoring and evaluation DPL Development policy loan PDO Project development objective ESW Economic and sector work PREM Poverty Reduction and Economic FSSP Female School Stipend Program Management (Punjab) PSIA Poverty and Social Impact Analysis GDP Gross domestic product PWP Public works program IBRD International Bank for RSR Rapid social response Reconstruction and Development SSN Social safety net IDA International Development Agency UCT Unconditional cash transfer IEG Independent Evaluation Group v Acknowledgments This evaluation is a product of the Independent Evaluation Group (IEG). Jennie Litvack was the task leader and principal author of the evaluation. Victoria Monchuk (deputy task team leader), Ursula Martinez, Javier Baez, and Sonya Sultan all made important contributions throughout the report. This evaluation was conducted under the overall direction of Vinod Thomas, Cheryl Gray, and Monika Huppi. This report is based on contributions from a team of IEG staff and consultants who con- ducted a portfolio review analysis, desk- and field-based country case studies, surveys, and several background thematic papers. Marie-Jeanne Ndiaye was responsible for all adminis- trative aspects of the evaluation. William Hurlbut provided editorial support. The portfolio review was conducted by Ursula Martinez and Manisha Modi, with input from Shampa Sinha. Andaleeb Alam conducted the econometric analysis throughout the report, and Tu Chi Nguyen assessed the countercyclical nature of social safety net (SSN) lending, as well as the distributional analysis of SSN lending under the supervision of Javier Baez. Suzana Abbott and Ursula Martinez analyzed the results framework of the portfolio. Anna McCord assessed the public works programs in the portfolio and prepared a paper on SSNs in low-income countries. The 26 desk-based country case studies and 4 country case studies were prepared by Dayl Donaldson (East Asia), Victoria Monchuk (Middle East and North Africa, Jamaica), Hjalte Sederlof (Latin America and the Caribbean, Eastern Europe and Central Asia), Rachel Slater (Ethiopia), Sonya Sultan (Africa, South Asia), and Carlos Eduardo Velez (Colombia). Analy- sis of the case studies and their application to the many evaluation questions and subques- tions posed by this evaluation was done by Victoria Monchuk and Sonya Sultan. Melvin Vaz designed the data analysis template. Hjalte Sederlof analyzed the application of the so- cial risk management framework. Javier Baez and Tu Chi Nguyen conducted the literature review of impact evaluations on SSNs. Two new impact evaluations on the long-term impacts of conditional cash transfers were conducted by Andaleeb Alam, Javier Baez, and Ximena del Carpio for Pakistan; and Javier E. Baez and Adriana Camacho, with support from Andaleeb Alam, Tu Chi Nguyen, Roman A. Zarate, and Roman D. Zarate, for Colombia. These three reports are available as separate IEG studies. Ursula Martinez conducted the staff survey of SSN response to the food, fuel, and financial crises, as well as the analysis of trust funds, analytic and advisory activities and the history of SSNs at the World Bank. She also conducted client interviews to assess the sharing of global knowledge, along with Victoria Monchuk. Robert Gillingham assessed the Bank’s support for economic reform through development program loans and their treatment of SSNs, with inputs from Ximena del Carpio and Ursula Martinez. Domenico Lombardi con- ducted extensive interviews with more than 40 Bank staff and managers to assess how the Bank’s organizational structure affects its work on SSNs. William Ascher wrote a back- vii ACKNOWLEDGMENTS ground paper on the political economy of SSNs. Jonathan Rodden examined the issues in- volved in supporting SSNs in decentralized settings. This evaluation greatly benefitted from financial support from NORAD. Norwegian funds helped fund consultants’ time and travel for conducing one in-country case study (Georgia) and several background papers. The Norwegian support greatly enhanced the quality and depth of data collection and analysis for the evaluation. IEG colleagues provided helpful guidance and comments, including Martha Ainsworth, So- niya Carvalho, Anis Dani, Catherine Gwin, Ali Khadr, Mark Sundberg, and Aristomene Va- roudakis. Many World Bank staff and managers provided useful comments and support during the evaluation. Among them were: Harold Alderman, Colin Andrews, Arup Banerji, Aline Coudouel, Maria Conception Steta Gandara, Roberta Gatti, Margaret Grosh, Jesko Henschel, Polly Jones, Kathy Lindert, Bassam Ramadan, Mansoora Rashid, Laura Rawlings, Helena Ribe, Lynne Sherburne-Benz, Kalanindhi Subbarao, Dominique van de Walle, So- phie Warlop, Adam Wagstaff, Ruslan Yemtsov, and Hassan Zaman. The team thanks the hundreds of individuals in government, civil society, the donor community, and the World Bank who were interviewed for background papers, project evaluations, and country case studies, especially in countries where field-based studies were undertaken—Colombia, Ethiopia, Georgia, Indonesia, and Jamaica. The external advisory panel for this evaluation consisted of Armando Barrientos (University of Manchester), Ravi Kanbur (Cornell University), and Gordana Matkovic (Centre for Liber- al-Democratic Studies, Belgrade, Serbia). Martin Ravallion (Director of Research, World Bank Development Research Group) served as peer reviewer from the approach paper stage through the final report. Jonathan Hoddinott (Senior Research Fellow, International Food Policy Research Institute) was peer reviewer for the approach paper, and Martin Rama (Lead Economist, East Asia, World Bank) was peer reviewer for the final report. The many thoughtful and thorough comments from all these reviewers improved the report and are much appreciated. viii Foreword Recent crises—fuel, food, and financial—have underscored the urgency of developing social safety nets (SSNs) in all countries. These crises have pushed millions of additional people into extreme poverty, reversing previous gains in poverty reduction. Even before they were hit by the crises, many countries were not on track to achieve the Millennium Development Goals because of chronic poverty and inequality. The consequences of increasing weather variability, climate change, and natural disasters have made things more difficult everywhere. When households are hit by shocks, SSNs can be used to protect particularly vulnerable people by providing liquidity, offering short-term employment, and discouraging negative mechanisms for coping with the setbacks. Many countries, especially high- and middle-income countries, have some form of targeted SSN programs, and they are spreading to low-income countries, too. During fiscal years 2000–10, the World Bank supported SSNs with $11.5 billion in lending and an active program of analytic and advisory activities and knowledge sharing, much of it during the last two years in response to the triple crisis. Prior to the crises, the Bank focused its SSN support mainly on addressing chronic poverty and investing in the human capital of the poor. The crises pointed out weaknesses in countries’ SSNs, as many middle-income countries found that their poverty-targeted SSNs were not flexible enough to increase coverage or benefits as needed, while low-income countries lacked poverty data and systems to target and deliver benefits. Bank support evolved in positive directions over the decade. Since the triple crisis hit, the Bank in- creased lending and analytical support greatly as it helped countries cushion their impact on their poor and vulnerable households and prepare SSNs to address future shocks. This recent crisis- related expansion of support included both low- and middle-income countries, increasing SSN activity greatly in many low-income countries. In addition, over the decade, the Bank began to move from a project-focused approach that emphasized delivery of social assistance benefits toward an approach that focused on helping countries build SSN systems and institutions to respond better to poverty, risk, and vulnerability. To build on results achieved to date, the Bank needs to engage now, when future shocks can be anticipated and appropriate systems put in place to help countries develop SSNs that effectively protect the poorest and most vulnerable. To this end, the Bank would want to focus on streng- thening its engagement in low-income countries, emphasizing SSN systems and institutional capacity, and improving the results frameworks of operations that support the development of SSN systems. Vinod Thomas Director-General, Evaluation ix Executive Summary Events of the past decade have underscored the vital need for social safety net (SSN) programs in all countries, especially in times of crisis. Many countries have some form of targeted SSN program, especially in high- and middle-income countries, but SSNs are increasingly spreading to the lowest income countries. Over fiscal years 2000–10, the World Bank supported SSNs with $11.5 billion in lending and an active program of analytical and advisory services and knowledge sharing, much of it during the last two years of the decade in response to the food, fuel, and fi- nancial crises. Yet the crises also pointed out weaknesses, as many middle-income countries (MICs) found that their poverty-targeted SSNs were not flexible enough to increase coverage or benefits as needed, and low-income countries (LICs) lacked poverty data and systems to target and deliver benefits. Bank support evolved in positive directions over the decade. The Bank began to move from a project-focused approach that emphasized delivery of social assistance benefits toward an ap- proach that focused on helping countries build SSN systems and institutions to respond better to poverty, risk, and vulnerability. Stronger demand for SSN support in MICs led to significantly stronger engagement there than in LICs. However, the recent crisis-related expansion of support included also LICs and permitted initiation of Bank support in 15 new countries. The Bank’s support to SSNs throughout the decade has relied strongly on both lending and knowledge shar- ing to engage clients. Bank support has largely accomplished its stated short-term objectives and helped countries achieve immediate impacts. But to achieve the longer-term goal of developing country SSNs, short-term objectives need to be better defined, effectively monitored, and anchored in a longer- term results framework. Weaknesses in poverty data, program designs, and monitoring indica- tors need to be addressed to ensure target groups are adequately reached. The Bank made substantial progress over the decade, but key areas of Bank support need streng- thening. First, the Bank needs to engage consistently during stable times to help countries devel- op SSNs that address poverty and can respond to shocks. Second, continued emphasis is needed on building SSN systems and institutional capacity. Third, stronger engagement is needed in LICs. Fourth, short- and longer-term results frameworks for Bank SSN support need to be strengthened. This involves improving the quality of objectives, design, and monitoring within projects, as well as developing a longer-term results framework for building effective SSNs to protect the poor and vulnerable. Finally, continued effort is needed to ensure strong cross- network coordination on SSNs. The Social Protection strategy currently under preparation is picking up on several of these issues and is seeking strategies for addressing them. Background and Context derscored over the past decade. It was particu- larly evident following the food, fuel, and finan- SSNs—programs designed to protect the poor cial crises, which pushed an additional 65 mil- and vulnerable from shocks and contribute to lion people into extreme poverty around the reducing poverty—are important elements of a world. country’s poverty reduction and growth strategy. The critical need for such programs in all coun- The Bank defines SSNs as noncontributory tries, especially in times of crisis, has been un- transfers targeted in some way to the poor and x EXECUTIVE SUMMARY vulnerable. This is a narrow definition and con- levant and important, the Bank and its client stitutes only one component of a country’s so- countries did not focus much on developing cial protection system; it is also just one of the flexible SSNs appropriate for responding to sys- many tools available for poverty reduction. temic shocks. When the food, fuel, and financial SSNs address chronic poverty and inequality, crises hit, lessons from previous crises were help the poor invest in developing human capi- once again underscored. Those countries that tal, and protect the poor and vulnerable from had developed SSN programs or institutions individual and systemic shocks, including during during “stable times,� such as Indonesia, were economic reforms. better positioned to scale up—and the Bank was better able to help them—than those that had SSN programs are a dynamic and growing area not. Results from IEG’s survey of Bank staff of World Bank work. Over fiscal years 2000–10, indicated that only 16 percent of countries’ the Bank loaned $11.5 billion to support such SSNs were well positioned to respond to the programs in 244 loans to 83 countries. Half of crises by being able to identify and reach af- the lending occurred in the last two years of the fected poor households. The two most common decade, when the SSN portfolio doubled from constraints for Bank support were weak country three to six percent of the Bank’s lending. It institutions and inadequate data. also conducted 297 analytic and advisory activi- ties focused on SSNs and 92 impact evaluations The lack of adequate SSN programs in of 24 Bank-supported SSN programs (the larg- many countries led the Bank to support SSN est body of evidence from impact evaluations of instruments that were not designed for crisis any sector) and produced hundreds of training response. Although existing instruments seminars, study tours, and publications. The enabled countries to provide benefits to various Bank’s lending and analytical support interact subsets of poor and vulnerable people, modify- with economic, political, and institutional condi- ing the target groups or scaling up the program tions to enable the Bank to have an impact in a to address new needs proved difficult. Moreo- country. ver, experience has shown that it is often diffi- cult to scale back benefits once a crisis subsides, The Independent Evaluation Group (IEG) has especially when SSN programs are not designed never evaluated SSNs before, but with the de- to be flexible or are delivered on a temporary mand for Bank support at an all-time high, an basis. Staff survey results indicated that 80 per- assessment of the Bank’s work in this area is cent of countries now have plans to strengthen timely. The purpose of this evaluation is to assess their SSNs to respond better to crisis. For ex- how effective and relevant the Bank has been in ample, in the Republic of Yemen, the Bank its support for SSNs and to draw lessons that can supported a labor-intensive public works pro- be applied to future support. Although the evalu- gram as a direct crisis response. At the same ation covers the whole decade, trends within the time, efforts were made to strengthen the cash- decade have also been examined, as experience in transfer program to serve as the main SSN for this area is evolving quickly. poverty reduction and crisis response in the fu- ture. Major Findings During this period, the Bank began to make Throughout the decade, countries and the an important shift from an approach based Bank focused SSN support mainly on ad- mainly on projects that emphasize delivery dressing chronic poverty and human devel- of social assistance benefits to helping opment and focused less on SSNs that can countries build SSN systems and institu- address shocks. During much of the decade, tions that can respond better to various most countries enjoyed strong and stable eco- types of poverty, risk, and vulnerability with- nomic growth. SSNs focused on addressing the in a particular country context. The systemic needs of the chronically poor or vulnerable as approach involves developing various capacities, well as developing the human capital of the such as data to identify vulnerable groups, tar- poor. Although these areas of support were re- geting systems to ensure the right groups receive xi EXECUTIVE SUMMARY benefits, payment mechanisms to channel re- countries’ low level of attention to SSNs. Bank sources to beneficiaries, and monitoring and resources for dialogue generally accompany evaluation systems to measure program imple- Bank preparation and supervision of operations; mentation and impact, as well as a set of coor- thus, countries with active lending programs are dinated programs appropriate for different provided more opportunities for dialogue (and groups of poor and vulnerable people. Only 14 potential for future lending) than those without percent of the 30 case study countries that IEG lending programs. This is indicated in results examined embraced a systemic approach to ad- from regression analysis, which show that coun- dressing different types of risk in the first half of tries receiving greater volumes of SSN loans are the decade, but 46 percent did so by the second more likely to receive further Bank SSN lending. half of the decade. Although this reflects client satisfaction with Bank services and enables the steady engage- The Bank was most effective in helping ment so important for SSN development, it countries where it had had steady engage- perpetuates a situation of low Bank engagement ment through lending, analytic and advisory for countries with initially low demand. activities, and dialogue over the decade. This longer-term engagement enabled the coun- Country demand for SSNs may be higher in try to develop SSN institutions and the Bank to MICs, but SSNs are also important in LICs develop a deeper understanding of country dy- to protect against systemic shocks and to namics. This was evident in such countries as help alleviate extreme poverty and food in- Brazil, Colombia, Ethiopia, and Moldova. security. The largest scale SSN supported by the Bank in a LIC is the Ethiopia Productive The Bank lending, analytical, and capacity- Safety Net Program, which benefited greatly building support for SSNs was significantly from capacity building, adapting the design as more concentrated in MICs than LICs the program progressed, and an exemplary ex- throughout the decade. The type of assistance perience in donor coordination. The Bank needs varied as well, with greater emphasis on institu- to engage in LICs, at least through analytical and tion building in MICs (57 percent of operations) advisory services, to help countries understand than in LICs (24 percent of operations). Despite the value of SSNs and improve their capacity to a recent increase in LIC engagement, SSNs re- design and implement programs that are appro- main a much less significant part of the devel- priate in their context. opment agenda in LICs than in MICs. Five per- cent of MIC lending and 13 percent of MIC Operations supporting SSNs generally have projects are focused on SSNs; only 2 percent of achieved satisfactory outcomes. The positive LIC lending and 6 percent of LIC projects focus results are particularly notable in LICS, where 88 on SSNs. percent of projects achieve at least moderately satisfactory ratings compared with an average of The stronger emphasis on MICs is driven 74 percent for other sectors. Consistent with by both country demand for SSN support project ratings, the evidence from impact evalua- and Bank supply of support. On the demand tions on SSNs shows positive short-term impacts side, MICs have a higher capacity to borrow and on measures such as household consumption, spend than LICs and can invest in more costly school attendance, and children’s health. Howev- institutional development and scale-up pro- er, these impact evaluations are for programs grams, rather than short-term relief programs overwhelmingly concentrated in the Latin Ameri- and pilots, which are typically used in LICs. In ca and the Caribbean Region. Some positive im- addition, with high poverty rates, tight budgets, pacts have been recorded for SSNs for protecting and many competing pro-poor needs, LICs may the poor during shocks, but very few studies have place less emphasis on SSNs as elements of examined this question. their poverty alleviation programs. Although short-term achievements have been On the supply side, the Bank’s low level of en- encouraging, results frameworks of opera- gagement in LICs may have perpetuated those tions supporting SSNs need to be further im- xii EXECUTIVE SUMMARY proved. Results frameworks have focused insuf- evident in IEG’s examination of conditional cash ficiently on the poor and vulnerable. By the transfers (CCTs), where both project results and Bank’s definition, SSNs support programs that impact evaluation evidence from more than 60 target the poor or vulnerable in some way. Yet impact evaluations measured short-term IEG found that objectives and performance achievements. Although CCTs often accom- indicators were often not specific enough to plished their limited objectives of transferring ensure effective monitoring of the effects of the cash to poor families and getting children to at- SSN project on the poor or vulnerable. Only 59 tend school, the achievement of the longer-term percent of operations supporting SSN had objec- objective of improving learning outcomes over tives that specifically targeted the poor and vul- time was rarely documented. Because the ratio- nerable, and 47 percent of operations supporting nale for CCTs in many countries is to invest in SSN did not have even one indicator to monitor the human development of poor children so they progress on reaching the poor. When the poverty can escape intergenerational poverty traps, moni- focus was mentioned, it was often in general toring the achievement of the longer-term objec- terms of “poverty reduction� rather than as part tive is necessary for success. of a time-bound objective directed toward a spe- cific subset of the poor. Results frameworks have Much effort has gone into rigorously eva- improved throughout the decade, but further luating the impact of various SSN programs improvements are necessary. on specific outcomes, but very little has been done to examine the cost of achieving The quality of results frameworks matters those outcomes and weighing alternative because clear objectives and monitoring policy options. The goal for SSNs, as in all indicators are necessary for coherent project areas of public policy, is to have the biggest im- design, in particular regarding coverage, pact possible for the least cost. For this reason, level and duration of benefits, and targeting. cost-effectiveness analysis can be useful in as- As an illustration, IEG examined in depth pub- sessing the cost per unit of impact of a particu- lic works programs, where documentation lar policy and comparing that result to other lacked important detail regarding which groups options. Yet only 12 percent of SSN operations of poor and vulnerable people would be em- assessed the cost and anticipated outcomes ployed by the public works and how the partici- from alternative programs. This may be because pation of and benefit to these groups would be of the lack of specific objectives and indicators, monitored. the presence of indirect effects, or the difficulty in obtaining data on benefits and cost. Howev- Short-term SSN project objectives have not er, even where impact evaluations have been been adequately anchored in a longer-term conducted and have provided precise evidence results framework. Whereas Bank-supported of benefits, this information has rarely been operations are short term in nature, the develop- used to weigh options and see where the largest ment of effective SSNs often takes longer, in- impact can be achieved for the least cost. volves multiple sequential phases of support, and requires complementary inputs from other sec- SSN programs have a unique set of fiscal, tors. Bank-supported operations can accomplish political, and institutional risks, in part be- limited objectives (and be considered satisfactory) cause of their need to expand and contract and still not lead to effective country SSNs to to meet needs. The size of the Bank’s financial protect the poor and vulnerable unless this long- contribution to SSN programs ranges from 9 er-term development is anticipated, implemented, percent of Brazil’s Bolsa Familia program to 100 and monitored. Whereas Bank supported percent of the CCT component of Macedonia’s projects may be formally accountable for only cash transfer program. Programs in MICs are those achievements occurring only during the life often supported largely by government re- of the project, the effectiveness of the Bank’s sources, whereas programs in LICs are usually support for SSN development also needs to be heavily donor dependent, given countries’ li- measured against longer term objectives that ex- mited capacity for redistribution. In LICs, the tend beyond the project cycle. This concern was major risk to fiscal sustainability is reduction in xiii EXECUTIVE SUMMARY donor support; in MICs it is change in govern- new political leadership was responsible for SSN ment priorities and budget support for SSNs. In reform (positive or negative) in half of the 30 a few upper-middle-income countries, such as countries. IEG also found that new political Colombia and Mexico, Bank-supported SSN space for SSN reform often opened up during programs throughout the decade have gone to crisis: 70 percent of cases where SSN programs the national scale, and there are concerns about were introduced occurred after a major country pressure on national budgets and questions of transition or crisis because there was political fiscal sustainability. These programs are still rel- appetite for it. SSNs are among the most politi- atively small (around 0.5 percent of gross do- cally sensitive areas of development policy, as mestic product), but Bank support to program they involve redistributing resources. Therefore, expansion, as in all areas of government ex- understanding how politics affect a country’s penditure, should be consistent with budget ability to design and implement SSNs is impor- management strategies. It should be monitored tant for the Bank’s effectiveness. The Bank’s to ensure sound design to achieve stated objec- support to Brazil’s Bolsa Familia was technically tives, and to avoid creating large, permanent oriented but politically savvy—it provided ap- entitlements. propriate input, in the right way, when most needed. Donor support for SSNs in LICs is often fragmented. Only in one of the LIC case Strategic engagement has strengthened the studies undertaken by this evaluation— effectiveness of Bank support for SSNs in Ethiopia—did IEG find strong Bank partic- many countries. Strategic engagement involves ipation in a well-coordinated, jointly fi- situating the SSN dialogue in wider country di- nanced effort. This is a missed opportunity for alogue, having a vision of how to build an effec- the Bank, as donor-dependent countries need a tive overall SSN system (for example, how vari- well-coordinated effort to harness the various ous SSN programs complement each other in sources of aid and advice, improve the efficien- the country context), and using the right in- cy and effectiveness of their programs, as well as struments at the right time with the right stake- enhance their fiscal sustainability. holders. In almost two-thirds of countries stu- died, the Bank supported SSNs in the context of In some SSN programs, fiscal pressure aris- country economic dialogue, Country Assistance es from the difficulty of scaling back a pro- Strategies, or sector discussions. However, in gram that has been expanded unless the less than one-third of the countries is there evi- benefits of the program itself are known to dence of the Bank having an overall vision of be temporary. Argentina faced such a challenge how its support helps build an effective SSN with its emergency Heads of Households Pro- system. In some countries, the Bank was able to gram throughout the decade, but after several take advantage of political openings for SSN years of effort (and Bank support), the country advancement when broader economic reforms successfully wound down the program designed were introduced. Bank and country experience for crisis response while continuing to streng- suggests that associating SSNs with economic then its permanent SSN. In other cases, crisis reform is an excellent way of garnering political provides an opportunity to introduce or enlarge support for both the reforms and the SSN a SSN program that is needed at the time and agenda (Colombia, Indonesia, Mexico, Turkey, remains in place to address the needs of specific and Yemen). groups among the chronically poor and vulner- able when the crisis subsides. SSNs are a challenge within the Bank, as they fall within the supervisory responsibili- Political economy issues have been impor- ty of the Social Protection Sector family, yet tant in determining the shape of SSN pro- less than half of projects containing SSN grams. An important factor in the Bank’s ability components are managed by staff in that to be relevant and effective in its support for sector family. Sources of tension exist with SSNs is its knowledge of political economy and regard to budget arrangements, task manage- how it is used. IEG’s case studies found that xiv EXECUTIVE SUMMARY ment, and accountability, although the extent  Improve the results frameworks of Bank varies by Region. supported SSN projects to (1) more clear- ly identify and address the needs of specific Recommendations groups of poor or vulnerable, and (2) identi- Because this evaluation assesses the World fy how project objectives fit within longer- Bank’s support for SSNs, the recommendations term objectives for development of country are for the Bank, with the objective of improv- SSNs. This involves improvements in the ing SSNs in countries. quality of objectives, design, and monitoring within projects, as well as development of a longer-term results framework or roadmap  Engage during stable times to build for building effective SSN programs and SSNs that can help countries respond systems. effectively to shocks. This requires steady country dialogue and support for develop-  Clearly define objectives and assess ing SSNs, whether by lending, country- benefits, costs, and feasibility of policy specific analytic and advisory activities, or alternatives to ensure the most appro- engagement in global knowledge and learn- priate use of SSNs. To ensure the most ing. Although the Bank’s focus on systemic appropriate use of SSNs, objectives must be shocks has accelerated since the most recent clearly defined and cost effectiveness type crises, greater attention is needed to design analysis used to assess policy alternatives. SSNs (in combination with other relevant Comparing alternative options for reaching programs) that adequately address systemic the specific objective(s) is particularly im- shocks. As such shocks are transitory in na- portant in contexts of high poverty and ture, an important characteristic is the abili- tight budgets and encourages specific SSN ty to expand and contract to reach different objectives. population groups as needed. Access to reli-  Improve internal coordination of SSNs. able poverty data, crisis monitoring systems, Budget systems need to be reviewed to see and flexible targeting systems are important if the incentives they create for managers elements to develop appropriate SSNs. and task team leaders constrain cooperation  Support the development of SSN institu- between Bank units. Expertise should be tions and systems. Institution building shared across sectors and networks to en- needs to be accelerated, particularly in LICs, hance Bank support. where capacity constraints are severe and the building blocks for SSN administrative systems may need to be built from scratch. In MICs, the approach will require continu- ing the effort to harmonize programs within the broader social protection framework.  Increase SSN engagement in LICs. The Bank needs to maintain special efforts (fi- nancing and internal incentives) for LICs to develop SSNs that will protect their poorest and most vulnerable and prepare for shocks. Depending on the country context, these may include improving country capac- ity, adapting SSN programs to the institu- tional environment, improving poverty data and analysis to identify the particularly vul- nerable groups, and assuring effective donor coordination for SSNs (for financing and technical assistance) to increase efficiency of government programs. xv Management Response Introduction Management welcomes this comprehensive review of the World Bank's support for social safety nets (SSNs) over the last 10 years. It is careful, detailed, and fair. Its analysis of how the Bank’s safety nets work has successfully met client needs—especially during the recent food, fuel and financial crises—is particularly timely. The Independent Evaluation Group’s (IEG) recommendations for how we can improve our safety nets work going forward close- ly match management's own strategic directions for Social Protection and Labor more broadly (as represented in the concept note for the proposed Social Protection and Labor strategy for 2012–22, endorsed by the Committee on Development Effectiveness in January 2011). Moreover, the detailed analysis in the report will be useful to add to the diagnostics and background work being done for the strategy. Management welcomes the overall findings of the review, which documents several achievements of the Bank’s work over the past decade: • Substantial portfolio. The World Bank has loaned $11.5 billion to support SSNs through 244 loans to 83 countries and has conducted nearly 300 analytic and advisory activities (AAA). The World Bank has also created a substantial inventory of knowledge products and related activities, including flagship global training courses and South-South learning fora. • Ability to help countries respond to crisis. In response to the food, fuel, and financial crises, the World Bank’s lending volume for SSNs increased from $1.2 billion in FY06–08 (precrisis) to $9.0 billion in FY09–11 (post-crisis). Between FY08 and FY10, 58 borrowers re- ceived lending and grant assistance for SSNs. The Rapid Social Response Multidonor and Catalytic Trust Funds and Japanese Social Development Fund Emergency Window (contri- buted by Japan, Russia, Norway, and the United Kingdom), as well as the Global Food Cri- sis Response Program, allowed a significant increase in capacity building and AAA in low- income countries (LICs), especially in LICs with which the Bank had not previously had di- alogue on SSNs. • Solid portfolio performance. Even with its rapid expansion, the SSN portfolio remains one of the best in the Bank. IEG rated 86 percent of projects with an SSN objective as mod- erately satisfactory or higher, compared with 78 percent of all other Bank projects. This higher performance is particularly striking for LICs, where 88 percent of projects with an SSN objective were rated moderately satisfactory or higher, compared with 74 percent of all other operations. • Use of impact evaluation. Over the past decade, 92 impact evaluations have been car- ried out on 24 SSN programs supported by the Bank, which in turn has generated a large, MANAGEMENT RESPONSE substantial evidence base not available, as yet in other sectors. These evaluations show posi- tive short-term results on household consumption, school attendance, and children’s health. Only a small number of studies have examined long-term impact, but there is some evi- dence that the immediate improvements in welfare created by SSN programs may be sus- tainable over time. The companion piece to this IEG evaluation presents a valuable sum- mary of all the available evidence while adding two new long-term evaluations. That said, it is important to recognize that the world of safety nets is extremely dynamic, and new knowledge and new practices are constantly evolving. Over the course of the last 10 years, the size and number of effective programs, together with the amount of safety nets know-how and evidence, have expanded at great speed. This performance has caught the close attention of the international community as well as the media and other development stakeholders. Countries are devising new SSN approaches every year, guided by the conti- nual harvesting of new evidence as to what works and what does not. This does suggest the vital importance of analyzing trends over time and not just settling on average findings over the course of the last 10 years. It is also important to recognize the degree to which the SSN revolution has been, and continues to be, led by the South, especially some large middle- income countries (for example Brazil, India, Mexico, and South Africa), which implies that knowledge work and South-South learning in particular is highly significant to the ongoing success of SSNs worldwide. Nonetheless, the importance ascribed to safety nets varies greatly from country to country and is often low in low income countries. SSNs’ redistributive and poverty-reducing role is well understood, but countries take very different positions about how much redistribution they want in their societies, and whether SSNs are the right tools to deliver it. SSNs’ role in risk management and resilient growth is also powerful, but the evidence for this is being built only slowly and is only beginning to filter out to central ministries, politicians, and the general public. As a result, there is less social consensus around the desirability of safety nets than, for example, around the goals for universal primary education or availability of clean water supply. Consequently, the World Bank will work with many countries to ana- lyze what role safety nets could play in their contexts, and to gather evidence, but should not expect to have an active lending portfolio in every country. Management Comments Management broadly agrees with the findings and recommendations of this review. In fact, three of the recommendations, namely: to increase work in the LICs, to move to a systems approach, and to improve results frameworks, are fully reflected in the main pillars of our new Social Protection and Labor Strategy, which is currently being shaped by a series of global consultations. This section provides management’s specific comments on each of the recommendations. Recommendation 1: Engage during stable times to build SSNs that can help countries re- spond effectively to shocks. Management agrees that this is a desirable direction and one on which the Bank can act. Indeed, the Bank has been working more vigorously in this area since the food, fuel, and finance crises of recent years hit. Having said that, the focus of the Bank’s assistance depends on the priorities assigned to the topic. However, management is xviii MANAGEMENT RESPONSE optimistic about being able to continue to engage in crisis preparedness at higher than pre- crisis levels, because of the following factors: • With respect to crisis preparedness, the triple wave of crises in 2008 and 2009 has raised countries’ demand for SSNs—a demand that is likely to be reinforced by the more recent resurgence of high food prices. • With respect to both crisis preparation and engagement in LICs, the Japanese Social De- velopment Fund Emergency Window, the Global Food Crisis Response Program, and the Rapid Social Response Multidonor and Catalytic Trust Funds have been extremely impor- tant in giving the Bank important means of involvement, funding 66 projects in 41 countries, of which 21 are in Sub-Saharan Africa. • We have a number of concrete actions in hand that can build on these opportunities to expand the development potential of SSN programs, as detailed in the Management Action Record. Management expects the Bank to sustain continuous engagement through a country-specific and time-varying blend of lending and all forms of nonlending support: formal economic and sector work, nonlending technical assistance, capacity building through training or South-South learning, involvement in donor coordination or impact evaluation, and so on. Measuring progress on this front will be principally a qualitative exercise. Looking only at lending volumes will not be enough to show progress. In fact, we expect our financing to taper off eventually as countries that have already borrowed extensively from the Bank re- cover eventually from crisis. Recommendation 2: Support the development of SSN institutions and systems. The Social Protection and Labor Strategy that is under preparation stresses this as a key focus area. The Bank will engage in this work at two levels: • Assisting countries to establish the “building blocks� of administrative capacity. These would include a targeting mechanism so programs reach the right beneficiaries, payment, and management information systems, and monitoring and evaluation. This work is already prominent in SSN operations and can be readily tailored to help in all country settings. • Understanding and improving the positive synergies between different social protection programs and regulations, such as social assistance programming, unemployment insur- ance, and contributory pensions. This is partly a design issue, but it can be assisted by shared technical platforms for eligibility, payments, and so on. Working at the systems level in this second sense is more feasible in countries that have multiple social assistance pro- grams in place or those where the Bank has a well-established dialogue. In other settings, an appropriate entry point may be to work on establishing or improving a single program, as a first step toward building more complex systems and their supporting technical elements. Recommendation 3: Increase SSN engagement in LICs. This is also a desirable direction, one that is highlighted in the Social Protection and Labor Strategy. Management expects that engagement will be sustained for LICs, as in Recommendation 1 for countries more general- ly, by a country-specific blend of lending and all forms of nonlending support. Work in this xix MANAGEMENT RESPONSE area must be sensitive to the very different visions from one country to another with respect to the role of safety nets, the capacity to finance them, and consequent differences in the choice instruments selected and the details of their implementation. The Bank will continue its engagements with countries to customize their strategy and take the next step pertinent in each context. Measuring progress on these fronts will be principally a qualitative exercise. Because the Bank has already engaged with a high percentage of LICs in the last three years, a simple country count of engagement is unlikely to reveal the extent of progress; rather, progress will need to be measured through the depth, breadth, and longevity of engagement in coun- tries. There are, however, two caveats worth noting with respect to the specifics of the SSN engagement in LICs: • The Bank’s level of SSN engagement at the country level will always be determined by demand and policy selectivity. The latter point on selectivity is especially relevant, given the limited amount of the Bank’s highly concessional IDA resources available for country use. Countries may also lack the broad national agreement on the role of safety nets necessary for their effective, sustainable operation. • The World Bank is actively involved in donor coordination through existing global and country-level frameworks; but it must be acknowledged that the degree of engagement may vary by country (as some national governments may assign other donors to the lead role in some cases), and successful coordination requires the active and willing engagement by other donors and partners. With respect to improvements in data collection as a means to guide social safety net as- sessments, it is important to recognize two different sides of the issue. The first is the collec- tion of basic poverty data to inform strategic choices, and the second is the development of targeting systems. These are both important and constrained in LICs, but progress can be made on the two independently. It is unlikely that rapid progress can be made in increasing the number of countries with robust coverage of safety net participation in their household survey data. Representative information on social assistance programs is typically only a feature of large, stable social assistance programs, which for the most part are missing in most LICs. Progress in the collection of multipurpose household survey data providing information on poverty has been steady but slow. National household surveys are produced throughout the world, and the Bank supports and maintains these databases. In many cases, Bank teams have been heavily involved in the process of designing and conducting these surveys. These efforts, coordinated by the Global Poverty Working Group sponsored by the Poverty Reduc- tion Board and Development Economics, has recently resulted in the single largest update ever of the database of poverty indicators—disseminated in the World Development Indica- tors report and on-line via the Open Data initiative (from 230 data points covering 104 coun- tries to almost 600 data points covering 115 countries). However, it should be noted that these are rarely Bank-led activities, but very complex national endeavors, where Bank sup- port involves funding (in most cases through trust funds), technical assistance, and capacity building. As such, progress is slow, particularly in LICs with low administrative and tech- nical capacity. In Africa and to a lesser extent in South Asia, poverty figures in particular, xx MANAGEMENT RESPONSE and household surveys in general, are available on average with considerable lags, low fre- quency, and problems of comparability. Steady support from the Bank and donors is re- quired to continue national efforts to improve household surveys data collection. Recommendation 4: Improve the results frameworks of Bank-supported SSN projects. Man- agement fully appreciates this recommendation. That the Bank is already moving in this di- rection is borne out by Human Development Network Anchor’s recent “Results Readiness in Social Protection and Labor,� which is now being disseminated to project teams (cited in the IEG report). This internal review shows a positive trend: more recently approved projects have better results frameworks. However, there are also notable areas in which task teams can do better: formulating project development objectives and aligning monitorable indicators along the results chain. Building on the recommendations in the HDNSP and IEG reviews, the HDNSP anchor’s results team will support task teams in applying lessons learned to the design of results frameworks for projects in the SSN portfolio. Management would also point out, however, that the long-term results and outcomes of SSN interventions go well beyond the scope of individual projects. Management views its range of instruments to help governments build a long-term vision in an inclusive way— including AAA, training and South-South learning as well as lending—and the vehicles that would contain long-run SSN strategies as country Poverty Reduction Strategies, national social protection strategies, and the like. Where such strategies exist, they may be referred to in the Project Appraisal Documents of specific operations. But it is unrealistic to expect that all Project Appraisal Documents will be able to make such references, particularly in the first operation of engagement in a country with only a nascent safety net sector or in crisis response projects. These longer-term outcomes can sometimes be picked up in follow-on projects but are more commonly being dealt with by specially designed impact evaluations carried out by Bank staff or by researchers outside the Bank. The report on impact evaluations accompanying the main IEG SSN evaluation offers some examples of such long-term assessments, which bypass the regular project cycle (for example, the evaluations of Colombia’s Familias en Ac- cion and Pakistan’s Punjab Secondary School Stipend Program). Given that most evidence in impact evaluations of SSNs comes from conditional cash transfer programs, IEG recom- mends expanding impact evaluations to other programs. Management agrees and has al- ready reformulated the clusters under the Bank’s Development Impact Evaluation initiative to include unconditional cash transfers and public works. Management therefore commits to continue its efforts to increase staff and client awareness of the importance of undertaking such longer-term impact evaluations and of making data from projects accessible to the re- search community at large. Recommendation 5: Clearly define objectives and assess benefits, costs, and feasibility of policy alternatives to ensure the most appropriate use of SSNs. While management agrees conceptually with the importance of comparing alternative means of reaching strategic goals, it has two problems with the application of this recommendation in specific project contexts. • First, the comparison of alternate policies or programs at the higher level (for example, SSNs versus other types of interventions), or of alternative SSN interventions (for example, xxi MANAGEMENT RESPONSE public works versus conditional cash transfers), is addressed not during project develop- ment, but through strategic analytic work—safety net assessments, public expenditure re- views, poverty assessments, support for countries as they prepare poverty reduction strate- gies and the like—undertaken at strategic junctures of continuous engagement (see recommendation 1). The Human Development Network has developed methodological tools (such as the “Human Development in Public Expenditure Reviews� guidance note, guidelines for Risk and Vulnerability Assessments, criteria for safety net assessments in Grosh et al. 2008) and literature to benchmark such work (Fiszbein and Schady 2009 on con- ditional cash transfers; del Ninno, Subbarao, and Milazzo 2009, and Subbarao and others 2010 on public works; Bundy and others 2009 on school feeding; and so forth). IEG’s com- panion volume, the meta-evaluation, will add to this literature. • Second, the task of comparing policy options is complicated by SSNs’ multiple objec- tives, which rule out simple rates-of-return analyses. The Bank’s conceptual definition of SSNs views their prime objective as reducing poverty through cash and in-kind transfers, and the SP ADEPT program developed by Human Development and Development Eco- nomics provides robust, comparable measures of this. But it is not sufficient to capture other objectives of SSNs. To help World Bank staff and country clients build the basis of more regular application of cost-effectiveness analysis, Human Development will provide en- hanced training as part of the ongoing suite of impact evaluation courses. In the Management Action Record, management largely follows IEG’s suggested focus on the formulation for lending operations, but with the understanding that at that stage of de- cision making, the range of options governments can consider is rather narrow, essentially focusing on alternative features of a given type of program or, more commonly, further de- velopment of an existing program. Management will focus its efforts on the core objectives of income support and effective institutions for the delivery of SSNs. To work toward being able to do better assessments, management proposes three areas: • To foment cost-effectiveness analysis in more of the significant body of impact evalua- tions • To conduct a global benchmarking of targeting performance of transfer programs of many types and using different targeting instruments • A series of knowledge pieces on the cost-effectiveness of different aspects of delivery systems to identify the most effective design options. By carrying out results readiness work and strengthening operational teams, management is taking actions to assist client countries in prudent and strategic choices about the choice of instruments, the size and selection of target populations and benefit levels and hence is helping to promote sustainability of safety nets. . Recommendation 6: Improve internal coordination of SSNs. The fact that SSN components occur in projects mapped to many sectors other than Social Protection implies a need to en- sure that the technical quality of the work is equally high and the advice consistent, no mat- ter which sector is in the lead. This IEG evaluation report provides somewhat limited analy- sis of whether this has been achieved, but raises no explicit flags and provides some small xxii MANAGEMENT RESPONSE reassurances. A number of mechanisms and practices are already in place to ensure that the knowledge function for SSNs cuts across boundaries: for example, cross-board membership, cross-sectoral thematic group and Global Expert Team membership, and consultation and collaboration on a series of flagship initiatives. As part of ongoing business modernization, management is strengthening sector boards to enable them to better manage the allocation of expertise across the Bank and revising the budgeting system for cross support to remove disincentives and encourage cross-unit collaboration and will report progress in the context of regular Board updates. Management proposes to continue and reinforce some of these mechanisms and to focus on improving the outreach of knowledge and quality assurance services. xxiii MANAGEMENT RESPONSE Social Safety Nets: An IEG Evaluation of World Bank Support, 2000-2010 Draft Management Action Record Acceptance by IEG findings and conclusions IEG recommendations management Management response 1. SSNs for Shocks: Throughout the Engage during stable times to Agree Management agrees with the recommendation: The Bank’s client countries decade, countries and the Bank focused build SSNs need broad-based and flexible SSN systems that address chronic poverty SSN support on addressing chronic that can help countries and its consequences but can also be scaled up to protect the poor from poverty and human development and respond effectively economic shocks—whether at global, country or household level. focused less on SSNs to address to shocks. This requires steady During the 2008–09 crisis, increased demand from clients and the shocks. During the last few years of the country dialogue and support for extraordinary amount of resources deployed by the World Bank have helped decade, the focus of countries and the developing SSNs, whether by to build a strong pipeline of activities that will persist into the immediate Bank expanded as addressing systemic lending, country-specific AAA or future, including in many countries that did not have prior SSN dialogue. The shocks took on greater importance. The engagement in global knowledge Bank will continue to provide client-driven assistance to respond to crisis urgency of preparing for systemic and learning. As such shocks are situations, and to help countries build systems that alleviate chronic poverty, shocks became particularly clear after transitory in nature, an important through its full menu of products—investment and policy lending (including the food, fuel, and financial crises, characteristic of a SSN is its new instruments, such as P4R), grants, and analytical activities. A key during which lessons from previous ability to expand and contract to element of this would be capacity building and knowledge exchanges. crises were underscored: reach different population groups These all will be financed through normal Bank budget allocations and those countries that had prepared as needed. Access to reliable supplemental seed funding from donors. during good times—by developing poverty data, crisis monitoring It will also launch new activities focused on helping countries build greater permanent SSN programs or systems, and flexible targeting capacity to respond to future crisis. Specifically: institutions—were better positioned to systems are elements to develop scale up, as needed—and the Bank appropriate SSNs. The proposed 2012–22 Social Protection and Labor strategy emphasizes was better able to help them—than continuing the move towards building country Social Protection (including those that had not. The Bank was most SSN) scalable and flexible systems that can be used to address both effective in helping countries in which it shocks and chronic poverty. New Social Protection and Labor (SP&L) had been steadily engaged through strategy (including results framework) developed, expected to be lending, AAA, or dialogue over the submitted to the World Bank Board (FY12). decade. Such engagement enabled the For the poorest client countries (who usually have inadequate SSN country to develop SSNs and the Bank systems), the Bank will use demand-driven grant financing from the first to develop a deeper understanding of round Rapid Social Response (RSR 1) multidonor trust fund to help country dynamics. countries to increase crisis preparedness—by building monitoring systems, flexible targeting and scalable SSN delivery systems, or devising crisis contingency plans. To meet unfulfilled country demand, the Bank will also xxiv MANAGEMENT RESPONSE Acceptance by IEG findings and conclusions IEG recommendations management Management response seek additional donor support for a second round of Rapid Social Response (RSR 2). Donor consultations for second round of RSR (FY12–13). The Bank will continue capacity building to enhance crisis resilience of client countries (LICs and middle-income countries [MICs]) through: South-South learning events (between MICs and LICs clients), continued offering of core training programs targeted to the representatives of client countries and staff of donor agencies. Organize new South-South learning events, and continue the annual SSN core course (FY12 and ongoing). 2. Systems Approach: During this Support the development of Agree Management agrees with the central role of the systems approach for period, the Bank began to make an SSN institutions and systems. safety nets and for social protection in general, as reflected by its support for important shift from an approach based Further accelerate institution the shift in project designs noted by IEG. The Bank will continue supporting mainly on projects that emphasize building, particularly in LICs, the systems approach in both LICs and MICs, adapted to country needs and delivery of social assistance benefits to where capacity constraints are capacity levels, and is aiming to implement this recommendation as follows: helping countries build SSN systems often severe and the building and institutions that can respond better blocks for SSN administrative The proposed 2012–22 Social Protection and Labor strategy puts systems to various types of poverty, risk, and systems may need to be built as a central theme for the SP&L practice in the World Bank in its vulnerability within a particular country from scratch. In MICs, the engagements with clients and partners. The strategy will emphasize context. The institutional development approach will require continuing deepening the Bank’s knowledge and practice for building effective SP&L approach was driven mainly by MICs, the effort to harmonize programs systems in different country contexts and capacity levels. It will also for which this was the most common within the broader social emphasize labor-market continuity of SSNs programs and will coordinate objective of Bank-supported SSN protection framework. with the IFC on private sector job creation. New SP&L strategy developed, programs. submitted to World Bank Board (FY12). The World Bank will develop and apply new County Policy and Institutional Assessment measures, with guidelines, for benchmarking Social Protection, including a new measure for Social Protection systems. This will become part of the regular process of monitoring progress towards system building. New County Policy and Institutional Assessment criteria and accompanying guidance note on Social Protection systems (FY12 onward). For LICs in particular, the Bank will draw lessons on developing the building blocks for SSNs in LICs through its current activities (supported by RSR and other crisis-related instruments) focused on building administrative systems, institutions and delivery capacity for SSNs. These lessons will be disseminated internally and externally. Success stories notes on good xxv MANAGEMENT RESPONSE Acceptance by IEG findings and conclusions IEG recommendations management Management response practices in LICs (launched FY12, fully posted and disseminated FY13). The Bank will develop new knowledge tools and data products to assess performance of SSN/SP&L systems through a global inventory of social programs, their rules and their interactions in delivering benefits at the household level. Global inventory of SSNs developed and maintained, and existing SP&L systems mapped with the new “SP Atlas� (first posted on web FY12, fully launched and updated by FY13–14). 3. SSN Engagement in LICs: The Increase SSN engagement in Agree Management agrees with the importance of continuing focusing efforts on Bank focused its lending, analytical, and LICs. The Bank needs to improving capacity of LICs to deliver sustainable SSNs for the poor and also capacity-building support for SSNs maintain special efforts (financing to protect access to basic services by the poor through all available significantly more on MICs than in LICs and internal incentives) for LICs instruments: lending, grants, AAA, training and knowledge-sharing activities. throughout the decade. Despite a recent that permit countries to develop To continue efforts to improve capacity of LIC SSNs, the Bank is committed increase in LICs engagement, SSNs SSNs that will protect their to the following actions: remain a much less significant part of poorest and prepare for shocks.  The proposed 2012–22 Social Protection and Labor strategy identifies the development agenda in LICs than in Depending on the country context increased engagement in LICs as the key action to increase coverage MICs. Country demand for SSNs may these may include: improving of the poor by social protection. The strategy will emphasize deepening be higher in MICs, but SSNs are also country capacity, adapting SSN the Bank’s knowledge and practice for supporting work in LICs, important in LICs, as they help protect programs to the institutional adapted to different contexts and capacity levels. New SP&L strategy against systemic shocks as well as help environment, improving poverty developed, submitted to World Bank Board (FY12). the poorest and most vulnerable people. data and analysis to identify the  To implement this strategic direction, the Bank will continue and The Bank’s low level of engagement particularly vulnerable groups and enhance capacity building and knowledge sharing activities for may have perpetuated countries’ low assuring donor coordination for operational staff of the Bank and client countries focused on LICs. level of attention to SSNs. The Bank SSNs (for financing and technical South South learning events and knowledge exchanges (FY12– needs to be engaged (at least through assistance) to increase efficiency 14). analytical and advisory services) to help of government programs.  To maintain the existing momentum in IDA countries, the Bank will countries understand the value of SSNs seek to sustain the engagement with LICs through continuous country and improve their capacity to design dialogue, strategic AAAs, and other forms of capacity building. The and implement the programs, as Bank will commit to report annually on the range of activities in LICs appropriate in each country’s context. and specifics of major country level engagements in LICs. Annual reports will provide details on country level engagement by regions and central vice presidential units in lending and nonlending services (FY12 onward).  To go beyond core work, the Bank will proactively seek additional donor resources to supplement Bank budget funding for SSNs capacity xxvi MANAGEMENT RESPONSE Acceptance by IEG findings and conclusions IEG recommendations management Management response building in LICs. Donor consultations for second round of RSR (FY12– 13).  The Human Development practice with Poverty Reduction and Economic Management will invest in a suite of knowledge products in the next two fiscal years, including tools to improve data adequacy on SSN targeting and delivery of benefits (including gender differentiated data). New knowledge products on building blocks of SSNs in LICs produced and posted on safety net website (analysis conducted in FY12–13, posted FY14).  The Bank will continue its yearly coordination meetings on social protection together with the International Labour Organisation, United Nations agencies and all key bilateral and multilateral donors, as well as actively participate in country-specific donor coordination meetings. Ongoing annual donor coordination meetings and participation in other donor coordination activities (FY11 onward) 4. Project performance and quality of Improve the results frameworks Partially agree Management partially agrees with this recommendation. Management results frameworks: The Bank’s sup- of Bank supported SSN agrees with and fully appreciates the need for improving results frameworks port for SSNs has been effective in projects to (1) more clearly iden- and of identifying project impacts on beneficiaries, as was borne out by our helping countries reach short-term ob- tify and address the needs of 2010 internal HDNSP results-readiness review. However, as long-term re- jectives and achieve short-term impacts, specific groups of poor or vulner- sults and outcomes go beyond the scope of individual projects, manage- such as increased school attendance or able, and (2) identify how project ment will focus on strengthening the application and consistency of medium immediate consumption. However, objectives fit within longer-term term indicators in the results chain to monitor performance of projects, and short-term SSN project objectives have objectives for development of through strategically targeted knowledge work. The Human Development not been adequately anchored in long- country SSNs. This involves network will also promote strengthened and consistent quality across the er-term results frameworks. Anticipating improvements in the quality of Bank through tools, analysis of outputs, impact evaluation and support to and planning for complementary inputs objectives, design, and monitoring task team in results. The Bank has agreed to the following: from other sectors (including inputs on within projects, as well as devel- monitoring poverty and vulnerability) opment of a longer term results  The Social Protection and Labor Sector Board will introduce the use of needs more attention. There is also a framework for building effective core sector indicators, as appropriate, for all new investment projects. New need to mobilize resources from outside SSN programs and systems. core sector indicators for social protection developed, validated and the project cycle to track progress to- introduced (FY12). ward the longer-term objectives, includ- ing selective longer-term impact evalua-  The HDNSP anchor will review and, as necessary, amend the existing tions. Results frameworks for projects Results Readiness Guidance Note for SSN to strengthen the focus on me- with SSN components focused insuffi- dium term key performance indicators and their fit within longer term objec- xxvii MANAGEMENT RESPONSE Acceptance by IEG findings and conclusions IEG recommendations management Management response ciently on clearly identifying the poor tives. and vulnerable target groups. (By the Bank’s definition, SSNs support pro-  HDN will reach support operational staff working on the results frame- grams targeted to the poor or vulnerable works of Specific Investment Loans, Development Policy Loanss and P4Rs in some way.) Yet IEG found that objec- and evaluation of safety nets components with tools, training and custo- tives and performance indicators were mized support via its toolkits, publications, website, training events, quality often not specific enough to ensure enhancement clinics and the like. effective monitoring of the effects of the  Regarding the above three points, support for results in projects project on the poor or vulnerable. (FY12 and onward). Project objectives need to be defined more precisely, monitorable key perfor- mance indicators need to be better aligned with those objectives and ac- companying M&E arrangements need to track their performance. 5. Weighing Policy Options and Us- Clearly define objectives and Partially agree Management only partially agrees with this recommendation for reasons ing Cost- assess benefits, stated in the accompanying text. Management recognizes that at the broad Effectiveness Analysis: The goal for costs, and feasibility of policy intersectoral level, choices about whether to invest in safety nets or other SSNs, as in all areas of public policy, is alternatives sectors are made at the level of Poverty Reduction Strategy Papers, or with to have the biggest impact possible for to ensure the most appropriate respect to country engagements, in the Country Assistance Strategy or the least cost. For this reason, cost use of SSNs. Comparing alterna- Country Partnership Strategy. In its proposed actions, management follows effectiveness analysis can be useful in tive options for reaching the spe- IEGs’ suggested focus on the formulation lending operations, but with the assessing the cost per unit of impact of cific objective(s) is particularly understanding that at that stage of decision-making the range of options a particular policy and compare that important in contexts of high po- under consideration by government is rather narrow, essentially focusing on result to other options. SSN operations verty and tight budgets and en- alternative features of a given type of program or more commonly, further have usually been undertaken without courages specific SSN objectives. development of an existing program. In the tools to improve consideration at explicitly considering alternatives for this stage, the Bank will focus its efforts on the core objectives of poverty achieving specific objectives (for exam- reduction through cash or in-kind transfers and building effective institutions. ple, by using cost-effectiveness analy- sis). Three specific actions proposed are: This may be because of the lack of specific project objectives and indica-  A series of SSN primers assessing the cost-effectiveness of different tors, the presences of indirect effects or delivery systems for cash transfers, namely: beneficiary registry systems, the difficulty in obtaining data on bene- payment systems and management information systems. These will be fits and cost. However, even where available to strengthen the economic analysis and design of the capacity xxviii MANAGEMENT RESPONSE Acceptance by IEG findings and conclusions IEG recommendations management Management response impact evaluations have been con- building or service delivery component of SSNs projects. Deliver a series ducted and provide precise evidence of of SSN Primer Notes on the cost-effectiveness of delivering cash benefits, this information has rarely transfers: beneficiary registries, payment mechanisms and MIS (FY12– been used to weigh various options or 14). assess where the largest impact can be  The Social Protection ADEPT module reports on targeting effective- achieved for the least cost. ness of SSN transfers. It will be applied across countries of the world to produce summary outcome indicators made available in Social Protection Atlas (including gender disaggregated data). “Social Protection Atlas� (first posted on Web FY12, fully launched and updated by FY13–14).  To encourage more cost-effectiveness analysis within the panoply of impact evaluations, a new module on the topic will be developed and in- cluded as a component of the general impact evaluation regional training courses being coordinated by HDN Chief Economist’s office. Include a cost-effectiveness module in the general impact evaluation training courses (FY12 forward). 6. Bank Internal Coordination: SSNs Improve internal coordination Agree Management agrees with this recommendation. are a challenge within the Bank, as they of SSNs. fall within the supervisory responsibility Review budget systems to see if  As part of ongoing business modernization, Management is streng- of the Social Protection sector family, the incentives they create for thening sector boards to enable them to better manage the allocation of yet less than half of projects containing managers and task team leaders expertise across the Bank and revising the budgeting system for cross sup- SSN components are managed by staff constrain cooperation between port to remove disincentives and encourage cross-unit collaboration and will in that sector family. Bank units. report progress in the context of regular Board updates. Sources of tension exist with regard to Share expertise across sectors Bank management is committed to continued strengthening of the SSN budget arrangements, task manage- and networks to enhance Bank community of practice and its intersectoral linkages by: ment, and accountability, although the support. extent varies by Region.  Ongoing review by Social Protection Sector Board of thematic coding system for new social protection lending and AAA activities, which will im- prove capacity to monitor portfolio and cross-sectoral linkages in operations with Social Protection components.  Maintaining cross-sector board membership and ongoing coordination and dialogue between Human Development Network Social Protection and Labor and Poverty Reduction and Economic Management Poverty and Eq- uity. Cross-Board membership and informal dialogue (continues FY11 and onward). xxix MANAGEMENT RESPONSE Acceptance by IEG findings and conclusions IEG recommendations management Management response  Reaching out to other sector teams with key SSN knowledge products and providing operational support to all teams engaged in SSN regardless of their sector affiliation. Dissemination across sectors of existing and new knowledge products, including proactively reaching out to task team leaders managing lending operations with SSN themes (FY12 onward).  Continuing consultation and collaboration in topical and key knowledge products by Social Protection and Labor, Education, Health, Nutrition, and Population; Poverty Reduction and Economic Management poverty, Agricul- ture and Rural Development, Sustainable Development Network as topical, for example, Social Protection and Labor staff will be involved in on-going products managed by other networks. Ongoing examples include:  Fuel subsidy reform and the role of safety nets—with the Energy Anc- hor, Sustainable Development Network.  Climate change adaptation and safety nets—with Social Development.  Nutrition and Social Protection/scaling up nutrition—with Health, Nutri- tion, and Population.  Food price monitoring—with Poverty Reduction and Economic Man- agement poverty  Labor market work on informality—with Poverty Reduction and Eco- nomic Management poverty and Finance and Private Sector Development. xxx Chairperson’s Summary: Committee on Development Effectiveness On May 25, the Committee on Development Effectiveness (CODE) considered the document Social Safety Nets—An Evaluation of World Bank Support, 2000-2010 prepared by the Independent Evaluation Group (IEG) and Draft Management Response. Summary In its statement, IEG provided the global context, underscoring the urgency of developing adequate social safety nets as a cushion against shocks on the poor and vulnerable and to countries’ poorest when not in crisis. The evaluation assessed the Bank’s effectiveness and relevance in helping countries to develop and implement SSNs in the past decade and to determine trends to help inform future support, drawing on evidence from the portfolio, country cases, and other sources. In its opening remarks, Management welcomed this timely and relevant evaluation, which echoes Management’s strategic directions for the proposed Social Protection and Labor Strategy for 2010-2022, (SP&L Strategy), under preparation--the Concept Note was discussed by CODE in January 2011. Management broadly agreed with IEG’s recommendations including the need for greater emphasis on SSN programs and systems to prepare countries better for shocks and greater emphasis on low-income countries. Management agreed with the need to strengthen the results framework for SSN projects and evaluate longer-term impacts of projects, but noted minor differences about how this might be achieved. IEG emphasized that development impact of SSNs requires a country team effort, and accountability extends beyond those working on social protection. It also underlined the need to ensure that Bank-supported interventions be appropriately anchored in a strong results framework and viewed as part of a road map of helping achieve countries’ longer-term development objectives aspired with SSNs. The Committee appreciated IEG’s comprehensive evaluation and Management’s Draft Response. Members stressed the need to focus on the long-term development impacts of SSNs, while recognizing the difficulty in designing a comprehensive and systemic results framework that could assess both short-and-long-term needs. They agreed with enhanced focus on low-income countries (LICs), while underscoring that this focus should not divert attention away from middle-income country (MIC) needs. Also highlighted was the need to leverage the Bank’s SSN experience to date and to put in place systems/cushions for unexpected shocks and crises. As the IEG evaluation noted, when hit by the recent food, fuel and financial crises, the vast majority of countries were not well prepared to respond. Members raised questions about the sustainability of SSNs, emphasizing the need for prudent choices of instruments, target groups —including addressing gender issues-- benefit levels, and the use of innovative financing schemes. They stressed due consideration of political economy issues. The need for a systemic approach to SSNs, based on country-appropriate institution building over time and complimentary interventions was noted, as were the importance of South-South learning and technical assistance. Questions were raised regarding broadening the narrow definition of SSNs, including looking at more traditional systems of social protection. Members also highlighted the importance of addressing inter-generational transmission of poverty. Anna Brandt, Chairperson This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank Group authorization. Statement of the External Advisory Panel Armando Barrientos, Ravi Kanbur, Gordana Matkovic Social safety nets (SSNs) are an important component of effective de- velopment strategies in a world where chronic poverty and inequality persist and where global economic developments present opportuni- ties as well as risks. As part of an overall framework of poverty reduc- tion, protecting the poor and most vulnerable from the consequences of these new risks and from the normal risks they face, as well as tar- geting resources for poverty reduction through transfers, is a key role for governments in developing and transition economies. It should therefore be a central plank in the World Bank’s support to its mem- ber countries. This Independent Evaluation Group (IEG) assessment will be an important input to the ongoing discussions on developing the World Bank’s new strategy for social protection. We follow the Bank’s definition of SSNs as “noncontributory transfers targeted in some way to the poor and vulnerable…[to]… address chronic poverty and inequality, help the poor invest in developing human capital, and protect the poor and vulnerable from individual and systemic shocks, including during economic reforms.� Of course this is just one part of an overall system of public intervention in pur- suit of poverty reduction, but it is an important and growing compo- nent. The Evaluation provides very useful information on the extent to which the Bank has expanded its support to SSNs, and shows that the major expansion has come in middle-income countries (MICs) ra- ther than in low-income countries (LICs). The evaluation finds that the Bank’s SSN projects do better on average than Bank projects in other sectors; further, the gap is more marked in LICs. This is an encouraging basis on which to examine the details of Bank support to SSNs, with a view to improving its effectiveness. There are several dimensions along which improvements can be made. Perhaps the most important recommendation of the evaluation is that support for SSNs needs to be treated systemically and systematically, rather than as one-off interventions in response to crises or particular political exigencies. The emphasis needs to be placed on institutional building. This recommendation flows from the evaluation’s detailed assessments on the design and performance of SSNs supported by the Bank, and to some extent it encapsulates the other recommendations: to engage governments in dialogue on SSN in “normal times,� to de- velop results frameworks that incorporate short term objectives with xxxii STATEMENT OF THE EXTERNAL ADVISORY PANEL the goal of reaching longer-term development impacts, and to im- prove the internal coordination of the Bank’s support to SSNs. Although the actual modality of financial support for SSNs will be through individual operations, the evaluation makes clear that suc- cess of these operations depends on locating the support within an overall assessment of the needs left uncovered or inappropriately covered by the current collection of interventions, and an agreed strategy to move toward a system of social protection more broadly conceived. A comprehensive social protection assessment, done not under crisis conditions but in the framework of an ongoing dialogue with the government, can reveal gaps in coverage, inefficiencies in the existing interventions, and the data and monitoring and evaluation requirements. It can also lay the foundations for a results framework for SSN operations, which the evaluation emphasizes have been in- sufficiently focused on the poor and vulnerable. Such an assessment, done jointly with the government, is essential in developing a SSN strategy for the country and for identifying further interventions, project or analytical, that the Bank can support, including institutional strengthening for analysis design and implementation of SSNs. With this background, we would like to highlight three issues on which Bank management should commit a response in light of the findings and recommendations of the evaluation. First, management should commit to raising the issue of such a sys- temic assessment with country governments and, where there is a demand, work with the government and local analysts to produce an operationally oriented comprehensive social protection assessment as part of the overall dialogue on growth and poverty reduction. As the findings of the report indicate, it is important that the assessments include close attention to the political economy and institutional envi- ronment within which SSNs work or are expected to work, in addi- tion to more technical design and implementation issues. Second, in the light of the findings in this report, it makes sense for the Bank to reflect further on the factors preventing a stronger en- gagement with SSNs in LICs and to explore ways in which a more balanced approach could be achieved. There is already sustained di- alogue on social protection in some MICs, and this can be built on. But it is particularly important to extend this to other countries, and it is particularly important in LICs where, as the analysis in the evalua- tion shows, the lack of experience with SSNs, for the Bank and the government, may become a self-fulfilling prophesy. The issue may be left underexplored and underinvestigated, despite the manifest needs of the poor. xxxiii STATEMENT OF THE EXTERNAL ADVISORY PANEL Third, Bank management should commit itself to improving coordi- nation across various units that engage in SSN activities. As the eval- uation makes clear, SSNs are a challenge within the Bank, as they fall under the general responsibility of the Social Protection Sector, yet less than half of projects containing SSN support fall under that Sector board. SSNs involve all networks and 13 sector boards…. Analytical work is often undertaken by regional Poverty Reduction and Economic Manage- ment (PREM) Network Poverty units, investment projects and tech- nical knowledge of SSNs are housed in the regional Social Protection department (which also produce considerable analytic and advisory activities), and multisectoral Development Policy Loans involving SSN components are supported by the regional economic policy unit within PREM. IEG research on the Bank’s organizational structure and its affect on SSN work indicates that although the matrix system enables expertise to be drawn from various areas, it also risks making coordination challenging. It creates weak incentives for collaboration, and ultimately affects the quality of the work in countries…. Yet ac- countability mechanisms are inconsistent with the multisectoral na- ture of the work involved, as there is no clear mechanism of coopera- tion, recognition, or quality control procedures for cross-sectoral situations. The evaluation itself does not have much in the way of concrete sug- gestions for improvement, although it does point to good practices like the Global Expert Team. Moreover, issues of matrix management are of course broader than for SSNs alone. However, Bank manage- ment should respond to the findings of the evaluation by setting out how it proposes to address the organizational challenges identified by the evaluation in the specific context of SSNs. To conclude, we welcome this evaluation. The Bank has responded vigorously to the needs for SSNs, especially during the recent crises. IEGs findings and recommendations can serve to underpin a streng- thening of the Bank’s support to SSNs, especially in taking a systemic and systematic approach to the problems, expanding coverage in LICs, and in improving internal Bank coordination in working on SSNs. xxxiv Evaluation Essentials � The definition of SSNs used throughout this evaluation is government programs involving Chapter 1 noncontributory transfers targeting in some way the poor Introduction and vulnerable. � The past decade has highlighted the need for SSNs in all countries. Social Safety Nets and the Functions They Serve � Bank engagement in SSNs has expanded rapidly since the Social safety nets (SSNs) are critical to reducing poverty because they food, fuel, and financial crises. support inclusive growth and provide resources to the most vulnera- ble in society. Although the World Bank traditionally viewed them as � The purpose of the evaluation is to assess how effective and mechanisms for redistributing income and improving the welfare of relevant the Bank has been in those unable to participate in productive activities, SSNs are now also helping countries develop seen as critical for the growth agenda. 1 SSNs. This evaluation is guided by the operational definition of SSNs most often used by the World Bank: a set of noncontributory transfers tar- 2 geted in some way to the poor and vulnerable. This definition is quite narrow, as it refers to only targeted programs and focuses only on the poor and near poor. SSNs are a subset of broader social protec- 3 tion programs supported by the Bank as well as broader poverty al- leviation programs. This definition corresponds to five “functions� 4 (or objectives) of SSNs:  Function 1: Reduce chronic poverty and inequality. Function 2: Encourage more and better human capital invest- ments among the poor to provide the opportunity to exit po- verty.  Function 3: Enable the poor to manage risk from individual shocks.  Function 4: Enable the poor to manage risk from systemic shocks.  Function 5: Protect the poor if necessary during broader eco- nomic reforms. SSN programs in the Bank’s portfolio support the following SSN in- struments: cash transfers (conditional and unconditional); in-kind transfers; education/health subsidies; energy, water, and housing subsidies; and public works programs. The operational definition of SSNs is intended to delineate a specific type of Bank support. In practice, however, it is often difficult to draw lines between different types of government policies and programs designed for social protection, poverty reduction, risk management, 1 CHAPTER 1 INTRODUCTION 5 or improving equity. For example, some countries have universal programs protecting the poor and non-poor, yet these would not be included in the Bank’s definition of SSNs because they are not tar- 6 geted to the poor despite serving as important components of a coun- tries’ poverty alleviation strategies. SSNs are but one of a variety of tools designed for poverty reduction, social risk management, and social protection. They can contribute to achieving each of these goals, but they do not constitute all the tools available to achieve any of them (Holtzman 2009). In addition to poverty objectives, SSNs also serve important political, social, and security purposes in some countries. In fragile states or countries that are in conflict or emerging from conflict, SSNs are often used to demonstrate state power and the ability to provide basic pro- grams to citizens and thus maintain some level of authority and popu- larity. In Latin America and the Caribbean, the political agenda has in- volved tackling the high level of inequality through SSNs. In the Middle East and Africa, and some other countries, such as Jamaica, SSNs have sometimes aimed to keep young people employed to mi- nimize political and security concerns. In Bosnia and Herzegovina, Nepal, and several fragile states in Africa, SSN transfers have also been targeted at ex-combatants to help reintegrate them into society. Although nonpoverty objectives can motivate the design and imple- mentation of SSNs, the Bank, with the overall mission of poverty re- duction, generally does not directly support such objectives. This evaluation, through its country case studies, acknowledges the relev- ance of addressing critical issues in a particular country context; how- ever, it primarily evaluates the Bank’s support to SSNs based on ob- jectives that focus on poverty and vulnerability. The recent food, Recent crises have underscored the urgency of developing SSNs in all fuel, and countries. The importance of SSNs was highlighted over the past dec- financial crises ade as a series of international economic crises—in East Asia, the Rus- have sian Federation, and South America—and the recent global crises underscored the sharply increased poverty. The fuel, food, and financial crises of 2007– need to develop 7 08, which affected a majority of Bank clients, provided the most dra- SSNs in all countries. matic justification for effective SSNs. The World Bank estimates that the three crises pushed approximately 64 million additional people into extreme poverty (below $1.25/day) in 2010, and 40 million additional 8 people went hungry in 2009 because of the crises. These crises re- versed previous gains in poverty reduction and intensified the hard- ship of those who were already trapped in poverty. Even before the economic downturn, however, many countries were not on track to achieve the Millennium Development Goals and were suffering socioeconomic consequences of increasing weather variabili- 2 CHAPTER 1 INTRODUCTION ty, natural disasters, and internal conflicts. In such situations, SSNs may be used to protect vulnerable people by providing liquidity, of- fering short-term employment, and discouraging negative coping me- chanisms. Countries generally spend 1–2 percent of gross domestic product 9 (GDP) on poverty-targeted SSNs. In Latin America and the Carib- bean, where reforms have focused on consolidating existing SSN pro- grams and implementing transfer programs to serve as the backbone of social assistance, total SSN spending is around 1.3 percent of GDP (Weigand and Grosh 2008). Individual SSN programs supported by the Bank are usually less than 0.5 percent of GDP, with the Bank’s contribution ranging from very small (9 percent of total program cost in Brazil’s Bolsa Familia conditional cash transfer [CCT] program) to 10 very substantial (100 percent in Macedonia’s CCT program ). Some SSN programs are ongoing, established programs integrated into state budgets, and others are more ad hoc, donor-driven projects. Some governments spend significantly more on programs to protect the poor (and others) using broad instruments, such as general subsi- dies, rather than narrowly targeted ones, and provide compensation to categorical groups considered vulnerable. This is especially com- mon in the Eastern Europe and Central Asia and the Middle East and North Africa Regions. In Bosnia and Herzegovina, for instance, veter- ans’ benefits represent more than 2 percent of GDP, and in the Repub- lic of Yemen, general subsidies cost close to 8 percent of GDP in 2009. Untargeted programs are not SSNs as defined by the Bank, yet they are treated as such by governments because they are among the in- struments intended to protect the poor and vulnerable. Table C.2 in appendix C shows spending levels for targeted and untargeted SSNs in selected countries. Bank Engagement in Social Safety Nets The Bank’s first strategy for social protection was produced in 2001 and included a detailed elaboration of the objectives of SSNs as well as the broader social protection agenda (World Bank 2001). The strat- egy focused on a “social risk management� framework that identified the key sources of risk faced by households and sought to develop a range of instruments, rather than free-standing projects, to address those risks. Although the Bank’s definition of SSNs is narrow, the social protec- tion strategy encourages a more fluid approach to addressing risk, with SSNs just one of several possible types of instruments to develop a social protection system serving the poor and non-poor. This strate- 3 CHAPTER 1 INTRODUCTION gy was intended to guide the Bank’s lending, analytical work, and dialogue in the sector throughout the past decade. A revised strategy Bank is now being prepared to assess past performance of the sector and engagement in indicate new or renewed directions for Bank attention. SSNs grew rapidly over the During the past decade, World Bank engagement in SSNs expanded decade, rapidly, particularly since the food, fuel, and financial crises. From particularly after FY00 to FY10, the Bank committed more than $11.5 billion to support the recent SSNs in 244 loans in 83 countries and spent more than $60 million on crises. analytic and advisory activities (AAA). More than half of the volume of the Bank’s lending and a third of its loans have occurred since FY09 as part of the Bank response to the ongoing crises and a major in- crease in government interest in Bank support. The use of Rapid So- cial Response trust funds and the International Development Associa- tion (IDA) crisis window has enabled engagement to spread to low- income countries (LICs), as reflected in the spike in projects and AAA, particularly in the Africa Region. Objectives, Scope, and Methodology of the Evaluation This evaluation The purpose of this evaluation by the Independent Evaluation Group assess how (IEG) is to assess how effective and relevant the Bank has been in relevant and helping countries develop SSNs to protect the poor and vulnerable. It effective the seeks to understand what drives both SSN performance and the Bank has been Bank’s effectiveness. It offers lessons for informing future support. in helping countries Although the Bank has been actively involved in SSNs for the past develop SSNs. 11 decade, IEG has never evaluated this work. But with the accelera- tion of demand for SSN support by the Bank, it is timely that IEG eva- luate this area of assistance to help inform future support. And with the most recent crises fading, it is important to look again at country and Bank responses to learn how well countries were positioned to protect their poor and vulnerable and what lessons there may be for future support. The evaluation examines the Bank’s support to SSNs during the past decade (FY00–FY10). Given the rapid growth and evolution of SSN work, every effort is made to examine trends within the decade. The It draws on evaluation covers the full portfolio of SSN lending operations ap- evidence from 12 proved since FY00 as well as AAA, global knowledge sharing, and the portfolio, the Bank’s engagement in SSN dialogue at the country level. country cases, and a variety of The evaluation draws evidence from several sources (described in other sources. more detail in appendix B):  A portfolio review of 244 loans (investment and policy based) supporting SSN development approved during FY00–10, trust 4 CHAPTER 1 INTRODUCTION funds, and AAA. For a description of the portfolio review con- struction, coding, and analysis, see appendix B, § I.  Thirty country case studies, 25 of which were randomly se- lected, in-depth desk studies, and 5 purposefully selected field studies. The purpose of having a large number of country case studies is to better understand the context of the Bank’s en- gagement and when it has been most effective. It enables as- sessment of the Bank’s relevance in countries—that is, whether the Bank has addressed the right issues given country econom- ic, political, institutional, and cultural circumstances. Random selection (stratified by region) was done to ensure that a wide range of SSN experiences was captured, from very active to less active Bank roles. Case studies involved extensive desk reviews as well as interviews with Bank staff, management, and clients. For a description of the methodology, including sources used for the case studies, structured and semi-structured questions, and how case studies were used for quantitative and qualitative analysis, see appendix B, § VI.  An electronic survey of staff working on all Bank client coun- tries to better understand Bank and country response to the food, fuel, and financial crises with regard to SSNs. Responses were received from staff working on almost half of Bank client countries (65). For a full report of the survey findings, see ap- pendix B, § VII.  An in-depth literature review of 137 impact evaluations on SSNs. This review examined evidence regarding what works and under what conditions, distribution of benefits, sustainabili- ty of benefits, and emerging lessons. Of the programs evaluated, 24 were Bank supported. Two new impact evaluations (in Co- lombia and Pakistan) examined longer-term impacts of CCTs. Impact evaluations have focused heavily on SSNs, and thus this evaluation has a great advantage of being able to draw from this large body of literature on development impact. For the full re- port on impact evaluations, see “Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets� (IEG 2011c); “Do Conditional Cash Transfers Lead to Medium-Term Impacts? Evidence from a Female School Stipend Program in Pakistan� (IEG 2011b); and “Assessing the Long-Term Effects of Condi- tional Cash Transfers on Human Capital: Evidence from Co- lombia� (IEG 2011a).  A series of background studies that draw on the literature, port- folio analysis, case studies, and additional interviews to examine specific topics: public works programming in the Bank’s SSN portfolio, social safety nets in low-income countries, develop- ment policy loans (DPLs) for SSNs and their distributional im- pacts on the poor, the political economy of SSN programs, how the Bank’s organizational structure affects assistance for SSNs, 5 CHAPTER 1 INTRODUCTION the impact of country decentralization on SSNs, the application of the Bank’s 2001 strategy for SSNs, SSN results framework analysis, and SSN global knowledge. Methodologies and key findings from each source are listed in appendix B.  Semistructured client feedback on the Bank’s contribution to their country SSNs. This was received through various sources, including five field-based case studies (including two Project Performance Assessment Reviews), tracer interviews with participants of past global or regional learning events (such as the Istanbul CCT conference and the Latin America and the Caribbean Region CCT Learning Circle), and inter- views conducted for two new impact evaluations. Organization of the Report The rest of the report is organized as follows. Chapter 2 examines the trends in World Bank support for SSNs. Lending is only one tool that the Bank has to assist countries with the development of SSNs: AAA (both economic and sector work (ESW) and nonlending technical assis- tance), trust funds, and global knowledge are also examined. Chapter 3 evaluates the effectiveness and sustainability of the Bank’s lending for SSNs. The SSN portfolio has grown during the past dec- ade and includes several different types of SSN programs (cash trans- fers, public works programs, and others) supported through various lending instruments (investment loans, emergency loans, DPLs) in both International Bank for Reconstruction and Development (IBRD) and International Development Association (IDA) countries. Under- standing the context of these disbursements, including objectives set for the operations, is important for learning what works best and for assessing the use of more than $11.5 billion. This chapter also ex- amines risks to fiscal, political, and institutional sustainability of Bank-supported projects and development impacts of SSN programs. Chapter 4 looks more broadly at the Bank’s engagement on SSNs in two ways. First, it looks beyond Bank lending to overall Bank assis- tance (lending, AAA, knowledge sharing, and dialogue) and ex- amines how relevant and effective the Bank has been in supporting countries as they build SSNs. Second, it assesses how well the Bank has helped countries develop SSN systems and institutions to address a range of poverty and vulnerability issues. Chapter 5 lays out the most important factors related to the nature of the Bank’s engagement in SSNs. It does this by examining political economy factors within the country and how the Bank took country factors into account when engaging on SSNs. The evaluation further 6 CHAPTER 1 INTRODUCTION examines the nature of the Bank’s engagement on SSNs by looking at the timing of Bank involvement; the awareness of institutional condi- tions within the countries; the flexibility and adaptability of Bank’s support within a context of sustained engagement; internal and exter- nal coordination of Bank support; and the link to the broader Bank di- alogue and country strategy, including poverty reduction and growth. Chapter 6 draws lessons and presents recommendations for future Bank support to countries in developing effective and relevant SSNs. 7 Evaluation Essentials � SSNs are a dynamic and growing area of Bank work. � The Bank’s assistance to SSNs Chapter 2 has been countercyclical and has increased sharply in response to the food, fuel, and Allocation of the World Bank’s financial crises. � SSN lending has been Support for SSNs dominated by a few large middle-income countries each year, although many others engage with the Bank through The World Bank supports countries in developing and implementing smaller loans or technical work. SSNs through a range of channels, including lending and AAA (ESW, nonlending technical assistance, global knowledge sharing, and re- � SSNs represent a larger share of lending and nonlending search and evaluation). All these tools provide opportunities for di- activity in middle-income alogue. They are usually used in combination to advance the SSN countries than in low-income agenda in a country. These channels interact with the country’s eco- countries. nomic, political, and institutional conditions to enable the Bank to � IDA trust funds have enabled have an impact. This chapter examines trends in the use of each of more engagement in lower- these channels over the past decade (FY00–FY10). income countries, especially in Africa. Overall, IEG found that SSNs are a growing and evolving area of Bank work. This is reflected in Bank engagement as measured by lending, projects, AAA (particularly nonlending technical assistance), research studies and evaluations, and the use of lending instruments. The Bank’s involvement in SSNs increased sharply in the past two years, as these constitute one of the main mechanisms through which the Bank responded to the food, fuel, and financial crises. Indeed, lending for SSNs jumped from three percent of total Bank lending throughout much of the decade (FY00–08) to six percent (FY09–10).The Bank played a positive countercyclical role in financing SSNs, but the volume of its lending over the decade has been dominated by a small number of middle-income countries (MICs). Trust funds and the IDA crisis window have been critical in countering the asymmetry in Bank lend- ing, enabling broader engagement in lower-income countries and an increase in the number of Bank-supported SSN programs, particularly Between FY00 in Africa. and FY10, the Bank loaned $11.5 billion to support SSNs. The SSN Lending Portfolio Between FY00 and FY10, the World Bank loaned $11.5 billion to sup- port SSNs through 244 loans to 83 countries (see list of projects in the portfolio in appendix B.I, table B.2). Most operations that support SSNs are multisectoral and involve several components, including those focused specifically on SSNs as well as those that are not. IEG 9 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS carefully reviewed project documents to determine the characteristics and performance of each specific program. SSN lending figures represent only the SSN component of the larger operation. (More de- tails of the filters used to extract this portfolio can be found in appen- dix B.I.) The portfolio includes investment loans and development policy lending (including, for example, DPLs, to IBRD countries and Poverty Reduction Support Credits, to IDA countries). COMMITMENTS Bank lending for Throughout the decade, Bank lending for SSNs increased during pe- SSNs has riods of crisis, with a particularly strong spike in 2009 and 2010 with increased 1 the food, fuel, and financial crises (figure 1). The Latin America and during periods the Caribbean Region dominates, with more than 60 percent of lending of crisis. (figure 3). In terms of number of projects, the regional distribution is spread mainly over Latin America and the Caribbean (31 percent), Africa (25 percent), and Europe and Central Asia (21 percent) (figures 2 and 3). Figure 1. Trends in World Bank Commitments and Project Approvals ($ millions), by Fiscal Year of Approval US$ million committed  to SSNs # of SSN projects USD 4,000  50 USD 3,500  45 40 USD 3,000  35 USD 2,500  30 USD 2,000  25 USD 1,500  20 15 USD 1,000  10 USD 500  5 USD � 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: IEG portfolio review. The Bank has actively supported SSNs in low-income countries 2 3 (LICs) and MICs, but its support has been larger for MICs. MICs ac- counted for 62 percent of total Bank projects supporting SSNs and 79 percent of total SSN lending over the period, whereas LICs represented 38 percent of total projects and just 21 percent of total lending. As loans to MICs have been substantially larger than those to 4 LICs, the share of lending has been higher than the share of projects in MICs. 10 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Figure 2. Projects Supporting SSNs by Figure 3. SSNs Commitments by Region Region ($ millions) Source: IEG portfolio review. Source: IEG portfolio review. Despite the greater emphasis on MICs, more LICs had Bank- supported SSNs than MICs. Sixty-eight percent of LICs had at least one operation supporting SSNs throughout the decade; 59 percent of MICs did. MICs that borrowed from the Bank for SSNs were more likely to have a series of Bank-supported operations (and AAA) over time supporting institutional development. In contrast, in LICs, projects were distributed more thinly over more countries, with greater emphasis on emergency response than ongoing institution building. Over the past decade, SSN lending has been dominated by one or two The larger large borrowers each year, and a small number of countries have portion of dominated the SSN lending portfolio over the decade. The top 10 bor- support has rowers for SSNs over the decade represent 70 percent of Bank SSN gone to MICs, lending (see figure 4). Yet these countries represent only 15 percent of though the top 5 borrowers poor people in Bank client countries. This concentration of lending account for a has been mainly to MICs with large Bank lending programs, with the relatively small exception of Ethiopia and Pakistan. (When the SSN lending is com- portion of global pared to overall Bank lending, the results are reversed: the top 10 bor- poverty. rowers represent 52 percent of Bank lending and 68 percent of the 6 poor). 11 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Figure 4. Top 10 SSN Borrowers A. 70 Percent of Borrowing, But only 15 Percent of Client Poverty Top Ten SSN Borrowers 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% % of total committed  to SSNs  % of total of poor people of Bank client countries  B. Mainly Middle-Income Countries 6% % of total of poor people in Bank client  5% INDONESIA 4% PAKISTAN countries  3% ETHIOPIA 2% BRAZIL PHILIPPINES 1% COLOMBIA MEXICO ARGENTINA PERU TURKEY 0% 0% 5% 10% 15% 20% % of total commited to SSNs  Sources: IEG portfolio review and World Development Indicators. Note: Poverty Data: FY00–07. Commitment data: FY00–10. Appendix D, table D.3, presents SSN SSNs have lending and poverty by countries. played a more important role in In addition to more Bank resources going to MICs, SSNs have played the Bank’s a more important role in the Bank’s engagement in MICs than in LICs. engagement in Over the decade, SSNs represent 2 percent of total LIC commitments MICs than in and 6 percent of LIC operations, compared with 5 percent of MIC LICs. commitments and 13 percent of MIC operations (figure 5). 12 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Figure 5. SSN Operations and Lending as a Percentage of Total World Bank Operations and Lending, by Country Income Level 30% 25% 20% 15% 13% 10% 6% 5% 5% 2% 0% SSN % of total LICs  SSN % of total  SSN % of total LICs  SSN % of total  operations MICs operations Lending MICs Lending Source: IEG portfolio review. There are several reasons why SSN lending for programs that by de- finition target the poor and vulnerable goes to clients with fewer poor people: MICs generally borrow more than LICs, given their higher borrowing and spending capacity. (See appendix C, table C.1 for dif- ferences between LICs and MICs.) The Bank’s lending ceilings for MICs are higher than for LICs, which have more constrained IDA al- locations. Also, SSN programs (both Bank supported and non-Bank supported) already exist in MICs to some extent, though they are of- ten inefficient, so countries borrow for institutional strengthening and scaling up. (Government institutional capacity building was an objec- tive in only 24 percent of the projects supporting SSNs in LICs, com- pared with 57 percent of the projects supporting SSNs in MICs.) SSN support to MICs is often through a series of loans over time, whereas in LICs, projects are thinly spread over more countries and are more limited in scope. Regression analysis indicates that volume of Bank SSN lending is correlated with the past volume of SSN lending, vo- lume of overall Bank lending, and countries that underwent an eco- 7 nomic crisis earlier in the decade. LICs, in contrast, have high levels of poverty, weak administrative The capacity of capacity, and many pro-poor programs competing for support. So LICs to borrow even if their overall focus on poverty reduction is greater than in for targeted MICs, their capacity to borrow for targeted SSNs may be lower. SSN SSNs is programs in LICs have focused more on immediate crisis response constrained. and piloting different SSN programs and less on institution building and scaling up or restructuring existing programs; these types of projects often require more resources over time than short-term or pilot efforts. Finally, in LICs, other donors often provide grant financ- 13 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS ing for SSNs (but not for other “hard investments�), and thus coun- tries may prefer to borrow, even at IDA rates, for other sectors. In addition to the differences in country demand by MICs and LICs, the Bank’s internal structure has served to accentuate, rather than mi- tigate this disparity. Greater ongoing lending to MICs provides addi- tional Bank resources (for SSN loan preparation and supervision) with which to engage in dialogue and further stimulate country de- mand for SSNs. So the existing lending patterns favor engagement with MICs. (As mentioned above, a significant predictor of Bank SSN lending is the volume of prior SSN lending.) In addition, whereas Bank-funded AAA could help strengthen coun- try capacity, it has also been directed heavily toward MICs, which re- ceive about three times as much of these resources as LICs (see Coun- tercyclicality of World Bank Support, below). Indeed, until recently (see Trust Funds, below), the Bank’s country presence in LICs on SSNs has been very modest, providing fewer opportunities to engage in dialogue, offer capacity building, and stimulate interest in SSN programs. This was particularly evident during the food, fuel, and financial crises, when the Bank was constrained in its ability to pro- vide support because of weak institutions, lack of data, and lack of SSN programs (see next paragraph). TRUST FUNDS 8 Over the past decade, recipient-executed trust funds provided $141 million for SSNs. After the food, fuel, and financial crises, activities supported by trust funds increased significantly, mainly in LICs. That increase is attributable to the Rapid Social Response (RSR) Program (figure 6). The RSR Program leveraged trust fund support for a total value of $61.3 million for SSNs, along with the IDA crisis window ($1.3 billion, to support additional efforts in LICs). Trust funds and In the latter part of the decade, RSR trust funds and the IDA crisis IDA crisis window have enabled more engagement in LICs and increased the funding have number of projects in Africa. Responses to the IEG food, fuel, and fi- enabled nancial crises staff survey indicate that 23 percent of LICs received increased SSN support from trust funds during the crises. (For a full report of the engagement in survey findings, see appendix B, § VII.) The RSR used three main LICs. 9 funding vehicles: the Catalytic Trust Fund ($2.8 million), the SSN 10 Central Contingency Fund ($ 3.92 million) and the Multidonor Trust 11 Fund ($58.5 million). These trust funds primarily supported Bank- executed AAA (ESW, country-level technical assistance, and capacity building) and direct grants for pilot activities and for scaling up pro- gram benefits in IDA-eligible countries, as well as knowledge man- agement activities in both IDA and IBRD countries. Around 76 per- cent of these funds were used in SSN activities; the rest were divided 14 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS between access to basic services for the poor and labor market inter- ventions. Figure 6. Trends in RETF Expenditures ($ millions) USD 70  25 USD 60  20 USD 50  15 USD 40  USD 30  10 USD 20  5 USD 10  USD � 0 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 USD millions Committed Number Source: IEG portfolio review. Note: RETF = Recipient-executed trust fund. Evaluating the use of these funds is not possible at this time, because of the short time since their allocation. Nevertheless, building capacity, data, and institutions for SSNs in LICs is a very positive development, given that these were all cited in responses to IEG’s survey as con- straints to the Bank during the recent food, fuel, and financial crises. (For a full report of IEG’s survey findings, see appendix B, § VII.) OPERATIONAL OBJECTIVES Improving government institutional capacity was the top objective for operations supporting SSNs. Although 45 percent of operations sup- porting SSNs involved institutional capacity building, clear differences exist between MICs and LICs. More than half of projects in MICs sup- ported institutional capacity building as a stated objective, but only a quarter of projects in LICs pursued this objective. Other objectives (in 12 decreasing order) included the following: access to public services, quality of service delivery, human development outcomes, temporary income support, improving efficiency of public expenditures, address- ing the effects of crisis, and improving access to economic opportuni- ties. Notable differences in objectives between LICs and MICs include fiscal efficiency (26 percent of projects supporting SSNs in MICs versus 13 percent in LICs) and community empowerment and nongovern- SSNs have ment capacity building (18 percent in LICs versus 9 percent in MICs). mainly been used to reduce Over the past decade, the Bank supported SSNs mainly as instru- chronic poverty and inequality. ments to reduce chronic poverty and inequality and improve human 15 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS capital investment of the poor, despite the emphasis of the 2001 social protection strategy on addressing risks (figure 7). Nevertheless, the Bank’s operational support (lending and nonlending) has contributed to the development of SSNs that address all five functions mentioned in chapter 1. (This is consistent with the Bank’s overall country SSN objectives assessed in chapter 4.) Figure 7. SSN Functions Served by Projects Supporting SSNs F1: Chronic Poverty and inequality  81% F2: Human Capital Investment 66% F4: Systemic Risk 53% F3: Individual Risk 19% F5: Macro reforms 10% 0% 20% 40% 60% 80% 100% Source: IEG portfolio review. SSN INSTRUMENTS 13 Support to SSNs was delivered through a range of SSN instruments. Often more than one instrument was supported through a single loan (thus the total of SSN instruments, 406, exceeds the number of loans supporting SSNs, 244). Countries’ use of SSN instruments differed somewhat depending on their income level and region (table 1). For example, the most common SSN instrument in LICs was in-kind transfers (55 percent of projects with SSN components, compared with 23 percent in MICs). Likewise, public works programs (PWPs) were supported in 45 percent of LIC projects focusing on SSNs but in only 23 percent in MICs. In contrast, 37 percent of MICs supported unconditional cash transfers (UCTs) and 27 percent supported CCTs, whereas 27 percent of LIC projects with SSN components supported UCTs and only 12 percent supported CCTs. The analysis by regions indicates some differences there as well. Latin America and the Caribbean was the leading region for CCTs, but the Africa region had a large share of in-kind transfers and Europe and Central Asia led in UCTs. The most common SSN instruments in East 16 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Asia and Pacific, South Asia, and the Middle East and North Africa were CCTs and in-kind transfers. Table 1. Share of Projects Supporting SSNs in LICs and MICs Containing Specific SSN Instruments Instrument participation Instrument participation SSN instruments in LICs (%) in MICs (%) CCTs 12 27 UCTs 27 37 In-kind transfers 55 23 Education/health subsidies 25 22 Energy, water, housing subsidies 18 21 PWPs 45 23 Source: IEG portfolio review, Note: CCT = conditional cash transfer; LIC = low-income country; MIC = middle-income country; PWP = public works program; UCT = unconditional cash transfer. SECTOR BOARDS AND LENDING INSTRUMENTS Although the Bank’s SSN work is housed in the Social Protection Sec- Fewer than half tor family in the Human Development Network and about 70 percent of SSN projects of commitments are attributable to it, fewer than half of projects sup- are implemented porting SSNs fall under that sector family. The second largest sector by the Social board for work involving SSNs is Poverty Reduction and Economic Protection Management (PREM) Economic Policy, which is responsible for 12 Sector Board. percent of commitments (15 percent of loans). There are many small- scale SSNs in the portfolio reporting to a wide variety of sector boards. The use of DPLs to support SSNs increased over the decade. About two-thirds of projects with SSN components were supported by in- vestment loans and one-third by DPLs; however, the operations sup- porting SSNs now rely much more on DPLs: more than half of pro- grams were supported by DPLs in FY10, largely driven by crisis response operations (figure 8). Almost 80 percent of the projects sup- porting SSNs in the Social Protection Board were investment loans (regular investment and emergency loans); the rest were DPLs. 17 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Figure 8. Trends in Projects Supporting SSNs, by Lending Instrument 50 45 40 35 24 30 25 DPL 11 13 20 9 2 IL 5 15 6 8 2 10 1 3 21 17 17 17 15 17 14 5 10 10 11 11 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: IEG portfolio review. SAFETY NETS IN FRAGILE SETTINGS The Bank actively supports SSNs in many fragile and post-conflict settings. Throughout the decade, the Bank supported 26 of the 33 14 countries considered fragile by the Bank’s definition. Bank sup- port sought to strengthen or create SSNs programs—using lending, AAA, or trust funds. In the past decade, support to fragile settings represented 13 percent of total projects with SSN components and 2 percent of total SSN lending. The Bank used various instruments in fragile settings. The most common were public works schemes, usually implemented through community-driven social funds (42 percent of projects supporting SSNs in fragile states) and in-kind transfers (42 percent of projects supporting SSNs in fragile states), followed by UCTs and education, health, water, and housing subsidies. PWPs and so- cial funds have been used in fragile states mainly to create short- term employment in response to urgent labor demand shortages; they have frequently been implemented in South Asia and Africa to respond to disasters and conflict and are related to political and social stabilization objectives. In-kind food programs provide a safety net of last resort. When markets are not functioning, food is not readily available locally, or inflation is very high, it may be pre- ferable for SSNs to provide food instead of cash. COUNTERCYCLICALITY OF WORLD BANK SUPPORT Certain groups of chronically or seasonally poor people need SSNs on a regular basis; however, the need for SSNs spikes during systemic shocks, when a large share of the population is negatively affected by an event such as a natural disaster or economic crisis and traditional systems of 18 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS coping fall apart. During times of economic contraction, when demand decreases and underemployment and unemployment rise, SSNs are needed to protect those thrown further into poverty as well as the tran- sient poor (those newly affected). Automatic stabilizers in industrialized countries include SSNs that ensure extra protection for growing numbers of poor and vulnerable during economic decline, but in developing countries, social services—particularly SSNs—are often among the first items to be cut (Ravallion 2002). The Bank can help protect SSNs by in- cluding them in the crisis support it provides. The Bank’s support for SSNs has been countercyclical, particularly in The Bank’s SSN response to the food, fuel, and financial crises. That means lending and support during AAA in many countries was significantly increased. Following the crises, the food, fuel, 15 between FY08 and FY10, 42 percent of Bank borrowers received lend- and financial ing assistance for SSNs. More broadly, for the rest of the decade, there is crises has been also indication that the Bank seems to be responsive to shocks with its countercyclical. SSN lending. However, this support has been countercyclical only for some countries. IEG compared the top 10 SSN borrowers and their borrowing trends throughout the decade to the change in their annual growth rate. Spikes in World Bank SSN aggregate lending were driven by certain large bor- rowers responding to economic contraction (with borrowing sometimes lagging a year behind the contraction). This is evident in Argentina, Ethiopia, Mexico, Turkey (figure 9) and Colombia (figure 10). For these countries, the Bank has been able to serve a countercyclical financing role. It has not played such a role in the other countries in the top 10. Figure 9. Safety Nets Commitments and GDP Growth in Turkey Turkey 180 12 160 10 140 8 6 120 4 $million 100 2 % 80 0 60 �2 40 �4 20 �6 0 �8 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Commitments  to SSN Annual GDP Growth Rate Source: IEG portfolio review and World Development Indicators. Note: GDP = gross domestic product. 19 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Colombia illustrates how the Bank initiated SSN support in response to an economic crisis at the start of the decade, continued to support SSN development during a period of stable growth (through both lending and AAA), and then increased lending for SSN dramatically in response to the economic shock of 2008 (figure 10). Figure 10. Safety Nets Commitments and GDP Growth in Colombia Colombia 350 10 300 8 6 250 4 $million 200 2 % 150 0 100 �2 50 �4 0 �6 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Commitments  to SSN Annual GDP Growth Rate Source: IEG portfolio review and World Development Indicators. Analytic and Advisory Activities ECONOMIC AND SECTOR WORK AND NONLENDING TECHNICAL ASSISTANCE The World Bank spent $60.5 million on SSN studies (ESW) and non- The Bank spent lending technical assistance over FY00–10. A total of 210 ESW and 87 $60.5 million on nonlending technical assistance activities were identified over the AAA for SSN decade. Nonlending technical assistance spiked in the last two years over the decade. (FY09–10) in the Latin America and the Caribbean, East Asia and Pa- cific, and Middle East and North Africa Regions. The Bank’s broader nonlending technical assistance portfolio also increased, but the in- crease is particularly pronounced for SSNs (figure 11). Trust funds Trust funds have enabled more AAA, especially in the past three years enabled more (FY08–10). The predominant source of funding for SSN ESW and non- AAA in the most lending technical assistance activities has been the Bank budget, except recent three- in the East Asia and Pacific Region, where the major source of funds was year period. trust funds. However, this trend changed during the crisis years, from an average of 84 percent of Bank budget and 16 percent trust fund precrises, 16 to 70 percent Bank budget, 30 percent trust fund postcrises. 20 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Figure 11. Trends in SSN ESW and Nonlending Technical Assistance Activities 50 45 40 35 30 25 20 15 10 5 0 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 ESW TA Source: IEG portfolio review. Note: In 2002, Operations Policy and Country Services changed the theme coding system and recoded lending operations that had a status of closed, active, and in the pipeline. However, it did not recode closed nonlending activities, which may explain why there are almost no AAA activities between FY00 and FY01. There is still a considerable gap between AAA provided to MICs and that provided to LICs, despite the focus of the RSR Program on the Afri- ca and South Asia Regions. Over FY00–10, MICs represented 55 percent 17 of the ESW activities, and LICs represented just 17 percent. Likewise, MICs represented 63 percent of the nonlending technical assistance and 18 LICs only 18 percent. SPREADING GLOBAL KNOWLEDGE ABOUT SSNS Also among the Bank’s analytical services are global knowledge prod- ucts. The SSN anchor has played an active role supporting knowledge sharing across the Bank’s client countries globally both indirectly via Bank regional staff and directly to clients through a number of channels. These include core training courses, international and regional south- south learning events, flagship publications and toolkits, and study tours and bilateral visits. Although conferences, courses, and study tours do not automatically lead to new or reformed SSN programs, for some Bank clients, exposure to global knowledge has directly influenced country developments (box 1). This global knowledge sharing also emerged as relevant, to varying degrees, for SSN development in many of the coun- try case studies. Global knowledge sharing, and AAA more generally, enable the Bank to remain engaged on SSNs in countries regardless of lending. 21 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS Box 1. International Conferences and the Impact on Country Policies As experience with CCTs grew throughout the decade, the Bank organized international conferences to share experiences. These included Mexico (2002), Brazil (2004), and Istanbul (2006). The 2006 CCT conference in Istanbul and subsequent Bank-organized study tours have had an impact on CCT program development in both Indonesia and the Philippines. Indonesia introduced a targeted UCT program in 2005 in an effort to replace regressive fuel subsidies. That was followed by a pilot CCT program started in 2007. Bank staff and Indonesian officials reported to IEG that their atten- dance at the Istanbul conference, as well as on a study tour to Mexico, helped them solidify the technical details of the CCT program. Bank staff reported that the information presented at the conference sparked the interest of Philippine officials in CCTs. A subsequent study tour to Co- lombia for high-level policy makers further developed government support and technical knowledge. This led to the introduction of a (Bank-supported) CCT program in the Philippines that covers more than a million households. Source: IEG client and Bank staff interviews. A notable example of regional information exchange is the Latin Ameri- can CCT Learning Circle. In late 2006, client countries with the most de- veloped CCT programs in the region (Brazil, Chile, Colombia, El Salva- dor, and Mexico) asked the World Bank to act as a regional facilitator of knowledge, learning, and innovation for CCT programs. In response, the Latin America and the Caribbean Region created the CCT Learning Community, which client countries say has been a vehicle for innovation and just–in-time advice regarding CCT programs (box 2). Box 2. The Latin America and the Caribbean CCT Learning Circle All client country representatives participating in the CCT Learning Circle reported to IEG that the initiative has had a positive influence on decision making for their programs. The learning circle was very important for Mex- ico’s Oportunidades expansion into urban areas. Chile used it to brainstorm about the integration of Chile Solidario into the Chilean Social Protection Sys- tem. El Salvador used it to design the expansion of its program into urban areas (based on the Mexican experience). Finally, Brazil’s Bolsa Familia based its “family monitoring� strategy—a partnership among income transfer, so- cial assistance, health, and education—on the experience of Chile’s Solidario, which it learned about through the Learning Circle. Exposure to Source: IEG client country interviews. global knowledge has The World Bank has also helped disseminate CCT knowledge global- directly influenced SSN ly via study tours. Since 2001, the Bank has organized study tours of developments in Oportunidades in Mexico, Familias en Acción in Colombia, and Bolsa some countries. Familia in Brazil for more than 30 other countries. The study tours are tailored to participant needs and have covered all aspects of CCT 22 CHAPTER 2 ALLOCATION OF THE WORLD BANK’S SUPPORT FOR SSNS programs, such as the design of programs for achieving poverty and human capital objectives, the role of CCTs in social protection frame- works, and a wide range of technical design and implementation de- tails. IMPACT EVALUATIONS OF BANK-SUPPORTED SSNS Over the past decade, 92 impact evaluations have been done on 24 programs supported by the Bank, providing a large body of evidence not available for other sectors (see figure 12). Creating this large body of evidence to assess results from SSNs is an innovation and achieve- ment of the Bank’s work in this area. These evaluations are concen- Ninety-three impact trated in Latin America, especially Mexico, because of the large num- evaluations ber of impact evaluations on CCT schemes in the region. In particular, have been done 75 percent of the studies evaluated programs in Latin America, with on 25 Bank- the rest shared among East Asia and the Pacific (9 percent), South supported SSN Asia (8 percent), Africa (5 percent), and Eastern Europe and Central programs. Asia (3 percent). There is no completed impact evaluation of any program focusing of SSN in the Middle East and North Africa Region. Similar to the re- gional distribution, the majority of impact evaluations are on CCTs (69 percent). School feeding, UCT, and workfare schemes each only account for 7–9 percent (seven to eight evaluations). The rest are on education fee waivers, employment subsidies, and noncontributory pension programs (see appendix C.) Figure 12. Impact Evaluations of Bank-supported SSN Programs, by Region ECA, 3% AFR, 5% SAR, 8% EAP, 9% LCR, 75% Source: IEG portfolio review. 23 Evaluation Essentials Chapter 3 � Projects supporting SSNs perform better than Bank Effectiveness and Sustainability of averages. Short-term impacts have been demonstrated through a large number of SSN Programs impact evaluations. � Projects’ results frameworks are weak, often not poverty This chapter focuses on the effectiveness of Bank lending for SSNs focused, and aim at narrow, and the sustainability of the programs it has supported. It assesses short-term results. how effective the Bank has been in achieving the stated objectives of � Although results on short-term SSN programs it supports and what explains that performance. impacts are encouraging, there is little evidence on how SSN Because projects supporting SSNs are multisectoral and sometimes outcomes link to longer-term development impact. have diverse objectives (see chapter 2), only those projects containing an explicit SSN objective are included in the analysis of performance. Of the 92 closed operations in the portfolio, 71 had an explicit SSN objective and were used to assess effectiveness and project perfor- mance. Effectiveness of the SSN Portfolio On average, operations with an SSN objective perform better than the rest of the Bank’s portfolio. The World Bank’s objectives-based evalu- ation methodology rates the performance of closed projects on three criteria: relevance (of objectives and design), efficacy (the extent to which the objectives were achieved), and efficiency in achieving ob- jectives. Throughout the decade, IEG rated performance of projects with an 1 SSN objective significantly higher than other Bank projects. Eighty- six percent of operations supporting SSNs were rated moderately sa- tisfactory or higher, compared with 78 percent for all other Bank 2 projects. This higher performance is particularly striking for LICs, where 88 percent of projects with an SSN objective were rated mod- erately satisfactory or higher, compared with 74 percent of operations Bank-wide (figure 13). (See appendix B, § IV, for methodology.) It is important to note that this performance rating reflects the whole project performance rather than that of the specific SSN objective. 25 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Figure 13. Percentage of Projects with a Performance Rating of Moderately Satisfactory or Higher, by Country Income Level 100 90 80 88 85 86 81 78 70 74 60 50 40 30 20 10 0 LIC (Bank�wide N=343;  MIC (Bank�wide N=447;  Total (Bank�wide  N=790;  SSNs N=24) SSNs N=47) SSNs N=71) Bank�wide Safety net projects Source: IEG portfolio review. Note: LIC = low-income country; MIC = middle=income country. To better understand the characteristics that are associated with strong performance, operations supporting SSN objectives were ana- lyzed using regression analysis. IEG regressed the overall perfor- mance rating of projects with SSN objectives on project characteristics and country controls and found that strongly performing projects were generally of short duration (typical of most DPLs but also some investment loans) and had good-quality results frameworks (see ap- 3 pendix B, § V). Again, these are characteristics of the project as a whole and cannot necessarily be attributed to the SSN. Whereas the overall outcome rating indicates that operations with an SSN objective perform better than other operations at the Bank, to as- sess how different SSN instruments perform, it is helpful to examine the rating of the particular SSN objective. It is important to note that the overall performance (outcome) rating is based on a six-point scale (highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory), and the rat- ing of the achievement of the particular SSN objective(s) is based on a four-point scale (high, substantial, modest, and low/negligible). The two rating scales are not comparable. The extent to which the 71 projects achieved their specific SSN objec- tives varied according to the SSN instrument that was supported (ta- ble 2). Projects that support CCTs achieve their SSN objectives more consistently than projects that support other SSN instruments. 26 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Projects supporting energy, water, and housing subsidies are the low- est performing program type. (See appendix B for methodology.) Table 2. Projects with Substantial or High Efficacy Ratings of SSN Objectives, by SSN Program Type Projects supporting SSN instruments Substantial or higher (%) CCTs 77 Education/health subsidies 76 PWP 70 UCT 65 In-kind transfers 54 Energy, water, housing subsidies 40 Source: IEG portfolio review,. Note: An SSN instrument cannot be compared with another SSN instrument—for example, CCTs against UCTs—because SSN instruments in the portfolio are not mutually exclusive. Therefore, the table compares each instrument type individually against the rest of the SSNs and does not compare instruments against each other. CCT = conditional cash transfer; PWP = public works program; UCT = unconditional cash transfer. Aligning Choice of SSN Instruments, Design, and Context 4 When the Bank supports an SSN program, extensive IEG evidence indicates that at least as important as the choice of SSN instrument is 5 a clear sense of objectives, country context, and program design. However, these details are often not considered together. Moreover, adequate poverty data are critical for well-designed SSN programs, yet data in most countries are often inadequate to distinguish among the poor, which is particularly important when a large share of the population falls under the poverty line. More generally, Bank- supported SSN projects have objectives and monitoring indicators that often lack specificity with regard to target groups of poor and vulnerable (see Results Framework analysis section). To explore these issues in greater depth, IEG examined two popular SSN instruments in more detail: PWPs and CCTs; this is followed by an analysis of project-level results frameworks. PUBLIC WORKS PROGRAMS PWPs are defined as government programs that provide temporary employment at low wages, mainly to unskilled workers. The em- ployment is mainly on labor-intensive projects such as road construc- tion and maintenance, soil conservation, and waste disposal (Subba- rao and others 2010). A PWP may or may not be a program that supports SSNs, depending on whether it is designed to ensure that 27 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS poor people participate in the works and that the wages serve as a cash transfer to them. Of the Bank’s projects with public works com- ponents, 75 projects (for 52 PWP programs) were classified by the Bank as SSNs, so they are included in the portfolio IEG reviewed in this evaluation. PWPs have been the second most popular SSN in- strument in LICs and the second most used by all Bank clients to re- Public works spond to the food, fuel, and financial crises. The most common objec- programs are tive of the PWPs (90 percent of the projects) was short-term the second most popular type of employment creation. The second most common objective of PWPs in SSN. the SSN portfolio was infrastructure creation (in 38 percent of projects). Program design details matter in the performance of PWPs, yet project documents do not pay adequate attention to design issues. Grosh and others (2008) identifies four design aspects that are important for PWPs to serve as effective SSNs: defining how it is going to target benefits to the poor; setting the wage rate appropriately so that it provides ade- quate income support for participants while not discouraging other work or attracting non-poor to the worksite; ensuring timely receipt of the transfer, especially linked to food insecurity; and considering the relationship between cost implications for labor intensity and the need to provide adequate budget to ensure the quality of assets created. However, Grosh and others (2008) found that many project documents did not specify poverty targeting or wage setting criteria or provide da- ta for determining labor and asset creation trade-offs. IEG examined the employment generation objective of a random 6 sample of half the Bank’s SSN operations supporting PWPs to assess target groups and how well key performance indicators were de- signed to measure the employment generation for these groups. The sample of PWP projects included a mix of investment-, emergency-, development policy- and learning and innovation operations (all of which comprise the SSN portfolio of PWPs). The extent to which PWP design details were elaborated varied greatly throughout the project Although PWPs documents. have poverty- related Even though most projects indicated what kind of targeting method objectives and would be used to allocate the works, less than half of the project doc- use various targeting uments stated which specific groups of the population were being methods, few targeted. Of the projects that did mention a specific target group, mention which some of the groups were quite broad (for example, newly unem- groups are ployed families with children). In only 14 percent of sampled projects being targeted (in loans to Argentina, Colombia, and Honduras) did the project docu- and even fewer ments (Project Appraisal Documents and Implementation Completion track Reports) indicate that employment was targeted to poor people or vul- participation of nerable groups. Rather, indicators used for monitoring the perfor- these groups. mance of PWPs were focused on asset creation or number of em- 28 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS ployment opportunities or person-days generated. Key performance indicators (KPIs) did not require any verification of whether the poor or vulnerable were successfully targeted or what remuneration they were offered. They relied instead on crude data on the number of people who have participated in the program, an exclusively process- related indicator that offers no insight into whether the program has had the required SSN impact. The Bank has placed more emphasis on targeting methods than it has on clearly identifying a specific group of target beneficiaries and assur- ing that the benefits (that is, work opportunity) reach them. Poverty targeting methods such as geographic poverty maps may be a good way of reaching the poor, but this will vary between settings, depend- ing on such factors as distribution of poverty, broader labor market op- portunities and public accountability of officials involved in targeting.If PWPs are meant to serve as a SSN by providing work opportunities to poor people or vulnerable groups, they must identify those groups and ensure that the program reaches them. Assessing the characteristics of those employed through PWPs can be done, as appropriate for the country, through various means such as: administrative data, house- hold surveys, and/or rapid social assessments of a sample of commun- ities. Assuring the benefits reach the intended groups is particularly impor- tant in situations of rationing (that is, greater demand than availability of jobs). In these situations, program administrators or politicians may distribute the jobs to enhance their own position. Transparent mea- surement of who actually receives the jobs is a strong incentive for offi- cials to maintain strict targeting criteria. PWPs can serve many purposes, given the country context and pro- gram design. If they are to serve as SSNs rather than as an infrastruc- ture or general employment projects, issues such as target group, cov- erage, seasonality, wage rate, length of program, and beneficiary participation are all important. For PWP to address chronic poverty, for example, duration of employment needs to be sufficiently long or repeated opportunities for employment need to be assured. Yet, of the 29 projects reviewed, only three provided details in design documents on the duration of employment. Greater clarity is needed across the Bank about the objectives of PWPs, the form and design of PWP for each purpose, and how PWPs fit in the system of available social pro- tection programs. There is a gap between the quality of technical analysis (AAA) and project design. IEG’s assessment of AAA related to PWPs indicates that the Bank has a great deal of technical, nuanced knowledge of PWP design issues and SSN objectives (Subbarao and others 2010; Del 29 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Ninno and others 2009; Ravallion and Datt 1994; Ravallion, Datt, and Chaudhuri 1993; Jalan and Ravallion 1999). Despite this knowledge, lessons learned are not often applied in project design. Although po- litical economy issues within countries affect the governments’ choice on various design issues, interviews with government officials found that the weak link between knowledge and practice can also be ex- plained by the timing of different stages of Bank-supported project development. The Bank agrees on the SSN program and broad objectives before project approval, yet often many of the key design details are not de- termined until the operational manual is prepared sometimes only after negotiations with the Bank and after Board approval as a condi- 7 tion of effectiveness. At this point, the Bank’s intense focus and knowledge about design does not get translated into the project de- sign. Nevertheless, these are critical issues that determine if a PWP can serve an SSN objective. If it cannot, the country would be better served by considering another SSN program. CONDITIONAL CASH TRANSFERS Conditional CCTs achieve their SSN objectives more consistently than other SSN cash transfers instruments (table 2). CCTs have become increasingly popular with are the best 8 governments because they transfer money to poor households under performing SSN conditions usually related to investment in the human capital of their instrument. children. As in PWPs, the Bank’s experience indicates that it is the alignment of objective, design, and country circumstances rather than the type of program that drives success. Although The Bank’s experience with CCTs generally has been positive as re- experience with flected in the portfolio review and in many impact evaluations, but CCTs has been the objectives and impacts are short term. The objectives of CCTs are positive, their oriented toward outputs or intermediate outcomes. Improving school objectives and enrollment or attendance is positive, and in this regard the CCT is ac- demonstrated complishing its limited objective. But the goal of developing the hu- impacts are man capital of poor people is improved learning. Although this is a short term. challenging objective because of the multiple factors required to achieve long-term objectives, moving further down the results chain is important for ultimate CCT success (IEG 2011a). Objectives of CCTs tend to vary depending on country income levels, which has implications for program design. The IEG portfolio review found that some countries, mainly MICs (Brazil, Colombia, Jamaica, Mexico, and Turkey), use CCTs to reduce poverty and inequality. These programs have large coverage and emphasize both poverty re- duction and human capitals goals. 30 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Some of these countries have consolidated several smaller programs and rely on the CCT as the principal poverty alleviation program for the country. In contrast, other countries, mainly LICs, use CCTs prin- cipally to encourage human capital formation through promoting school enrollment, especially for girls or other specific groups. To achieve their human development objectives, these programs should place great emphasis on monitoring compliance with the conditions and ensuring quality of education or health services. These programs have been most common in South Asia (Pakistan and Bangladesh) and East Asia and Pacific (Lao People Democratic Republic [PDR]) but are increasingly also being piloted in Africa (Kenya and Nigeria). CCTs often face significant challenges to fully achieve their human 9 development and poverty alleviation objectives. According to the publication Achieving Social Protection for All in Latin America and the Caribbean (World Bank 2010), the Latin America and the Caribbean Region’s targeted income support systems (largely CCTs) need to be strengthened. Four major areas are identified to improve CCTs: coor- dinating between CCTs and the supply side in health and education and between CCTs and the social welfare system; refining procedures for enrolling and “graduating� beneficiaries in a timely fashion; adapting programs to urban settings; and strengthening the crisis re- sponse capacity of the programs. The remaining challenges for CCTs in Latin America and the Carib- bean highlight two important points: the remaining agenda for achieving longer-term development impacts and the need for caution in replicating CCTs in new countries. As CCTs are increasingly used in other contexts, where country circumstances differ more dramati- cally, aligning objectives and design features with the necessary coun- try conditions needs to be a high priority to enhance the chances of continued and deeper effectiveness. As country circumstances change, the objectives of a country’s SSN As country can change as well, but this should be followed by appropriate circumstances changes in design. An SSN program designed for one purpose may or change, SSN may not be amenable for use in other situations. This is a key point of objectives can a recent World Bank publication, “Conditional Cash Transfers: Re- also change, but the design may ducing Present and Future Poverty� (Fiszbein and Schady 2009). It also need to reviews a large body of empirical evidence regarding the performance change. of CCTs and identifies circumstances under which CCTs make sense and when additional SSN programs would be necessary to address other types of vulnerability. SSN INSTRUMENTS IN THEORY AND PRACTICE: THE REALITY OF SECOND-BEST OPTIONS Sometimes—and often during crisis—the Bank has taken a practical approach and supported existing programs that were less than ideal 31 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS for addressing transient poverty, including CCTs. As noted by Fisz- bein and Schady, having a CCT in place, though not ideal, is better than not having any large-scale social assistance program at all when there is a crisis and an immediate need to protect the poor. Many countries borrowed from the Bank in response to the food, fuel, and financial crises using instruments not designed to respond flexi- bly to shocks. Although this may have helped protect the poor during the crisis, there were drawbacks. Mexico, for example, increased the benefit level of its CCT program Oportunidades as a response to the food crisis. This helped protect the chronic poor, who are typically the most affected by these types of shocks, against rising food prices. However, what was meant to be a temporary increase in the benefit level has become a permanent increase. In addition, Colombia, El Salvador, Mexico, and Panama all have used their existing CCT program structures (all supported by the 10 Bank) as the basis for responding to crises between 2008 and 2010. When the food crisis hit the Republic of Yemen, the Bank had to react by using an existing second-best program—a labor intensive public works program—because the preferred program—the cash transfer program managed though the Social Welfare Fund—was not operat- ing well or effectively targeting those in need. Identifying when Identifying when a particular SSN is a second-best solution is diffi- an SSN is a cult. The Bank has handled this issue with varying degrees of success. second-best In Niger, the Bank’s emphasis over most the decade has been on agri- solution is cultural and rural development and not on SSN development. Thus, 11 difficult and has support to Niger in response to the food, fuel, and financial crises was been handled channeled to the irrigated rice fertilizer subsidy program, although with varying success. the poor neither consume rice as a staple food nor engage in irrigated rice production and therefore did not benefit. Also, social funds created in many countries throughout the decade may not have been the best choice for creating sustainable SSNs to assist the poor during stable times. Nevertheless, during times of crisis they were the only existing instrument able to channel additional funds to poor areas. SSNs established during stable times provide options for policy makers SSNs and the Bank during times of crisis. This was confirmed in the IEG staff established survey on SSN response to crises. (For a full report of the survey find- during stable times provide ings, see appendix B, § VII.) However, many countries found them- the Bank and selves without viable SSNs during the recent crises. The Bank provided policy makers support through second or third best instruments or nongovernmental with options organizations, but follow-up work was initiated to improve the institu- during times of tions to enable programs to better respond to crisis (as in Mexico, Mol- crisis. dova, Tajikistan, and Yemen). Indeed, 80 percent of survey respondents said that governments in their countries were planning to modify the 32 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS design of SSN programs to serve the country better during times of cri- sis, and 75 percent reported that governments were planning to streng- then institutions to enable more effective SSNs. The most common instruments to address the food, fuel, and financial crises were, in decreasing order of usage, UCTs, PWPs, in-kind trans- fers, and CCTs (see appendix D, figure D.5). Countries that had these instruments in place, as well as the ability to identify and deliver bene- fits to those in need, were better positioned to respond to the crisis than those lacking this capacity. Nevertheless, as countries develop SSNs to address shocks, they need to seek a mix of programs that will provide flexibility to adjust the selection of beneficiaries and expand the scale of coverage as necessary during crisis, and then contract the program af- ter the crisis. Indeed, experience has shown that contracting programs after a crisis can be challenging for SSNs, particularly CCT programs. Depending on the country context and the nature of the crisis, some instruments are more appropriate than others. For example, case study evidence (Indonesia and Sri Lanka) indicates that UCTs were the the most appropriate SSN instrument to use in a crisis situation. UCTs maybe the most appropriate instruments for three reasons: (1) UCTs can be expanded or contracted without the constraint of “eligi- bility� limitations (for example, health or education conditionalities, or the condition of unemployment as a qualifier); (2) UCTs are admi- nistratively simpler than other programs and can be implemented more quickly (PWPs have procurement issues, and CCTs require not only supervision of conditionality compliance but also that the neces- sary social services be in place); and (3) UCTs are often more political- ly acceptable during times of crisis, as people are aware of the need to protect the population affected by the crisis. That said, UCTs are not a panacea: in an inflationary crisis, cash may not be the most effective tool. Ultimately, other social protection in- terventions, such as unemployment insurance, may be more ade- quate, depending on the crisis transmission channels and the degree of informality in the country. Results Frameworks To assess the effectiveness of SSN operations and how well they are measuring the intended results, IEG examined the results frameworks of 12 projects supporting SSNs. For an SSN program to achieve its intended results, it should have a clear statement of objectives (project develop- ment outcome [PDO]) and intermediate outcomes to move toward the PDOs. Progress toward these objectives needs to be monitored with KPIs involving outputs, intermediate outcomes, and outcomes to ensure 13 progress toward the intended results. Given the short-term nature of 33 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Bank-supported projects, often the project supports the achievement of only one stage of a longer-term effort (or “results chain�) that involves other complementary inputs, with the goal of achieving development 14 outcomes. A clear project results framework, therefore, increases the chances that the stated objectives will be achieved, and a longer-term re- sults framework can enable realization of the longer-term SSN objectives. Analysis reveals areas of concern despite recent progress toward better results frameworks (see figures D.6 and D.7 in appendix D). Given that SSNs are by definition programs targeted to the poor, it is surprising that only 59 percent of SSN operations had objectives that targeted the poor and vulnerable and that 47 percent of projects containing SSN compo- nents did not even have one indicator to monitor progress on reaching the poor. The project development objectives were actually outputs ra- ther than outcomes in about a third of operations, and progress was measured more by achievement of outputs than of outcomes: only 46 percent of such projects contained outcome indicators. Further, given that the most common objectives of projects with SSN components are institutional development and service delivery, it is noteworthy that only 25 percent of these projects had intermediate indicators with which to measure progress toward these objectives. Project Projects supporting SSNs also lacked specificity in their monitoring monitoring frameworks. Only about half the operations included target values for frameworks the outcome indicators; 48 percent mentioned the target population; 32 lacked percent had time-bound indicators and 30 percent contained some base- specificity. line data before project effectiveness. This situation is improving gradu- ally (see figure 14). More than half (54 percent) of all operations ap- proved by the Board in FY09–10 included baseline data in the Project 15 Appraisal Document. Figure 14. Project Objectives and Performance Indicators Coded as Outcome Driven by Loan Type 80% 70% 73% 60% 64% 62% 50% 55% 50% 40% 30% 36% 20% 29% 10% 18% 0% IL ERL DPL PDO Outcome indicator Intermidiate outcome indicator Source: IEG portfolio review. 34 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Investment loans had stronger results frameworks than DPLs. In- Investment vestment loans had a higher proportion of operations with outcome- loans have driven development objectives (73 percent) than either emergency stronger recovery loans (64 percent) or development policy loans (62 percent). frameworks DPLs are vaguer than investment loans with regard to target popula- than DPLs. tion and target value for the objective. There is no significant differ- ence among the quality of results frameworks across sector boards. Development Impact Impact evaluations from 24 Bank-supported programs provide evi- dence on the short-term impacts of SSN interventions. When assess- ing effectiveness of the Bank’s support for SSNs, impact evaluations focus specifically and rigorously on the impact of a specific interven- tion and provide a useful complement to a broader evaluation of the Bank’s effectiveness. SSNs subject to impact evaluation consistently are found to protect Impact and improve households’ investments in human capital, immediate evaluations consumption, poverty status, productive investment strategies, and, consistently to some extent, abilities to mitigate the negative impacts of shocks. show that SSNs This has the potential to enhance their level of human and physical have important effects. capital and future earnings. Further, the findings from a few evalua- tions that examine medium-term impacts provide evidence of in- creased educational attainment and income growth. Table 3 summa- rizes the directions of impacts found in Bank-supported projects 16 covering SSNs. The table shows that the Bank’s support to SSNs has helped bring about positive impacts in a number of outcome areas. Nevertheless, these results, typical of impact evaluation results more generally, need to be handled carefully, because several other facts are essential for meaningful interpretation, such as size of impact, importance of the issue measured, how the issue fits in the results chain and what else needs to happen in the results chain, and the cost of obtaining this impact. This last issue is particularly important and remains a critical gap in the analysis of SSNs (as it does in many sectors). 35 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Table 3. Average Impacts of Bank-Supported SSN Programs Number of Positive No Negative Outcome areas Indicators projects impact impact impact Education School enrollment—primary 12 11 1 School enrollment—secondary 9 8 1 Dropout rates 5 5 Secondary/high school 33 graduation School attainment 4 4 Health and Health care usage 66 nutrition Immunizations 63 2 Infant height growth, HAZ, and 72 4 stunting Infant weight gain, WAZ, and 63 2 underweight Infant WHZ and wasting 5 1 4 Income, Income 4 3 1 consumption, Consumption 7 511 poverty, and Poverty (head-count ratio) 6 6 employment Labor participation 9 3 4 Coping with Systemic shocks 55 shocks Idiosyncratic risk 2 2 Source: IEG 2011c. Notes: Projects with mixed impact on the indicated outcome are not reported as positive, negative, or no impact. HAZ = height for age; WAZ = weight for age; WHZ = weight for height. LONG-TERM IMPACTS A key assumption in the results chain of several safety net interven- There is evidence that tions (for example, CCTs and fee waivers for use of education and the immediate health services) is that investing in the human capital of the poor will improvements help them break out of the intergenerational poverty traps prevalent in in welfare those societies. Yet little is known about whether the longer-term de- created by SSN velopment impacts can be achieved through these projects. Even programs may though safety nets are typically justified on the basis of their immediate be sustainable. and sustained impacts on poverty, most impact evaluations investigate their effects within one or two years after the intervention. The majority of studies have found that programs increase school participation or health care use, but only a few have shown that these additional inputs actually translate into increased human capital. Only a few studies examine the effects on long-term impacts, but there is some evidence that the immediate improvements in welfare created by SSN programs may be sustainable over time. A few evalu- ations of CCT programs indicate that they increase the rate of second- ary school completion among beneficiaries. Consistent with this, the available studies have found that children covered by a particular SSN accrue more years of education. 36 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS 17 But results on learning outcomes are mixed. Another area in the lite- rature with emerging longer-term evidence looks at the impacts over time on investment behavior and incomes. The evidence indicates that participation in some SSN programs is associated with more and bet- ter investments in productive capital. Moreover—and probably re- lated to this—there are also SSN programs that led to gains in income and consumption in both the short and medium term (IEG 2011c). In addition, the impact evaluation evidence has been able to record valuable indirect effects from some programs that can have long-term development impact. When CCTs in Pakistan succeed in keeping girls in school longer, which delays their marriage and reduces fertility, this is a positive long-term impact (see box 3). Box 3. Indirect Effects of SSNs By giving cash and in-kind transfers, which are often tied to conditions that require house- holds to comply with certain behaviors, SSNs help reduce budget constraints of beneficiaries and introduce changes in the relative prices they face. This is in line with the primary goals of SSNs that seek to reduce poverty, protect the poor and vulnerable, and induce changes in be- haviors to promote more and better investments in the human capital of children. However, in principle, the benefits and conditions of the programs may also lead to further changes with unintended effects. It is important for program design and operation to assess these effects, because they may undermine or contribute to the main objectives of SSNs. The existing evidence about the indirect effects comes largely from impact evaluations; al- though it is not large yet. Some suggestive patterns emerge in the literature in a variety of as- pects that range from remittances and other private transfers to marriage, sexual behavior, and fertility, to the reallocation of resources within the households and empowerment, to credit markets, and to spillover and general equilibrium effects:  With some exceptions, most large cash benefits, such as conditional and unconditional cash transfers, do not appear to crowd out private transfers.  CCTs and education fee waivers, by keeping adolescent girls in school longer, seem to en- courage them to adopt safer sexual practices and delay early marriage and childbirth.  There is little or no reallocation of calories within households whose children receive food transfers in a few school-feeding schemes.  There are examples of cash conditional programs with positive, negative, and no effects on the school enrollment and work of ineligible siblings within participant households.  Programs that increase women’s control over the resources of the family seem to help them gain more decision-making power within the household when the benefits are given to them.  CCTs provide additional liquidity to the communities with beneficiaries, leading to in- creased savings and credit available to nonparticipant households.  There are both positive and negative spillover effects of safety nets on nonparticipants in terms of education and employment. Source: IEG 2010c. 37 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS There is In an effort to help fill gaps on long-term impacts, IEG conducted two evidence that impact evaluations to examine whether the immediate positive impacts indirect effects of two Bank-supported CCT programs on school inputs remained over can have long- time. The first of these evaluations looked at the Punjab Female School term Stipend Program in Pakistan, a targeted CCT program that was imple- development mented in the context of a larger education sector reform and in re- impact. sponse to gender gaps in education, mostly in middle school (IEG 2011b). Results from a previous impact evaluation undertaken two years after the program started found significant increases in female enrollment (Chaudhuri and Parajuli 2008). IEG’s evaluation shows that after four years of implementation, the program continues to help ado- lescent girls complete middle school while having other positive indi- rect effects: eligible girls located in areas with the program reduce labor participation, delay marriage, and have fewer children. The second IEG impact evaluation, in Colombia (IEG 2011a), ex- amined the education outcomes of children from poor households who benefited from the CCT program Familias en Acción. A short-term evaluation had already found that the program leads to higher enrollment and more time devoted to studying (Attanasio and others 2006; Attanasio, Fitzsimons, and Gomez 2005). The new evaluation shows that the program also helps children, particularly girls and be- neficiaries in rural municipalities, to accumulate more years of educa- tion. This effect is seen through increases in high school completion. Taken together, the positive impacts of both programs on close-to- final outcomes in education have the potential to improve opportuni- ties for employment and earnings prospects of participants, though these outcomes still need to be monitored and confirmed. The results of longer-term impacts of SSNs are encouraging in these two cases. However, external validity of impact evaluations is weak because the design details and country context differ for each SSN pro- gram (as described throughout this chapter). Thus, monitoring and evaluation within countries needs to extend over many years to ensure that programs can be modified to achieve their longer-term objectives. Mexico is a notable example of a country continuing to monitor and evaluate performance of its SSN program Oportunidades over time to assess if increased schooling has led to higher incomes. Although the outcomes of WEIGHING THE COSTS AND TRADE-OFFS OF DIFFERENT POLICY OPTIONS SSN programs have been Although much effort has been spent on rigorously evaluating the im- evaluated, little pact of various SSN programs on specific outcomes, very little has been has been done done to examine the cost of achieving these outcomes. If a sizable im- to examine the pact is achieved at great cost, the program may not be desirable. cost of achieving those The goal for SSNs, as in all areas of public policy, is to have the biggest outcomes. impact possible for the least cost. For this reason, cost-effectiveness 38 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS analysis can be undertaken to assess the cost per impact of a particular policy and compare that result to other options. Like all areas of public policy, SSN investments will have direct and indirect impacts. Direct objectives for the project should be clear and results measured against them. A program should not be justified by its direct or indirect effect without assessing alternatives for achieving them. For example, if the major development impact of a CCT program is reducing fertility (by keeping girls in school longer), it is important to assess whether this is the most cost-effective way of achieving that objective. Doing this type of analysis will require precision in defining objectives and weighing alternatives for achieving the principal stated objective. 18 This precision is often missing in SSN programs. The major technical challenge in doing cost-effectiveness analysis is being able to measure the impacts. A secondary challenge is obtaining the cost information. Impact evaluations measure impact but rarely use the data to assess costs relative to benefits, let alone compare the cost-benefit results of alternative choices. IEG’s recent evaluation, Cost-Benefit Analysis of World Bank Projects (IEG 2010b), highlights the lack of cost-effectiveness analysis for social protection projects. Al- though the use of some form of cost-effectiveness analysis has been minimal with regard to SSNs (12 percent of projects supporting SSNs), cost-benefit analysis has been used in 23 percent of projects supporting SSNs throughout the decade. However, the analysis is usually applied to the infrastructure component of PWPs rather than to its SSN component (for example, enabling consumption smoothing 19 for the poor). The lack of cost-effectiveness analysis for projects subjected to impact evaluations is only the most glaring situation. The failure to do this type of analysis to assess various types of SSN or other pro-poor in- terventions is an important issue for all countries. Risks to Sustainable Outcomes SSN programs, in part because of their need to expand and contract, have a unique set of fiscal, political, and institutional risks. Of the 53 20 completed projects supporting SSNs rated by IEG, for risk to devel- opment outcome, one-quarter were considered to have high or signif- icant risks (compared with 39 percent for all other Bank projects) and 62 percent were considered to face moderate risks (compared with 45 percent for all other Bank projects). 39 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS FISCAL RISKS The importance of the Bank’s financial contribution to programs that support SSNs ranges considerably. Programs in MICs are often large- ly supported by government resources; programs in LICs are usually heavily donor dependent. The level of Bank support as a share of the program ranges widely (for example, 100 percent of Macedonia’s new 21 CCT program and 9 percent of the Bolsa Familia CCT program in Brazil). Because the programs and countries vary considerably, the risks to fiscal sustainability do as well. The most important distinction is between LICs and MICs, with the former generally not having the redistributive capacity to support SSNs without donor support (Ravallion 2009b). In LICs, the major risk in the short term for many countries, generally, would be reduction in donor support. Over the longer term, countries would need to plan for SSN programs within the overall envelope of their poverty reduc- tion expenditures and thus consider policy trade-offs of achieving ob- jectives. In MICs, the major fiscal risk would be a change in govern- ment priority and budget support for SSNs. Donor support Evidence from the country case studies highlights experiences of for SSN in LICs fragmented donor support for SSN in LICs, including that of the has been 22 Bank. LICs are generally dependent on donor funding for SSNs fragmented. (given their very limited capacity for redistribution), and it is widely and reasonably believed that donor support will continue to be an important source of financing for them for the medium term. Unless donor support is carefully coordinated by the government, a series of donor-supported programs is unlikely to form an effective SSN for the country. The Bank’s experience with donor coordination (from the 30 case stu- dies discussed further in chapter 4) appears much more favorable for preparation of strategic documents than for implementation of pro- grams. Ethiopia is a notable exception, as donor coordination has enabled the transition of annual ad hoc emergency relief appeals into a government-owned, predictably funded and organized SSN sup- ported by several donors. This is a missed opportunity for the Bank, as donor-dependent countries need a well-coordinated effort to har- ness the various sources of aid and advice and enhance the fiscal sus- tainability of the programs. The fiscal risk issues in MICs are generally related to government Fiscal risks in commitment to SSN programs within country budgets rather than MICs are their ability to fund them. The programs that the Bank supports in generally related MICs generally represent less than 0.5 percent of GDP (appendix C, to government table C.2). A program of this size can be continuously funded by a commitment. MIC. In the short term, governments and the Bank tend to focus on 40 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS expenditure priorities; however, over the medium term, it is possible for many countries (including LICs) to raise revenues by improving tax effort, thus providing the potential for additional resources to support SSNs. For many Bank MIC clients, SSN programs save rather than cost countries (compared to alternative policies) and thus improve their For many MIC fiscal sustainability. Removing broad food and fuel subsidies or re- clients, SSN forming poorly targeted social assistance and implementing targeted programs save programs represents significant fiscal savings. money compared to For example, fuel subsidies in Jordan have been a large fiscal drain alternative since 2003 (6 percent of GDP in 2006) but were fully removed in 2008. policies. To compensate the poor, the Bank’s technical assistance seeks to strengthen targeting of the poverty benefit. The overall SSN spending in Jordan (reduced at the end of the decade to about 1 percent of GDP) is considered adequate for protecting the poor and vulnerable (World Bank 2007). In a few countries, Bank-supported SSN programs have grown throughout the decade as they have assumed a larger role as the prin- cipal national SSN. There is potential concern about pressure on na- tional budgets and questions of fiscal sustainability, unless budget management is handled appropriately. For example, Familias en Ac- ción, Colombia’s CCT program, grew from 0.1 percent of GDP in 2005 to 0.27 percent in 2009 and around 0.5 percent in 2010, expanding coverage especially to urban areas. The costs for the Oportunidades program in Mexico grew from between 0.30 to 0.37 percent of GDP annually between 2004 and 2008, to an estimated 0.46 percent of GDP 23 in 2011, as it was scaling up from 5.2 to 5.8 million households. In Jamaica, the Program for Advancement through Health and Educa- tion expanded throughout the decade, increasing benefits and ex- panding beneficiaries by 47 percent as projected from the program initiation. Even if the government is increasingly assuming the costs of the program, with high levels of external debt and poor tax admin- istration, the fiscal sustainability of the program funding is dependent on continued external support to national budgets. POLITICAL RISKS Political risk is a relevant issue for SSNs, and the Bank needs to be particularly aware of the political context in which it is providing support. SSNs are relatively new in many countries and lack perma- nent budget sections or even home ministries (for example, health and education); thus, they are more susceptible to political influence. SSNs introduced by one politician or political party can become close- ly associated with that leader and jeopardize political support for the 41 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS program from other politicians or parties. SSNs in more than a quar- ter of the 30 case study countries were affected by political shifts or political groups (among them, Argentina, Brazil, Bosnia and Herze- govina, Georgia, Nepal, Pakistan, Sri Lanka, and Uruguay). Fiscal pressure One particular political issue that can create fiscal pressure involves can arise from the difficulty of scaling back a program that has been expanded, un- the difficulty of less the benefits or program are known to be temporary. In recogni- scaling back a tion of this challenge, Sri Lanka designed its tsunami response pro- program that gram to be temporary, given its experience with another SSN has been program, Samurdhi, program and the entitlement nature of SSNs in expanded, that country. In Jamaica (in response to Hurricane Ivan), and in Chile unless the and Ecuador, the increase of benefits was announced as temporary benefits or one-time payments. In Indonesia, to cushion the 2005–06 fuel subsidy program are removal, a temporary one-year cash transfer program was put in known to be place. temporary. If a crisis response program is put in place without plans to scale back benefits after the crisis, a country can face strong political pressure to maintain those benefits. Argentina faced such a difficult challenge with its emergency Heads of Households program throughout the decade. The Bank-supported program was initiated in response to the 2001–02 economic crisis. Although it was never intended as a genral SSN, the program grew quickly because of enormous need and ex- ceeded its planned cost. The country began a strong economic recov- ery in 2003, and the government closed beneficiary enrollment and froze nominal benefit levels to contain costs rather than terminate benefits to existing households. As a result, a well-targeted program (Galasso and Ravallion 2003) that effectively addressed transient po- verty during an emergency became a less effective general SSN. Be- tween 2005 and 2009, the Bank provided two additional loans to Ar- gentina to support its transition away from this program, which it 24 eventually achieved at the end of 2009. Anticipating the potential politicization of SSN reform in the initial 25 design would enhance the Bank’s technical advice. The IEG case studies indicated that programs sometimes start off as reasonably ef- ficient and effective but become less so over time. It is difficult to as- sess when the Bank should support flawed programs and when it should not, given the complexity of each country’s political situation, as well as longer-term objectives of developing social protection sys- tems. Nevertheless, the Bank must consider that SSNs are very politi- cal instruments (as they result in strong support or opposition for pol- iticians), and it should support countries to provide more efficient and effective SSNs. 42 CHAPTER 3 EFFECTIVENESS AND SUSTAINABILITY OF SSN PROGRAMS Given the difficulty of contracting benefits and the tendency to ex- pand successful programs, the Bank’s support to program expansion should be consistent with budget management strategies and should be monitored to ensure that the design can achieve stated objectives, as well as to avoid creating large, permanent entitlements. INSTITUTIONAL RISKS One institutional risk in many SSNs is finding an appropriate institu- tional home for the programs and a regular, predictable financing source. Projects with SSN components are sometimes introduced by a One institutional particular political leader and housed outside the normal bureaucratic risk for SSNs is structures. The program then enjoys advantages that enable it to be finding an more efficient than a regular government program; however, it some- appropriate times does so at the potential risk of political sustainability. home for the program and a Social funds are a good example of this. Institutionalizing programs predictable that support SSNs involves changing from a project to a program, source of funding. creating the appropriate legal framework, having a long-term pre- dictable budget and wide buy-in across different political parties so that changes in government do not jeopardize the existence of the programs. Finding the right institutional home for a new SSN program has been difficult in many contexts. In Tanzania, the Tanzania Social Action Fund remains partly outside government structures, though there are attempts to mainstream it through new legislation. In Colombia, Fami- lias en Acción is still under the president’s office and therefore suscept- ible to political changes. In Pakistan and Argentina, new governments have installed new SSN programs rather than reform existing pro- grams associated with previous governments. Building Building the credibility of a program by showing that it is accounta- program ble, transparent, and fair and that it has a concrete development im- credibility helps pact helps ensure the sustainability of SSN programs. This has been ensure evident in Brazil, where the media has played a critical role in creat- sustainability. ing incentives for good governance of the program and disseminating the encouraging results of Bolsa Familia (Lindert and Vincensin 2008). 43 Chapter 4 Evaluation Essentials � In the past decade, operations Beyond Project Support: The Bank’s supporting SSN focused most on reducing chronic poverty and Assistance to Countries to inequality and on encouraging human capital investments among the poor. Establish Sound SSNs � Since the food, fuel, and financial crises, the Bank’s support for developing SSNs to The Bank’s engagement and impact in a country may be significantly address systemic shocks has increased significantly and is different than indicated by the volume of its lending for SSNs or even promising for both LICs and by the success of those programs. Ultimately, what matters is not the MICs, given its relevance and success of Bank-supported projects, but how well the Bank has used demonstrated effectiveness. its lending, knowledge, and dialogue to help countries develop SSNs � Bank SSN support for to protect their poor and vulnerable, improve their welfare, and ad- addressing chronic poverty must dress the risks they face. be designed more selectively and used to reach particularly This chapter takes a broader approach to assessing the Bank’s effec- poor or vulnerable groups in tiveness in helping countries establish SSNs to serve their poor and society while weighing programs for general poverty reduction vulnerable. It examines the full package of World Bank SSN support carefully against other pro-poor and assesses, from a country perspective, whether the Bank has fo- investments. cused on the right set of issues, given each country’s needs, and if its 1 � The Bank is increasingly support has been designed appropriately to achieve SSN objectives. adopting a more systemic It also assesses the Bank’s effectiveness in achieving results in these approach in its SSN support, areas as well as in building the systems and institutions necessary to which should be encouraged. protect households from various sources of risk. Several sources were � Building sound, sustainable used to address this set of issues, most notably the 30 country case SSN systems requires furthering studies and the staff survey on the food, fuel, and financial crises and the Bank’s recent gains in SSN response. (For a full report of the survey findings, see appendix institution building in MICs, and B, § VII.) stepping up efforts in LICs. Focus and Effectiveness of Bank Support for Country SSNs Support for SSNs has Country case studies and the portfolio review suggest that the Bank’s focused mostly support for SSN has focused most frequently on chronic poverty al- on chronic leviation and inequality and less on developing SSNs to protect poverty against shocks. Not all five SSN functions are equally important in alleviation and every country or at all times. Therefore, IEG examined the relevance inequality and of the Bank’s SSN efforts within the context of each country’s SSN less on 2 needs. Although the Bank’s support in case study countries is rela- developing tively closely aligned with country needs to address chronic poverty, SSNs to protect inequality, and human capital investments, the gap between the need against shocks. for SSNs to help address shocks and the Bank’s support on these func- tions is much larger (figure 15). Since the recent crises, however, the 45 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS Bank’s focus has shifted significantly toward addressing systemic shock using SSNs (figure 16). Figure 15. Average Ratings of the Relevance and Bank Objectives of SSN Functions 5 = Very great 5 extent Relevance: This  function is a  4 relevant need for  SSNs in the  country 3 Bank Focus: The  Bank's support to  SSNs has an  objective to  2 address this  function in the  country 1 = Not 1 at all F1: Chronic  F2:Human  F3:  F4: Systemic  F5:Macro  poverty and  capital  Idiosyncratic  risk reform inequality investment risk Source: IEG case studies. Figure 16. Trends in SSN Functions 50 45 40 35 30 25 20 15 10 5 0 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 FY10 F1: Chronic Poverty and inequality  F2: Human Capital Investment F3: Individual Risk F4: Systemic Risk F5: Macro reforms Source: IEG portfolio review. 46 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS REDUCING CHRONIC POVERTY AND INEQUALITY To effectively address chronic poverty and reduce inequality, SSN programs need to clearly identify target groups and distinguish among specific groups of the poor. This distinction is rarely made in project objectives or monitoring indicators (chapter 3). The problem stems from weak poverty data and lack of specificity in project objec- 3 tives. Evidence from the country case studies indicates that monitor- ing and evaluation (M&E) systems do not routinely collect data on depth of poverty or key characteristics of target groups. When such information is collected, it is rarely used in the design or monitoring of Bank-supported projects with SSN components. Across the portfo- lio, only 42 percent of operations supporting SSNs have performance indicators that mention specific groups of the poor, and just 52 per- cent mention “the poor.� SSN programs are rarely weighed against other pro-poor investments SSN programs (such as basic health care and clean water) to ensure they are the most are rarely effective way to achieve poverty alleviation objectives. As indicated in weighed against chapter 3, very few Bank-supported projects undertake cost- other pro-poor effectiveness analysis, and impact evaluation findings are rarely used investments to ensure they are to compare the results to alternative policy choices. the most The Bank has made great strides in improving the evidence base for effective way of assessing results of different SSN interventions, but it has not focused achieving analytical work on assessing the best policy options for achieving spe- poverty-related objectives. cific poverty-related objectives. To do this, SSN interventions would have to be compared to other possible means to achieve the same objec- tives. This is particularly important in LICs, where poverty is wide- spread and SSN investments must compete with many other develop- ment investments. Identifying where SSNs can best serve a country’s poverty strategy can enable project design to reflect the specific need it seeks to fill. Relatively few countries have documented changes in chronic poverty due to SSNs. Yet there is evidence from impact evaluations that the Bank helped some countries develop SSNs that produced short-term impacts on income, consumption, and poverty and led to increased household spending on food, education, and health care. Existing evi- 4 dence, mainly from MICs, is from a large range of interventions, in- cluding CCTs, UCTs, workfare schemes, food aid, and pensions. Im- pacts are often directly linked to the value of the transfers, but they also 5 reflect some behavioral changes by the households (IEG 2011c). Only a few countries have monitored changes in the depth of poverty (Brazil, Ecuador, Ethiopia, Jamaica, and Mexico). The evidence sug- gests that changes in the depth of poverty are not always reflected in a reduction of the poverty headcount index, pointing to the impor- 47 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS tance of going beyond monitoring the poverty rate in assessing the impacts of SSNs on poverty. Although SSNs increase short-term household spending, there is still little evidence about their longer- term impacts on poverty. DEVELOPING HUMAN CAPITAL The Bank has The Bank has supported SSNs that develop the human capital of the supported SSNs poor more often in MICs than in LICs. This is generally appropriate, that develop the given supply constraints for health and education in LICs. Whether human capital of SSNs are relevant for investment in human capital—rather than an the poor more intervention in the education or health sector—depends on the coun- often in MICs try context, such as the quality of health and education services, as than in LICs. well as the household constraints to accessing them. The portfolio re- view found that 20 percent of projects supporting CCTs are in LICs. MICs are more likely to have the appropriate infrastructure and re- sources to invest in SSNs that develop the human capital of poor children. Given the tremendous needs of the health and education sectors in LICs, the Bank supported governments both through supply-side in- terventions in the health and education sectors and, to a lesser extent, through SSN programs, such as school feeding programs and other in-kind assistance. Efforts have been made to transplant the expe- rience with CCTs in MICs to LICs with six new programs introduced since FY09. But caution is warranted to ensure that the prerequisite supply-side conditions for CCT programs are adequate in LICs. Approaches that address both demand-side and supply-side factors of human capital investment (such as in Guatemala, Lao People’s Democratic Republic, and Tanzania) may be promising. In Guatemala, the approach com- bines a CCT with strengthening supply of services and markets. SSNs have Bank-supported SSNs have produced positive impacts on human cap- produced ital investment (intermediate outcomes), but evidence of impacts on positive impacts final outcomes is limited and mixed (appendix C, table C.3). There are on human positive and significant effects on intermediary school and health out- capital comes, such as enrollment, attendance, progression, health service investment, but 6 usage, and growth monitoring (IEG 2011c). Bank-supported SSNs evidence of impacts on final sought what might be realistic within the period of the project, that is, outcomes is to improve indicators such as school enrollment or attendance, limited and though usually not within a larger results chain seeking to improve mixed. final outcomes. Although there is some evidence that SSNs also have an effect on longer-term outcomes (such as years of schooling, age or marriage, and childbirth), results are thin for others (Fiszbein and Schady 48 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS 7 2009). Complementing SSNs with other supply-side interventions is commonly called for to improve results (chapter 3). The evidence does not point to any particular program type as most effective at promoting investment in the human capital of the poor. Impact evaluations, overwhelmingly on CCTs in the Latin America and the Caribbean Region, also show positive impact for programs such as school feeding, take-home rations, fee waivers, and UCTs on children’s access to healthcare and schooling (IEG 2011c); case studies; see table 3). Bank-supported school feeding programs in Bangladesh, Burkina Faso, and the Philippines have shown improved enrollment rates, drop-out rates, and anthropometric measures. PROTECTING AGAINST IDIOSYNCRATIC SHOCKS Case studies and the portfolio review indicate that Bank support for Bank support SSNs seldom explicitly targets protection of the poor against individ- for SSNs ual risks (for example, illness, injury or death of a family member, seldom fire, and so forth), but it has been assumed in many cases that this explicitly targets protection of the would occur as a result of other types of social protection interven- poor against tions. Depending on design details, programs that support SSNs and individual risks. other functions may also be able to help poor and vulnerable house- holds absorb idiosyncratic shocks. For example, a PWP that seeks to address chronic poverty (box 4), or a CCT with objectives of poverty reduction and investment in human capital, may also help with con- sumption smoothing and insurance for the household when a shock occurs. However, this would also depend on design details, such as the level and length of benefit (for example, sufficiently high or regu- lar benefits such that households can generate savings to help absorb shocks), flexibility to allow new beneficiaries to enter the program, and indicators to measure progress toward addressing shocks. The limited existing evidence shows that SSNs can help poor people respond to and recover from individual shocks. In the few cases where the effects of SSNs on absorbing individual shocks have been eva- luated, evidence indicates that SSNs, even if not explicitly designed with this objective in mind, can protect the poor from individual 8 shocks. Evaluations of three CCT programs in the Latin America and the Caribbean Region (Honduras, Mexico, and Nicaragua), for exam- ple, show that CCTs helped households keep children in school and prevented them from engaging in child labor when the families were faced with individual shocks. However, the majority of projects with SSN components have not iden- tified protection against idiosyncratic shock as an explicit objective, nor have they designed performance indicators to monitor progress. 49 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS Hence, it is unknown (but unlikely) whether widespread progress to- ward this objective has been achieved. Box 4. Unemployment Insurance for Rural Areas: The Mahatma Gandhi National Rural Employment Scheme in India The Mahatma Gandhi National Rural Employment Act guarantees all rural households 100 days of work on demand. The public works program pro- vides all rural households with general insurance. It is expected that house- holds will turn to the program in times of crisis but also serves as a mini- mum income floor. Anyone who is unemployed or faces a family crisis (rising household expenses due to a sudden illness of a family member or wedding expenses for a child, for example) can participate in the program. An early Bank assessment of its precursor program—the Maharashtra Em- ployment Guarantee Scheme (Datt and Ravallion 1992)—demonstrated that having the possibility to participate in the program enabled families to take more risks in their income-generating activities. This analysis, one of the first rigorous program evaluations undertaken by the Bank, helped provide the first evidence of the benefits of this approach to policy makers in India as well as globally. Source: IEG case studies. PROTECTING AGAINST SYSTEMIC RISKS The food, fuel, and financial crises highlighted the importance of being prepared against the adverse effects of shocks. Although the Bank’s di- alogue throughout the decade in many country case studiesrecom- mended working on crisis preparedness during normal times, only re- cently has it been able to engage clients on this agenda. IEG’s SSN staff survey, which focused on the SSN response to the recent crises, found that only 16 percent of Bank clients were well positioned to respond with SSNs (see appendix B, § VII) to the food, fuel, and financial crises (that is, “able to identify and address those hit by crisis�). Another 40 percent were “somewhat� prepared. In LICs in particular, the most common constraints were inadequacy of data and weak country institutions. But the food, fuel, and financial crises stimulated a large-scale response that led to many more countries developing SSNs appropriate for crisis response. For example, case stu- dies of countries in Europe and Central Asia found that those countries did not have programs to address systemic shocks throughout much of the decade. However, since the crises, AAA and lending has spiked in the Region as countries seek to develop fiscally sustainable SSN pro- grams that can be responsive to systemic shocks. Specifically, there have been efforts to strengthen targeting, identification, and payment sys- tems, which were lacking in both MICs and LICs. 50 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS The portfolio review reveals that Bank support for SSNs to address shocks rose sharply after the food, fuel, and financial crises, and did so proportionately more than the other areas of SSN focus (see figure 16). IEG’s SSN staff survey confirmed this finding, as 80 percent of countries now have plans to strengthen their SSNs to be better able to respond to crises. Increasing volatility in global markets and climate change indi- cates that the occurrence of shocks is likely to increase rather than dimi- nish. Preparing countries for the future, the SSN agenda needs to be The Bank’s more strongly oriented toward shock mitigation. ability to support countries during The Bank actively supported countries during and after the recent cris- and after the es, although its ability to do so varied depending largely on SSN “pre- recent crises paredness� in the country. According to results of the SSN staff survey, varied, in more than two-thirds of countries, the Bank supported scaling up of depending an existing SSN (whether it was one being supported by the Bank) or largely on the establishment of a new SSN program to cushion the impact of the country crises on the poorest (see appendix B). preparedness for SSNs. More broadly, the Bank was able to increase its engagement (lending or technical assistance) on SSNs after the crises in 86 percent of the res- ponding countries. However, in 25 percent of countries, governments responded to the crises by introducing a generalized price subsidy, re- gardless of whether they had targeted SSN instruments available. Gen- eral subsidies are costly and politically difficult to remove, as demon- strated by country case studies (for example, Indonesia, Iran, Yemen). Case studies show that countries that have SSNs flexible enough to re- Countries that spond to shocks (Argentina, Ethiopia, Indonesia, Jamaica, Pakistan, and have SSNs Sri Lanka) have faced repeated large natural disasters or economic flexible enough shocks in the past 10–20 years and have established programs or sys- to respond to tems enabling them to respond quickly. The evidence indicates that a shocks generally have very quick (and therefore relevant) response to a systemic shock is only faced repeated possible where there is strong capacity to establish new programs quick- shocks and ly (as in Indonesia), where knowledge and systems exist (as in Argenti- have na), or when programs—although not always perfectly suited to deal established with crisis—and systems are in place already and can be scaled up (as in programs or Ethiopia and Jamaica). In Pakistan, experience with shocks throughout systems that the decade has led to the development not only of programs that sup- enable a quick port SSNs but of systems, so that the country can respond better to fu- response. ture shocks (box 5). Although it is difficult to measure the impact of SSNs addressing sys- temic shocks (because of their unanticipated nature), there is some evi- dence, from impact evaluation (appendix C, table C.3), on the outcomes produced by SSNs that they protect the poor and vulnerable against systemic shocks. The two workfare programs in Argentina (Trabajar II 51 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS and Heads of Households) were found to help mitigate the effect of the economic crisis of the early 2000s on income, employment, and poverty (Galasso and Ravallion 2003; Galasso, Ravallion, and Salvia 2001). In Indonesia, a scholarship program appeared to mitigate the effects of the Asian crisis (late 1990s) on school enrollment, child labor, and consump- tion (Sparrow 2007). Box 5. Pakistan—Learning from Experience and Developing SSN Systems to Address Shocks In 2005, a massive earthquake hit two of the most remote and poorest areas of Pakistan. The Bank and other donors started a number of emergency relief operations, including several programs to deliver cash to affected needy households. These temporary safety nets proved very effective and showed that SSNs can be launched for short periods to address specific issues—they subsequently rolled back. This experience motivated the development of a national social protection strategy in 2007, which sets out a comprehensive approach to social protection for the country. In 2007–08, the international food and fuel crisis hit Pakistan hard. In October 2008, the government launched a targeted cash benefit program, the Benazir Income Support Program (BISP) as the country’s main SSN program. Through BISP, the Bank has supported the development of a new system for beneficiary identification. The process is under way, but not yet operational. It is expected that the targeting tools and databases (in particular the poverty score database linked to the national identification numbers) can be used by both the federal and provincial governments to deliver relief to populations affected by future disasters on a much larger scale and more quickly than was possible in 2005. With the disastrous floods of 2010, the government and provincial disaster management authorities were able to fall back on the experience of the earth- quake. The strategies adopted are relatively straightforward: the government quickly initiated the Citizen’s Damage Compensation program, which pro- vided more that 1.4 million flood affected families with a cash grant of about $230 each by using the national identification database to identify the benefi- ciaries and provide cash grants through debit cards provided by the private banking sector. The government has now asked development partners to help finance additional cash grants to affected households. As the BISP poverty scorecard is being rolled out (to be completed in 2011), it will provide a basis for directing relief and recovery support in future disaster situations. Although political support for SSNs has been strong in Pakistan, each political party has often felt the need to develop and launch its own program. Given BISP’s strong association with the ruling party, there is a danger that the pro- gram could be dismantled to give way to a new program, should the opposi- tion take over in the next elections. The Bank’s efforts to update systems and build institutions are therefore wise, as these can be used even if a new pro- gram comes into place. There is a need to garner broad political and stake- holder support for the underlying systems so that progress can be maintained when the political winds shift. Sources: IEG case studies. 52 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS The CCTs in Nicaragua (Red de Proteccion Social) protected the con- sumption of coffee-growing households when they were affected by a considerable fall in international coffee prices. In Ethiopia, an impact evaluation found that despite droughts and high food prices in the past few years, Productive Safety Net Program workfare beneficiaries were more likely to be food secure, borrow for productive purposes, use improved agricultural technologies, and operate non-farm busi- ness activities (Gilligan, Hoddinott, and Taffessee 2008). Moreover, the Productive Safety Net Program prevented beneficiary households from sliding deeper into poverty and selling household assets, thus protecting them from the worst effects of the shocks. CUSHIONING EFFECTS OF ECONOMIC REFORMS In contrast to other SSN functions, developing or strengthening SSNs to protect the poor during macroeconomic and structural reforms has not been relevant in all countries. About 10 percent of the Bank’s 244 projects supporting SSNs in the past decade were undertaken to help cushion the effects of reforms. These projects were mainly in MICs Developing or (Europe and Central Asia and Latin America and the Caribbean), with strengthening a few each in the East Asia and Pacific, Africa, the Middle East and SSNs to protect North Africa, and South Asia Regions. the poor during macroeconomic Case studies indicated that these broader economic and structural re- and structural forms are often associated with larger development transitions, such reforms has not as regime changes (such as independence of Europe and Central Asia been relevant in countries after the collapse of the former Soviet Union, or the transi- all countries. tion from monarchy to democracy in Nepal); economic transition (for example, from socialist toward market-oriented economies in Europe and Central Asia, the Middle East and North Africa, and Tanzania); changes in governance structures (such as “big-bang� decentraliza- tion in Indonesia); or economic policy change (such as removing sub- sidies in oil-producing countries, changing land policy in East Asia). In reality, several of these changes often occur jointly. IEG’s case studies indicate that the Bank has supported SSNs during reforms in certain countries such as Indonesia and Colombia and in some countries in Europe and Central Asia. In all cases, whereas the introduction of SSNs was likely motivated by political expediency, it was an important development for social policy programs that con- tinued to be refined over subsequent years. This experience would highlight the potential for marrying the introduction of economic re- forms and SSNs more often. In the Middle East and North Africa Region, although the Bank has been involved in dialogue on subsidy reform for many years, it was not until the onset of the food, fuel, and financial crises that there was 53 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS a stronger push for SSNs to help support the poor and vulnerable during subsidy reform. However, throughout the decade, this has been an extremely difficult agenda to advance, given the politics of The Bank reform. With the continuing rise in food prices and recent security assesses the concerns and volatility in the region, governments may be pressured distributional to reintroduce general subsidies or other widely targeted compensa- implications of tion, putting targeted SSNs further on hold. economic reform Although the Bank does not support a large number of SSNs to pro- programs it tect the poor during economic reforms, it assesses the distributional supports and implications of economic reform programs that it supports and re- recommends commends measures to protect the poor when appropriate. The 9 measures to Bank’s efforts to protect the poor during economic reforms are usual- protect the poor ly part of economic reform DPLs (often focused on subsidy removal) when rather than SSN operations. appropriate. A review of 46 economic reform DPLs (selected randomly from among sectors likely to have distributional implications for the poor) carried out for this evaluation found that the vast majority of Bank- supported DPLs were probably not having a significant negative im- pact on the poor. (See appendix B for methodology.) Fewer than half of the DPLs had measures that could negatively affect the poor, and of those, half did not get implemented, most often because of political sensitivities. When they did get implemented, two thirds involved follow-up ac- tions, although only one-third of those involved actually compensat- ing the poor (whereas the rest involved only studies). In short, there is much more emphasis on assessing the situation than on addressing it, when necessary. (This is consistent with the findings of IEG’s safe- guards evaluation, which showed much less emphasis on country compliance and follow-up than on anticipating the problem areas [IEG 2010c].) Even Bank evaluations of Poverty and Social Impact Analysis (PSIA; World Bank 2009a, 2009b; IEG 2010a) do not assess the implementation of mitigating measures. Greater follow-up re- garding implementation of recommended measures to protect the poor during Bank-supported macro reforms should be required. IEG’s assessment of a sample of 46 economic reform DPLs indicates that the Bank often conducts PSIAs or “PSIA-type� analyses to assess the distributional effects of reforms supported by DPLs. However, although the Bank has the potential to play a relevant role with regard to protecting the poor during economic reforms, the distributional analysis varies greatly in quality and style. Moreover, policy recom- mendations are often too general and challenging to operationalize. 54 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS These findings are consistent with the results of IEG’s PSIA evalua- tion, which showed that implementation of PSIAs had limitations and that it only had moderate effects on Bank operations and country pol- icies. Improving the operationalization of the analytic work is impor- tant and can lead to more relevant loan design, as evidenced in IEG’s evaluation of Bank ESW (IEG 2009c). The Bank’s Support for Country SSNs Systems and Institutions: The Building Blocks of SSNs An integral part of the Bank’s assistance for SSNs (through lending, ESW, technical assistance, and global knowledge sharing) involves helping build systems and institutions for delivering social assistance. Beneficiary identification and targeting systems, payment systems, M&E systems, and data systems (to name a few) serve as the building blocks for SSNs; they enable identification of and response to different types of household risk and vulnerability. These structures and institutions can be established to implement a particular program or to use across a range of programs. For example, the single-household registry developed for Brazil’s Bolsa Familia can help the government target beneficiaries in a range of programs. Or, once a country has developed an efficient cash payment system, it can be used for emergency disbursements after a natural disaster, for reg- ular social pension benefit disbursements, or for a new CCT program to encourage children’s attendance in school. Moreover, country SSN and social protection programs and institu- tions need to be well coordinated to serve the varying and evolving needs for protecting the poor and vulnerable in LICs and MICs. This section analyzes how relevant and effective the Bank’s support has been in establishing the core SSN building blocks as well as the devel- opment of a coordinated set of programs and policies to protect 10 against household risk. The focus on institution building has been stronger in MICs than in The focus on LICs. The portfolio review found that 45 percent of projects support- institution ing SSNs had government institutional capacity building as an objec- building has tive (57 percent in MICs and 24 percent in LICs). Only about one- been stronger in quarter of the country cases involved systems adequate for managing MICs than in SSN programs, another half of the countries had SSN systems consi- LICs. dered “somewhat adequate.� Not surprisingly, most of the adequate systems are found in MICs with long-term SSN programs. 55 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS The country case studies indicate that Bank staff and country stake- holders see institution building as the strongest aspect of Bank sup- port for SSNs. This is particularly true in MICs in Latin America and the Caribbean and in Europe and Central Asia, which place more em- phasis on it. The Bank relies on a range of instruments to do this, in- cluding lending, AAA, training and global knowledge sharing, and dialogue. Brazil is an example where the Bank has played a big role in building institutions through a range of inputs throughout the decade (see box 8 in chapter 5). The importance of institutional capacity for addressing different types of risk was highlighted during the recent food, fuel, and financial crises. According to IEG’s staff survey, three key factors limiting Bank support after crisis included: weak country data (33 percent), weak country institutions (33 percent), and inadequate systems or instru- ments (24 percent). Identifying different characteristics of the poor, including the chronic and transient poor remains a challenge in many countries. The difficul- ty of obtaining these data should not be underestimated. Nevertheless, the data are essential for effective design. Case studies indicate that to improve the effectiveness and relevance of the Bank’s SSN support for chronic poverty, better data are needed in many countries to be clear about which problem the SSN is trying to address—chronic, transient, or seasonal poverty. Where Bank-supported efforts have not distinguished between chronic and transient poverty and interventions have targeted “the poor� broadly, short-term or seasonal measures have been less relevant to address chronic poverty and have had limited impact in reducing po- verty. For example, until the Bank started conducting detailed analysis of safety net transfers in Tanzania, the country’s main SSN did not reach the most vulnerable, chronically poor groups, such as children and the elderly. Instead, the main safety net (the temporary public The Bank has works programs administered by the Tanzania Social Action Fund) supported the targeted able-bodied adults. improvement of poverty data in The Bank has supported the improvement of poverty data in two- 66 percent of thirds of the country cases studied. This has involved either improve- the countries ments in the diagnostic level (usually through household surveys) or studied. the operational level (that is, being able to identify the welfare and needs of applicants to a SSN program). The Bank has supported greater reliance on country systems for monitoring poverty and has emphasized cross sectoral capacity building in production and use of poverty data. The Bank participates in “PARIS21,� an international 56 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS partnership established to influence and facilitate statistical capacity 11 development and the better use of data. IEG’s staff survey on the food, fuel, and financial crises found that one-third of countries had inadequate data to inform crisis response, thus constraining the Bank’s support. Client interviews indicated that improving poverty data was perceived as a very important contribu- tion to country capacity to design effective and efficient SSNs, improve transparency in countries, and garner political support within countries for targeting. Bank AAA and research are more nuanced in their appreciation for different types of poverty and the implications for program design, but Bank-supported M&E systems tend to track levels of poverty (“headcount� measures) rather than assess whether they are reaching the chronic or transient poor through their programs. This problem has been highlighted by the Human Development Network (Grosh and others 2008). There are signs of improvement, as the Bank has fo- cused recent efforts on improving targeting and data collection not only in lending operations (for example, in Albania, Bosnia, Jordan, Moldova, and the Philippines) but also with analytical work and re- search. For example, the Development Economics department has re- cently developed software that can help countries analyze household welfare and thus help guide the collection of meaningful poverty da- ta, and this has been adapted specifically for social protection with ADePT SP. Obtaining quality data with which to measure SSNs is improving but remains a challenge in many countries. TARGETING Identifying and reaching intended beneficiary groups (“targeting�) is an important component of the institutional capacity needed to design and implement SSNs. Both the impact and cost-effectiveness of target- ing have been richly debated inside and outside the Bank. In each coun- try context, the poverty situation, political economy issues, and admin- istrative capacity will vary, affecting the most appropriate choice of reaching the poor—from narrow targeting to universal (untargeted) programs. (In the latter case, these are not SSNs, as defined by the Bank, but other poverty alleviation instruments.) The Bank was involved in the analysis and dialogue on targeting in 80 percent of the country case studies. Targeting is very politically sensi- tive, as it determines who will benefit from programs with SSN com- ponents. Success for the Bank, then, is not simply about providing the best technical advice on targeting; it includes finding a targeting me- chanism that is politically acceptable to the government and the elec- 57 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS torate, and that can be implemented as efficiently as possible within the institutional context of that country. In 75 percent of the country cases, interviews with Bank staff indi- cated that they were at least somewhat knowledgeable about the po- litical factors around targeting and tried to take them into account. In 56 percent of the country cases, the Bank encouraged as fine-tuned targeting as possible to maximize program efficiency. Governments will make targeting decisions on their own, or in response to domestic political pressures, and the Bank’s technical advice can help in that process if it takes into account the political dynamics involved (as it did in Argentina, Costa Rica, and Guatemala). The Bank has been aware of political sensitivities with targeting, yet it has also realized that its technical role can serve to de-politicize the issue. For example, when the Bank encourages moving toward tighter and more objective poverty-based targeting, it introduces transparen- cy that is publicly valued and useful for gaining support. This has generally been done by providing poverty and vulnerability assess- ments that clearly show which groups need to be targeted through SSN assistance, followed by technical work that helps the government assess the best way to identify these groups and individuals. In Pakistan, for example, the Bank was instrumental in developing an identification system to make beneficiary selection more transparent. Before Bank involvement, the BISP program stipulated that Members Establishing a of Parliament identify potential beneficiaries. When the Bank agreed sound system to provide technical assistance for the program, however, a new, ob- for delivering jective, and transparent targeting mechanism was agreed on. In Ja- cash benefits is maica, the introduction of the proxy-means-tested formula and Bene- critical to ficiary Identification System removed the possibility of special ensure the interest groups nominating people for benefits. These are instances effectiveness where the Bank used technical skills and in-depth knowledge of the and credibility of many SSN political and institutional environment to encourage reforms to im- programs. prove the efficiency and effectiveness of the SSN program while en- couraging public sector good governance more generally. PAYMENT SYSTEMS Establishing a sound system for delivering cash benefits to recipients is critical for ensuring the effectiveness and credibility of many SSN programs (such as UCTs, CCTs, and PWPs). The portfolio review re- veals that 10 percent of the SSN operations supported the develop- ment of payment systems. In this area, countries have been eager to learn from each others’ experiences, and the Bank has facilitated this south-south learning through various forums (including the Latin 58 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS America and the Caribbean CCT Learning Circle, Istanbul CCT confe- rence, and study tours to Mexico and Colombia). MONITORING AND EVALUATION Building and improving M&E systems has been an important aspect Building and of institution building, and systems have improved significantly improving M&E throughout the decade. The Bank has supported partner governments systems has in setting up M&E systems in half of the case study countries. At the been an start of the decade, Bank projects containing SSN components tended important to have M&E frameworks, but there was less emphasis on helping aspect of governments monitor and evaluate their wider SSN interventions and institution impacts across the sector. By the end of the decade, almost half of the building, and case study countries had adequate and measurable indicators for as- systems have sessing outcomes. improved. Despite recent improvements in M&E, the information generated dur- ing the decade has often not been used for program management de- cisions, such as design modification or whether to scale up programs. Only 22 percent of case study countries were found to use M&E fully for program decisions; another quarter of countries used their M&E systems to some extent. Support to SSNs within Broader Social Protection Systems Bank-supported SSN efforts have been examined for relevance and ef- fectiveness. However, unless these programs are part of a larger con- text of improving SSN systems, a country may have a reasonably well- functioning program but many needy households that still lack protec- tion. Because sound SSN systems require more than successful projects, the 2001 Social Protection Strategy (World Bank 2001) recommended a systemic approach to social protection based on a social risk manage- 12 ment framework, rather than reliance on individual instruments and The Bank’s social programs. The strategy advocated looking at how programs over- risk management lapped and interacted and how social protection could contribute to has been partially poverty reduction beyond passive income redistribution. implemented, and improved IEG assessed the application of the social risk management frame- throughout the work by examining full country programs (beyond SSNs) and their decade. 13 contexts for the 30 country case studies. Each country was assessed for three periods throughout the decade (2000–04, 2005–10, and 2000– 10); then summary statistics were compiled. IEG’s assessment indicates partial implementation of the social risk management framework, with low levels of implementation early in the decade but considerable improvement throughout the decade. Al- 59 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS though only 14 percent of the 30 Bank programs reviewed supported the social risk management approach between 2000 and 2004, this rose to 46 percent between 2005 and 2009 (figure 17). The in-depth reviews indicated that countries that have embraced a more systemic approach have engaged over a longer period and have relied on a series of loans (investment loans or DPLs), AAA, dialogue, and global knowledge sharing. These countries span MICs and LICs and all regions. In about half the In almost half of the country cases, the Bank has been supporting the countries implementation of certain programs, but there may be large, unad- studied, the dressed gaps in the social protection system (the second-rank category Bank has in figure 17). Over the decade, about 20 percent of country cases fell supported into two other categories that reflect low engagement in social protec- implementation tion. These countries (Algeria, Cameroon, Iran, and Malaysia) have had of certain inconsistent dialogue with the Bank during the decade because of programs, but broader political or governance issues affecting Bank relations. there may be large, Figure 17. Rankings of the Bank’s Application of the Social Risk Management unaddressed Framework, by Period gaps in the social 16 protection 50 % system. 14 46 % 12 Number of countries 36 % Rank 0 10 29% Rank 1 8 Rank 2 6 14 % Rank 3 4 11 % 7 % 7 % 2 0 Year 2000�04 Year 2005�09 Source: Case studies, social risk management analysis. Note: Benchmarking was determined (see methodology and results in appendix B) in consultation with new Country Policy and Institutional Assessment criteria for assessing country social protection systems as well as with World Bank experts working on these issues. Countries were placed in three categories for each time period. Rank 3: The Bank is implementing a strategy for social protection that is based on the social risk management framework and complements government efforts to introduce a comprehensive social protection system, including safety nets, social insurance mechanisms, and labor market policies. Rank 2: The Bank is supporting government efforts to implement social protection programs, but there is no comprehensive social risk management framework, and there are large gaps in or uneven development of the social protection system that are not being addressed. Rank 1: The Bank is offering advice on elements of social protection in the context of economic, financial, or sector AAA, but there is little or no follow-up through lending. Rank 0: There is no dialogue between the Bank and the government on social protection. 60 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS There is a discernable trend over the past five years as the Bank has moved away from simply supporting successful projects or individu- al programs with SSN components (a particular CCT or PWP) to look- ing more broadly at the SSN sector and SSN systems. Such an ap- proach has made it clear that many countries have a plethora of small programs, often having the same objectives but run by different min- istries, that are ineffective, drain resources, and would benefit from consolidation and strengthening. Equity and fairness considerations also drive this agenda. Beneficia- Over the past ries in different parts of the country want to know that there is a stan- five years the dard way of selecting eligible households and determining levels of Bank has moved benefits. The Bank’s global knowledge products, such as conferences toward looking and study tours, have also helped expose countries to various ap- more broadly at proaches and options to address SSN needs. the SSN sector and SSN With this new systemic perspective (rather than a project orientation), systems. the Bank has started examining the streamlining and integration of SSN programs much earlier in the dialogue. In some cases, where the government has relied on a single program, the discussion focuses on the development of other complementary programs to address a range of household vulnerabilities. For example, Ethiopia has focused narrowly (and effectively) on food insecurity in rural areas; it is now engaged in the preparation of a broader social protection strategy that will address other sources of vulnerability as well. The Bank is en- couraging countries to develop national social protection strategies, which will force governments to move away from a focus on individ- ual programs to set out broader policy objectives in a more strategic and coordinated manner through a national social protection policy. Some countries are also developing the capacity to assess their social protection systems to see how they respond to different types of shocks and which groups may or may not be covered. For example, a recent project in Latvia, through Bank simulation work, assessed the likely impact on the poor of potential SSN programs to be imple- mented after the food, fuel, and financial crises. Simulations showed that scaling up coverage of unemployment insurance would lower the poverty headcount but not reach those deeper in poverty, and the scaling up and increased benefit levels of the UCT and the new public works program would reduce the poverty gap but not the headcount. Thus, the government was able to know which households would be protected by which programs and plan accordingly. This type of technical analysis is a very promising use of Bank technical resources. For some upper-middle-income countries, the Bank is supporting a broad social protection approach. The challenge of this lies in harmo- 61 CHAPTER 4 BEYOND PROJECT SUPPORT: RELEVANCE AND EFFECTIVENESS OF OVERALL BANK ASSISTANCE TO HELP COUNTRIES ESTABLISH SOUND SSNS nizing the SSN with broader social protection policies, so that a strong SSN does not create incentives for households to remain in the infor- mal sector (Ribe, Robalino, and Walker 2010). Developing SSNs within a coherent, broader social protection system has emerged as an important issue in Colombia and Mexico, for ex- ample, where social protection systems contain a range of SSN pro- grams and benefits for the poor and a variety of programs for those in the formal sector. Yet in between lies a segment of society that is not covered by social protection. The rising concern is that this incomplete social protection system creates a dual system that is not equitable or efficient. Moreover, incentives for people to remain in the informal sector, to access benefits, and to not pay taxes and payroll contribu- tions remain a drag on growth rather than a driver (IEG forthcoming; Ribe, Robalino, and Walker 2010; Levy 2008). 62 Chapter 5 Evaluation Essentials � Political economy drivers are Understanding How the Bank Works crucial for SSN developments and have played important to Build SSNs roles in shaping SSNs. � The Bank has had sustained engagement in SSNs in some countries, especially MICs, enabling an integration of The Bank’s ability to have a positive impact on SSN developments SSNs in the country dialogue depends not only on the appropriate technical design of programs, as as well as a better understanding of political discussed in chapters 3 and 4, but also on the country context and economy realities. how the Bank works within that context. The country case studies re- veal the importance of political economy issues for SSNs, as well as � Sustained engagement has helped the Bank be more factors related to the Bank’s engagement in a country. This chapter 1 strategic in its efforts to provide explores those issues. the appropriate advice to the right people at opportune moments through a range of Political Economy Issues in the Design, Implementation, and channels, as well as to support longer-term institution building. Reform of Programs and Systems of SSNs � In many LICs, the Bank was SSNs are among the most politically sensitive areas of development unable to engage in a strategic manner until recently, and SSN policy. This is because SSNs involve redistributing resources toward support has not been grounded the poor. Views of this vary not only between countries but within in wider country economic countries. Therefore, understanding how politics affects the ability of dialogue such as on growth. a country to design and implement SSNs is important for the Bank’s effectiveness. This chapter examines how politics has affected SSN reform in countries and how the Bank has used knowledge of political economy to guide its work. Societal views and preferences on poverty and SSNs have shaped polit- Politically, SSNs ical discourse and the decisions governments have made. Views vary are highly by region, as does the nature of the social contract between the state sensitive, and and its citizens. For example, in countries in Europe and Central Asia, social views on citizens have much higher expectations of government benefits and poverty and SSNs have support (all five case studies in this Region are about streamlining ben- shaped political efits rather than setting up new SSN systems, and SSN benefits are seen discourse and as an entitlement). In contrast, in the Africa and Latin America and the government Caribbean Regions, the political discourse is mainly about avoiding decisions. “handouts� and instead focuses on the need for people to earn their benefits in some way (for example, by fulfilling social conditions of CCT programs). Political factors have also driven changes in SSN programs. SSNs were used for various political objectives in the countries examined. Among those objectives were making difficult economic reforms more accepta- ble politically, helping win support in elections, demonstrating gov- 63 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS ernment legitimacy to gain social peace, and addressing high inequality to meet the demands of certain underprivileged groups. New political leadership was responsible for SSN reform (positive or negative) in half of the case studies. In several cases, new political leaders wanted to distance themselves from previous parties’ success- ful programs and cancelled those SSN programs to establish new ones under the brand of their own political party (this occurred in Argenti- 2 na, Mexico, and Pakistan). In some countries, closely contested elec- tions have prevented SSN reform that was thought to be unpopular (especially when it involved removing beneficiaries from the pro- gram). For example, in Sri Lanka, no party has taken on reform of the Samurdhi cash transfer program, because it is seen as too costly polit- ically (box 6). New leadership has also sometimes meant a new vision for SSNs, as was the case in Brazil. Box 6. Difficult SSN Reforms in South Asia The Samurdhi cash transfer program is Sri Lanka’s largest SSN program. The program targeting system suffers from inclusion and exclusion errors. Although the program covered 40 percent of the population at one point, beneficiaries were not all from the poorest income quintile. There have been several attempts to reform program eligibility and exit criteria, to make it more objective and transparent. For example, the government has experi- mented with reforms such as using proxy means testing and community- based targeting. But reform is challenging because of various political econ- omy considerations and potential political costs of removing non-poor bene- ficiaries from the program. The Bank has attempted to help the government improve Samurdhi targeting throughout the decade, but without much trac- tion until 2009, when the government requested Bank technical assistance. India’s public food distribution system provides subsidized food across the country, but at great cost to the federal government. It has high levels of both targeting and leakage errors, causing little of the subsidized items to reach the benficiaries. However, it has been running for decades and is seen as an entitlement. The political economy of reforming the program is highly com- plex also due to difficulties with public grain procurement and the ongoing debate on food versus cash-based support for food security. All political par- ties have shifted away from reforming the system, because it might involve removing millions of people from the program and, consequently, result in a loss of political support. The Bank has recognized the difficulties of reform- ing this system and has offered technical support and evidence on the flawed targeting methodology to help inform the debate. Source: IEG case studies. New political space for SSN reforms has Evidence indicates that new political space for SSN reform has often often opened up opened up during crises. In 70 percent of countries, there was some during crises. forward movement to install/reform SSN programs after a crisis be- 64 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS cause of increased need and political appetite. When people expe- rienced negative repercussions from a lack of SSNs, strong support arose to establish SSN systems even among the non-poor, and gov- ernments were empowered to set up new systems. In Argentina, not until the economic crisis in 2001, when a quarter of the workforce was unemployed, did SSN initiatives finally become politically feasible. Traditionally there had been strong opposition to any programs that handed out cash to the poor. In fragile contexts, governments have found it important to establish or scale up SSNs to gain popular support and credibility, as well as to promote security during peace processes and political transition and adjustment. The Bank must tread a fine line within the state-building agenda: on one hand, state building is essential for the country’s prospects for economic development and poverty reduction; on the other hand, the Bank does not support particular political groups. The country cases indicate that the Bank is generally sensitive to this ten- sion and seeks to support fragile states with SSNs that incorporate pro-poor elements that can be developed further over time, albeit of- ten more loosely than in more stable countries. Decisions on targeting of SSN programs have been particularly sensi- Targeting tive to political influence. Whereas one government might be elected decisions have on a platform of improving efficiency and thus seek to tighten SSN been targeting, another government might have a more populist appeal particularly and move toward more universality of benefits. The evidence from sensitive to the case studies, however, shows a distinct move toward tighter tar- political geting of social programs, with only two countries (Nepal and Argen- influence. 3 tina) moving toward more universal benefits. It seems that transparent and tighter targeting is good politics in Targeting has many countries, as was found in Nigeria, for example, where SSN been credibility with the middle class required proper targeting to the particularly poor. Otherwise the program would be seen as a cash handout for sensitive to party supporters. Targeting (and retargeting) of SSN programs has political been used in various ways to gain political support for politicians. influence Country case studies found cases of politicians using well-targeted programs to expand access (for example, expanding from rural to ur- ban areas), particularly before elections in Colombia, India, Indonesia, and Pakistan. As discussed in chapter 4, whether narrowly targeted programs are the best way of reducing poverty in a particular situa- tion, the reality is that policy deliberation usually involves both con- siderations of efficiency (minimizing leakages) and political accepta- bility. 65 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS THE BANK’S USE OF POLITICAL ECONOMY KNOWLEDGE The case studies (including interviews with staff and other country stakeholders) indicate that the Bank is aware of political economy con- siderations at least to some extent in most countries. In about half of the cases, the Bank identified the key stakeholders in SSN reform, and in two-thirds of countries there was an SSN “policy champion� that the 4 Bank identified to help move SSN reform forward. The Bank used political economy knowledge most successfully in coun- Bank support tries where it has supported large, successful programs (Brazil, Colom- for SSNs bia, Ethiopia, and Indonesia). The common factors are willingness to responded provide AAA at a politically opportune moment, including core minis- quickly to political shifts in tries (for example, finance and planning) in SSN dialogue and early rec- 40 percent of ognition of what was politically feasible and what was not. In some cas- the countries es, this has meant supporting the expansion of SSN programs and the studied. weakening of targeting. This has involved case-by-case judgment based on the overall development of a country’s social protection system. Bank support for SSNs adapted quickly to political shifts in 40 percent of 5 the countries studied. According to the case studies and the SSN staff survey (also see appendix B, § VII), it is critical to have specialized social protection staff in a country who have both political economy know- ledge and the ability to move dialogue on social protection forward by having good working relationships with members of government and other stakeholders. In some countries, the Bank has been able to take advantage of political openings for SSN advancement when broader economic reforms are in- troduced. Evidence from the case studies suggests that when the SSN can be installed at the same time as economic reform, that is an excellent way to garner political support for both the reforms and the SSN agenda (for example, in Colombia, Indonesia, Mexico, Turkey, and Yemen). Where SSNs were introduced or scaled up to make reforms more politi- cally acceptable, the evidence shows that countries placed little emphasis on tight targeting of the poor. Broad coverage of benefits is often politi- cally important for the feasibility of reform implementation without jeo- pardizing security and stability. The Bank’s support in these circums- tances has sought to institute programs quickly with large coverage (as in Indonesia after 2005, Nepal, Tanzania, and Yemen). Yet once the SSNs were in place, there was potential to improve design and strengthening targeting (box 7). Given the reality that SSNs can be very attractive forms of garnering support for politicians, the Bank’s support for technical systems can insu- late programs from political patronage. For example, the Bank’s support often addresses issues such as the development of transparent systems of targeting, grievance mechanisms, and controlling fraud and error. In ad- 66 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS dition, innovations in payment systems, such as those that go directly through the banking systems are increasingly in use and help reduce oc- casions for politicians to associate themselves with SSN benefits. Box 7. Indonesia—SSNs from Supporting Macro Reforms to Permanent System The Asian Financial Crisis in the late 1990s caused a sudden and severe deterioration of Indonesia’s economy. Indonesia pursued a stabilization program with substantial external support, and within two years of the crisis’ onset, GDP growth was again positive. It returned to precrisis levels by 2003. Before the crisis, there were no formal SSNs. The government introduced a series of subsidies (food, fuel, and electricity) as a response, as well as new safety net programs known collectively as the Japing Pengaman Sosial. In 2005, because of the fluctuations in oil prices, an effort was undertaken to reduce the highly regressive fuel subsidies (which cost 5 percent of GDP). The government was able to secure political support for the economic reform in part by committing to a pro-poor agenda. The government used fiscal space freed up from the subsidy reduction to fund a UCT. The introduction of the UCT allowed the government to remove fuel subsidies without major public protests Between 2005 and 2007, the SSN focus (by both the government and the World Bank) shifted from programs to protect people during emergencies toward building SSN systems that were able to respond to a range of household risks. Although the food, fuel, and financial crises have not had a major impact on Indonesia, the government was well prepared to face the prospects of a significant economic downturn. Fiscal balance had been substantially improved, and over the decade the government invested in moving toward a permanent system of social protection. Sources: IEG case studies; Grosh and others 2008. Creating solid systems for SSN management can help insulate SSNs from politics, as these technical institutions can persist over time, even if politicians choose to recast (or rename) programs in their political favor. For example, Mexico’s Progresa CCT program evolved into Oportunidades after the 2000 elections, yet the design and institutions remained intact. In this regard, the Bank’s support for rigorous and credible impact eval- uations with transparent dissemination of results can help strengthen popular support for programs and further insulate SSN programs from the winds of political change. At the project level, the new Operational Risk Assessment Framework provides managers and project teams with a tool to potentially assess the risk to achievievement of PDOs. The analysis includes a comprenhensive analysis of stakeholders and country and institutional risks that has the 67 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS potential to to help inform the political economy knowledge on a project- by-project basis. Nature of Bank Engagement The nature of Bank engagement is a critical determinant of its ability to work with the right partners with the appropriate advice at the oppor- tune time. Because of the many political economy issues surrounding SSNs, the manner in which the Bank and other donors engage with countries is important. This section examines evidence related to the Bank engagement, such as whether it has maintained a sustained dialo- gue on SSNs, engaged strategically, linked SSNs with the growth agenda and overall country strategy, and maintained internal and external coor- dination. SUSTAINED BANK ENGAGEMENT In 57 percent of In 57 percent of the case study countries, the Bank’s SSN dialogue was case study 6 sustained throughout the decade. This was more evident in MICs (70 countries, the percent) than in LICs (30 percent). The reason for sustained engagement Bank’s SSN was demand for the Bank’s expertise and comparative advantage in dialogue was three areas of SSN development: helping consolidate and coordinate sustained SSN and social protection programs, making programs more cost- throughout the decade. effective and efficient, and providing technical guidance. There are several cases of sustained and integrated engagement throughout the decade, notably Brazil and Colombia. In those countries the Bank supported governments that were actively implementing, eva- luating, and refining SSN programs and institutions while expanding coverage as key instruments of human development and poverty reduc- tion (box 8). Programs in both Brazil and Colombia featured muliyear AAA. Brazil’s engagement featured a multiyear AAA that enabled the Bank to deliver many policy notes, technical papers, expert visits, and workshops to accompany the government along its path of developing Bolsa Familia. IEG’s case study of Brazil found that the Bank’s use of poli- cy-based lending, investment lending, AAA, global knowledge sharing, and dialogue was considered very effective support for the country’s Upper-middle- SSN agenda. income countries found Whether large loans were necessary to support these programs, given the Bank’s AAA the fungibility of budgetary resources, upper-middle-income countries highly relevant found the Bank’s analytical and technical services highly relevant and and sought to sought to borrow from the Bank to secure that support. The reality of the borrow from the Bank’s operations is that countries involved in some lending (regardless Bank to secure of volume) usually receive greater Bank resources to develop and super- its support. vise the projects and thus are offered a continuous level of technical sup- port. The amount each country borrowed was related more to its need 68 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS for general budgetary resources than to what was required to develop the sector. Box 8. Brazil—Sustained Engagement and Building SSN Institutions through a Range of Inputs Brazil, which established several CCTs at the beginning of the decade, has pur- sued a strong social agenda. When the Lula government came into office in 2002 on a platform of social welfare, several highly visible safety net initiatives were undertaken. These included programs to eradicate hunger, promote youth em- ployment, and unify numerous CCT-type programs under the flagship Bolsa Fa- milia. A Ministry of Social Development was created, and a national family regi- stry and program eligibility system, the Cadastro Unico, was put into place. The Bank was closely involved in consolidating four CCT programs into Bolsa Familia and in helping the government develop the program. Important features included timely, technically advanced, and politically savvy interventions at the critical early stages of the reform, directed at key stakeholders and including a strong in-country specialist staff presence. Support for the government’s social agenda was framed by a programmatic hu- man development reform loan supported by an integrated package of analytical and technical work through AAA. For the CCT the Bank partnered with the Bolsa Familia program in two results-based adaptable program loans focusing specifi- cally on that program, coupled with two pieces of AAA: Brazil Social Assistance Program and Brazil Evaluation AAA. The emphasis of the multiyear programs was gradually shifted from the “what� of policy making to the “how� of imple- mentation. The combination of approaches addressed key issues as they were identified, while also responding to government needs, including themes such as targeting mechanisms and tools; M&E; and more strategic, longer-term safety net devel- opment issues. It allowed for a long-term, ongoing partnership between the gov- ernment and the Bank and permitted deepening of the technical aspects of the program. Through the flexible and innovative results-based lending mechanisms, programmatic AAA, and the strong performance of staff, the package of Bank support helped establish and strengthen a major government program. Another key determinant of the sustained success of the program was the very strong government commitment and close alignment of the program objectives to the country’s and the Bank’s priorities of reducing extreme poverty and inequali- ty. The client sought, listened to, and internalized the Bank’s assistance, especially technical assistance, and the program’s visibility in the media ensured transpa- rency and credibility of the government’s and the Bank’s efforts. As a result, Bolsa Familia is a well-targeted program and has had a significant impact on reducing poverty and inequality and improving school attendance and demand for health services by the poor. Source: IEG case studies. Although the Latin America and the Caribbean countries exemplify si- multaneous and sequential use of lending and AAA, Moldova is a case of sustained engagement in a different form. Throughout the decade, the Bank provided technical assistance and ESW to Moldova to help build capacity for reforming the SSN system (inherited from the former Soviet 69 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS Union) toward providing effective protection to legitimately poor and at- risk households (box 9). Based on this capacity, when the circumstances changed after the food, fuel, and financial crises, the government was able to move ahead with reforms. The Bank supported this financially. Box 9. Moldova—Combining Early-Period AAA and Technical Assistance with a Multidonor Lending Program in Preparation for Government Reform Although Moldova is the poorest country in Europe, poverty levels dropped from greater than 63 percent in 2001 to close to 12 percent in 2007.a The Moldovan social protection system, inherited from the former Soviet Union, includes 15 types of so- cial assistance programs. The government’s main objective for its social assistance system was to improve the targeting system and de-institutionalize social care services. The Bank sustained its engagement in the process throughout the decade, mainly through AAA. Bank support included recommendations on reshaping safety nets to provide effective protection to legitimately poor and at-risk households. Recommendations also aimed at reforming social care services by emphasizing social integration and com- munity and family care. A series of sector pieces and technical papers built understanding and appreciation of targeted benefits and formed the basis for a number of reforms aimed at rationa- lizing and targeting the social assistance programs via two policy loans and a Bank- led multidonor program (Poverty Reduction Support Credit 1 and 2; Health Servic- es and Social Assistance Project) in FY07 and FY10. Overall, the Bank’s approach of carefully considering the government’s political concerns regarding the speed of reforms appears to have been effective. In practice, this entailed developing an understanding of important issues and building capaci- ty during the first half of the decade. This later enabled the successful implementa- tion of reforms with Bank financial assistance, when circumstances permitted. Source: IEG case studies. a. World Bank poverty headcount ratio at $2 a day (PPP) (% of population). In the latter part of the decade there were also cases of increasing Bank engagement in SSNs. The main reason was the heightened attention to SSNs because of the food, fuel, and financial crises, which led to greater country demand and greater Bank supply (with additional resources available through the RSR program, IDA crisis window, and other trust funds). Demand was also stimulated by high-profile natural disasters such as the Asian tsunami (2004) and the Pakistan earthquake (2005), and supply increased through the formation of new social protection units in the South Asia and East Asia and Pacific Regions in the middle of the decade with new sector managers, additional staff, and budget. The Bank’s re- gional knowledge events (co-organized with the World Bank Institute), such as in Africa, Europe and Central Asia, and South Asia, have also contributed to the increased country interest in developing and reform- ing SSN targeted to the poor. 70 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS Another reason the Bank was able to support SSNs more in the second part of the decade is that the popularity of community-driven develop- ment (CDD) operations declined. The case studies point to a strong pres- ence of CDD-type operations (including social funds) in the first half of the decade. These CDD operations seemed to address government and Bank desire for pro-poor operations but without developing linkages to any overall social protection or SSN framework and without specifically designing or monitoring programs to reach the very poor and vulnera- ble. The Bank should have been proactive and flexible in exploring earli- er whether the instruments and design chosen—for example, social funds with public works components—were effective as instruments for protecting the poor and vulnerable. In LICs, country studies indicate that the Bank should have found ways to engage in SSNs earlier, rather than waiting until the food, fuel, and LIC cases financial crises made SSNs a priority. For example, food insecurity is not indicate that the a new problem in Niger, and although the Bank has focused on agricul- Bank should tural development, it could have started to engage earlier in providing have found flexible SSNs, as it has done in Ethiopia. All programs in Niger are high- ways to engage in SSNs earlier, ly dependent on donor financing, but it was only with the onset of the rather than food, fuel, and financial crises that donors, including the Bank, saw pro- waiting until the grammatic SSN support as a means to address food insecurity. Also, in food, fuel, and Nepal, despite the government’s interest in strengthening safety nets for financial crises. the poor and vulnerable, the Bank did not significantly engage in SSNs until recently. In addition to cases of high levels of sustained engagement, there are cases of very low and decreasing engagement. (The selection of case study countries for this evaluation was done randomly to ensure that a variety of situations was reviewed, not only the high-profile, flagship programs.) In some MICs, the level of Bank engagement has been low or decreasing. This lack of engagement is mainly due to international poli- tics, which limits Bank-country cooperation, as well as lack of interest by the country. In Algeria and Iran, higher-level cooperation between the Bank and the country determined the Bank’s entry and exit points in the SSN sector and in the country. Today, the Bank does not engage with Iran and in Algeria does so through fee-based services. In Malaysia, the government has not been interested in working with the Bank during the past decade although the engagement is increasing as of recently. In Cameroon, SSNs were not viewed as relevant to the country’s poverty reduction efforts during the decade. Therefore, the Bank did not engage with the country on building SSNs. 71 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS STRATEGIC ENGAGEMENT AND LINKS TO BROADER COUNTRY DIALOGUE Strategic engagement involves situating the SSN dialogue in wider coun- try dialogue, having a vision of how to build an effective overall SSN system (for example, how various SSN programs complement each other in the country context), and using the right instruments at the right time with the right stakeholders. In almost two-thirds of countries studied, the Bank supported SSNs in the context of country economic dialogue, Country Assistance Strategies, or sector discussions. However, in less than one-third of the countries is there evidence of the Bank having an overall vision of how its support helps build an effective SSN system. The issues are different in MICs and LICs. The Bank’s engagement in case study MICs was initiated opportunisti- The Bank’s engagement in cally in most cases, often with project-based lending and dialogue. How- case study MICs ever, in a growing number of countries, there was a shift toward a fairly usually has strategic approach involving coordinated programs linking to a broader been initiated social protection strategy and economic policy. This happened mainly opportunistic- because many MICs had already developed programs before the start of ally, often the decade or during the first half of the decade and had used the second initiated with part of the decade to consolidate, expand, and otherwise coordinate the project lending programs. and dialogue. Additionally, MICs have “new� social protection needs and demands, arising from increased urbanization, which has weakened traditional, informal SSNs; increased integration with the global economy that has made them prone to covariate shocks; and rapid growth of senior popu- lations. These new needs lead to increased demand for a more significant government role with respect to social protection. Each region tends to have a different focus, depending on the SSN needs, country history, and the social contract between the government and the population. For example, in Latin America and the Caribbean, the Bank’s strategic focus has been on strengthening SSN systems (building blocks) and investing in the human capital of the poor to break intergenerational poverty traps and to reduce inequality. Hence, strengthening SSN sys- tems has become a political imperative in the Region. In Europe and Central Asia, with the transition away from the large state role, there has been less interest in supporting poverty-focused SSNs. The Bank provided countries in the Region with AAA for several years until there was greater political opportunity to support a more efficient poverty-focused SSN system or programs. The dialogue has centered on both improving the efficiency of existing SSN programs and systems and strengthening institutions to provide SSNs for European Union accession. 72 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS In the Middle East and North Africa, country preferences and high-level political decisions have limited the strategic nature of Bank dialogue on SSNs. But the Bank has to some extent helped strengthen SSN systems in the context of subsidy (food and fuel) removal. In East Asia and Pacific (both for MICs and LICs), because of popular attitudes of self-reliance, government places low priority on SSNs; thus, the Bank has not put social protection at the top of country dialogue. De- spite strong support of pro-poor growth strategies, large-scale systemic shocks (Asian crisis and the recent crises) have made countries realize the benefits of SSNs for protecting people during situations beyond their control. In many LICs, case studies indicated that issues related to SSNs were In many LICs raised within the context of Poverty Reducation Strategy Papers and in- the Bank has cluded in poverty assessments, yet did not have much traction for a only recently number of reasons. With chronic poverty widespread and budgets tight, been able to be it was not evident to governments or sometimes to Bank country direc- strategic, and SSN support tors what role SSNs play in a poverty reduction and pro poor growth has not been agenda. firmly grounded In addition, the reality of aid dependency and donor response to shocks, in wider country economic such as drought and famine, led many LICs to rely on ad hoc responses dialogue. rather than developing SSN systems to address this critical need. Also, as was evident in IEG’s examination of the Bank’s organization structure and SSNs, given that Bank dialogue is often enabled by budget provided through supervision and lending, Bank resources to support SSNs and institution building and to stimulate country demand were lacking in LICs. As additional funds were provided through the RSR trust funds, engagement in LICs increased, and the Bank and countries focused more on institutional strengthening. LINKS TO THE GROWTH AGENDA Including SSNs in broader country discussions, including with core min- In most country istries such as finance and planning, was usually a prerequisite for pro- case studies, ductive Bank support of SSNs. Although the case studies indicate that the including SSN discussions in wider Bank dialogue on national economic, government’s fiscal, and poverty issues does not necessarily lead to more fruitful colla- SSN strategy is borations on SSNs, not including SSNs in the wider Bank dialogue limits part of its wider the Bank’s effectiveness. It can make it more difficult to advocate for SSN poverty programs and spread support to the core finance and planning minis- reduction tries, or it can lead to a less strategic approach overall in how the Bank agenda and is links SSN work with the rest of its engagement. included in the poverty In most country case studies (80 percent), the government’s SSN strategy reduction was part of its wider poverty reduction agenda and was included in the strategy. poverty reduction strategy. In 60 percent of country cases, Bank dialogue 73 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS on SSNs was initiated within the context of broader economic, fiscal, and poverty issues. In other cases it was initiated sectorally (as in Cameroon, India, Jamaica, Nepal, Niger, Nigeria, Pakistan, Tanzania, and Yemen). In certain regions—Africa in particular—the SSN agenda generally was not part of the wider Bank economic and poverty dialogue with the gov- ernment, reflecting the limited interest of governments and the Bank in this area. Although Justifying SSNs by their contribution to the growth agenda can be a way various growth for the Bank to help expand country support for them. In 40 percent of arguments were cases, no growth argument was used to encourage the adoption of SSNs; made in the instead, it was justified in terms of poverty reduction and equity/social country case justice. studies, investing in Various growth arguments (box 10) were made to different degrees in human the country case studies. Investing in human development was used development most frequently (probably reflecting the large number of CCTs sup- was used the ported by the Bank), followed by improving the efficiency of public ex- most frequently, followed by penditures. The risk insurance rationale for growth was used least often, improving the reflecting the lack of focus on shocks in the Bank’s support for SSNs. Al- efficiency of so, this rationale is less well studied than the others, so there is less evi- public dence. However, the risk insurance function has gained more impor- expenditures. tance and credibility (particularly within the Bank) with the food, fuel, and financial crises, as the need for consumption smoothing and avoid- ing negative coping mechanisms became clearer. Box 10. Links between SSNs and Growth The Bank has used a variety of links between SSNs and growth.  Improving human capital strengthens the labor market.  Creating productive infrastructure while paying people wages enables higher productivity (PWP).  Improved efficiency of SSNs can free fiscal space (particularly with universal subsidies), which can be used to improve immediate consumption of poor and invest in pro-poor items that will help these people rise out of poverty.  SSN can make difficult economic reforms more acceptable (not just throughsub- sidy removal), and by preventing strong political opposition (even violent pro- tests) can help with economic stability and growth.  SSN can help populations graduate from poverty and help spur entrepreneur- ship and employment.  SSNs can compensate for insurance market failures in developing countries (agricultural, health, or unemployment) and credit market failures (lack of credit for the poor) and help the poor smooth consumption during individual or sys- temic shocks. If people are aware that this SSN exists as a form of insurance, they are more likely to make more productive decisions about their livelihoods as well as their investment in the human capital of their families. Source: IEG case studies. 74 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS INTERNAL COORDINATION SSNs are a challenge within the Bank, as they fall under the general SSNs are a responsibility of the Social Protection Sector, yet less than half of challenge within projects containing SSN support fall under that sector board. SSNs the Bank involve all networks and 13 sector boards. Most notably, the three because so main departments involved in SSN work include Social Protection many of the (within the Human Development Network), Poverty Reduction and projects fall Economic Management (PREM) Poverty, and PREM Economic Policy. outside of the Analytical work is often undertaken by regional PREM Poverty units, supervisory investment projects and technical knowledge of SSNs are housed in responsibility of the regional Social Protection department (which also produce consi- the Social derable AAA), and multisectoral DPLs involving SSN components are Protection Sector Board. supported by the regional economic policy unit within PREM. IEG research on the Bank’s organizational structure and its affect on SSN work indicates that although the matrix system enables expertise to be drawn from various areas, it also risks making coordination challenging. It creates weak incentives for collaboration, and ultimate- ly affects the quality of the work in countries. Semistructured interviews with 45 Bank staff and managers across the Bank uncovered sources of tension in the Bank’s current organiza- tional structure with regard to SSN work. These sources of tension Among the relate to current budgetary arrangements, task-management ar- sources of rangements, and accountability mechanisms for staff. tension are budgetary arrangements, Interviewees reported that in theory the matrix structure is meant to task foster cooperation across sectors and between sectors and country management management units. In practice, however, budgetary arrangements arrangements, bring about the opposite response. Three-quarters of respondents and pointed out that there is no formal mechanism for sharing credit or accountability budget across the organization, because project budgets are allocated mechanisms. by default to the sector with which the team leader is affiliated. The management of a project is usually granted to the staff member who initiated the dialogue with country authorities, preparing the way for the operation in question. Because PREM typically leads the dialogue on broader country strategic issues, this favors its position when managing DPLs. In practice, because of the “contestability� of SSN-related activities, country directors are forced to choose between competing units, which spurs a sense of uncertainty and “turf� among staff. IEG’s SSN staff survey of the food, fuel, and financial crises and SSN response corroborates this finding, with only 52 per- cent of respondents reporting that the countries on which they work have very effective internal cooperation (that is, between networks). (For a full report of the survey findings, see appendix B, § VII.) 75 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS Without a sufficient budget, managers and directors from sectors and Without a sufficient units that do not have opportunities to manage projects with SSN budget, components face the prospect of having to cut down on their staff. managers and This exerts a powerful drag on staff in that it undermines incentives directors from for cooperation while it pressures them to provide resources for the sectors and respective managers and directors as actively as possible. Moreover, units that do not because Bank units are under pressure to fund the work program of have their staff, they chase the largest sources of demand, fuelling a sort of opportunities to “reactive mode.� manage SSN projects face the For instance, the Latin America and the Caribbean Region, consisting prospect of mainly of MICs, is responsible for 52 percent of the SSN lending and having to cut thus has been allocated larger Bank budgets to support the develop- down on their ment and supervision of these operations. This has provided oppor- staff. tunities for sustained engagement that has not been possible in coun- tries lacking lending programs. This is also suggested by regression analysis, which finds that past SSN borrowing is a significant predic- tor of future SSN borrowing. Recently, however, the strengthening of centrally funded initiatives has provided significant opportunities to step up work on SSN- related activities in LICs, particularly in Africa, as shown by the re- cent increase in the number of operations and the size of commit- ments in IDA countries. This has provided a counterweight to the in- trinsic asymmetry of budget arrangements, had they been allowed to fully operate in the Bank’s matrix structure. There is no clear Another issue that emerged from IEG’s SSN interviews with Bank staff mechanism of had to do with quality assurance mechanisms (or the lack of them) in cooperation, the context of multisectoral work. SSN-related operations tend to be recognition, or multisectoral and, as about half of the respondents noted, this means quality control that the Social Protection Sector Board cannot manage all SSN-related procedures for operations. Yet accountability mechanisms are inconsistent with the cross-sectoral multisectoral nature of the work involved, as there is no clear mechan- situations. ism of cooperation, recognition, or quality control procedures for cross- sectoral situations. Given the multisectoral nature of SSNs, facilitating collaboration between networks is important. Budget practices should be reviewed to see if they constrain cooperation. The SSN Global Ex- pert Team is comprised of Human Development and PREM SSN ex- perts and could be used throughout the Bank as a source of quality control for projects containing an SSN component. EXTERNAL COORDINATION The Bank is an important actor in most of the countries where it has operations, although it is usually only one of several donors trying to help the country with its poverty reduction goals. In both MICs and 76 CHAPTER 5 UNDERSTANDING HOW THE BANK WORKS TO BUILD SSNS LICs, a government’s ability to coordinate donors is important for best use of the financial and technical resources. The case studies indicate that donors (including the Bank) have coordi- Donors have nated well in slightly over half the countries throughout the decade. coordinated well The IEG staff survey data, however, reveal good collaboration during in slightly over the food, fuel, and financial crises in only about a fifth of cases. In 41 half of the percent of countries, survey respondents said there was little or no do- countries, but nor coordination in their countries following these crises; in 37 percent there was good of countries, respondents said donors coordinated only somewhat. collaboration Again, the responses varied by Region and were most evident in South during the recent crises in Asia, where 60 percent of countries had no or little donor coordination only about 20 in response to the crises. (For a full report of the survey findings, see percent of appendix B.) Donor coordination may be particularly challenging dur- cases. ing crisis, when rapid response is required. However, given the huge increase in donor involvement during a crisis, the need for donor coor- dination is particularly heightened. The Bank can be particularly useful in providing financial resources The Bank can be to LICs that have not benefitted greatly from bilateral donors. Some particularly LICs have drawn very successfully on donor funds to finance SSNs, useful in but most have not. The most notable positive example is Ethiopia, providing which receives a large amount of aid and has a highly coordinated financial resources to mechanism for channeling donor funds and technical assistance to LICs that have support the country’s principal SSN. not benefitted as greatly from In contrast, in many LICs, donor-supported projects with SSN com- bilateral donors. ponents are small scale, fragmented, and scattered throughout the country; they address various pockets of need, yet lack the coverage and cohesiveness necessary to build SSN institutions over the longer run or to serve as a potential mechanism for addressing other shocks that might arise. Moreover, the reality of international geopolitics is that some countries receive a great deal more donor support, regard- less of their economic need. These politically important LICs have greater potential for improving their SSNs; those LICs that receive less donor interest face tremendous challenges for developing SSNs. Finally, in practice, the Bank’s analytical work often plays an impor- tant coordinating role by providing intellectual leadership on poverty issues, and this helps other donors form their strategies. 77 Chapter 6 Major Findings and Recommendations This evaluation covers a wide range of issues related to the effective- ness and relevance of the Bank’s lending support to specific SSN pro- grams, as well as its broader analytical and advisory support to coun- tries for developing SSN systems. Events of the past decade have underscored the critical need for SSN programs in all countries, especially in times of crisis. Over fiscal years 2000–10, the World Bank supported programs to protect poor and vulnerable people through noncontributory transfers targeted to the poor and vulnerable. This amounted to $11.5 billion in lending and an active program of analytical and advisory services. Given the growing importance of SSNs, IEG has evaluated the relev- ance and effectiveness of the Bank’s support in this area. Because this evaluation assesses the World Bank’s support for SSNs, the recom- mendations are for the World Bank but with the broader objective of improving SSNs in countries. Substantial progress has been made over the decade, yet the evalua- tion found areas of Bank support that need strengthening to improve their effectiveness and relevance. Continued effort is needed to devel- op SSNs that are flexible and able to respond to shocks; to build coun- try institutional capacity to address various sources of poverty, risk, and vulnerability; to engage more on SSNs in LICs; and to increase the precision and poverty focus in Bank-supported SSN programs by improving the quality of objectives, design, and monitoring. Following are the major findings and recommendations of this evalu- ation: 1. SSNs for shocks: Throughout the decade, countries and the Bank focused SSN support on addressing chronic poverty and human development and focused less on SSNs to address shocks. During the last few years of the decade, the focus of countries and the Bank expanded as addressing systemic shocks took on greater importance. The urgency of preparing for systemic shocks became particularly clear after the food, fuel, and financial crises, during which lessons from previous crises were underscored: those coun- tries that had prepared during “good times� by developing per- manent SSN programs or institutions were better positioned to 79 CHAPTER 6 MAJOR FINDINGS AND RECOMMENDATIONS scale up, as needed—and the Bank was better able to help them— than those that had not. The Bank was most effective in helping countries in which it had been steadily engaged through lending, AAA, or dialogue over the decade. Such engagement enabled the country to develop SSNs and the Bank to develop a deeper under- standing of country dynamics. Recommendation: Engage during stable times to build SSNs that can help countries respond effectively to shocks. This requires steady country dialogue and support for developing SSNs, whether by lending, country-specific AAA, or engagement in global knowledge and learning. As such shocks are transitory in nature, an important characteristic of an SSN is its ability to ex- pand and contract to reach different population groups as needed. Access to reliable poverty data,crisis monitoring systems, and flexible targeting systems are elements to develop appropriate 1 SSNs. 2. Systems approach: During the review period, the Bank began to make an important shift from an approach based mainly on projects that emphasize delivery of social assistance benefits to helping countries build SSN systems and institutions that can re- spond better to various types of poverty, risk, and vulnerability within a particular country context. The institutional development approach was driven mainly by MICs, for which this was the most common objective of Bank-supported SSN programs. Recommendation: Support the development of SSN institutions and systems. The Bank can further accelerate institution building, particularly in LICs, where capacity constraints are often severe and the building blocks for SSN administrative systems may need to be built from scratch. In MICs, the approach will require con- tinuing the effort to harmonize programs within the broader so- cial protection framework. 3. SSN engagement in LICs: The Bank focused its lending, analyti- cal, and capacity-building support for SSNs significantly more on MICs than in LICs throughout the decade. Despite a recent in- crease in LICs engagement, SSNs remain a much less significant part of the development agenda in LICs than MICs. Country de- mand for SSNs may be higher in MICs, but SSNs are also impor- tant in LICs, as they help protect against systemic shocks as well as help the poorest and most vulnerable people. The Bank’s low level of engagement may have perpetuated countries’ low level of attention to SSNs. The Bank needs to be engaged (at least through analytical and advisory services) to help countries understand the 80 CHAPTER 6 MAJOR FINDINGS AND RECOMMENDATIONS value of SSNs and improve their capacity to design and imple- ment the programs, as appropriate in each country’s context. Recommendation: Increase SSN engagement in LICs. The Bank needs to maintain special efforts (financing and internal incen- tives) for LICs that permit countries to develop SSNs that will pro- tect their poorest and prepare for shocks. Depending on the coun- try context, these may include improving country capacity, adapting SSN programs to the institutional environment, improv- ing poverty data and analysis to identify the particularly vulnera- ble groups, and assuring donor coordination for SSNs (for financ- ing and technical assistance) to increase efficiency of government programs. 4. Project performance and quality of results frameworks: The Bank’s support for SSNs has been effective in helping countries reach short-term objectives and achieve short-term impacts, such as increased school attendance or immediate consumption. How- ever, short-term SSN project objectives have not been adequately anchored in longer-term results frameworks. Anticipating and planning for complementary inputs from other sectors (including inputs on monitoring poverty and vulnerability) needs more at- tention. There is also a need to mobilize resources from outside the project cycle to track progress toward the longer-term objec- tives, including selective longer-term impact evaluations. Results frameworks for projects with SSN components have fo- cused insufficiently on clearly identifying the poor and vulnerable target groups. (By the Bank’s definition, SSNs support programs targeted to the poor or vulnerable in some way.) Yet IEG found that objectives and performance indicators were often not specific enough to ensure effective monitoring of the effects of the project on the poor or vulnerable. Project objectives need to be defined more precisely, monitorable key performance indicators need to be better aligned with those objectives, and accompanying M&E arrangements need to track their performance. Recommendation: Improve the results frameworks of Bank- supported SSN projects to (1) more clearly identify and address the needs of specific groups of poor or vulnerable, and (2) identify how project objectives fit within longer-term objectives for devel- opment of country SSNs. This involves improvements in the qual- ity of objectives, design, and monitoring within projects, as well as development of a longer-term results framework for building ef- fective SSN programs and systems (see Box 11). 81 CHAPTER 6 MAJOR FINDINGS AND RECOMMENDATIONS Box 11. Improving Results Frameworks for SSNs: Key Questions to Ask of Each Program The results frameworks and design of Bank-supported SSN programs can be improved by answering the following questions about each operation:  Why were these particular objectives selected, and how will they help the country move toward the longer-term goal of establishing effective and efficient SSNs and development objectives (such as the poverty, health, and education Millennium Development Goals)?  Which particularly poor or vulnerable groups does this SSN seek to help?  How will the SSN program and components selected help achieve the shorter-term objectives and contribute to longer-term objectives?  What complementary or subsequent measures would be necessary for achievement of the short- or longer-term objectives?  How will the design details selected help the program achieve the shorter-term objectives?  How will the KPIs help track progress toward the shorter-term pro- gram objectives as well as the longer-term SSN objectives? 5. Weighing policy options and using cost-effectiveness analysis: The goal for SSNs, as in all areas of public policy, is to have the big- gest impact possible for the least cost. For this reason, cost- effectiveness analysis can be useful in assessing the cost per unit of impact of a particular policy and compare that result to other op- tions. SSN operations have usually been undertaken without ex- plicitly considering alternatives for achieving specific objectives (for example, by using cost-effectiveness analysis). This may be because of the lack of specific project objectives and indicators, the presences of indirect effects, or the difficulty in obtaining data on benefits and cost. However, even where impact evaluations have been conducted and provide precise evidence of benefits, this in- formation has rarely been used to weigh various options or assess where the largest impact can be achieved for the least cost. Recommendation: Clearly define objectives and assess benefits, costs, and feasibility of policy alterntives to ensure the most ap- propriate use of SSNs. Comparing alternative options for reach- ing the specific objective(s) is particularly important in contexts of high poverty and tight budgets and encourages specific SSN ob- jectives. 6. World Bank internal coordination: SSNs are a challenge within the Bank, as they fall within the supervisory responsibility of the Social Protection Sector family, yet less than half of projects con- taining SSN components are managed by staff in that sector fami- ly. Sources of tension exist with regard to budget arrangements, 82 CHAPTER 6 MAJOR FINDINGS AND RECOMMENDATIONS task management, and accountability, although the extent varies by Region. Recommendation: Improve internal coordination of SSNs. It will help to review budget systems to see if the incentives they create for managers and task team leaders constrain cooperation be- tween Bank units. Expertise across sectors and networks needs to be shared to enhance Bank support. 83 Appendix A History of Social Safety Nets at the World Bank Social safety nets (SSNs) have a long history as a protective mechanism for the poor that can be traced back as far as Ancient Egypt and the Roman Empire. However, in the context of the Independent Evaluation Group (IEG) evaluation, the history of SSNs at the World Bank 1 began in the 1980s following global and Bank events that influenced the SSN agenda. In the 1980s Latin America suffered the most severe crisis in its history. The crisis led to a considerable drop in incomes, economic growth stagnated, unemployment soared, and in- flation reduced the buying power of the middle class. To help address the crisis, the Interna- tional Monetary Fund (IMF) and the World Bank proposed and financed structural adjust- ment and stabilization programs throughout the region. Under these programs, countries implemented market-based reforms in exchange for financial assistance. The stabilization programs and the crisis itself had a high social cost. In 1987, to cushion the adverse effects of the stabilization and structural adjustment programs on the poor, Bolivia, with the support of the Bank, put in place the first Emergency Social Fund (ESF) aimed at providing emer- gency relief by creating temporary employment and improving income. After the ESF, a wave of social funds spread to more than 60 countries. In the 1990s, the dissolution of the Soviet Union and the East Asia financial crisis high- lighted the importance of sound SSN systems. After the break-up of the Soviet Union, East- ern and Central Europe suffered economic and political instability, with high social cost. The World Bank stepped in to provide assistance to eastern European countries to build SSN programs. In 1993, the 47th Development Committee meeting of the joint World Bank- IMF recognized the importance of SSNs for mitigating major transitory adverse effects of economic reform on vulnerable groups and to enhance the political viability of reforms. In 1994, two events helped shape the thinking on SSNs, the Mexican crisis (Tequila Crisis) and the transition of South Africa into a democratic nation. The Tequila Crisis set the stage for the first cash transfer program to the poor, and the new South African government ex- tended social pensions and child support to all its citizens, without regard to race. The same year, the World Bank published Averting the Old-Age Crisis: Policies to Protect the Old and Promote Growth, which recommended a three-pillar approach to pension reform: a man- dated, publicly managed, unfunded, defined-benefit pension; a mandated, funded, privately managed, defined-contribution scheme, and voluntary retirement savings. In 1995, the first conditional cash transfer (CCT) programs were launched in Brazil: the Bolsa Escola program in the Distrito Federal on January 3, 1995, and the Guaranteed Minimum Family Income Program (PGRFM) was launched by the mayor in the Campinas Municipali- ty on January 6, 1995. These programs became a model that multiplied in manymunicipali- ties and states in Brazil and led to the current program Bolsa Familia. In 1996, the Bank created the Social Protection and Labor Sector as a part of the Human De- velopment Network, recognizing the importance of social protection in development. The 85 APPENDIX A SOCIAL SAFETY NETS’ HISTORY same year there was a major shift in Bank policies toward financing food expenditures. Food expenditures were allowed to be financed, on an exceptional basis, in poverty allevia- tion, health and nutrition, and SSN programs that provided temporary compensatory measures during fiscal and economic reform programs. In 1997, Mexico launched its CCT program Progresa, now known as Oportunidades. The pro- gram used conditional cash transfers to help reduce short-term poverty, while giving households incentives to invest in the human capital of their children. Oportunidades had a series of impact evaluations that showed intermediate outcomes in human development and consumption smoothing. CCTs now have been expanded in more than 30 countries. In 1998, the global financial crisis hit East Asia and then the Russian Federation and Brazil. SSNs were created, scaled up, and used as a mitigation measures. Indonesia, for example, started SSN programs that now cover a large portion of the poor population. In 2000, the World Development Report (WDR), Attacking Poverty: Opportunity, Empower- ment, and Security, highlighted the need to develop national programs to prevent, prepare for, and respond to macro shocks—financial and natural. In particular, it stressed the need for permanent safety nets that could serve a counter cyclical role when countries are hit by a shock. The same year the Bank launched its first social safety net core course. That course has been fundamental to the learning and knowledge exchange agenda on SSNs. In 2001, Argentina entered an economic crisis causing the destitution of the De La Rua Ad- ministration leading to high social costs. The same year, the Bank published its strategy pa- per “From Safety Net to Springboard,� setting the conceptual foundations of the Social Pro- tection and Labor sector strategy. The SP&L strategy placed particular emphasis on risk and risk management. It also articulated the efficiency motive for SSN to help households man- age risk, help households invest, and help governments reform other sectors. 2001 was also the first time the Bank provided financing for a CCT program (Colombia, Jamaica, and Tur- key). An interpretation of the Bank bylaws was necessary as it was not clear if the Bank could finance cash transfers. The human development argument was crucial for the legal department to consider cash transfers a development investment. In 2002, Ethiopia suffered a major drought that left 13 million people in need of food aid. The crisis shifted the country emergency response from a system dominated by humanita- rian aid to a donor-coordinated, predictably financed, productive and development- oriented safety net to address the urgent needs of chronically food insecure households. Currently, the Ethiopia Productive Social Safety Net Program is one of the largest in Sub- Saharan Africa (covering 5 million people). In 2003, the Bank’s SSN portfolio increased considerably, when the Jefes del Hogar (Heads of Household) Project received $600 million (total cost $1.9) billion as a response to the Argen- tinean crises. In 2004, the Bank’s OP 8.60 operational policy for development policy operations (DPLs) recognized that DPL operations can have distributional effects. Consequently, it required an evaluation of poverty and social impacts, as well as mitigation measures when needed. 86 APPENDIX A SOCIAL SAFETY NETS’ HISTORY In 2005, a new Bank publication on pensions, Old Age Income Support in the 21st Century: An International Perspective on Pension Systems and Reform recognized the central role of social pensions in countries’ social protection systems. In 2008, the food and fuel crises hit developing countries, 36 countries were in crisis as a re- sult of higher food prices. The Bank established the Food Crisis Response Program with and initial line of credit of $1.2 billion, giving priority to the most fragile states. The available funds were increased to $2 billion. The same year, the first ministerial conference on social development in Africa was held in Namibia. Ministers and senior officials from the African Union’s 53 member states participated. African Union members agreed to develop social protection schemes for disadvantaged groups such as children, elderly, and people with disabilities, by, among other things, introducing and extending public-financed non- contributory cash transfer schemes. In 2009, the financial crisis reached emerging economies. Sixty-four million more people dropped below the $1.25 day poverty line (extreme poverty) and 40 million more people went hungry as a consequence of the food, fuel, and financial crises. The Bank established the Rapid Social Response Program (RSR) to respond to the crises, leveraging the Bank’s own resources through IBRD and IDA ($12 billion for SSNs and other social protection pro- grams in health and education), and donor resources ($65.22 million).The Bank’s SSN lend- ing portfolio hit a record $3.4 billion, with a single operation (Mexico’s Oportunidades pro- gram) of $1.5 billion. As of October 2010, $71 billion (IBRD/IDA) has been disbursed to countries in need of fi- nancial assistance as a consequence of the crises—including $21 billion to the world’s 79 poorest countries. SSNs represented 9 percent of the total Bank disbursements post-crises ($6.3 billion). Twenty years ago, SSNs were put in place to cushion against the adverse effects that stabili- zation and structural adjustment programs have on the poor. Now, SSNs fill a greater role: reducing inequality and extreme poverty; helping households prevent, prepare, and re- spond to shocks—macro or idiosyncratic—so they can better manage risk; and helping households invest improve their livelihoods. 87 APPENDIX A SOCIAL SAFETY NETS’ HISTOR Table A 1. Timeline Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events Employment Guarantee Scheme (EGS) India • The EGS pilot launched in 1965. It was subsequently expanded, precipitated by an extended drought in 1971. This culminated in the EGS Act and its 1979 implementation 1971 in Maharashtra.a • With the implementation of EGS, India was one of the few countries in the developing world to implement a public works program (PWP) • 31 percent of the FY00–10 SSN portfolio had a PWP component. b Chile invents the Proxy Means Test (PMT) • PMT is an alternative method to Verified Means Testing in developing countries that have a high degree of informality in the labor market. c • If designed and executed correctly, PMT 1980 can have a high degree of accuracy, while also being cost efficient and transparent. • 53 percent of the countries with a conditional cash transfer (CCT) program use PMT as a target method. d Latin America Debt Crisis World Bank support to Latin • The 1982 debt crisis was one America and the Caribbean of the most severe in Latin (LCR) America’s history: incomes • WBG provided support for 1982 dropped, economic growth structural adjustment and stagnated, unemployment rose stabilization programs in to high levels, and inflation LCR reduced the buying power of the middle class.e 88 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events • The crisis made social assistance and targeting priorities on the agendas of the World Bank and state governments. First Bank loan for social costs of adjustment • Bolivia’s Emergency Social Fund (ESF) was established to cushion the adverse effects on the poor of the economic crisis and the subsequent stabilization program of the 1980s, as well as to 1987 facilitate transition through the phases of structural adjustment.f • After ESF, a wave of social funds (SF) began, quickly spreading to 50+ countries. SFs have been used, in the context of SSNs, primarily to fund PWPs. Dissolution of the Soviet Union • Eastern and Central Europe started a transition period, introducing market 1991 liberalization policies.g First Bank SSN operation in ECA • Post-dissolution of the Soviet Union, 1992 the Bank provided assistance building SSN systems for the first time to Albania.h Social Security Reforms and 1993 Social Safety Nets Report 89 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events presented at the 47th meeting of the joint World Bank- International Monetary Fund Development Committee meetings. Helped shape thinking and work program on safety net programs. Recognized the importance of SSNs for mitigating major transitory adverse effects of economic reform on vulnerable groups to assist the poor and enhance the political viability of economic reforms. Analytical work on SSN increased considerably because of the support of the development community to SSNs. i Tequila Crisis (Mexico) • December 1994, one of the worst economic Averting the Old-Age Crisis: crises in Mexican history began with the Policies to Protect the Old sudden devaluation of the Mexican peso. and Promote Growth The crisis had ramifications for the Southern (Report) Cone.j • The report recommended a • 50 percent of the Mexican population fell three-pillar approach to into poverty. pension reform: a mandated, 1994 publicly managed, unfunded, defined-benefit pension; a South Africa democratic elections mandated, funded, privately South Africa transitioned from apartheid to managed, defined- constitutional democracy. The new contribution scheme, and government modified the social pensions voluntary retirement savings. and child support systems, raising benefit l levels of blacks to the level of whites. The social grant system, which primarily 90 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events supports children, has expanded coverage from 2.5 million beneficiaries in 1999 to over 13 million in 2009 and it has influenced thinking in the region. k CCT programs started Two CCTs were launched in two Brazilian municipalities (representing two different political parties): the “Bolsa Escola� pro- gram was launched in the Distrito Federal 1995 (Brasilia) and the “Guaranteed Minimum Family Income Program (PGRFM) in the Campinas Municipality. These programs were the orginis of the current program Bolsa Familia. m Creation of the Social Protection and Labor Sector • The Social Protection and Labor Sector was created as a part of the Human Development Network. Bank financing of food 1996 expenditures was permitted for poverty alleviation, health and nutrition, and to improve targeting and effectiveness of projects with SSN components that provided temporary compensatory measures during fiscal and economic reform programs.n Mexico’s CCT program started • Mexico initiated its incentive-based 1997 poverty reduction program, called Progresa in response to the 1995 crisis. Progresa, now known as Oportunidades, uses conditional 91 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events cash transfers to help reduce short-term poverty through transfers, while giving households incentives to invest in the human capital of their children and thereby reduce poverty over the long run. o Global financial crisis hit East Asia then Russia and Brazil • The global financial crisis resulted in the implementation 1998 of large-scale social protection measures with World Bank assistance (for example, Indonesia).p WDR 2001 Attacking First Social Safety Nets Core Course: Poverty: Opportunity, The Design and Implementation of Empowerment, and Security Effective Safety Nets • The report highlighted the • The SSN core course has been need to develop national delivered for 10 years now, and has been programs to prevent, prepare fundamental to the learning and for, and respond to macro knowledge exchange agenda on SSNs. 2000 shocks—financial and natural. In particular, the need for counter-cyclical safety nets to be permanent and ready to be deployed when countries are hit by a shock. q Argentina Crisis World Bank Social First time Bank lends for CCT • The Argentine crisis hit its worst point Protection Strategy Paper: • The WBG starts lending programs to when massive numbers of people began From Safety Net to support CCTs in Colombia, Jamaica, and withdrawing money from the Bank, causing Springboard. Turkey.t 2001 a run on the banks. The government then • This paper set the • The Human Development argument enacted a set of measures that effectively conceptual foundations of the was key to getting legal clearance for froze all bank accounts for twelve months, SP&L sector strategy CCTs. allowing for only minor sums of cash to be • It put particular emphasis withdrawn.r on risk and risk management: 92 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events the Social Risk Management (SRM) framework complemented social protection’s more traditional emphasis on equity and basic needs, and recognized the demands and consequences of the rapidly changing global economy. • Articulates efficiency motive for SSN to: – help households manage risk – help households invest – help governments reform other sectors.s Ethiopia drought crisis 1st International Conference on CCT 13 million people required emergency food Programs ( Mexico) aid due to a severe drought. The • The conference, first in its kind, government of Ethiopia shifted from a provided a forum for government 2002 system dominated by emergency officials and other main stakeholders to humanitarian aid to a productive and share experiences—both successes and development-oriented safety net to address challenges—on CCT programs. the urgent needs of chronically food insecure households. u First time that the Bank lends more than half a billion dollars for a project containing SSN components 2003 • As a response to the Argentina crisis, the Jefes del Hogar (Heads of Household Project) received $600 million. The total cost of the project was $1.9 billion.v World Development Report Ethiopia Productive Safety Net 2004: Making Services Work Program. 2004 For Poor People • First Bank investment financing an • The WDR 2004 dealt with unconditional cash transfer in low- 93 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events the crucial role of transfers income Africa.x and social services.w • OP 8.60, operational policy for development policy operations (DPLs) recognized that DPLS operations can have distributional effects. Consequently, it requires an evaluation of poverty and social impacts. Indian National Rural Employment Old Age Income Support in Guarantee Act (NREGA) the 21st Century: An • The scheme provides a legal guarantee for International Perspective on 100 days of employment in every financial Pension Systems and year to adult members of any rural Reform (Report) household willing to do public work-related Modified the conceptual unskilled manual work at the statutory framework for the Bank’s minimum wage of Rs.100 per day.y pension work, adding the following pillars: • “zero pillar�—(social • Rights-based approach to social pension, or general social protection. assistance typically financed by the local, regional or 2005 national government), fiscal conditions permitting, to deal explicitly with poverty- alleviation objectives in order to provide the elderly with a minimal level of protection. • “first pillar�—contributory pension, linked to (and intended to replace) part- time earnings; • mandatory “second pillar�—individual savings accounts; • voluntary “third pillar�— 94 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events could be individual or employer-sponsored; • A non-financial “fourth pillar,� which includes access to informal support (such as family support), other formal social programs (such as health care and/or housing), and other individual financial and non-financial assets (such as home ownership and reverse mortgages where available). z Livingstone Call for Action-- A World Bank WDR: Equity Third International Conference on Transformative Agenda for the and Development Conditional Cash Transfer Programs 21st Century: Examining the • Re-establishes within the (Turkey) case for basic social protection World Bank the equity • Formation of the LCR learning circle. in Africa motive for safety nets Ministers and senior • Safety nets now seen as representatives from 13 African having both protection and countries (Ethiopia, Kenya promotion as goals.aa Lesotho, Madagascar, Malawi, Mozambique, Namibia, Rwanda, South Africa, Tanzania, Uganda, 2006 Zambia, and Zimbabwe) together with Brazil, development partners, UN agencies, and NGOs participated • Called for more “social transfer� programs, more capacity building, and long- term funding commitments; • Sets schedule for developing local plans and bi-annual south- south conference. 95 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events Food and Fuel Crises hit • High volatility in food prices threatened to further increase food insecurity and exacerbate the hardships faced by the poor. • 36 countries in crisis as a result of higher food prices.bb First ministerial conference on social development in Africa (Namibia): “Towards a Food Crisis Response Sustainable Development Program (FCRP) Agenda for Africa.� • On May 29, 2008, the Board Ministers and senior officials of Executive Directors from the African Union’s 53 approved a $1.2 billion rapid member states participated financing facility to address Commitment at the ministerial immediate needs arising level to develop social 2008 from the food crisis, protection schemes for including $200 million in disadvantaged groups such as grants from IBRD surplus children, elderly, and people targeted at the vulnerable in with disabilities. Measures IDA countries. This program included: had the mandate to give • Extending existing social priority to the most fragile insurance schemes (providing states. subsidies for those that cannot contribute) • Setting up community-based or occupation-based insurance schemes on a voluntary basis • Providing social welfare services and employment guarantee schemes • Introducing and extending public-financed non- contributory cash transfer schemes.cc 96 APPENDIX A SOCIAL SAFETY NETS’ HISTORY Year World Events SSN’s Country Events Bank-Wide Events Bank-SSN Events FCRP Bank SSN commitments dramatically • On April 2009, in response increased to high demand, the Bank • The Bank announced that its Food and Fuel Crises increased the GFRP ceiling to investments in SSNs and other social • Although food grain prices $2 billion. protection programs in health and declined from their peaks in education are projected to triple to $12 2008, major food grain prices Rapid Social Response billion over the next two years in order 1 Countries borrowing for the first time for a Program (RSR) to protect the most vulnerable people are still above average. project of SSNs • The RSR was established in from the worst effects of the global • As a result of the food, fuel and financial response to the triple wave of economic crisis. Financial Crisis hits 2009 • Crisis spreads rapidly crises, increased attention was given to food, fuel, and financial SSNs: 17 new countries borrowed from the crises. The RSR Program First time the Bank lent more than a through the United States and 3 leverages the Bank’s own billion dollars for a single project European financial sectors. Bank for SSNs. resources through IBRD and focusing on SSNs Major banks draw back on (Oportunidades-Mexico) financing for trade and foreign IDA, and donor resources • As a response to the crises, the Bank investment. (Norway, the Russian lent $1.5 billion to support the • Crisis begins to hit emerging Federation, and the United Oportunidades program. The total cost of economies. 2 Kingdom for a total of $65.22 4 the project was $6.3 billion. million) . • 72 percent of the funds • The Bank’s SSN portfolio hits lending were used in SSN activities. record of $3.4 billion Impact of the food, fuel, and Bank commitments financial crises on the poor • The Bank has committed $138 billion • The Bank estimates that 64 to its members since the crises began. As million more people are living of October 2010, $71 billion (IBRD/IDA) in extreme poverty (below has been disbursed, including $21 billion $1.25/day) in 2010 and 40 to the world’s 79 poorest countries. million more people went 2010 hungry in 2009 due to the • SSN represented 9 percent of the total 5 Bank disbursements post- crises ($6.3 crises. billion). • Oportunidades received $1.2 billion in additional financing, hitting a record of $2.7 billion project. a. Gaiha Raghav (2006), The Maharastra Employment Guarantee Schem, ODI. b. SSNs Portfolio Review. 97 APPENDIX A SOCIAL SAFETY NETS’ HISTORY c. Larrañaga Osvaldo (2005), El Sistema CAS en Chile, World Bank. d. SSNs Portfolio Review. e. Khadiagala, Lynn S. (1995), Social Funds: Strengths, Weaknesses, and Conditions for Success, World Bank. f. Khadiagala, Lynn S. (1995), Social Funds: Strengths, Weaknesses, and Conditions for Success, World Bank. g. World Development Report: From plan to market (1996), World Bank. h. SSNs Portfolio Review. i. Subbarao K, et al. (1997), Safety Net Programs and Poverty Reduction, World Bank. j. Lederman Daniel (2000), et al., Mexico: Five Years after the Crisis, World Bank. k. Palacios Robert, Oleksiy Sluchynsky (2006) Social Pensions: Their Role in the Overall Pension System, World Bank. l. World Bank (1994), Averting the Old-Age Crisis: Policies to Protect the Old and Promote Growth, World Bank m. Lindert Kathy, et all (2007), The Nuts and Bolts of Brazil’s Bolsa Família Program: Implementing Conditional Cash Transfers in a Decentralized Context, World Bank. n. Operational Memorandum 33262 (1991), World Bank. o. Fiszbein Ariel and Norbert Schady (2009), Conditional Cash Transfers: Reducing Present and Future Poverty, World Bank. p. http://web.worldbank.org/WBSITE/EXTERNAL/NEWS/0,,contentMDK:20013299~menu PK:34464~pagePK:64003015~piPK:64003012~theSitePK:4607,00.html. q. Bank (2001), World Development Report Attacking Poverty: Opportunity, Empowerment, and Security, World Bank. r. http://web.worldbank.org/WBSITE/EXTERNAL/COUNTRIES/LACEXT/0,,contentMDK:20786193~pagePK:146736~piPK:146830~theSitePK:25 8554,00.html s.World Bank (2001), Social Protection Strategy Paper: From Safety Net to Springboard, World Bank. t. SSNs Portfolio Review. u. http://web.worldbank.org/WBSITE/EXTERNAL/EXTABOUTUS/IDA/ 0,,contentMDK:21395349~menuPK:4754051~pagePK:51236175~piPK:437394~theSitePK:73154,00.html v. SSNs Portfolio Review. w. World Bank (2004), World Development Report: Making Services Work For Poor People, World Bank. x. SSNs Portfolio Review. y. Gaiha Raghav (2006), The Maharastra Employment Guarantee Schem, ODI. z. World Bank (2005) Old Age Income Support in the 21st Century: An International Perspective on Pension Systems and Reform, World Bank. aa. World Bank (2006), World Development Report: Equity and Development, World Bank. bb. http://web.worldbank.org/WBSITE/EXTERNAL/NEWS/0,,contentMDK:21665883~menu PK:34460~pagePK:34370~piPK:34424~theSitePK:4607,00.html. cc. http://www.helpage.org/es/Site/News/Latestnews/2008/@72903. 98 Appendix B Methodology I. Portfolio Reviews The World Bank’s SSN portfolio was used throughout the evaluation to determine many kinds of trends in lending and performance. Project variables were taken from the Bank’s operational database, and many other variables were coded based on project documents and IEG reviews of Implementation Completion Report. This enabled a deeper analysis of project portfolio performance as well as an assessment of project results frameworks and design details. LENDING PROJECTS To identify projects supporting SSN, IEG followed the selection criteria used by the Human Development Network (HDN) for its publication Social Safety Nets in World Bank Lending and Analytical Work: FY2002–2007. HDN included all projects thematically coded as 54 (Social 1 Safety Nets). To avoid errors of exclusion, HDN also identified all projects approved by the Social Protection Sector Board not assigned theme code 54, as well as all projects approved by other Sector Boards assigned theme code 56 (Other social protection and risk manage- ment). HDN reviewed the design documents of these additional projects and selected all those with at least one safety net component. HDN later updated the portfolio with another 2 45 safety net projects approved between FY08 and FY09. Using the same selection criteria, IEG Table B.1. Number of Approved Projects with SSN Components identified 30 IBRD/IDA safety net Summary Table Total projects approved during FY00–01, and another 45 approved in FY10 (in- IEG (FY00–01) 30 cluding food crisis response projects). Anchor (FY02–07) 121 IEG also included projects that were Anchor (FY08–09) 45 assigned the new theme code 91 IEG (FY10) 45 (Global Food Crisis Response) and IEG othera 3 have a social safety net component. In Total 244 addition, IEG added projects that in- Source: World Bank internal database. cluded components to address health a. “Other� includes projects the provide health insurance for the poor, care for the poor, which were not in- which are not typically assigned a safety net theme code. cluded in the HNP portfolio but were considered to be strong SSNs. Three projects were add- ed to the SSN database. IEG reviewed project design documents and, when available, ICRs to code variables not available in an internal Bank database. IEG coded whether projects had various safety net 3 programs; targeting method employed (means testing, geographic targeting, community- based targeting, and so forth); monitoring and evaluation, including whether baseline data were collected, whether an impact evaluation was planned or implemented, and key per- formance indicators. Other variables coded included number of responsible ministries, pro- 99 APPENDIX B METHODOLOGY curement arrangements, financial management arrangements, safeguard policies triggered, targeting method used, and Results Framework variables. SSN Functions and Other Variables IEG coded five functions of social safety nets:  Function 1: reduce chronic poverty and inequality  Function 2: encourage more and better human capital investments among the poor to provide the opportunity to exit poverty  Function 3: enable the poor to manage risk due to individual shocks  Function 4: enable the poor to manage risk due to systemic shocks  Function 5: protect the poor if hurt during broader economic reforms. Most interventions have more than one function. (For example, a CCT for education aims to both mitigate chronic poverty and invest in the human capital of the poor.) In most cases, the safety net function is not stated, so it was inferred from the project objectives, the justifi- cation for the project, the country context, and the specific safety net program. In addition, by definition, IEG considered all CCTs to serve functions 1 and 2. All projects with training activities were also coded function 2. Finally, workfare projects were coded function 1 and function 3. All projects with health insurance for the poor were coded function 4. Finally, all emergency recovery loans (ERLs) were coded function 3. All DPLs were reviewed specifical- ly to determine if they were implemented to support some type of macro reforms, and if so, 4 were coded function 5. Commitments IEG used the following criteria to calculate the commitments:  For projects theme code 54, the SSN amount corresponds to the share of the theme code multiply by the full dollar value of the project.  For projects where the task team did not assign code 54, IEG reviewed the design documents. When SSN was a component in the project and the project stated the dollar value per component, IEG considered the SSN amount the total amount of the SSN component. When SSN was a subcomponent of the project or the dollar amount was not stated in the project design documents, the share assigned to the other social protection codes (51, 56, and 87) was considered the SSN share.  For additional financing, IEG counted the amounts committed to safety nets by these projects. Only projects for which the project development objective changed counted as free-standing projects for the purpose of project counts. 100 APPENDIX B METHODOLOGY Table B.2. Approved Projects with SSN Components Approval  Project ID  Projectname  Country  year  2000  P070337  EC�SAL  Ecuador  2000  P057952  SIF 2  Armenia  2000  P042039  MZ�Railway & Port Restr (FY00)  Mozambique  2000  P056393  2nd Social Action Fund (FAS II)  Angola  2000  P068830  RY�Second Social Fund for Dev  Yemen  2000  P064510  BI�Social Action II (BURSAP)  Burundi  2000  P040649  COMMUNITY REINTEGRATION & REHABILITATION  Sierra leone  2000  P050613  NI SECOND BASIC EDUCATION PROJECT  Nicaragua  2000  P068394  EMG EARTHQUAKE RECOV � EERL  Turkey  2000  P068762  CO� COMMUNITY WORKS (MANOS A LA OBRA)  Colombia  2000  P062932  PE�HEALTH REFORM PROGRAM  Peru  2001  P070920  KE�HIV/AIDS Disaster Resp (FY01)  Kenya  2001  P069964  CO� Human Capital Prot.� Cash Transfers  Colombia  2001  P070650  SOTAC  Bosnia�Herzegovina  2001  P044674  ER�Emerg Reconstr ERL (FY01)  Eritrea  2001  P050772  BR LAND�BASED POVRTY ALLEVIATION I (SIM)  Brazil  2001  P052021  PA BASIC EDUCATION II  Panama  2001  P071243  PE�Programmatic Social Reform Loan I  Peru  2001  P073196  ET�Demob & Reinteg ERL (FY01)  Ethiopia  2001  P004850  VIETNAM � POVERTY REDUC.SUPPORT CREDIT  Vietnam  2001  P068463  ER�Integ Early Childhood SIL (FY01)  Eritrea  2001  P050342  ET�Women Dev Initiatives LIL (FY01)  Ethiopia  2001  P064895  HN FIFTH SOCIAL INVESTMENT FUND PROJECT  Honduras  2001  P064906  NI Poverty Red.&Local Dev. FISE  Nicaragua  2001  P069894  PRIV SOC SUPPRT  Turkey  2001  P071113  LB � Community Development Project  Lebanon  2001  P065372  TZ�Soc Action Fund SIL (FY01)  Tanzania  2001  P070995  COMM DEVT  Bosnia�Herzegovina  2001  P064536  CHILD WELFARE REF  Bulgaria  2001  P008783  SOC SECT DEV (SSD)  Romania  2002  P076712  COMM DEVT  Macedonia, FYR  2002  P073531  MA�Social Development Agency  Morocco  2002  P073604  ER�Emerg Demob & Reint ERL (FY02)  Eritrea  101 APPENDIX B METHODOLOGY 2002  P067774  JM� Social Safety Net Project  Jamaica  2002  P069858  SIF  Ukraine  2002  P068808  SDF 2 (APL .2)  Romania  2002  P073591  AR SAL 1 (Federal Prov Fiscal Relations)  Argentina  2002  P074408  SRMP  Turkey  2002  P044876  Female Secondary School Assis. II  Bangladesh  2002  P069935  PRSC  Albania  2002  P069923  VC Disaster Management  St Vincent and the  Grenadines  2002  P071371  BI�MultiSec HIV/AIDS & Orph APL (FY02)  Burundi  2002  P008860  POV ALLV 2  Tajikistan  2003  P069861  CO� Social Sector Adjustment  Colombia  2003  P080345  MG�Emerg Econ Recovery Crdt (FY03)  Madagascar  2003  P077172  UY Structural Adjustment Loan  Uruguay  2003  P073671  UG�PRSC II  Uganda  2003  P079156  ID�KECAMATAN DEV. 3  Indonesia  2003  P073817  PE�Programmatic Social Reform Loan II  Peru  2003  P069532  SIEP  Bulgaria  2003  P071374  RW�MultiSec HIV/AIDS (FY03)  Rwanda  2003  P083074  AR Economic & Social Transition  Argentina  2003  P079335  SL�Natl Soc Action (FY03)  Sierra Leone  2003  P080263  UY SSAL  Uruguay  2003  P076837  JM National Community Devt. Project  Jamaica  2003  P074361  SIF 2  Georgia  2003  P075911  MW�MASAF APL 3 (FY03)  Malawi  2003  P074762  GY Public Sector Tech. Assistance Credit  Guyana  2003  P069374  EMPLYMT PROMO LIL  Serbia  2003  P082700  BO Social Safety Net SAC  Bolivia  2003  P082472  Natn'l Emergency Emp. Prog for Rural  Afghanistan  2003  P078053  MZ�HIV/AIDS Response SIL (FY03)  Mozambique  2003  P002952  UG�N Uganda Soc Action Fund (FY03)  Uganda  2003  P082395  EC�FIRST PROGRAMMATIC HUMAN DEV. REFORM  Ecuador  2003  P080612  ZM�Emergency Drought Recovery ERL (FY03)  Zambia  2003  P067575  PSAL 2  Romania  2003  P073578  AR Social Protect VI (Jefes de Hogar)  Argentina  102 APPENDIX B METHODOLOGY 2003  P081773  ET�Emerg Drought Recovery ERL (FY03)  Ethiopia  2003  P078288  AO�Emerg Demob & Reinteg ERL (FY03)  Angola  2003  P074602  BI�ERC (FY03)  Burundi  2003  P078390  SOSAC (SERBIA)  Serbia  2004  P078951  PE�Programmatic Social Reform III  Peru  2004  P087841  BO�Social Sector Programmatic Credit  Bolivia  2004  P072637  AR�Prov. Maternal�Child Hlth Adj PMCHSAL  Argentina  2004  P087661  BO Emergency Economic Recovery Credit  Bolivia  2004  P049892  PENSION & SOC ASST  Azerbaijan  2004  P084596  EMPLYMT 2  Bosnia�Herzegovina  2004  P087713  BR Bolsa Familia 1st APL  Brazil  2004  P082976  RY�THIRD PUBLIC WORKS  Yemen  2004  P081964  BI�Demobilization & Reint Prj (FY04)  Burundi  2004  P078658  DRC�Emerg Demob Reintegr ERL (FY04)  Congo, Dem. Rep.  2004  P082037  CL�Social Protection TA Loan  Chile  2004  P079060  CO 1st PSAL Labor & Soc Ref  Colombia  2004  P087620  SOC PROT ADMIN  Armenia  2004  P078088  CL�Social Protection Adjustment Loan DDO  Chile  2004  P074358  SOC PROT  Macedonia, FYR  2004  P085433  DO Social Crisis Response Adjustment Loa  Dominican Republic  2004  P082498  RY�SOCIAL FUND FOR DEVELOPMENT III  Yemen  2005  P070668  KH�Cambodia Education Sector Support  Cambodia  2005  P084583  ID�UPP3  Indonesia  2005  P092019  ID Kecamatan Development Project 3B  Indonesia  2005  P092473  3A�Afr Emergency Locust Prj (FY05)  Africa  2005  P085786  TZ�Soc Action Fund 2 SIL (FY05)  Tanzania  2005  P082328  BR�Integ.Munic.Proj.�Betim Municipality  Brazil  2005  P087707  ET�Productive Safety Nets I (FY05)  Ethiopia  2005  P077749  ET�PRSC 2 DPL (FY05)  Ethiopia  2005  P078457  SAC 2 (SERBIA)  Serbia  2005  P094513  India Tsunami ERC  India  2005  P083932  North East Housing Reconstruction Progra  Sri Lanka  2005  P069207  SN�Casamance Emerg Reconstr Supt (FY05)  Senegal  2005  P081484  BJ�Natl CDD SIL (FY05)  Benin  2005  P074065  SN�PRSC 1 DPL (FY05)  Senegal  103 APPENDIX B METHODOLOGY 2005  P095028  Uruguay Social Program Support Loan DPL  Uruguay  2005  P087807  MINE CLOSURE, ENV & SOCIO�ECO REG  Romania  (CRL)  2005  P079628  PH�2ND WOMEN'S HEALTH & SAFE MOTHERHOOD  Philippines  2005  P078716  BR(CRL1)Prog Growth for Housing  Brazil  2005  P088857  CO (CRL2) TAL to support the 2nd PSAL  Colombia  2005  P091365  BO�Social Sector Programmatic Credit II  Bolivia  2005  P069937  SOC WELF DEVT  Croatia  2005  P082018  KAZAN MUNICIPAL DEVT  Russia  2005  P083968  PE�Programmatic Social Reform Loan IV  Peru  2005  P078523  Integrated Human Development Project  Maldives  2005  P083746  MA�Housing Sector DPL  Morocco  2005  P094193  Post Tsunami Emergency Project  Maldives  2006  P099110  Pakistan Earthquake ERC  Pakistan  2006  P096594  ID Second Development Policy Loan  Indonesia  2006  P057880  URBAN HEAT  Armenia  2006  P095523  HT Transport and Territorial Devopment  Haiti  2006  P093096  SOC INCL PROG (CRL)  Romania  2006  P096411  NE�Rural & Social Policy Reform I (FY06)  Niger  2006  P082242  HN Nutrition and Social Protection  Honduras  2006  P084567  Education Sector Dev. Support Credit II  Bangladesh  2006  P090832  Bangladesh Development Support Cr. III  Bangladesh  2006  P050761  BR�Housing Sector TAL  Brazil  2006  P079316  DPL  Ukraine  2006  P098167  EC (APL1) Bono de Desarrollo Humano  Ecuador  2006  P089443  CO Social Safety Net Project  Colombia  2006  P075231  SOC ASST SYS MOD  Ukraine  2006  P055483  AR�Heads of Household Transition Project  Argentina  2006  P086775  HN (CRL1) Rural Infrastructure Project  Honduras  2006  P088732  MX Access to Land for Young Farmers  Mexico  2006  P096605  DO Youth Development Project  Dominican Republic  2006  P099453  AVIAN FLU (AICHPPCP)  Kyrgyz Republic  2006  P075464  PH�NP Support for HNP  Philippines  2007  P094869  DM GSPTAC  Dominica  104 APPENDIX B METHODOLOGY 2007  P090967  Second Higher Education Project  Nepal  2007  P095250  HEALTH SERVICES AND SOCIAL ASSISTANCE  Moldova  2007  P099882  PRSO 3  Georgia  2007  P098963  NE�Rural & Social Policy DPL 2 (FY07)  Niger  2007  P102541  Education Sector Dev.Support Credit III  Bangladesh  2007  P094097  CO� 3rd Prog.Labor and Social Sector  Colombia  2007  P104097  VN � Program 135 Phase 2 Support Credit  Vietnam  2007  P100026  MA�National Initiative for Human Dev.  Morocco  2007  P100390  Sri Lanka: Puttalam Housing Project  Sri lanka  2007  P089898  GT (CRL1) Education Quality and Sec. Edu  Guatemala  2007  P090690  PK PRSC II  Pakistan  2007  P096861  PUBLIC SECTOR REFORM TA  Tajikistan  2007  P086294  DRC�Education Sector Project (FY07)  Congo, Dem. Rep.  2007  P098093  ET�Productive Safety Nets II (FY07)  Ethiopia  2007  P103606  MG�Sust. Health System Dev. (FY07)  Madagascar  2007  P100846  Sindh DPC  Pakistan  2007  P083927  UY First Prog. Reform Implement. DPL  Uruguay  2007  P099918  HT (APL1) Education For All  Haiti  2008  P087925  BO Land for Agricultural Dev  Bolivia  2008  P096823  Delivery of Improved Local Services  Serbia  2008  P101084  BO Investing in Children and Youth  Bolivia  2008  P110762  Peace Support Project  Nepal  2008  P105116  SOCIAL PROTECTION DEVELOPMENT  Azerbaijan  2008  P100546  JO � Social Protection Enhancement  Jordan  2008  P105024  JM Social Protection  Jamaica  2008  P109964  BI�Second HIV/AIDS MAP (FY08)  Burundi  2008  P094288  LB—Reform Implementation DPL  Lebanon  2008  P098328  PA Social Protection project  Panama  2008  P107254  ER�Integrated ECD II SIL (FY08)  Eritrea  2008  P090010  DO Social Sectors Investment Program  Dominican Republic  2008  P088837  PPDPL 2  Turkey  2008  P110446  MW�3rd Social Action Fund APL II (FY08)  Malawi  2008  P105002  National Program for Community Empower  Indonesia  2008  P052608  CO—Antioquia Secondary Education Projec  Colombia  2009  P110191  Fifth Development Policy Loan  Indonesia  105 APPENDIX B METHODOLOGY 2009  P116125  Empl. Entrepreneurship & HCDP DPL  Poland  2009  P105075  PPAF III  Pakistan  2009  P115737  MN�Development Policy Credit  Mongolia  2009  P101171  AR SOC&FISC NTL ID SYS II  Argentina  2009  P102119  NG�HIV/AIDS Prog. Dev. II (FY09)  Nigeria  2009  P112312  GT Fiscal and Institutional DPL I  Guatemala  2009  P111633  UG�SEC N�Uganda SAF (NUSAF2) (FY09)  Uganda  2009  P113002  NP Social Safety Net � Food Crisis Respo  Nepal  2009  P115177  PA Protect Poor Under Glob Uncert DPL  Panama  2009  P112761  Bangladesh Food Crisis DSC  Bangladesh  2009  P113492  Philippines GFRP DPO  Philippines  2009  P113301  GH�EGPRC (fast�track)  Ghana  2009  P100657  SOCIAL INCLUSION PROJECT  Bulgaria  2009  P115938  SN�Nutrition/Cash Transfer Proj (FY09)  Senegal  2009  P106834  RW�1st Comm Living Standards (FY09)  Rwanda  2009  P095681  OECS (Grenada) Skill for Inclusive Growt  Grenada  2009  P101211  CO Second Social Safety Net Project  Colombia  2009  P115183  AR Basic Protection Project  Argentina  2009  P103974  Conditional Cash Transfers Project  Macedonia, FYR  2009  P115067  MX Support to Oportunidades Project  Mexico  2009  P113134  MG�Emerg. Food Sec. & Reconstr. (FY09)  Madagascar  2009  P111545  KE�Cash Transfer for OVC (FY09)  Kenya  2009  P114910  SV Public Finance and Social Sector DPL  El Salvador  2009  P113372  Poverty Reduction & Econ. Support Oper.  Pakistan  2009  P103160  Social Safety Net Technical Assistance  Pakistan  2009  P106332  Disability and Children at Risk  Bangladesh  2009  P106445  PA Hlth Equity & Performance Improvement  Panama  2009  P106502  CV � PRSC 4 �DPL  Cape Verde  2009  P111164  VN � PRSC 8  Vietnam  2010  P117758  RW�Second Community Living Standards Gra  Rwanda  2010  P102018  DPL 1  Romania  2010  P114236  BF:Agricultural Productivity & Food Sec.  Burkina Faso  2010  P117238  Malawi: PRSC�3  Malawi  2010  P117016  LC Economic and Social DPL  St Lucia  2010  P119214  IQ � Fiscal Sustainability DPL  Iraq  106 APPENDIX B METHODOLOGY 2010  P116226  MX Social Protection in Health  Mexico  2010  P112700  Georgia: DPO �1  Georgia  2010  P117608  RY SWF Institutional Support Project  Yemen, Rep. of  2010  P115638  Social Safety Nets DPC  Pakistan  2010  P113421  Afghanistan Pension Admin and Safety Net  Afghanistan  2010  P117023  JO�Recovery Under Global Uncertainty DPL  Jordan  2010  P117000  GD Economic and Social DPL  Grenada  2010  P117665  Fiscal, Social and Financial Sector DPL  Croatia  2010  P115732  Safety Net & Soc. Sector Reform Program  Latvia  2010  P113220  ET Productive Safety Nets (APL III)  Ethiopia  2010  P117949  RY:Social Fund for Development IV  Yemen  2010  P115592  HN� Social Protection  Honduras  2010  P115700  Belarus Development Policy Loan  Belarus  2010  P115626  AM DPO�1  Armenia  2010  P111546  KE�Youth Empowerment Project (FY10)  Kenya  2010  P116774  Social Safety Nets & Employment Support  Bosnia�Herzegovina  2010  P112625  Economic Recovery DPO  Moldova  2010  P116972  DO�1st Perform.&Accbilty SocSctrs DPL  Dominican Republic  2010  P106063  3A�West Africa Fisheries � Phase 1  Africa  2010  P107416  GT Expanding Opport. Vulnerable Groups  Guatemala  2010  P114987  SB � Rapid Employment Project  Solomon Islands  2010  P121686  LR � Youth, Employment, Skills Project  Liberia  2010  P108759  Public Expenditure DPL  Serbia  2010  P115052  ID PNPM�Rural III  Indonesia  2010  P082144  PH�Social Welfare and Development Reform  Philippines  2010  P118036  SV Sustaining Social Gains  El Salvador  2010  P116951  Public Expenditure Crisis DPL  Bosnia�Herzegovina  2010  P115247  GH�Social Opportunities Project (FY10)  Ghana  2010  P119825  Mongolia Multi�Sector TA  Mongolia  2010  P115145  DO Prog.PubFinance & Social Sector DPL  Dominican Republic  2010  P117440  SV�Income Support and Employability  El Salvador  2010  P112495  REGE DPL  Turkey  2010  P106708  CO Social DPL  Colombia  2010  P117203  KHSmallholder Ag & Social Protection Sup  Cambodia  2010  P121052  SL�Youth Employment Support (FY10)  Sierra Leone  107 APPENDIX B METHODOLOGY 2010  P120631  KM�Emergency Crises Response Project  Comoros  2010  P116984  Macedonia DPL 1  Macedonia, FYR  2010  P115318  DRC�Street Children Project (FY10)  Congo, Dem. Rep.  2010  P114822  SC DPL  Seychelles  ANALYTICAL AND ADVISORY ACTIVITIES (AAA) For the AAA portfolio review, IEG downloaded from an internal Bank database all the eco- nomic and sector work (ESW) and technical assistance nonlending projects approved be- 5 tween FY00 and FY10 that had been assigned codes 54 and 91. A total of 297 activities were identified, 70 percent of them were ESW and the other 30 per- cent of activities were nonlending technical assistance. For both ESW and nonlending technical assistance activities, the dollar amounts down- loaded correspond to expenditures and not commitments. The database only registers ex- 6 penditures. As of August 10, 2010, 10 percent of the ESW projects were active. Likewise, as 7 of that date, 34 percent of the nonlending technical assistance projects were active. TRUST FUNDS The exercise started with the objective of mapping all trust funds that contributed to SSNs 8 operations and AAA. None of the Bank systems permit the downloading of aggregate data 9 on trust funds by thematic code. IEG also consulted with the regions and the networks and the information (coded 54) was not available. IEG used the database to obtain partial data on trust funds. By this means it was possible to obtain information on Recipient-Executed Trust Funds stand-alone grants (N = 56) and Bank-Executed trust fund AAA activities (included in the AAA analysis). IEG was not able to obtain data on trust funds used for project preparation and supervision. For Recipient-Executed Trust Funds, IEG downloaded from the database projects under codes 54 (SSNs) and 91 (Global Food Crisis Response Program). In terms of donors, the Food Price Crisis Response trust fund financed 50 percent of the activities, followed by 10 11 IDF (30 percent) and other donors (20 percent). Eighty percent of the Food Price Crisis Response activities had IBRD as a donor. II. Incidence Analysis of Bank SSN Lending Bank lending was taken from an internal database. The top 10 borrowers over the decade were determined by aggregating Bank lending to each country. Poverty statistics were drawn from the World Development Indicators. Three poverty lines were considered in the analysis: Poverty Headcount under $1.25 a day (PPP), Poverty Headcount under $2 day (PPP), and, the national poverty line. The years considered in the analysis were 2000–07. 108 APPENDIX B METHODOLOGY IEG used the Poverty Headcount under $2 a day because the series for the national poverty line were incomplete and the Poverty Headcount under $1.25 measures extreme poverty. SSNs are targeted to the poor, not only the extreme poor. Two methods were used to derive the number of poor people:  Method 1: The number of poor people corresponds to the average number of poor people during the corresponding period (2000–07).  Method 2: The number of poor people corresponds to the number of poor in the most recent available year during the corresponding period (2000–07). The final analysis includes numbers of poor people from the average of the poor population at $2 a day (PPP) in the corresponding period 2000–07. ANALYSIS OF COUNTRY TAKE-UP OF BANK SSN LENDING Of the 130 Bank client countries during the period 2000–10, 82 countries received Bank lend- ing for SSN while 48 countries did not. We did a probit regression to determine which of the following country characteristics are strongly correlated with Bank support for SSN be- tween 2000 and 2010:  Income, proxied by log of GDP per capita in 2000. Alternatively, we also used country income categories (upper middle income, lower middle income, and lower income) to define country income  Location, represented by six regional dummies for East Asia and Pacific, South Asia, Latin America and the Caribbean, the Middle East and North Africa, Europe and Central Asia, and Africa  SSN borrowing in 1990–99. Alternatively, we also used a binary variable for SSN take up in 1990–99 (1 = Yes, 0 = No) as well as share of SSN in total country borrowing  Scale of Bank borrowing in 1990–99. Based on share of country borrowing in total bank lending during 1990–99, countries were divided into four categories: (a) smallest borrower, corresponding to the first quartile (b) second smallest borrower, corres- ponding to the second quartile (c) second largest borrower, corresponding to the third quartile, and (d) largest borrower, corresponding to the fourth quartile. Alter- natively, we also used share of country borrowing in total Bank lending.  Crisis. To measure if the country has had a crisis in 2000–10, we first demeaned the real GDP growth series for 1990–2010. Countries whose GDP growth was more than 2 standard deviations below the mean were defined as having a “crisis� or economic shock. This is however an imperfect and noisy measure of crisis.  Overall CPIA score during the period 2000–10. The results of the probit regression are given below (Table B.4). From the correlation analy- sis, it can be inferred that:  Countries with high income are less likely to have a Bank-supported SSN program. Compared to low income countries, upper-middle and lower-middle-income coun- tries are less likely to have a Bank-supported SSN program. 109 APPENDIX B METHODOLOGY  Africa, East Asia and Pacific, Europe and Central Asia, and the Middle East and North Africa are less likely than Latin America and the Caribbean to have a Bank- supported SSN operation.  12 Small borrowers are less likely to have Bank-supported SSN operations.  Crisis, prior SSN borrowing and country political and institutional capacity is not correlated with country take-up of Bank-supported SSN lending. Other very important factors correlated with SSN take-up, for example, poverty headcount or level of inequality in the country, or non-Bank funded SSN program, cannot be included because of lack of data availability for many countries in the sample. After looking at some of the correlates of country take-up of Bank-supported SSN lending, our next step was to estimate which of the above factors are most strongly associated with the volume of SSN lending. We define SSN lending several ways, in terms of value and share of SSN borrowing in total Bank lending to the country. However, we observe the de- pendent variable for a restricted, nonrandom sample (as the analysis above shows, there are statistically differences between countries that borrow for SSN programs form the Bank ver- sus those that do not). In an attempt to address this selection bias, we perform a Heckman selection correction, which uses information from non-takers, to adjust the estimates of the parameters in the volume of SSN lending regression. The results of this analysis are pre- sented in table B.2. This analysis suggests that:  Countries that have had a crisis in 2000–10 are more likely to get a higher volume of 13 SSN lending.  Big borrowers are more likely to have a higher volume of SSN lending than smaller borrowers.  Past volume of SSN lending is correlated with current volumes of SSN lending. So higher SSN lending in 1990–99 means that a country is more likely to have higher 14 SSN lending in 2000-2010.  . There are no regional and income effects. As in the case of SSN take-up, several important determinants of the volume of SSN lend- ing, for example, number of poor people or the level of inequality, cannot be included be- cause of data limitations. 110 APPENDIX B METHODOLOGY Table B.2. SSN Take-Up Analysis Table 1: Heckman Selection Model � Results Particpation Equation (Heckman Selection Model) Outcome Equation (Heckman Selection Model) SSN Take Up  SSN Take Up  SSN Take Up  SSN Take Up  SSN Take Up  Volume of  Volume of  Volume of  Share of SSN  Share of SSN  (1=Yes, 0=No) (1=Yes, 0=No) (1=Yes, 0=No) (1=Yes, 0=No) (1=Yes, 0=No) SSN Lending SSN Lending SSN Lending  in Total Bank  in Total Bank  (Log) Lending to  Lending to  Country Country (Log) Model (1) Model (2) Model (3) Model (4) Model (5) Model (1) Model (2) Model (3) Model (4) Model (5) Rea l GDP per Ca pita (Log) �0.617*** �0.560*** No �0.614*** �0.532*** �20.20 24.99 No 0.00186 0.000114 (0.1577) (0.1699) (0.1578) (0.1693) (62.3643) (58.5445) (0.0132) (0.0456) Region: Africa �1.437*** �1.422*** �1.174*** �1.442*** �1.449*** �126.2 �70.82 �0.162 �0.0301 �0.0705 (0.4610) (0.4822) (0.4501) (0.4615) (0.4840) (165.8229) (167.9614) (0.8369) (0.0360) (0.1365) Region: Ea st As ia  & Pa cific �1.742*** �1.516*** �1.728*** �1.750*** �1.537*** �167.0 �30.80 �0.602 �0.0149 �0.0534 (0.5430) (0.5319) (0.5580) (0.5443) (0.5369) (203.0946) (188.3932) (1.0886) (0.0417) (0.1503) Region: Europe & Centra l Asia �0.768* �0.770* �0.482 �0.766* �0.767* �132.0 �111.0 0.237 �0.0199 �0.00332 (0.4403) (0.4531) (0.4221) (0.4398) (0.4492) (94.0403) (95.9380) (0.4472) (0.0212) (0.0770) Region: Middle Ea st & North  �1.315** �1.487** �1.341** �1.334** �1.602*** �222.5 �114.8 0.131 �0.0606 �0.0693 Africa (0.5491) (0.5825) (0.5594) (0.5466) (0.5748) (180.9607) (188.7567) (1.0428) (0.0378) (0.1568) Region: South As ia �0.608 �0.403 �0.587 �0.614 �0.399 �218.7 �116.9 �0.190 �0.0170 0.0463 (0.7582) (0.7844) (0.7448) (0.7601) (0.7905) (150.0517) (130.7198) (0.7809) (0.0327) (0.1061) Cris is 0.471 0.233 0.479 0.477 0.326 188.1** 158.5* 0.187 0.0237 0.00134 (0.4501) (0.4723) (0.4480) (0.4480) (0.4565) (94.6157) (82.3041) (0.4874) (0.0200) (0.0651) Sha re of Country Borrowing in  79.88** No 82.21*** 83.64*** No 27908.3*** No 197.7*** 3.568* No Tota l Ba nk Lending (33.5457) (29.5820) (30.1553) (9845.4150) (47.8290) (1.8598) Sha re of Country Borrowing in  �805.6** No �788.3** �830.2** No �261836.7** No �2117.0*** �43.76* No Tota l Ba nk Lending Sqa ured (361.3896) (338.1567) (347.6431) (1.10e+05) (574.5575) (22.7761) SSN lending (1997�1999) 0.00179 0.00493 No No No 0.879 1.738*** No No No (0.0083) (0.0062) (0.5456) (0.4298) Borrower: Smalles t No �0.635 No No �0.826** No �323.8*** No No �0.452*** (0.4127) (0.3844) (120.8094) (0.1007) Borrower: Second Smallest No �0.553 No No �0.709* No �259.3** No No �0.189** (0.3866) (0.3626) (102.9457) (0.0853) Borrower: Second Larges t No 0.396 No No 0.263 No �282.1*** No No �0.100* (0.4284) (0.4123) (73.9691) (0.0535) Lower Middle Income Country No No �0.708* No No No No 0.0378 No No (0.3670) (0.5686) Upper Middle Income Country No No �1.407*** No No No No �0.312 No No (0.4484) (0.9067) Sha re of SSN in Total  Bank  No No 0.0232 0.119 2.519 No No �13.26 0.109 �0.225 Lending to Country (1990�1999) (6.9225) (6.8290) (6.8635) (10.9995) (0.4172) (1.4651) CPIA 0.394 0.334 0.317 0.389 0.310 No No No No No (0.2575) (0.2650) (0.2522) (0.2561) (0.2621) Consta nt 3.833*** 4.158*** 0.302 3.828*** 4.217*** 119.9 166.8 2.860*** 0.00770 0.606* (1.2924) (1.4074) (0.9070) (1.3039) (1.4180) (402.5506) (430.6981) (0.5533) (0.0881) (0.3453) Mills � Lambda 118.7 15.44 �0.556 0.0574* 0.0662 (169.1128) (185.7133) (1.0385) (0.0341) (0.1492) N 130 130 130 130 130 ***1% significance level **5% significance level *10% s igniffica nce level Number of obs: 130, Cens ored obs: 48, Uncensored obs : 82 111 APPENDIX B METHODOLOGY III. Countercyclical Analysis of SSN Lending To assess how responsive Bank lending was to changes in country growth rates, IEG ex- amined annual growth rates over the past decade for each of the top 10 borrowers as well as annual Bank SSN lending to that country. The lending commitments were used for the IEG portfolio review. Commitments were counted in the year disbursement started and not along the project life. Annual GDP growth rates were taken from the World Development Indicators for countries that have had SSN operations in 2000–10. IV. Explaining the Performance of SSN in the Portfolio Review: Statistical Analysis The purpose of the regression analysis is not to claim attribution, but to determine which factors are consistently associated very strongly with project performance as measured by project Outcome ratings. The analysis is limited by issues such as small sample size (n = 71) and some potential sources of bias that include the lack of data on several aspects of per- formance (for example, quality of implementation) and possible two-way associations be- tween dependent and independent variables, omission of relevant variables and measure- ment error. 15 The sample of analysis consists of closed SSN operations, which were approved between FY00 and FY09, and which are closed and have a completed ICR Review. Of the 92 SSN op- erations that meet these criteria, only 71 operations had an SSN objective. Because IEG rat- ings are objectives based, the analysis was restricted to the 71 projects with a stated SSN ob- 16 jective. Of the 71 operations in our sample, 29 are DPLs, 8 are ERLs, and 34 are other investment loans. In terms of regional distribution, LCR has the highest concentration (27 operations), followed by Africa (18), Eastern Europe and Central Asia (15), South Asia (6), 17 Middle East and North Africa (5). The projects in the sample do not always consist of only 18 one SSN intervention: 43 operations are host to between two to six instrument types; of these 22 operations implemented two instruments while 14 operations implemented three different instruments. SSN operations tend to involve multiple sectors (49 percent of projects are hosted by the Social Protection Board) and have multiple objectives (SSN objectives and non-SSN objectives). The overall IEG Outcome rating is an average rating of all the project development objectives (SSN and non-SSN), and considers three different aspects: achievement of objectives (efficacy), effi- ciency, and relevance of objectives and design. IEG then analyzed the efficacy (which measure achievement of objectives) rating of the specific SSN objectives. IEG separated the objectives into two categories: SSN objectives and “other objectives� (non- SSN objectives). Of the 92 closed projects evaluated a total of 71 projects had separate objec- tive and separate IEG ratings for efficacy (see tables B.3 and B.4). 112 APPENDIX B METHODOLOGY Table B.3. Projects of SSNs with Efficacy Ratings Efficacy rating Number Total Rating High-substantial SSN objective 46 71 64% High-substantial average “other� objectives 43 71 60% Source: IEG ICR review database.FY00–10. Table B.4. Intersection of Efficacy and Overall Outcome Rating (percent) Efficacy rating Overall outcome rating- moderately satisfactory and higher SSN objective Other objectives High-substantial 69 67 Source: IEG ICR review database.FY00–10. 19 The efficacy of SSN objectives was rated substantial or higher in 64 percent of the cases. “Other� non-SSN objectives were rated substantial or higher in 60 percent of the cases. When doing a cross check between the efficacy rating and the overall Outcome rating SSN 20 objectives were rated substantial or higher on efficacy and Moderately Satisfactory and higher on Outcome in 69 percent of the cases. “Other� non-SSN objectives were rated sub- stantial or higher and Moderately Satisfactory and higher in 67 percent of the cases. The dif- ference between these two ratings averages was not statistically significant in both methods. Hence, it can be assumed that the achievement of the overall outcome rating is not princi- pally driven by other non-SSN objectives. The dependent variable in the regression analysis explaining performance on projects sup- porting SSNs is the project outcome rating, as assigned in the ICR Review, by IEG. This de- pendent variable is defined in two ways: First, on an ordinal scale as originally assigned by 21 IEG. None of the 71 SSN operations in the sample was rated Highly Unsatisfactory, and 34 operations were rated Satisfactory, followed by 25 operations with a Moderately Satisfacto- ry rating. Because only 3 operations were rated Unsatisfactory and 7 that were rated Mod- erately Unsatisfactory, outcome ratings 2–3 were combined to form one category, outcome rating 4 remained a distinct category, while outcome ratings 5–6 were combined to form the 22 third category. Alternatively, the dependent variable was also defined as a binary variable, 23 that is, a rating of 5–6 (that is, Satisfactory or Highly Satisfactory) was assigned a 1 while all others were assigned a 0. The 36 projects that got a 1 were defined as Highly Perform- 24 ing. The correlates used in the analysis include type of lending instrument, number and type of SSN interventions, quality of results framework, regional effects, SSN focus, duration, and technical assistance. Because the sample size is relatively small and as there is insufficient data available, a full set of covariates cannot be included in the regressions. However, the inclusion of explanatory variables in the regression analysis is informed by the portfolio re- view and the latent underlying relationship with project performance. A summary of some key cross-tabulations between these variables and project performance is presented below in table B.5. 113 APPENDIX B METHODOLOGY Table B.5. Data Description and Summary No. of high performing No. of non-high operations performing operations Variables and definitions (outcome rating >4) (outcome rating <=4) DEPENDENT VARIABLE Project Performance Rating (Outcome) 36 35 INDEPENDENT VARIABLES Fast Disbursing Instrument 24 13 Definition: DPL and ERL Cash Transfer Definition: Project at least has a conditional or unconditional cash transfer 19 11 component Outcome Type Measurement 13 16 Definition: The outcome indicators actually measure outcomes not outputs Region South Asia 5 1 East Asia and Pacific 0 0 Eastern Europe and Central Asia 7 8 Latin America and the Caribbean 16 11 Middle East and North Africa 1 4 Africa 7 11 Quality of Results Framework Definition: The proportion of “good� attributes that the results framework has adopted. Good attributes are defined as: Baseline, Impact Evaluation, 14* 7 Linkage of output and outcomes to Development Objectives, Outcome and output targets are clearly articulated, are time-bound and specify target group Technical Assistance Definition: Project has a technical assistance component or technical 33 29 assistance activities SSN Focus 8 10 Definition: The project has a primary focus on SSN Number of SSN Interventions 2.2** 2.0** Definition: Number of functions supported by a project Function 25 25 Definition: Project is geared towards human capital accumulation Loan Size $53 million*** $40 million*** Definition: Commitment Amount in $million Duration 12**** 27***** Definition: Duration is less than or equal to 3 years * Number of projects that had more than 50 percent of “good� attributes incorporated in their results framework. **Average number of instruments per project. ***Refers to average loan size. ****Number of projects whose SSN share was greater than one-third. 114 APPENDIX B METHODOLOGY Regression analysis suggests conditional correlation between the performance rating (Outcome) and the following project attributes: quality of results framework and project duration:  On average, projects with SSN components that are of shorter duration performed better than programs with a longer duration. This is true for short investment loans 25 compared with longer ones within their own cohort.  On average, projects measuring outcomes are less likely to have higher performance compared to projects that measure outputs. Specifically, there is no statistically sig- nificant difference between the performance rating of fast disbursing loans and in- vestment loans if both measure outputs. Although investment loans measuring out- comes are correlated with lower performance rating compared to investment loans measuring outputs, fast disbursing instruments measuring outcomes do not have ratings that are statistically different from fast disbursing instruments measuring outcomes. These findings were robust to inclusion of different controls (see tables 26 B.6a and b, Model 2).  In some specifications regional dummy variables showed up with statistical signific- ance. However, no regional effect was consistently significant across different speci- fications. Also, the relationship between outcome ratings and size of loan, number of SSN instruments, type of SSN instrument, technical assistance, SSN functions in the project and Country Policy and Institutional Assessment were not statistically signif- icant. 115 APPENDIX B METHODOLOGY Table B.6a. Regression Results 116 APPENDIX B METHODOLOGY 117 APPENDIX B METHODOLOGY V. Results Framework Analysis This analysis was carried out as a background exercise to help explain the performance of SSN operations. Specifically, it attempted to answer the following questions. First, what are SSN operations measuring, and second, how well are they measuring their expected results? The analysis was carried out on 50 percent of the SSN portfolio, randomly selected from the 244 SSN operations and stratified by fiscal year of approval. ASSESSMENT OF PROJECT DEVELOPMENT OBJECTIVES The analysis classified development objectives by type into: access to public services and infrastructure (schools, health centers, roads, and so forth)—local public goods; access to economic opportunities (microcredit, active labor market programs, skills development, etc.); temporary income support (unconditional transfers, public works and temporary em- ployment, CCTs, wage subsidies, and so forth); human development outcomes (improved education, health, and nutrition, HIV status, and so on); community empowerment and ca- pacity building (nongovernment) government institutional capacity building; quality of ser- vices/ service delivery; Fiscal objectives/efficiency; Address effects of food, fuel, and finan- cial crises; and Other. IEG followed the same classification used by the Bank’s Human Development Network assessment of projects with SSN components, “Results Readiness in 27 Social Protection and Labor Operations.� In most cases, operations addressed more than one area of focus, and the analysis classified them according to all of the areas they ad- dressed. As a result, the classification is not mutually exclusive, so the sum of the classifica- tion by focus area is greater than the number of operations. Development objectives were also examined to determine if they were outcome driven (by clearly describing the specific development change that was expected as a result of the op- eration); whether the objectives mentioned the poor and vulnerable and whether, in addi- tion, they mentioned a specific target group within the broader classification of “poor and vulnerable,� such as poor-at-risk youth, poor rural communities, chronically food-insecure households, or poor uninsured mothers and children. ASSESSMENT OF KEY PERFORMANCE INDICATORS Key performance indicators (KPIs) were assessed to determine whether SSN operations had KPIs that were outcome-driven, measurable by having a target to achieve, mentioned the target population, were time-bound, and had baseline data at the time of project appraisal. For purposes of classification, outcome and intermediate outcome indicators were classified as indicators that reflect changes in the behavior or condition of the target population as a result of the project in the short and long run (for example, outcome indicator: percent in- crease in an average individual beneficiary’s net annual earnings; intermediate outcome in- dicator: number of low income workers employed as a result of the project). Output indica- tors were classified as project activities or outputs that would lead to the achievement of the outcomes (for example, number of employment subprojects approved under the project). 118 APPENDIX B METHODOLOGY In addition, IEG assessed whether project KPIs measured what could be considered SSN objectives. An operation was considered to have met this condition if it contained at least one KPI designed to measure an SSN objective, that is, to measure progress toward:  Government institutional capacity building: In social protection, human development, poverty measurement, and addressing risk through social protection interventions.  Access to public services: Within the social protection system (SSNs, pensions, labor market interventions) and also related to human development.  Quality of services/service delivery: Quality of health services, educational services, and social services in general.  Human development outcomes: School attendance, achievement, and health outcomes.  Temporary income support: Increase in income and/or consumption.  Addressing crisis: Improve the ability to mitigate the negative effects of shocks.  Fiscal/efficiency: Fiscal support or fiscal efficiency of SSN programs. ENSURING QUALITY AND CONSISTENCY IN CODING Since classification of development objectives and KPIs involve some degree of judgment, a team of two people judged each project. One evaluator was a team member thoroughly fa- miliar with the SSN portfolio, the other was a retired human development Lead Operations Specialist thoroughly familiar with results frameworks. Any doubts about classification were raised with the evaluation task manager and determination was made. VI. SSN Case Studies The evaluation undertook 30 in-depth country case studies to understand the nature of the Bank’s overall work (not just lending projects) on SSNs over the past 10 years and the re- sults in terms of SSN development in client countries. The large number of case studies and the random selection enabled an examination of a wide range of countries with different SSN systems and levels of Bank engagement. COUNTRY SELECTION To obtain a wide range of countries, some in which the Bank had been heavily engaged over the decade and some in which it had very little engagement, the sampling universe was all client countries of the Bank. Countries with a population of less than 1 million were ex- cluded. IEG randomly sampled 25 countries stratified by region. Additionally, five countries (Colombia, Ethiopia, Georgia, Indonesia, and Jamaica) were purposefully selected for in- depth case studies for two reasons: (1) the Bank has had long-standing engagement on SSNs in these countries and IEG wanted to learn from these experiences, and (2) the Bank sup- ported projects of SSNs that recently closed and could be subjected to an in-depth ex post project evaluation (Project Performance Assessment Review, PPAR). The regional distribu- tion of the final sample included seven counties in Latin America and the Caribbean; five in 28 Africa, East Asia and Pacific, and Europe and Central Asia; and four in the Middle East and North Africa and South Asia (table B.7). 119 APPENDIX B METHODOLOGY Table B.7. List of 30 Country Case Studies, by Region AFR EAP ECA LCR MNA SAR Cameroon Cambodia Albania Argentina Algeria India Ethiopiaab Indonesia a Bosnia-Herzegovina Brazil Iran Nepal Niger Lao Bulgaria Colombiaab Jordan Pakistan Nigeria Malaysia Georgiaa Costa Rica Yemen Sri Lanka Tanzania Philipp ines Moldova Guatemala Jamaicaab Uruguay Note: AFR = Africa Region; EAP = East Asia and Pacific Region; ECA = Europe and Central Asia Region; LCR = Latin America and the Caribbean Region; MNA = Middle East and North Africa Region; SAR = South Asia Region. a Purposely selected and included field work. b Included PPAR. APPROACH AND SOURCES The data produced by the case studies were based on substantial and in-depth reviews of Bank and non-Bank project and program documents (including Project Appraisal Documents, pro- gram documents, ICRs, ICR reviews, implementation status reports, PPARs, and country strat- egies), research documents, AAA documents, evaluations, and other formal and informal com- munication notes. The analytical material consulted included, among others, poverty assessments, public expenditures reviews, country economic memoranda, poverty and social impact analyses, beneficiary assessments, impact evaluations, country social assistance reviews, and country social protection strategies. In addition, IEG conducted a number of interviews with key Bank staff and mangers who have been involved in the SSN support to the countries (2-4 interviews per country). Each desk-based case study took approximately two weeks to complete while field-based case studies including PPARs took six to eight weeks and those without PPARs took an additional two weeks. The five countries that were purposefully se- lected also involved country visits and extensive interviews with client, other key stakeholders, and development partners and visits to SSN programs sites and local offices. METHODOLOGY The case studies used a detailed 24-page structured questionnaire laying out (a) the country context, (b) the SSN programs in the country, (c) government and Bank objectives for SSNs, and (d) the nature of Bank’s engagement in SSNs. Country-specific questions were posed that helped answer questions raised in the evaluation approach paper related to the relevance and effectiveness of Bank support and the nature of engagement. Thus, the questionnaire also in- cluded sections that linked information to assessment of relevance and effectiveness of Bank support. The final section of the questionnaire included ratings of the relevance and effective- ness of Bank assistance to SSNs and the nature of the Bank’s engagement. The questionnaire included structured qualitative questions and a number of discrete quantita- tive questions enabling assessment of both trends and contextual details. Some questions were factual while others required an assessment using data and evidence to support the assessment. The quantitative questions asked information that could be answered “yes,� “no,� “somewhat,� or “not applicable.� For example, “Is the government’ strategy on SSNs part of a poverty reduc- 120 APPENDIX B METHODOLOGY tion agenda?� or “Did the Bank introduce/pilot any programs?� or “Were benefit levels large enough to make an impact on poverty (such as on poverty gap or severity of poverty etc.)� The answers had to be justified and explained using evidence and data. The quantitative informa- tion was used to determine trends among the countries. This information was assembled into a detailed timeline for each country including country events and Bank engagement through var- ious instruments, including lending, AAA, high-level dialogue, and major workshops. This enabled a broader assessment of whether the Bank’s engagement had been sustained (conti- nuous) throughout the decade, and contributed to determining if Bank’s efforts were strategic (selected and sequenced to have as large impact possible). For example, it allowed the report to conclude that “In 57 percent of the countries assessed via case studies the Bank engaged in a sustained fashion on SSNs throughout the decade whereas for the rest it did not� (from chapter 5). The case study template also included a set of questions that rate the performance of the Bank- supported SSN programs and the nature of Bank’s support on a scale from 1 to 5 where 5 indi- cated the most desirable outcome. For example: “To what extent was Function 2 (encourage human capital investment for the poor and vulnerable) a relevant issue for SSNs in the coun- try?� (1 = Not at all, 2 = Very little, 3 = To some extent, 4 = A great extent, 5 = A very great ex- tent). Again, ratings were justified and explained using evidence and data. Thus, for each coun- try, each rating question at the end of the template referred back to sets of relevant factual/descriptive information throughout the questionnaire (sections a-d described above). This information was drawn upon in determining the ratings (as relevant for each country con- text). To ensure consistency across the countries and provide benchmarks for the case study rat- ings, the ratings were underpinned by information obtained through a number of factual ques- tions. Table B.8 lists the benchmarks used to determine ratings for relevance of SSNs reported in chapter 4. Table B.8. Benchmarks Used for Relevance Ratings in Chapter 4 Function Benchmarks Function 1  Chronic poverty or chronic food insecurity exists among specific, identifiable population groups for whom special efforts are needed as they are not being reached by other programs  Using SSNs to address specific groups among the chronically poor is cost effective compared to other pro-poor options  Mechanisms/systems/institutions for implementing SSNs to address chronic poverty or chronic food insecurity exist or could be put in place (feasibility). Function 2  The poor underinvest in human capital,  There are demand constraint to human capital investment,  The poor have access to human development quality infrastructure,  There is sufficient administrative capacity and intersectoral coordination in order to monitor and enforce conditionalities, where applicable. Function 3  The country and the poor are vulnerable to systemic shocks (for example, climatic, economic)  There is large transient poverty or seasonal food insecurity  Existing SSN mechanisms do not protect well against shocks (for example, they cannot can scaled up as necessary to identify newly vulnerable and/or expand benefits when a shock happens). Function 4  The poor are vulnerable to idiosyncratic shocks (for example, health, losing jobs and income)  There is large informality and social insurance systems do not adequately protect the poor, losing investments)  Existing SSN mechanisms do not well protect against shocks (e.g. they cannot absorb people in a timely manner when a shock happens, or existing benefits are too low to allow people to save and withdraw from these when a shock happens). Function 5  The country is undertaking broader economic reforms that will have distributional implications and hurt the poor (even if only temporary)  Existing SSNs do not adequately mitigate the consequences of reform  There are weak legal mandates to protect poor workers that are laid off (for example, if informality is high)  Mitigation programs are politically necessary in order for reforms to be feasible without causing riots, and so forth. 121 APPENDIX B METHODOLOGY QUALITY CONTROL AND CONSISTENCY ACROSS CASE STUDIES Five IEG staff and consultants undertook the 30 case studies. At the start of the work, IEG organized a two-day workshop for the team to review the case study questionnaire and get common understanding of what information was needed to answer the questions and come to a conclusion on the quantitative and rating questions. Each draft case study write-up was vetted by the evaluation task manager as well as two of the other case study authors for consistency and evidence base. Where questions arose about relative rankings, discussions were held to compare different country experiences (much like how Poverty and Social Im- pact Analysis ratings are determined within the Bank’s operational regions). USE OF DATA At the end of the process the team had gathered a broad set of very detailed information about SSNs in the 30 countries. The case study information fed into the evaluation though detailed analysis by themes (such as technical design, risk and sustainability, links to the growth agenda, and political economy factors of SSNs). IEG compiled the analytical pieces and triangulated evidence from quantitative and qualitative case study answers as well as information from background papers, the portfolio review, and findings from IEG’s parallel impact evaluation work. VII. Food, Fuel, and Financial Crises Survey PURPOSE OF THE SURVEY The recent food, fuel, and financial crises underscored the urgency of appropriate crisis res- ponses from governments and the World Bank to avert major escalations of global poverty. The aim of the survey was to obtain feedback from Bank staff working in different regions regarding not only countries’ experiences with SSNs in the context of the crises, but also Bank SSN assistance to countries in response to the crises An electronic survey was sent to each Social Protection sector manager, who then assigned the survey to a staff member responsible for each country in their region. Responses came directly to IEG and thus the responses remained confidential. Results are available by re- gion, but confidentiality is maintained by not releasing country-specific data. The sector manager followed up several times with staff to ensure survey completion; this process led to a high response rate. SURVEY RESPONSE The survey response rate is defined as the number of actual survey responses divided by the total number of countries as determined by lending classification in each region. The World Bank data group classifies countries by lending category into IBRD, IDA, and blend. IDA countries are those that have a per capita income in 2009 of less than $1,165, and lack the creditworthiness necessary to borrow from IBRD. Blend countries are eligible for IDA loans because of their low per capita incomes but are also eligible for IBRD loans because they are financially creditworthy. 122 APPENDIX B METHODOLOGY The response rate for the survey was 47 percent (Bank staff in 65 countries responded to the survey). Europe and Central Asia and South Asia had the highest response rates and the Africa Region the lowest. In order to assess if the response for the survey was representative of all Bank clients, we compared the characteristics of those countries that responded with those that did not. IEG regressed the response rates (1 = responded, 0 = did not respond) for countries against regional dummies and precrisis variables such as the log of GDP per capi- ta in 2007 and SSN take-up in 2000–07 (1=the country had an SSN program supported by a Bank project). IEG found that countries that did not respond were more likely to be in Africa and were al- so less likely to have a Bank-supported SSN operation. There were no significant differences between response rates of other Regions (South Asia, East Asia and Pacific, Europe and Central Asia, and the Middle East and North Africa) compared to Latin America and the Caribbean; nor were there any statistically significant differences between the income level of respondents and non-respondents. Within Africa, countries that responded were more likely to have an SSN program supported by the Bank but they were not different in income level from countries that did not respond. In conclusion, we found that the survey responses under represent African countries that do not have Bank supported SSNs. To correct for this response bias, IEG reweighted the survey responses with the inverse of the predicted values from the probit regression and found that survey responses were not sensitive to the use of weights to balance the sample due to differences in response rates. Therefore, all numbers presented for the survey are based on the original unweighted res- ponses. SURVEY ANALYSIS Survey data were analyzed in aggregate and by region and LIC-MIC. Aggregate results are presented below. 123 APPENDIX B METHODOLOGY 124 APPENDIX B METHODOLOGY 125 APPENDIX B METHODOLOGY 126 APPENDIX B METHODOLOGY 127 APPENDIX B METHODOLOGY 128 APPENDIX B METHODOLOGY 129 APPENDIX B METHODOLOGY 130 APPENDIX B METHODOLOGY 131 APPENDIX B METHODOLOGY 132 APPENDIX B METHODOLOGY 133 APPENDIX B METHODOLOGY 134 APPENDIX B METHODOLOGY VIII. Evidence and Lessons Learned from Impact Evaluations on SSNs Recent meta-analyses of impact evaluations of safety nets have captured rich evidence re- garding the results of individual interventions. However, they mostly focused on a few in- terventions with abundant data. Moreover, very few questioned the effects of safety nets beyond their immediate outcomes, the efficiency of programs, or the contribution of differ- ent aspects of programs to impacts. For its evaluation, IEG reviewed a comprehensive pool of impact evaluations on SSN interventions to address those questions and identify evidence across program types and development outcomes. An exhaustive search of the literature was conducted, and four separate filters were used to select a pool of 137 impact evaluations that had a development focus, applied rigorous me- thods (including the use of a credible counterfactual), demonstrated robust findings, and were consistent in their results. The papers reviewed cover 32 developing and transition countries in five regions. The 56 programs evaluated span 10 intervention types, but the ma- jority are conditional cash transfers. The evaluations explore a wide range of outcomes across multiple dimensions, including health, education, nutrition, consumption, income, poverty, labor, economic activities, risk-coping behaviors, and indirect effects (such as mar- riage and fertility decisions, private transfers). Less than half of the evaluations used experimental design, while the remainder used such quasi-experimental methods as matching, difference-in-differences, instrumental variables, and regression discontinuity design to construct the counterfactuals. The data, in most cases, were collected no more than two years into the program’s implementation, which limits the ability of the studies to explore long-term effects. The group of papers considered for this study was compiled through an exhaustive online search of both completed and ongoing impact evaluations of SSNs, drawn from the follow- ing sources: 135 APPENDIX B METHODOLOGY  The World Bank’s impact evaluation databases, including the Africa Impact Evaluation Initiative, the Development Impact Evaluation, the Spanish Impact Evaluation Fund, the Social Protection Publication Database, and the Poverty Impact Evaluation Database.  Electronic databases of academic journals on economics, health and nutrition, and social policy (mostly JSTOR and Google Scholar).  Websites of institutions involved directly in impact evaluations, including the Abdul Latif Jameel Poverty Action Lab, the International Food Policy Research Institute, the Innovations for Poverty Action Lab, the International Initiative for Impact Evaluation, the International Policy Centre for Inclusive Growth, the In- stitute of Development Studies, and other evaluation groups within international organizations.  Web sites featuring discussion paper series of research centers (in academia, in- ternational organizations, and think tanks) as well as publications with the pro- ceedings of relevant conferences.  Reference lists of evaluation reviews and related studies; and recommendations made by people familiar with the impact evaluation literature on SSNs inside and outside the Bank. TIME FRAME AND SCREENING PROCESS The review looked at papers published in or after 1999 and papers in progress. This period was chosen because a preliminary search revealed that the bulk of impact evaluations (94 percent) were carried out during this time. In order to confine the meta-review to rigorous impact evaluations of specific safety net programs the studies identified in the search process were subject to four filters:  Filter 1: Development focus, relevance, and consistency. In order to focus on the role of SSNs in development policy, only studies of developing and transition coun- tries were included. Papers that did not explicitly evaluate an SSN program or disen- tangle the impacts of safety net transfers from those of other interventions linked to broader social policies were excluded as well. This filter also discarded evaluations of programs that had a safety net objective but were neither targeted nor imple- mented in line with their original evaluation design.  Filter 2: Construction of a counterfactual and use of objective measures to estimate changes. This filter selected only those evaluations where changes in specific out- come indicators can be attributed to a particular safety net instrument. A paper has to demonstrate the use of a carefully and credibly constructed counterfactual, that is, a comparison group of nonbeneficiaries that resembles the without-program scena- rio for beneficiaries. An additional criterion for inclusion was the use of objectively measured and comparable indicators of outcomes. An example of an objective measure is the use of standardized test scores—rather than personal perceptions of cognitive abilities—to estimate the effects on cognitive development.  Filter 3: Robustness of the findings. Even the best efforts to eliminate potential biases that could contaminate the comparison between program participants and nonpartici- 136 APPENDIX B METHODOLOGY pants do not necessarily guarantee the legitimacy of the findings. Therefore, a third fil- ter was applied to select only those studies with results that were convincingly robust (not sensitive) to a variety of confounding factors. These factors include changes in econometric specifications and methods, endogeneity issues, characteristics of the population and context under analysis, and implementation aspects of the program.  Filter 4: Final inspection to double check. The studies that passed the three pre- vious filters were reviewed independently by different members of this team to en- sure that only studies that demonstrate relevance and technical rigor and provide consistent findings were included in the sample. Furthermore, during this stage, pa- pers were scrutinized to retain only the most recent version of the evaluation under analysis and avoid duplication. This screening process narrowed down the initial list of more than 350 published articles and working papers to a selection of 136 impact evaluations. In addition, to take stock of current research, an additional sample of more than 40 papers in progress was considered (and screened with the first two filters) to produce a final sample of 24 ongoing evaluations. MAIN FINDINGS OF THE REVIEW Despite the complexity of integrating the findings of numerous impact evaluations across a variety of policy instruments, implementation contexts, and intermediate and final out- comes, a number of patterns emerge:  In the short term, safety nets are found to improve households’ investments in hu- man capital, immediate consumption, current economic activities, and abilities to mi- tigate the negative effects of shocks.  The literature on long-term outcomes is less abundant, but the limited evidence sug- gests that the income and consumption gains are maintained, probably because of a positive effect on educational attainment and productive investments.  Impact evaluations rarely investigated the indirect effects of safety nets, but those that did found more positive than negative results  Many efforts have been devoted to understanding the distribution of program im- pacts across beneficiaries.  Little work has been done to investigate the balance of benefits versus costs and how program designs, implementation processes, and local contexts affect impacts. IX. Assessing the Medium- and Long-Term Impacts of SSN Programs Although there is significant evidence showing that SSNs can have positive immediate im- pacts on school participation, little is known about their long-term impacts on human capi- tal and related human welfare indicators. In the particular case of CCTs, evidence indicates that these interventions, aligned with their conditions, increased school enrollment and at- tendance in the short term.These impacts are expected to serve as inputs to more human capital accumulation, but the evidence of such connection is scarce. 137 APPENDIX B METHODOLOGY In an effort to help fill this gap, IEG undertook two new impact evaluations. One of the stu- dies assessed the Punjab Female School Stipend Program (FSSP) in public girls’ schools in Pakistan. The other focused on the Familias en Acción program for boys and girls in Colom- bia. DO CONDITIONAL CASH TRANSFERS LEAD TO MEDIUM-TERM IMPACTS? EVIDENCE FROM A FEMALE SCHOOL STIPEND PROGRAM IN PAKISTAN The Program The FSSP was designed to improve educational attainment among girls and decrease gender gaps at the middle school level (grades 6–8). The program targeted the 15 districts with the lowest literacy rates to address their educational disadvantages. Launched at the end of 2003, the program provides quarterly subsidies (around $10) to the families of girls enrolled in middle school with the condition that they attend at least 80 percent of classes. In 2006, the stipend was extended to girls enrolled in high school (grades 9–10). By 2007, 245,000 girls were covered by the program in middle school. Objectives The impact evaluation of the medium- and long-term impacts of the FSSP sought to answer three questions: 1. What are the program’s impacts on outcomes that might affect future productivity of adolescent girls who have been participating in the intervention for up to four years, in- cluding sustained school enrollment, transition and completion of middle school and high school, early labor market outcomes, and marriage and fertility decisions? 2. Does the program affect different beneficiary subgroups differently? 3. Are there positive or negative indirect effects of the program on boys who live in the same household with eligible girls? Empirical Research Design As the education stipend was not allocated randomly but targeted to the most disadvan- taged districts, the evaluation relies on quasi-experimental techniques to construct compa- rable treatment and control cohorts at the school and household levels. At the school level, the annual Punjab public school censuses from 2003 to 2009 were used to create treatment cohorts of girls who were enrolled in stipend schools during the period of the program. These girls should therefore have been exposed to the program for at least one year, and they were compared against the similar group of girls in no-stipend districts during this pe- riod to estimate whether the program impacts on enrollment are sustained. At the house- hold level, the analysis used two waves of the Punjab Multiple Indicator Cluster Survey in 2003 and 2007–08 to construct cohorts of girls in stipend districts who are likely to have at least one year of exposure to the program. These cohorts were compared with counterfac- tual cohorts in no-stipend districts in terms of the likelihood of completing middle school, transitioning to and completing high school, as well as their labor force participation and marriage and fertility decisions. 138 APPENDIX B METHODOLOGY The empirical research design was implemented with two econometric models: difference- in-differences (comparing changes between baseline and follow-up across treatment and control groups) and regression discontinuity design (using the cut-off rate of literacy for dis- trict eligibility). These techniques help address possible selection bias due to time-fixed dif- ferences between stipend and no-stipend districts induced by the non-random placement of the program. In addition, the econometric models include covariates that account for socioe- conomic differences between stipend and no-stipend districts, including differences in school facilities, access, and local infrastructure. The analysis also included models based on samples that that did not condition on enrollment in middle school to check the sensitivity of the findings to the potential issue of selection in participation. Main Findings  The positive short-term impacts of the program appear to be sustained over time given that, four years into the program implementation, adolescent girls in stipend districts are more likely to progress through and complete middle school and work less than four years after the program was implemented.  Although less significant in statistical sense, there is also some suggestive evidence that participant girls delay their marriage by more than a year and have fewer births by the time they are 19.  Girls who are covered by the expansion of benefits given in high school also increase their rates of matriculation into and completion of high school grades.  There is no evidence that the FSSP has adverse indirect effects on similar educational outcomes of boys living in the same household with eligible girls. ASSESSING THE LONG-TERM EFFECTS OF CCTS ON HUMAN CAPITAL: EVIDENCE FROM COLOMBIA The Program The program Familias en Acción was started in 2001–02 as an emergency safety nets program in response to a major economic crisis in the late 1990s. It then became a key element in the country’s poverty reduction strategy through improving children’s education and health. The program provides $7 and $14 monthly for children 7–18 years old in primary and sec- ondary school, respectively, conditional upon 80 percent school attendance. The grants tar- get the poorest 20 percent of households as identified by SISBEN, a proxy-means index that ranks households’ welfare status. The program has been expanding over the years from geographic targeting (small rural municipalities with adequate facilities) to national cover- age. Objectives This study evaluated the impacts of Familias en Acción on high school completion and learn- ing outcomes. In particular, it sought to answer four questions: 1. Does the additional use of educational inputs attributed to the program help participant children become more likely to complete high school? 139 APPENDIX B METHODOLOGY 2. In addition to the inherent positive effects in learning for the children that are brought into school due to the program (“new enrollers�), do those who finish high school and participate in the program perform better in academic tests than similar children not covered by the program? 3. What is the distribution of these long-term impacts across gender and rural and urban areas? 4. What are the indirect effects, if any, of Familias en Acción on similar educational out- comes for nonparticipant young adults living in the same households as participant children? Empirical Research Design The program was not randomly allocated but assigned to the poorest households in specific municipalities, so there may be substantial differences between beneficiaries and non- beneficiaries. To address this, the evaluation used two non-experimental techniques to create counterfactual groups that are comparable to the treatment groups. Propensity score matching This approach was based on the baseline household survey collected in 2002 for the short- term impact evaluation of the program. The units of analysis included eligible families se- lected randomly in stratified comparable treatment and control municipalities. The sample was limited to children 18 years old or younger at baseline who could have finished high school during the period 2003–09. Children in treatment and control areas were matched based on three different specifica- tions of pretreatment observable characteristics (household demographic and socioeconom- ic indicators, and community location and infrastructure) used to predict the probability of treatment. Various kernel techniques and bandwidths were used to define common sup- ports that exclude poor matches between the two groups. Assuming there are no differences in unobservable variables that jointly influence program participation and outcomes, this method eliminates any selection bias in observable characteristics due to the nonrandom targeting of the program and decision to participate. The weighted average difference for each outcome was therefore the average impact of the program. Regression discontinuity design This approach used data from the monitoring and evaluation database of program benefi- ciaries from 2001 to 2009, and from the SISBEN surveys collected between 1994 and 2003 to construct the proxy-means test. These data were merged to create a universe of individuals below and above the threshold of eligibility as defined by the score of the proxy means test. The sample was restricted to a group of treatment municipalities to make it comparable to the sample used in the matching analysis. Although the SISBEN score does not perfectly predict treatment status, those below the threshold are significantly more (around 65–70 percent) likely to participate in the program than those above. Program estimates were therefore obtained from that variation in pro- gram participation as the ratio of the jump in the outcomes to the jump in the probability of 140 APPENDIX B METHODOLOGY treatment at the cutoff (an evaluation strategy known as “fuzzy� regression discontinuity design). The strategy relies on the assumption that children just below and just above the threshold are similar except for their participation in Familias en Acción. Once the comparable treatment and control groups were identified through the two empiri- cal methods, they were merged with the administrative records of ICFES, a national manda- tory standardized test administered to students at the end of high school. Registration to ICFES was used as a proxy for high school completion because more than 90 percent of stu- dents in grade 11 take the test and more than 90 percent of the test takers end up finishing high school. Likewise, the test scores were a proxy for learning at the end of high school as they reflect the student performance in most major subjects. Main Findings  The analyses show that the program helps children, particularly girls and beneficia- ries in rural municipalities, to accumulate more years of education. On average, par- ticipant children were 4 to 8 percentage points (equivalent to 8–16 percent) more likely than nonparticipant children to finish high school.  Regarding impacts on tests scores, program recipients who graduate from high school performed at the same level as equally poor nonrecipient graduates in mathematics, Spanish, or the overall test. This result holds after restricting the analysis to beneficia- ries that would have finished high school even without the program to correct for se- lection bias induced by the children brought into school by the program. X. Additional Studies ASSESSMENT ON THE USE OF THE RISK MANAGEMENT FRAMEWORK UNDER THE SOCIAL PROTECTION STRATEGY 2001 Objective of the Study and Sample The Bank’s Social Protection Strategy 2001 was developed against a background of growing Bank engagement in social protection over the 1990s as the Bank responded to the need to cushion the negative effects of transition in the former communist countries and the global financial crisis in the late 1990s. At the time, social protection consisted of a collection of in- struments—safety nets, social insurance, development funds, labor market interventions— where the emphasis tended to be placed on individual instruments. The 2001 strategy at- tempted to introduce a more systemic approach that looked at the social protection system as a whole—at how these instruments overlapped and interacted and how social protection could contribute to poverty reduction beyond passive income redistribution. The Social Risk Management framework, developed in late 1998 and early 1999 (Holzman and Jorgenson 1999), provided a means of defining social protection as public interventions that assist in- dividuals, households, and communities to manage risk and provide support to the poor. The 2001 Strategy notes that “social risk management…regards social protection as a springboard as well as a safety net for poor people. While a safety net for all should exist, the program should also provide poor people with the capacity to climb out of poverty, or at least resume gainful work….It focuses less on symptoms and more on the causes of po- 141 APPENDIX B METHODOLOGY verty by making it possible for poor people to engage in activities that have higher risks but also higher returns� (World Bank 2001). The Social Risk Management framework provides typologies for types of risks, strategies, instruments, and institutions involved in risk management. Briefly, they are—  Types of income risk: Catastrophic versus non-catastrophic shocks; idiosyncratic versus covariant shocks; single versus repeated shocks  Types of strategies to address these shocks: risk reduction, mitigation, coping  Types of instruments: Informal/personal; formal/market; formal/publicly man- dated or provided  Types of institutions/actors: Individuals; communities; nongovernmental organiza- tions; market institutions; government. The objective of the study was to assess How the Bank went about applying the 2001 So- cial Protection Strategy—to what extent the Bank helped countries develop safety nets as part of a broader system of social protection. The analysis drew on the 30 country cases to assess the Social Risk Management strategies and the extent to which the 2001 Strategy was applied by the Bank, particularly from the perspective of SSNs. The Social Risk Management strategies are defined as follows:  Risk reduction—Strategies/instruments that reduce the likelihood of a risk occur- ring  Risk mitigation—Strategies/instruments that reduce the impact of a future risk  Risk coping—Strategies/instruments that relieve the impact once a risk has oc- curred. Taken together, these strategies prevent households from falling into poverty or deeper into poverty; and promote their moving out of poverty. Methodology IEG country assessment included review of Country Assistance Strategy documents, the country lending portfolio; AAA, including poverty, social protection, and relevant economic reports throughout the decade; as well as interviews with Bank staff and management. Benchmarking was determined in consultation with the new Country Policy and Institu- tional Assessment criteria for assessing country social protection systems as well as with Bank experts working on these issues. Countries were placed in three categories for each time period as shown in table B.9. 142 APPENDIX B METHODOLOGY Table B.9. Rank Values of Bank Social Risk Management Implementation Rank Definition The Bank is implementing a strategy for social protection that is based on the SRM framework and complements government efforts to introduce a comprehensive SP system, including safety nets, social insurance mechanisms and Rank 3. labor market policies. The Bank is supporting government efforts to implement SP programs; but there is no comprehensive SRM framework; Rank 2. and there are large gaps in or uneven development of the SP system which are not being addressed. The Bank is offering advice on elements of SP in the context of economic, financial or sector AAA, but there is little or Rank 1. no follow-up through lending. Rank 0. There is no dialogue between the Bank and the Government on SP Each country was assessed based on three time frames throughout the decade (2000–04, 2005–10, and 2000–10). The analysis used the Social Risk Management strategies as a start- ing point for looking at the extent to which the 2001 Strategy has been applied by the Bank “as a springboard as well as a safety net� in Bank social protection strategies in the country case studies from the safety net angle. For specific social protection instruments, the Social Risk Management matrix would generally look as shown in table B.10. Table B.10. SSN Instruments by SRM Strategies SRM strategy Instrument  School vouchers, scholarships Reduction  Fee waivers  Labor market policies (industrial relations, ALMPs)  Social insurance (contributory pensions, health insurance, unemployment insurance)  Severance Mitigation  Short term benefits  Non-contributory pensions  Wage/employment subsidies  Unconditional poverty-targeted cash and in-kind transfers  Poverty-targeted family allowances Coping  Workfare/public works  Targeted housing, utility subsidies  Disability benefits Mitigation and Coping  SIF/CDD  Conditional cash transfers Reduction and Coping  Social care services Note: SRM = Social Risk Management; SIF = social investment fund; CDD =community-driven development; ALMPs = acrive labor mar- ket programs. The country/regional matrices used a set of indicators to assess to what extent the 2001 strategy has been reflected in the social protection programs for the period 2000–09. XI. The Role of SSNs in Development Policy Operations OBJECTIVE OF THE STUDY The purpose of the study was to identify if and how either the existence of or need for a so- cial safety net shapes the structure of development policy lending. The study examined whether DPLs have considered the need for a social safety net—or, in the absence of a com- prehensive social safety net, similarly targeted interventions—to mitigate any adverse con- sequences the operations might have on particular groups. This study differed than others 143 APPENDIX B METHODOLOGY recently completed by Operations Policy and Country Services and Poverty Reduction and Economic Management in that it examined if an assessment of distributional implications was undertaken and if so, what happened with the results of that analysis in country. It fol- lowed the recommendations to see if they were discussed with government and then if they were implemented. SAMPLE The review period was January 2005 to November 2009, right after the promulgation of the operational Policy OP 8.60, which required a poverty and social impact assessment for opera- tions where there was a likely chance of having distributional implications for the poor. During that period, there were 336 DPL operations. IEG focused on DPLs that were more likely to have distributional side effects. IEG sorted the documents according to the major sector on which they focused. Where more than one major sector was identified, IEG used the sector that was mentioned first (table B.11). Table B.11. Program Documents by Major Sector Public administration, law and justice, education and health and other social services were not con- sider for the analysis as they are very unlikely to have unintended distributional conse- quences on poor people. The IEG sample included the following categories: (1) energy and mining; (2) agriculture, fishing and forestry; (3) water, sanitation and flood protection; (4) transportation; and (5) industry and trade. These categories were considered to be more susceptible to the type of immediate impacts—for instance, increased cost of living or loss of employment—that might negatively affect the poor and vulnerable. IEG chose a sample of one-third (29) of the projects in these categories for more detailed analysis. Seven of the 29 were dropped because the program documents were not available, leaving a sample of 22. To ensure that the focus on the “primary�—first mentioned—major sector did not bias re- sults, IEG also chose a sample of projects that listed one of our five categories not as first 144 APPENDIX B METHODOLOGY major sector. The impact of this secondary sample was primarily to identify projects that listed public administration, law and justice as the primary category, but also included one of our focus categories as a secondary major sector. This second sample yielded an addi- tional 23 projects. The final total sample was 46 development policy operations. METHODOLOGY IEG reviewed the program documents in the sample, emphasizing the discussion of distri- butional issues in the document as a whole, rather than focusing on individual prior actions. For a more in-depth analysis, IEG interviewed task team leaders of the sample of DPLs on the role of distributional issues in their operations. To capture follow-up actions on distributional analysis, IEG conducted a second set of in- terviews with country economists of 16 DPLs in which 27 distributional issues had been 29 identified in the design stage of the development policy operation. Country economists were the primary point of contact under the premise they were and are in the best position to monitor what happens if and when the DPL is initiated. XII. PWP Portfolio This analysis was carried out as a background exercise to elaborate the results framework analysis as well as to help explain the performance of public works programs. The SSN portfolio includes 52 programs that included public works components. By num- ber, almost 50 percent of the projects were in Africa, 25 percent in Latin America and the Caribbean, 10 percent in South Asia and Europe and Central Asia, with minimal program- ming in the Middle East and North Africa and East Asia/Pacific regions (table B.12). Table B.12. Distribution of World Bank-Financed PWPs, by Region Region Number of Bank-supported PWPs % of total number of PWP projects Africa 25 48% East Asia Pacific 1 2% Europe and Central Asia 5 10% Latin America and Caribbean 13 25% Middle East North Africa 3 6% South Asia 5 10% Total 52 100% Source: IEG portfolio review. Note: PWP = public works program. Out of this universe, 29 programs were randomly selected for detailed examination drawing on the available Bank documentation. The documents reviewed included World Bank Im- plementation Completion Reports and Project Appraisal Documents prepared by Bank staff and case study evidence when available. A review of World Bank AAA on public works programs (PWPs) was also carried out, drawing on 56 advisory reports prepared between 2000 and 2010. 145 APPENDIX B METHODOLOGY PWP TYPOLOGY The PWPs were divided in four categories, defined on the basis of core design features:  Type A: Programs offering short-term employment  Type B: Large-scale government employment programs offering repeated or ongo- ing employment and in some cases an employment guarantee  Type C: Programs prioritizing asset creation and the labor intensification of govern- ment infrastructure spending  Type D: Programs that enhance supply-side characteristics, promoting “employabili- ty.� Although some programs may include aspects of more than one of these types, most PWPs tend to have a primary identity which enables them to be located in one of the four catego- ries. The primary identity was used to analyze program design and the aspects of program activity which are prioritized. PWP OBJECTIVES Although the PWP typology offers some conceptual clarification, IEG examined PWP objec- tives in more detail, as a review of international evidence suggests that a wide range of ob- jectives are associated with PWPs, and a single program may include a diversity of objec- tives. Particular objectives are associated with certain forms of PWP, but different PWP types are not necessarily or exclusively linked with particular objectives, and most pro- grams have a range of associated objectives, which may be variously prioritized, and the relative balance of objectives in a PWP can result in significant differences in program de- sign and social protection outcomes. Based on a review of PWP objectives internationally, objectives typically found in PWPs can be placed into six broad groupings:  Social protection (SSN provision)  Employment  Skills development  Asset or service provision  Macroeconomic stimulation  Political stabilization. This diversity of objectives is indicative of the range of potential performance indicators and outcomes that are anticipated in relation to PWP, and it is important to note that SSN provi- sion is only one of six different groupings. PWP KEY PERFORMANCE INDICATORS A review of KPIs of PWPs was conducted separately from the Results Framework Analysis. The review was consistent with the general Results Framework Analysis. The review listed for each project the types of KPIs that were listed (such as number of assets created, number of people employed, poverty targeting of employment), whether the project objective was output or outcome focused, whether poverty targeting was applied, and which SSN func- 146 APPENDIX B METHODOLOGY tion the project was aiming to address. The review found that KPIs tend to focus on output or process indicators, rather than outcome indicators, and that only 3 of 29 PWP projects aimed to target employment to the poor. XIII. The Bank’s Organization and SSN-Related Activities OBJECTIVE OF THE STUDY This background study focused on how the Bank’s internal structure affects its own ability to support sound SSN systems. More specifically, the study focused on coordination within the Bank’s matrix management system to assess how units work together and how this af- fects their services to clients. SAMPLE DESCRIPTION Consultations with Bank staff on the relationship between SSN-related activities and the Bank’s own internal organization covered a sample of 34 staff members, taken to be broadly representative of the diversity of challenges faced by the Bank in its SSN-related interven- tions. Staff members were selected to include all regional sector managers for Social Protec- tion and PREM Poverty, as well as members from their respective sector boards and from the SSN Global Experts Team. In addition, the sample included key informants from Devel- opment Economics and the World Bank Institute, task team leaders of SSN-related opera- tions, experts who had worked on the country case studies prepared for the SSN evaluation, and senior experts on the Bank’s organizational structure. More specifically, the sample included 14 Operations and Sector Managers from the Regions and Human Development and PREM Networks, 14 Social Protection Sector Board members (including the former chair), 8 PREM Poverty Sector Board members, 5 members of the SSN Global Experts Team, 14 task team leaders, 3 country case experts, and 2 senior experts on 30 the Bank’s organizational structure. METHODOLOGY Selected members were invited to participate in the exercise. A one-to-one semistructured interview was conducted to respondents who were available. The detailed questionnaire is presented in table B.13. 147 APPENDIX B METHODOLOGY Table B.13. Semistructured Questionnaire 1. With which Network/Region/Sector Board/Team are you affiliated? 2. Can you summarize your experience in regard to SSN based on your current and, if related, previous positions in the Bank? 3. How would you describe the current division of labor/responsibilities across networks (Human Development, PREM, Sustainable Development) with regard to SSNs? Do you personally think this works well or are there sometimes overlaps on certain issues or gaps between issues given that the agenda is spread over different units? Can you give a couple of examples where coordination between different units has worked well with regard to SSNs, and a couple of experiences where it hasn’t worked so well? 4. Based on your experience, how well has the formulation of a recent Poverty Reduction Support Credit/DPL/project leveraged on a collaborative approach on SSN-related issues with the Bank’s relevant hubs (Social Protection, Sustainable Development, PREM, Deveklopment Economics, World Bank Institute)? Can you elaborate on the process? What has helped this to work well or limited it from working well? 5. Does the fact that the SSN agenda overlaps on Human Development/PREM/Sustainable Development issues make coordination challenging and, in your view, can this affect the quality of the Bank’s support to countries? 6. How would you describe the current division of labor/responsibilities between Social Protection and the Regions with regard to SSNs? 7. Although the SSN work is formally housed in the Social Protection Board, only one half of all projects with SSN components fall under its management. What are the budgetary arrangements to fund SNN- related activities? Who (from which sector/network) typically task manages SSN-related activities in your Sector/Region? 8. With reference to a recent SSN-related project in which you have been involved, were the Social Protection resources and the staff effectively employed? If not, what were the constraints? 148 Appendix C Supplemental Tables and Boxes Table C.1. Bank’s Engagement in SSNs by Country Income Level SSN Variables LICs MICs Total SSNs Countries with SSN operations 34 49 83 SSN lending $2,438 $9,117 $11,555 SSN lending as a percentage of total World Bank lending in LICs and MICs 2% 5% 4% SSN operations 91 151 242 SSN operations as a percentage of total World Bank operations in LICs 6% 1 3% 8% and MICs Top 10 borrowers 2 8 10 Recipient Executed Trust Funds operations 18 36 55 Recipient Executed Trust Funds expenditures $71 $70 $141 Most used SSN instrument In-kind UCTs In-kind transfer transfer AAA (SSN activities) 36 115 2,101 Source: IEG portfolio review. Note : The difference is explained by activities at the regional level. AAA = analytic and advisory activities; LIC = low- income country; MIC = middle-income country; UCT = unconditional cash transfer. Table C.2. IEG Social Safety Nets Evaluation—Design Matrix Evaluation questions Sub-questions Sources What has the What objectives has the Bank What have been the objectives Project portfolio review, Bank done to pursued in countries with its of Bank support? AAA review support SSNs? support for SSNs; how has it What are trends in lending and done so and how well has the AAA support? Bank support achieved its objectives? How effective, How well have the Bank’s How well has portfolio Portfolio review, PPARs relevant, and projects achieved their stated performed? sustainable objectives? What is known Development impact of Bank IE results, literature have Bank about the development impact of supported SSNs? supported Bank supported SSNs? Have projects been Bank supported efforts been Sustainable? (fiscally, politically, Political economy in LICs and sustainable—fiscally, politically, development impact) background paper, Desk MICs? and with regard to development review of random projects, impact? Have Bank supported IEG sustainability of impact projects been relevant for study, TTL/SM interviews, achieving their objectives as well PPARs, Country cases as for being compatible and the and development of broader LCR regional study, development of broader social social insurance systems? literature, Desk review of insurance systems? What random projects, Cases, explains the high success rate of PPARs Bank-supported projects with SSN components? 149 APPENDIX C SUPPLEMENTAL TABLES AND BOXES Evaluation questions Sub-questions Sources How relevant To what extent has the Bank’s How relevant and effective has Portfolio review, Literature, and effective assistance to LICs and MICs the Bank’s work been to the Cases, PPARs, background has overall been relevant by supporting development of safety nets to paper on social funds, Bank countries develop SSNs to help address the chronic poor? CCTs, and public works? assistance the chronic poor, protect the How relevant and effective has Portfolio review, Literature, been in helping poor and vulnerable from shocks the Bank’s work been to the Country cases, PPARs countries and compensate the poor and development of safety nets to establish sound vulnerable from negative effects address individual shocks? safety nets? of macro reforms? How effective was Bank’s support at each? How relevant and effective has IEG Crisis Response the Bank’s work been to develop review, CAS reviews, Crisis SSNs that protect the poor and papers, elasticity of SSN vulnerable in LICs and MICs spending paper, Cases, from systemic shocks? PPARs, Literature (Ravallion Argentina paper) How relevant and effective has PSIA evaluation, OPCS the Bank’s work been to develop DPL retrospective, CAS SSNs that protect the poor from review, Cases, TTL effects of reforms? interviews How well has Bank helped Literature review, Desk countries develop effective review of random projects, targeting systems (identify the IE results, Cases, PPARs, needs, design coherent TTL/SM interviews administrative structuring M&E) How can the How can the effectiveness and Nature and approaches to CAS reviews, Cases, effectiveness relevance of Bank support be engagement?/ To what extent PPARs, TTL/SM interviews and relevance explained and understood and are Bank programs country of Bank what lessons can be derived for specific? support be future support for SSNs? What Sustained engagement? Portfolio review, AAA explained and drives both project performance review, CAS reviews, understood and and the development of country Cases, PPARs, TTL/SM what lessons SSN systems? interviews can be derived for future Political econ. of SSNs to guide Political economy support for strategic dialogue? background paper, Desk social safety review of random projects¸ nets? CAS reviews, Cases, PPARs, TTL/SM interviews Integrating with poverty and AAA review, CASs, country institutional analysis and growth cases, PPARs, TTL dialogue? interviews Contribution of Bank’s global Cases, PPARs work in SSN? Cases, PPARs 150 APPENDIX C SUPPLEMENTAL TABLES AND BOXES Table C.3. Government SSN Spending and Cost SSN Programs Supported by the Bank (Percent of GDP) Total Fuel and Program Program Country SSN food Main targeted SSN program type costb spending subsidies a Argentina 1.5 . Heads of Households PWP 0.1 Brazil 1.4 . Bolsa Familia CCT 0.4 Colombia 0.6 . Familias en Acción CCT 0.27 Program for the Adv. through Health 0.2 (0.3- Jamaica 0.8 . CCT and Education (PATH) 0.5) Mexico 1.0 . Oportunidades CCT 0.4 (0.46) Indonesia 1.3 5.3 Bantuan Langsung Tunai (BLT) UCT 0.66 Jordan 1.3 3.7 National Aid Fund (NAF) UCT 0.55 Yemen 1.0 13.7 Social Welfare Fund (SWD) UCT 0.6 Turkey 0.9 . Social Risk Mitigation Project (SPMP) CCT 0.6 UCT with Albania 1.9 . Ndihma Ekonomike 0.4 work-fare Bosnia and 1.3 2.0 c Civilian benefit UCT 1.3 Herzegovina Productive Safety Net Program Ethiopia n.a. 2.5 PWP 1.2 (PSNP) Sources: SSN spending: For countries in the Europe and Central Asia Region: Europe and Central Asia Social Protection Database, World Bank 2010; for all other countries: Weigand and Grosh 2008. Fuel and food subsidies: IMF Fiscal Affairs Department Food and Fuel subsidy database, 2010. Program cost: Argentina: ICR 2010; Brazil: PAD 2010; Colombia PAD 2008; Jamaica ICR 2009; Mexico PID 2010; Indonesia PAD 2009; Jordan PAD 2008; Yemen SP Strategy Phase I; Turkey ICR 2008; Albania Mangiavacchi and Verme 2009; Bosnia and Herzegovina Europe and Central Asia Social Protection Database, World Bank; Ethiopia PAD 2009. a Actual or projected share of outlays in 2008. Data on fuel subsidies vary significantly between year to year depending on the market price of oil. b Estimates are from latest year available. Data in parenthesis indicate estimates for 2010 for Jamaica and 2011 for Mexico. c War veterans’ benefit 2007. Table C.4. SSN Impact Evaluations on Bank-Supported Projects World Bank Program Region Country Project name project code Conditional EAP Cambodia CESSP Scholarship P070668 cash transfer ECA Turkey Social Risk Mitigation Project P074408 LCR Brazil Bolsa Alimentacao, Bolsa Escola, Programa de Erradicacao de P080746 Trabalho Infantil, and Programma de Garantia de Renda Minima Colombia Familias en Acción P089443/P101211 El Salvador Red Solidaria P088642 Jamaica Programme of Advancement P067774 through Health and Education Mexico Progresa/Oportunidades P007689 Nicaragua Atencion a Crisis P064906 151 APPENDIX C SUPPLEMENTAL TABLES AND BOXES World Bank Program Region Country Project name project code Nicaragua Red de Proteccion Social P050613 SAR Pakistan Female Secondary School Stipend P102608 Program Fee waivers EAP Indonesia Jaring Pengamanan Sosial P063939 for education LCR Colombia Programa de Ampliacion de Cobertura de la Educacion P006866 Secundaria SAR Bangladesh Female Secondary School Stipend P044876 Program Workfare AFR Ethiopia Employment Generation Schemes P081773 and Gratuitous Relief ECA Poland Intervention Works P008582 LCR Argentina Plan Jefes y Jefas P073578 Argentina Trabajar II P049268 AFR Ethiopia Productive Safety Nets Program P098093/P082242 Social LCR Bolivia Bono Solidario/Bolivida P082700 pension School AFR Burkina School Canteen and Take Home P100887 feeding/take Faso Ration home rations EAP Philippines Dropout Intervention Program P004565 SAR Bangladesh Food for Education P009496 Unconditional EAP China Southwest China Poverty P003639 cash transfer Reduction Project LCR Ecuador Bono de Desarrollo Humano P098167 Source: IEG 2011c. Note: AFR = Africa Region; EAP = East Asia and Pacific Region; ECA = Europe and Central Asia Region; LCR = Latin America and the Caribbean Region. 152 APPENDIX C SUPPLEMENTAL TABLES AND BOXES Table C.5. Selected Design Features of Different CCT Programs Percent of Transfer as a households percent of pre- in to Frequency Education Health Program name Bank project objectives transfer poorest of conditions conditions consumption quintile payment of the poor receiving transfers Bangladesh Increase girls’ secondary 0.6 (among all 8 (2000) Twice a Attend 75% of None Female school enrollments, and beneficiaries, year school days, Secondary thus increase the, number not just the attains 45% of School of educated women poor) class-level test Assistance capable of participating scores Program fully in economic and social development of the country Brazil The broad objectives of 11.7 (2006) 56 (2006) Monthly School Children 0–6 Bolsa Familia the BFP are to reduce enrollment of vaccine poverty and inequality and all children 6– schedule, promote human capital 17, daily regular health investments among poor attendance at checkups, families through the least 85% each growth provision of direct month, monitoring; monetary transfers to participate in pregnant and poor families and parent teacher lactating incentives for investing in meetings women pre- human capital and post natal health checkups and participation in seminars Mexico Increase capacities in 33.7 (2004) 70 (2004) Bimonthly School Compliance by Oportunidades health, nutrition, and enrollment and all household (formerly education of poor families minimum members with Progresa) through human capital attendance the required investment by promoting rate of 80% number of regular health check-ups, monthly and preventive improving health status, 93% annually, medical and raising school completion of checkups, enrollment and middle school, attendance of attendance rates. Build completion of family member sustainable connections grade 12 older than 15 between Oportunidades before age 22 years at health and other social programs and nutrition of the Government of lectures Mexico in order to improve health and education outcomes for Program participants Source: Project documents, Fiszbein and Schady 2009. 153 APPENDIX C SUPPLEMENTAL TABLES AND BOXES Table C.6. Possible Motivations of Actors vis-à-vis Social Safety Net Programs Possible Motivations of Actors vis-à-vis Social Safety Net Programs Actor Possible motivations/ Possible Actions Arising from These Motivations Top national  Maintain/expand political support government leaders o Increase or decrease safety net programs according to preferences of potential political supporters o Strengthen/reward political networks by placing loyalists into safety net program administration o Strengthen political networks by requiring recipients to participate in government-support activities o Structure safety net administration to reward supporters  Enact sound safety net programs o Calibrate targeting to ensure sufficient support for the program  Minimize political disruption o Establish/expand safety net programs to reduce dissatisfaction of the poor  Maintain credibility regarding past criticisms of existing safety net programs o Change &/or re-label safety net programs  Accomplish other policy objectives o Establish/expand safety net programs to enhance support for other policies  Optimize development assistance levels & uses [secure aid for sound & politically rewarding projects & programs, without the reality &/or perception of excessive dependence] o Resist pressure from external development assistance agencies o Shift government funds away from safety nets insofar as they are fungible Top sub-national  Strengthen their political support government leaders  Secure a larger portion of national safety net funds for their jurisdiction o Exaggerate number of eligible beneficiaries  Protect discretion over the sub-national budget o Resist unfunded safety-net program mandates  Secure more safety net resources to be allocated on the sub-national level o Pressure for decentralization of safety net program administration  Support associated political leadership at the national level Agency heads  Enhance & protect the standing of the agency & its personnel o [sometimes] Increase the agency budget o [sometimes] Resist expansion of functions to preserve the agency’s culture  Maintain/improve effectiveness of the agency o Divert funds from programs to staff salaries & other agency-enhancing resources o Allocate funds to areas & beneficiaries to enhance the agency’s standing  Serve broader goals of the administration o Allocate funds to areas & beneficiaries to enhance the government’s standing  Accommodate broad community targeting preferences o Permit looser targeting o Permit community control of targeting Agency front-line  Enhance/maintain incomes administrators o Accept bribes to qualify recipients & refrain from disqualifying if ineligible or non-compliant  Minimize conflict with community members o Qualify recipients & refrain from disqualifying if ineligible or non-compliant according to community preferences o Permit looser targeting o Permit community control of targeting  Secure one’s personal safety o Qualify recipients & refrain from disqualifying ineligibles, to avoid angering community members Front-line service  Enhance/maintain incomes providers o “Privatize� service provision de facto  Minimize conflict with community members o Accept fraudulent vouchers or other indications of qualification for subsidized service  Maintain professional integrity o Resist agency demands to compromise services The poor  Maximize incomes o Strive for qualification for safety net coverage, possibly through fraud o Mobilize to establish/expand safety net programs o Oppose weakening targeting for the poor  Maintain positive community relations Source: IEG SSN evaluation background paper. 154 APPENDIX C SUPPLEMENTAL TABLES AND BOXES Political Economy: As in many areas of public policy, there are a wide range of stakeholders with interests in SSNs. General assumptions regarding stakeholder positions and behaviors on various issues related to design of SSNs can vary depending on the country circumstances. A careful anal- ysis of stakeholder positions and incentives in a particular country setting can enable a SSN to be designed in a way that would render the SSNmost effective and feasible. Table C.7 provides several examples. Table C.7. Conventional Assumptions Challenged by Political Economy Insights Assumption Reason why it is sometimes violated Implications or recommendations Likely b eneficiaries woul d s upport s ocial The c onditionalities may be re garded by Evaluate c onditionality op tions in t erms of safety net initiatives potential be neficiaries as ov erly burd en- the lik elihood t hat the init iative would s e- some; the p oor and th eir r epresentatives cure the targeted level of funding may try to hold out for greater benefits Eligible s afety ne t rec ipients woul d pref er Maintaining positive c ommunity r elations Consider d esigning in fl exibility in e ligibility stringent targeting may b e s een as requiring br oader benefi- criteria & modes of benefit allocation ciary pool Compliance wi th eligibility c onditions will be Beneficiaries may be able t o f abricate t he Increase th e m onitoring of c ompliance; ensured by the threat of disqualification reportage of c ompliance, s ometimes in raise th e p enalties for bot h b eneficiaries concert with program administrators and administrators for fabricating reports The net a ddition to th e wel lbeing of rec i- The for mal s afety net pr ogram m ay c rowd For a giv en t arget be nefit level, ad just for pients is equa l to the be nefits they rec eive out i nformal c ommunity arran gements that anticipated reductions in informal communi- from the formal safety net program provide some benefit for particularly vulner- ty-donated benefits able families The safety net i nitiative’s magnitude is what Non-poor may believe tha t the in itiative is Consider provisions in t he l egislation, or non-poor assume would be the magnitude if just a step to an even bigger program other for ms of as surance, t hat t he burden the initiative succeeds on the non-poor would not increase “Leakage� is ba d for the p oor because it Without “ leakage�, the political s upport f or Estimate the e ligibility bread th req uired t o reduces the resources available to them safety ne t f unding m ay be to o w eak to secure s ufficient s upport; &/or link it to provide the appropriate level of funding policies benefiting the non-poor (e.g., more secure unemployment insurance) The c ontributions of forei gn a ssistance Government fu nds for s afety net programs International fu nders s hould c onsider re- agencies to s afety net pr ograms are fu lly may be diverted to other purposes if exter- quiring matches or other m odes of c ommit- additive to th e to tal ma gnitude of the pro- nal contributions fill the gap ting the government t o fu nd a t a c ertain gram level Safety net agency heads would welcome the Agency he ads may try to pr eserve the Consider establishing s eparate u nits to largest possible e xpansion of their auth ority ethos, efficiency, and integrity of the agency administer safety net programs and resources by avoiding expansions that may risk these Pro-poor NGOs a re the best al lies for gov - Some pro- poor NGOs are antagonistic Consider w orking more c losely wit h gras s- ernments trying to establish or expand safety toward t he government, a nd wi ll criticize roots or ganizations rath er t han pro-poor net programs government ini tiatives as part of a gen eral NGOs strategy of opposition Support f or s afety net pro grams will be The non-poor, insofar as they are also more Consider building in provisions for automat- greatest when th ey are nee ded t he mos t: vulnerable un der thes e c onditions, may ic adjus tments of s afety net fu nding as 155 APPENDIX C SUPPLEMENTAL TABLES AND BOXES Assumption Reason why it is sometimes violated Implications or recommendations during tim es of s evere ec onomic do wnturns oppose inc reases in s afety net f unding, or economic conditions change &/or policy reforms that make the poor most even d emand decreases in or der t o div ert vulnerable budget funds to their own support The optimal degree and nature of decentrali- The t echnically o ptimal degree and nat ure Evaluate dec entralization o ptions in terms zation of s afety n et a dministration i s deter- of s afety n et dec entralization may be in- of th e lik elihood that t he i nitiative would mined by th e c riteria o f ad ministrative eff i- compatible with sufficient political support to secure the targeted level of funding ciency, allocative efficiency, and equity yield t he ap propriate lev el o f s afety net funding The o ptimal de gree a nd n ature of conditio- The t echnically o ptimal degree and nat ure Evaluate c onditionality op tions in t erms of nalities is de termined by their effectiveness of c onditionalities may res ult in l ower c ov- the lik elihood t hat the init iative would s e- in improving human resource development erage of o therwise elig ible be neficiaries, cure the targeted level of funding evasion and corruption, and reduce political support necessary to y ield t he appropriate level of funding The em phasis o n productivity ga ins will The failure to ac hieve demonstrable, signif- Exercise re straint i n j ustifying sa fety n et strengthen the att ractiveness of s afety net icant h uman res ource dev elopment gains , programs on gr ounds of t he p otential for programs or s uccessful infr astructure g ains, can be short- or medium-term productivity gains used to attack safety net initiatives The positions ex pressed by repr esentatives To fu lfill th eir mandate, re presentatives Appealing dir ectly to the no n-poor to s up- of the n on-poor are ac curate refl ections of often feel compelled to defend the interests port sa fety ne t ini tiatives m ay gain m ore the pr eferences of thos e wh om th ey rather than the preferences of th ose whom support than wor king thr ough re presenta- represent they represent tives The g overnment’s s tated p osition o n s afety “Bureaucratic pol itics� pits differe nt gov - External en tities can as sist th ose govern- net programs h as the c oncurrence of all ernment ag encies ag ainst one a nother in mental units th at are try ing t o promote units within government the c ompetition f or pol icy influ ence, re- sounder safety net programs sources, and jurisdiction Safety n et programs c an b e s wiftly ada pted Bureaucratic i nertia, r esistance from pr e- Consider automatic phase-out provisions or termi nated as top gov ernment offic ials existing beneficiaries, an d tec hnical o b- decide to do so stacles to ada ptation a nd ter mination may be obstacles The ma gnitude of fundi ng th at top govern- Program a dministrators m ay div ert mor e For a giv en t arget ma gnitude of benefits, ment po licymakers and ex ternal fun ders funds for the agency’s operations; adminis- planners o ught t o ad just for anticipated earmark for pr ogram b eneficiaries is the trators m ay div ert be neficiaries’ fu nds t o diversions magnitude these beneficiaries will receive others New gov ernments will c ontinue ex isting New gov ernments may feel c ompelled t o International funders c an as sist i n “ re- safety net programs deemed effective terminate or rad ically c hange pro grams to labeling� and l egitimizing s ound ad aptation gain c redit for in novation &/or av oid c riti- of pre-existing programs. cism for maintaining pr ograms t hey ha d criticized when in opposition Source: IEG. 156 Appendix D Supplemental Figures Figure D.1. Trends in SSN Commitments by Country Income Level ($ millions) Source: IEG portfolio review. Figure D.2. Trends in Projects with SSN Components by Country Income Level Source: IEG portfolio review. 157 APPENDIX D SUPPLEMENTAL FIGURES Figure D.3. Percent of Poor People in Bank Client Countries All Bank Client Countries % of total of poor people of Bank client  35% India 30% 25% countries 00�07 China 20% 15% 10% 5% Colombia Mexico Argentina 0% 0% 5% 10% 15% 20% % of total commited to SSNs FY00�FY10 Source: IEG portfolio review and World Development Indicators. Figure D.4. SSN ESW and Non-Lending TA Expenditures by Country/Region ($ millions) USD 4.5  USD 4.0  USD 3.5  USD 3.0  USD 2.5  USD 2.0  USD 1.5  USD 1.0  USD 0.5  USD � Source: IEG portfolio review, FY00–10. 158 APPENDIX D SUPPLEMENTAL FIGURES Figure D.5. Trends in the Choice of Instruments for SSN Operations 30 25 20 CCTs Cash transfers 15 In�kind transfers Education and health subsidies Energy, water, housing subsidies 10 PWP 5 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Source: IEG portfolio review. Figure D.6. Trends in PDOs and Outcome Indicators That Were Outcome Driven 80% 70% 60% 50% 40% FY00�FY05 30% FY06�FY10 20% 10% 0% PDO outcome Source: IEG portfolio review 159 APPENDIX D SUPPLEMENTAL FIGURES Figure D.7. Trends Inpoverty Focus of Projects with SSN Components 100% 90% 80% 70% 60% 50% FY00�FY05 40% FY06�FY10 30% 20% 10% 0% Poverty PDOs Poverty KPIs Source: IEG portfolio review. 160 Bibliography laboration with Ankara, Turkey: AGRIN Adato, M., B. de la Brière, D. Mindek, and A. Co. Ltd. Quisumbing. 2000. “The Impact of Ahmed, A.U. 2004. “Impact of Feeding Children PROGRESA on Women’s Status and in School: Evidence from Bangladesh.� Intrahousehold Relations.� Washington, Washington, DC: IFPRI. DC: International Food Policy Research Institute. Ahmed, A.U., C. del Ninno, and O.H. Chowd- hury. 2004. “Investing in Children Adelman, S., H. Alderman, D.O. Gilligan, and J. through the Food for Education Pro- Konde-Lule. 2008. “The Impact of Al- gram.� In The 1998 Floods and Beyond: ternative Food for Education Programs Towards Comprehensive Food Security in on Child Nutrition in Northern Ugan- Bangladesh, eds. Dorosh, P., C. Ninno, da.� Mimeo. Washington, DC: IFPRI. and Q. Shahabuddin, Chapter 10: 280- Adelman, S., H. Alderman, D.O. Gilligan, and K. 311. Washington, DC: IFPRI. Lehrer. 2008. “The Impact of Alternative Ahmed, A. U. and M. Arends-Kuenning. 2006. Food for Education Programs on Learn- “Do Crowded Classrooms Crowd out ing Achievement and Cognitive Devel- learning? Evidence from the Food for opment in Northern Uganda.� Mimeo. Education Program in Bangladesh,� Washington, DC: IFPRI. World Development, Elsevier, 34(4): 665- Adelman, S.W., D. O. Gilligan, and K. Lehrer. 84. 2007. “How Effective are Food for Edu- Alam, A., J. Baez, and X. del Carpio. 2010. “Do cation Programs? A Critical Assessment Conditional Cash Transfers Lead to Me- of the Evidence from Developing Coun- dium-Term Impacts? Evidence from a tries.� Food Policy Review 9. Washington, Female School Stipend Program in Pa- DC: IFPRI. kistan.� Mimeo. Washington, DC: The Afridi, F. 2010. “The Impact of School Meals on World Bank. School Participation: Evidence from Ru- Alderman, Harold and Haque, Trina. 2006. ral India.� Indian Statistical Institute, “Countercyclical Safety Nets for the Planning Unit, New Delhi Discussion Poor and Vulnerable�. Food Policy. 31 Papers 10-02. New Delhi: India Indian (4): 372-383. August. Statistical Institute. Alderman, Harold and Hoddinott, John. 2007. ———. 2009. “Child Welfare Programs and “Growth-Promoting Social Safety Nets�. Child Nutrition: Evidence from a Man- 2020 Focus Brief on the World’s Poor dated School Meal Program in India.� and Hungry People. IFPRI. Washington Journal of Development Economics. DC. October. Agüero, J. M., Carter, M.R. Carter, and I. Woo- Andersson, C., A. Mekonnen, and J. Stage. 2009. lard. 2007. “The Impact of Uncondition- “Impacts of the Productive Safety Net al Cash Transfers on Nutrition: The Program in Ethiopia on Livestock and South African Child Support Grant,� Tree Holdings of Rural Households,� Working Papers 39, International Policy EfD Discussion Paper 09-05, a joint pub- Centre for Inclusive Growth. lication of Washington, DC: The Envi- Ahmed, A.U., M. Adato, A. Kudat, D. Gilligan, ronment for Development Initiative and T. Roopnaraine, and R. Colasan. 2007. Resources for the Future. “Impact Evaluation of the Conditional Angelucci, M., and G. de Giorgi. 2008. “Indirect Cash Transfer Program in Turkey: Final Effects of an Aid Program: The Case of Report.� Washington, DC: IFPRI in col- 161 BIBLIOGRAPHY PROGRESA and Consumption.� Ameri- Attanasio, O., E. Fitzsimmons, and A. Gómez. can Economic Review. 2005. “The Impact of a Conditional Edu- cation Subsidy on School Enrollment in Angelucci, M. 2004. “Aid and Migration: An Colombia.� Report Summary: Familias Analysis of the Impact of Progresa on 01. London, UK: Institute for Fiscal Stu- the Timing and Size of Labour Migra- dies. tion.�IZA Discussion Paper No. 1187. Available at SSRN: Attanasio, O., C. Meghir, and A. Santiago. 2005. http://ssrn.com/abstract=564561. “Education Choices in Mexico: Using a Structural Model and a Randomized Angrist, J., E., Bettinger, E. Bloom, E. King, and Experiment to Evaluate PROGRESA.� M. 2002. “Vouchers for Private School- Working Paper EWP05/01. London, ing in Colombia: Evidence from a Ran- UK: IFS. domized Natural Experiment,� Natural Field Experiments 0004, The Field Expe- Attanasio, Orazio, Emla Fitzsimons, Ana Go- riments Web site. mez, Diana Lopez, Costas Meghir, and Alice Mesnard. 2006. “Child Education Angrist, J., E., Bettinger, and M. Kremer. 2004. and Work Choices in the Presence of a “Long-Term Consequences of Second- Conditional Cash Transfer Programme ary School Vouchers: Evidence from in Rural Colombia.� The Institute for Administrative Records in Colombia,� Fiscal Studies. London. NBER Working Papers 10713, National Bureau of Economic Research, Inc. Attanasio, O., E. Fitzsimmons, A. Gómez, D. López, C. Meghir, and A. Mesnard. Angrist, Joshua, Eric Bettinger, Erik Bloom, Eliz- 2006. “Child Education and Work abeth King, and Michael Kremer. 2002. Choices in the Presence of a Conditional “Vouchers for Private Schooling in Co- Cash Transfer Programme in Rural Co- lombia: Evidence from a Randomized lombia.� Working Paper W06/01. Lon- Natural Experiment.� American Economic don, UK: Institute for Fiscal Studies. Review 92: 1535-1558. Attanasio, O., L. C. Gómez, P. Heredia, and M. Angrist, Joshua, Eric Bettinger, Erik Bloom, and Vera-Hernández. 2005. “The Short-Term Michael Kremer. 2004. “Long-term Con- Impact of a Conditional Cash Subsidy sequences of Secondary School Vouch- on Child Health and Nutrition in Co- ers: Evidence from Adminstrative lombia.� Report Summary: Familias 03. Records in Colombia.� NBER Working London, UK: Institute for Fiscal Studies. Paper 10713. Baird, S., C. McIntosh, and B. Ozler. 2009. “De- Ardington, C., A. Case, and V. Hosegood. 2009. signing Cost-effective Cash Transfer “Labor Supply Responses to Large So- Programs to Boost Schooling among cial Transfers: Longitudinal Evidence Young Women in Sub-Saharan Africa,� from South Africa,� American Economic Policy Research Working Paper Series Journal: Applied Economics, American Eco- 5090. The World Bank: Washington DC. nomic Association, 1(1): 22-48. Baird, S., E. Chirwa, C. McIntosh, and B. Ozler. Attanasio, O. and A. Mesnard. 2005. “The Im- 2009. “The Short-term Impacts of a pact of a Conditional Cash Transfer Schooling Conditional Cash Transfer Programme on Consumption in Colom- Program on the Sexual Behavior of bia.� Report Summary: Familias 02. Young Women,� Policy Research Work- London, UK: Institute for Fiscal Studies. ing Paper Series 5089. Washington, DC: Attanasio, Orazio, Emla Fitzsimons, and Ana The World Bank. Gomez. 2005. “The Impact of a Condi- Barham, T. and J. Maluccio. 2008. “The Effect of tional Education Subsidy on School Conditional Cash Transfers on Vaccina- Enrolment in Colombia.� The Institute tion Coverage in Nicaragua.� Health for Fiscal Studies. London. 162 BIBLIOGRAPHY and Society Working Paper HS2008-01, Behrman, J. R., P. Sengupta, and P. Todd. 2000. Institute of Behavioral Science. Boulder, “The Impact of PROGRESA on CO: University of Colorado. Achievement Test Scores in the First Year.� Final Report. Washington, DC: Barham, T. 2005a. “Providing a Healthier Start IFPRI. to Life: The Impact of Conditional Cash Transfers on Infant Mortality.� Unpub- Bertrand, M., D. Miller, and S. Mullainathan. lished manuscript, Department of Agri- 2000. “Public Policy and Extended Fami- culture and Resource Economics. Berke- lies: Evidence from South Africa.� NBER ley, CA: University of California. Working Paper 7594, National Bureau of Economic Research, Inc. ———. 2005b. “The Impact of the Mexican Con- http://www.nber.org/papers/w7594. ditional Cash Transfer on Immunization Rates.� Unpublished manuscript, De- Betcherman, G., K. Olivas and A. Dar. 2004. partment of Agriculture and Resource “Impacts of Active Labor Market Pro- Economics. Berkeley, CA: University of grams: New Evidence from Evaluations California. with Particular Attention to Developing and Transition Countries.� Social Pro- Barrera-Osorio, F., M. L. Bertrand, L. Linden, tection Discussion Paper Series No. and F. Perez-Calle. 2008. “Conditional 0402. Washington, DC: The World Bank Cash Transfers in Education: Design Features, Peer and Sibling Effects Evi- Chen, Shaohua andMartin Ravallion. 2009. “The dence from a Randomized Experiment Impact of the Global Financial Crisis on in Colombia.� Policy Research Working the World’s Poor.� Development Re- Paper 4580. Washington, DC: The World search Group World Bank. Washington, Bank. DC. Behrman, Jere R., J. Gallardo- García, S. W. Corporate Secretariat. 2009. OM2009-0008. Parker, P. E. Todd, and V. Velez- Cameron, L.A. 2002. “Did Social Safety Net Grajales. 2005. “How Conditional Cash Scholarships Reduce Drop-out Rates Transfers Impact Schooling and Work- during the Indonesian Economic Cri- ing Behaviors of Children and Youth in sis?� Policy Research Working Paper Urban Mexico.�Mimeo. 2800. Washington, DC: The World Bank. Behrman, Jere R., S. W. Parker, and P. E. Todd. Cardoso, E. and A. P. Souza. 2004. “The Impact 2005. “Long-Term Impacts of the Opor- of Cash Transfers on Child Labor and tunidades Conditional Cash Transfer School Attendance in Brazil.� Working Program on Rural Youth in Mexico.� paper 04-W07, Department of Econom- Discussion Paper 122. Göttingen, Ger- ics. Nashville, TN: Vanderbilt Universi- many: Ibero-America Institute for Eco- ty. nomic Research. Case, A. 2001. “Does Money Protect Health Sta- Behrman, Jere R., P. Sengupta, and P. E. Todd. tus? Evidence from South African 2005. “Progressing through PROGRESA: Pensions,� Working Papers 268, Prince- An Impact Assessment of a School Sub- ton University, Woodrow Wilson School sidy Experiment in Rural Mexico.� Eco- of Public and International Affairs, Cen- nomic Development and Cultural Change ter for Health and Wellbeing. 54(1): 237–75. Cattaneo, M.D., S. Galiano, P.J. Gertler, S. Marti- Behrman, J. R. and J. Hoddinott. 2001. “An nez, and R. Titiunik. 2007. “Housing, Evaluation of the Impact of PROGRESA Health, and Happiness.� Policy Re- on Pre-school Child Height.� FCND search Working Paper Series 4214. Discussion Paper No. 104. Washington, Washington, DC: The World Bank. DC: IFPRI. 163 BIBLIOGRAPHY Chaudhury, Nazmul and Dilip Parajuli. 2006. on Household welfare: Comparing two “Conditional Cash Transfers and Fe- Diverse Cash Transfer Programmes in male Schooling: The Impact of the Fe- Rural Mexico.� Working Papers 02-10, male School Stipend Program on Public Agricultural and Development Econom- School Enrollments in Punjab, Pakis- ics Division of the Food and Agriculture tan.� World Bank Policy Research Organization of the United Nations Working Paper No. 4102, Washington, (FAO—ESA). DC de Brauw, A. and J. Hoddinott. 2008. “Must ------------. 2008. “Conditional Cash Transfers and Conditional Cash Transfer Programs Be Female Schooling: The Impact of the Conditioned to Be Effective? The Impact Female School Stipend Program on Pub- of Conditioning Transfers on School lic School Enrollments in Punjab, Pakis- Enrollment in Mexico.� Discussion Pa- tan.� Journal of Applied Economics. per 757. Washington, DC: IFPRI. Chen, Sh, R. Mu, and M. Ravallion. 2008. “Are Del Carpio, X.V. and K. Macours. 2009. There Lasting Impacts of Aid to Poor “Leveling the Intra-household Playing Areas ? Evidence from Rural China,� Field: Compensation and Specialization Policy Research Working Paper Series in Child Labor Allocation.� Policy Re- 4084. Washington, DC: The World Bank. search Working Paper Series 4822. Washington, DC: The World Bank. Coady, D., P. Olinto, and N. Caldes. 2004. “Cop- ing with the Coffee Crisis in Central Del Carpio, X. V. 2008. “Does Child Labor Al- America: The Role of Social Safety Nets ways Decrease with Income ? An Evalu- in Honduras.� Washington, DC: IFPRI ation in the Context of a Development and Departamento de Economía Agra- Program in Nicaragua.� Policy Research ria—ETSI Agrónomos—UPM. Working Paper Series 4694. Washington, DC: The World Bank. Coady, D. P. and S.W. Parker. 2004. “Cost- Effectiveness Analysis of Demand- and Del Ninno, C. and P. A. Dorosh. 2002. “In-Kind Supply-Side Education Interventions: Transfers and Household Food Con- The Case of PROGRESA in Mexico.� Re- sumption: Implications for Targeted view of Development Economics, 8(3): 440– Food Programs in Bangladesh.� Discus- 51. sion Paper 134, Food Consumption and Nutrition Division. Washington, DC: Dabalen, A., T. Kilic, and W. Wane. 2008. “Social IFPRI. Transfers, Labor Supply and Poverty Reduction: The Case of Albania,� Policy Del Ninno, C., Kalanidhi S. and Milazzo A, and Research Working Paper Series 4783. others. 2009. “How to Make Public Washington, DC: The World Bank. Works Work: A Review of the Expe- riences.� Working Paper Series 0905. Das, Jishnu, Quy-Toan Do, and Berk Özler. 2005. Washington, DC: The World Bank. “Reassessing Conditional Cash Transfer Programs.� World Bank Research Observer Departamento Nacional de Planeación. 2007. 20(1):57–80. “Evaluación de Políticas Públicas No. 5: Programa Empleo en Acción Impactos Datt, G., and M. Ravallion. 1992. “Growth and del Programa sobre los Beneficiarios y Redistribution Components of Changes sus Familias.� Bogotá, D.C., Colombia: in Poverty Measures: A Decomposition Departamento Nacional de Planeación. with Applications to Brazil and India in the 1980's� Journal of Development Eco- Departamento Nacional de Planeación. 2004. nomics 38:275–95. “Evaluación de Polìticas Públicas No.2: Documento Programa Empleo en Ac- Davis, B., S. Handa, M. Ruiz-Arranz, M. Stam- ción. Condiciones Inciales de los Benefi- pini and P. Winters. 2002. “Conditionali- ciarios e Impactos de Corto Plazo.� Bo- ty and the Impact of Programme Design 164 BIBLIOGRAPHY gotá, D.C., Colombia: Departamento Growth, and Development: An Analysis Nacional de Planeación. of Mexico’s Oportunidades.� The Lan- cet, 371: 828–37. Dercon, S. and P. Krishnan. 2004. “Food Aid and Informal Insurance.� Development and Ferreira, Francisco H. G., D. Filmer, and S. Nor- Comp Systems 0409026, EconWPA. bert. 2009. “Own and Sibling Effects of Conditional Cash Transfer Programs: Dubois, P. and M. Rubio-Codina. 2009. “Child Theory and Evidence from Cambodia,� Care Provision: Semiparametric Evi- Policy Research Working Paper Series dence from a Randomized Experiment 5001. Washington, DC: The World Bank. in Mexico.� CEPR Discussion Paper No. DP7203. Available at SSRN: Ferreira, Francisco and Norbert Schady, 2008. http://ssrn.com/abstract=1356432 “Aggregate Economic Shocks, Child Schooling and Child Health�. Policy Re- Dubois, P., A. de Janvry, and E. Sadoulet. 2004. search Working Paper 4701. World “Effects on School Enrollment and Per- Bank, 2008. formance of a Conditional Transfers Program in Mexico.� Economics Work- Filmer, D. and N. Schady. 2009a. “Are There ing Paper Archive (Toulouse) 36, French Diminishing Returns to Transfer Size in Institute for Agronomy Research Conditional Cash Transfers?� Policy Re- (INRA), Economics Laboratory in Toul- search Working Paper Series 4999. ouse (ESR Toulouse). Washington, DC: The World Bank. Duflo, E. 2003. “Grandmothers and Grand- ———. 2009b. “School Enrollment, Selection daughters: Old-Age Pensions and Intra- and Test Scores.� Policy Research Work- household Allocation in South Africa.� ing Paper Series 4998. Washington, DC: World Bank Economic Review 17(1): 1-25,. The World Bank. Duryea, S. and A. Morrison. 2004. “The Effect of ———. 2006. “Getting Girls into School: Evi- Conditional Transfers on School Per- dence from a Scholarship Program in formance and Child Labor: Evidence Cambodia.� Policy Research Working from an Ex-Post Impact Evaluation in Paper 3910. Washington, DC: The World Costa Rica,� RES Working Papers 4359: Bank. Washington, DC: Inter-American De- Fiszbein, Ariel, and Norbert Schady. 2009. velopment Bank. “Conditional Cash Transfers: Reducing Edmonds, E.V. and N. Schady. 2008. “Poverty Present and Future Poverty.� Policy Re- Alleviation and Child Labor.� Policy search Report, World Bank. Washing- Research Working Paper Series 4702. ton, DC. Washington, DC: The World Bank. Friedman, Jed and Norbert Schady. “How Many Edmonds, E. V. 2006. “Child Labor and School- More Infants are Likely to Die in Africa ing Responses to Anticipated Income in as a Result of the Global Financial Cri- South Africa.� Journal of Development sis?�. Policy research Working Paper. Economics, 81(2): 386-414. World Bank. Washington DC. Edmonds, E.V., K. Mammen, and D. L. Miller. Galasso, E. and M. Ravallion. 2003. “Social Pro- 2004. “Rearranging the Family? Income tection in a Crisis—Argentina’s Plan Support and Elderly Living Arrange- Jefes y Jefas.� Policy Research Working ments in a Low Income Country.� Paper Series 3165. Washington, DC: The NBER Working Papers 10306, National World Bank. Bureau of Economic Research, Inc. Galasso, E., M. Ravallion, and A. Salvia. 2001. Fernald, L., P. Gertler, and L. Neufeld. 2008. “Assisting the Transition from Workfare “Role of Cash in Conditional Cash to Work: a Randomized Experiment,� Transfer Programmes for Child Health, 165 BIBLIOGRAPHY Policy Research Working Paper Series Gulemetova-Swan, M. 2009. “Evaluating the 2738. Washington, DC: The World Bank. Impact of Conditional Cash Transfer Programs on Adolescent Decisions García, S. and J. Hill. 2009. “The Impact of Con- about Marriage and Fertility: The Case ditional Cash Transfers on Children’s of Oportunidades.� Dissertations availa- School Achievement: Evidence from Co- ble from ProQuest. Paper AAI3363363. lombia.� Universidad de los Andes, http://repository.upenn.edu/dissertati CEDE Working Paper Series. Available ons/AAI3363363 at SSRN: http://ssrn.com/abstract=1485841. Hamoudi, A. and D. Thomas. 2005. “Pension Income and the Well-Being of Children Gertler, P. 2000. “Final Report: The Impact of and Grandchildren: New Evidence from PROGESA on Health.� Washington, DC: South Africa.� On-Line Working Paper IFPRI. 043-05. Los Angeles, CA: California Gertler, P., S. Martínez, and M. Rubio-Codina. Center for Population Research, Univer- 2006. “Investing Cash Transfers to Raise sity of California. Long-Term Living Standards.� Policy Handa, S., M-C. Huerta, R. Perez and B. Straf- Research. Working Paper 3994. Wash- fon. 2001. “Poverty, Inequality, and Spil- ington, DC: The World Bank. lover in Mexico’s Education, Health, Gilligan, D. O. and J. Hoddinott. 2006. “Is There and Nutrition Program.� FCND discus- Persistence in the Impact of Emergency sion papers 101. Washington, DC: IFPRI. Food Aid? Evidence on Consumption, Hanlon J., Barrientos, A., and Hulme D. 2010. Food Security, and Assets in Rural Ethi- “Just Give Money to the Poor: The opia.� FCND Discussion Paper 209. Revolution from the Global South� VA Washington, DC: IFPRI. United States. Gilligan, D. O., J. Hoddinott, and A.S. Taffesse. Heinrich, C. J. 2007. “Demand and Supply-Side 2008. “The Impact of Ethiopia’s Produc- Determinants of Conditional Cash tive Safety Net Programme and its Lin- Transfer Program Effectiveness.� World kages.� Discussion Papers 839. Wash- Development 35(1): 121-43. ington, DC: IFPRI. Hoddinott, J. , E. Skoufias, and R. Washburn. Glewwe, P. and P. Olinto. 2004. “Evaluating the 2000. “The Impact of PROGRESA on Impact of Conditional Cash Transfers on Consumption.� IFPRI: Washington, DC. Schooling: An Experimental Analysis of Honduras.�Mimeo. Minneapolis, MN: Holzman, Robert, editor, 2009, “ Social Protec- University of Minnesota. tion and Labor at the World Bank 2000- 2008�, World Bank, Washington, DC. Grillenberger, M., C.G. Neumann, S.P. Murphy, N.O. Bwibo, P. van’t Veer, J.G. Haut- Holzman R., and Jorgensen S. 2000 “Social Risk vast, and C.E. West. 2003. “Food Sup- Management: Anew Conceptual plements Have a Positive Impact on Framework for Social Protection and Weight Gain and the Addition of Ani- Beyond� Social Protection Discussion mal Source Foods Increases Lean Body Paper No. 0006, The World Bank. Mass of Kenyan Schoolchildren.� The IEG (Independent Evaluation Group). Forth- Journal of Nutrition; 133: 3957S–64S. coming, “Project Performance Assess- Grosh, M. C. Ninno, E. Tesliuc, and A. Ouerghi. ment Report: Colombia, Social Safety 2008. “For Protection and Promotion: Net Project.� The Design and Implementation of Ef- IEG. 2009. What Can We Learn from Nutrition fective Safety Nets.� Washington, DC: Impact Evaluations? Lessons for Devel- The World Bank. opment Practitioners and Evaluators from a Review of Evaluations to Reduce 166 BIBLIOGRAPHY Child Malnutrition. Washington, DC: Institute for Fiscal Studies, Econometría, and World Bank. Sistemas Especializados de Información. 2006. “Evaluación del Impacto del Pro- ———. 2011a. “Assessing the Long-Term Effects grama Familias en Acción–Subsidios of Conditional Cash Transfers on Human Condicionados de la Red de Apoyo So- Capital: Evidence from Colombia.� cial.� Bogotá, Colombia: National Plan- ———. 2011b. “Do Conditional Cash Transfers ning Department. Lead to Medium-Term Impacts? Evi- Jacoby, H. G. 2002. “Is There an Intrahousehold dence from a Female School Stipend Pro- ‘Flypaper Effect’? Evidence from a gram in Pakistan.� School Feeding Programme.� Economic ———. 2011c. Forthcoming, “Evidence and Les- Journal 112 (476): 196–221. sons Learned from Impact Evaluations on Jalan, J., and M. Ravallion. 1999. “Income Gains Social Safety Nets.� to the Poor from Workfare—Estimates ———. 2011d. The World Bank Group’s Response to for Argentina’s TRABAJAR Program.� the Global Economic Crisis. Washington, Policy Research Working Paper Series DC: World Bank. 2149. Washington, DC: The World Bank. ———. 2010a. Analyzing the Effects of Policy Re- de Janvry, A., F. Finan, E. Sadoulet, and R. Va- forms on the Poor: An Evaluation of the Effec- kis. 2006. “Can Conditional Cash Trans- tiveness of World Bank Support to Poverty fer Programs Serve as Safety Nets in and Social Impact Analyses. Washington, Keeping Children at School and from DC: World Bank. Working When Exposed to Shocks?� Journal of Development Economics 79(2): ———. 2010b. Cost-Benefit Analysis of World Bank 349–73. Projects. Washington, DC: World Bank. ———. 2010c. Safeguards and Sustainability Poli- de Janvry, A., F. Finan, and E. Sadoulet. 2005. cies in A Changing World: An Independent “Using a Structural Model of Educa- Evaluation of World Bank Group Expe- tional Choice to Improve Program Effi- rience. Washington, DC: World Bank. ciency.� Berkeley, CA: University of California at Berkeley. ———. 2009a. Improving Effectiveness and Out- comes for the Poor in Health, Nutrition, and de Janvry, A., F. Finan, and E. Sadoulet. 2006. Population. Washington, DC: World “Evaluating Brazil’s Bolsa Escola Pro- Bank. gram: Impact on Schooling and Munici- pal Roles.� Berkeley, CA: University of ———. 2009b. “Lessons from World Bank California at Berkeley. Group Responses to Past Financial Cris- es.� Evaluation Brief 6. World Bank. Jensen, R. T. 2004. “Do Private Transfers ‘Dis- Washington, DC. place’ the Benefits of Public Transfers? Evidence from South Africa.� Journal of ———. 2009c. Using Knowledge to Improve Devel- Public Economics, Elsevier, 88(1-2): 89- opment Effectiveness: An Evaluation of 112. World Bank Economic and Sector Work and Technical Assistance. Washington, DC: Kazianga, H., D. de Walque, and H. Alderman. World Bank. 2009a. “Educational and Health Impacts of Two School Feeding Schemes: Evi- IFPRI and El Salvador Social and Economic De- dence from a Randomized Trial in Rural velopment Department. 2009. “Evalua- Burkina Faso.� Policy Research Working cion de Impacto Externa de la Red Soli- Paper Series 4976. Washington, DC: The daria: Informe de Impactos al Ano de World Bank. Implementacion.� Washington, DC: IFPRI. ———. 2009b. “School Feeding Programs and the Nutrition of Siblings: Evidence from 167 BIBLIOGRAPHY a Randomized Trial in Rural Burkina Childhood: Evidence from a Rando- Faso.� Mimeo. mized Experiment.� Policy Research Working Paper Series 4759. Washington, Khandker, Sh. R., M. M. Pitt, and N. Fuwa. 2003. DC: The World Bank. “Subsidy to Promote Girls’ Secondary Education: The Female Stipend Program Maluccio, J.A. 2007. “The Impact of Conditional in Bangladesh.� Mimeo. Washington, Cash Transfers on Consumption and DC: The World Bank. Investment in Nicaragua,� Middlebury College Working Paper Series 0722, Kim, J., H. Alderman, and P.F. Orazem. 1999. Middlebury College, Department of “Can Private School Subsidies Increase Economics. Enrollment for the Poor? The Quetta Urban Fellowship Program� The World ———. 2005. “Coping with the ‘Coffee Crisis’ in Bank Economic Review 13(J): 443-65. Central America: The Role of the Nica- raguan Red de Protección Social.� Dis- Kluve, J., H. Lehmann, and C. M. Schmidt. 2001. cussion Paper 188, Food Consumption “Disentangling Treatment Effects of and Nutrition Division. Washington, Polish Active Labor Market Policies: DC: IFPRI. Evidence from Matched Samples.� IZA Discussion Paper No. 355, Bonn, Ger- Maluccio, J. A., and R. Flores. 2005. “Impact many: Institute for the Study of Labor. Evaluation of a Conditional Cash Trans- fer Program: The Nicaraguan Red de Kluve, J., H. Lehmann, and C. M. Schmidt. 1999. Protección Social.� Research Report 141. “Active Labor Market Policies in Pol- Washington, DC: IFPRI. and: Human Capital Enhancement, Stigmatization or Benefit Churning?“ Marcano, L.L. and I.J. Ruprah. 2008. “An Impact LICOS Discussion Papers 8099, LICOS— Evaluation of Chile’s Progressive Hous- Centre for Institutions and Economic ing Program.� OVE Working Papers Performance, K.U. Leuven. 0608. Washington, DC: Inter-American Development Bank, Office of Evaluation Kristjansson, B. and others. 2006. “School Feed- and Oversight. ing for Improving the Physical and Psy- chosocial Health of Disadvantaged Stu- Martinelli, C. and S. W. Parker. 2008. “Do School dents.� Cochrane Database of Subsidies Promote Human Capital In- Systematic Reviews, Issue 1. Art. No.: vestment among the Poor?� Scandina- CD004676. DOI: vian Journal of Economics 110(2): 261- 10.1002/14651858.CD004676.pub2. 276. Levy, Santiago. 2008. Good Intentions—Bad Out- Martinez, S. 2004. “Pensions, Poverty and comes. Washington, DC: Brookings Insti- Household Investments in Bolivia.� Pre- tution. liminary Version. Department of Eco- nomics, University of California at Levy, D. and J. Ohls. 2007. “Evaluation of Jamai- Berkeley. ca’s PATH Program: Final Report.� Washington, DC: Mathematica Policy Milazzo, Annamari and Grosh, Margaret. 2008. Research. “Social Safety Nets in World Bank Lend- ing and Analytical Work, 2002-07�. SP Lindert, Kathy, and Vanina Vincensin. 2008. Discussion Paper No. 0810. World Bank. “Bolsa Família in the Headlines: An Washington DC. May. Analysis of the Media’s Treatment of Conditional Cash Transfers in Brazil.� Nieves Valdés, M. 2008. “Did PROGRESA Send World Bank. Washington DC. Drop-outs back to School?“ Economics Working Papers we085926, Universidad Macours, K., N. Schady, and R. Vakis. 2008. Carlos III, Departamento de Economía. “Cash Transfers, Behavioral Changes, and Cognitive Development in Early 168 BIBLIOGRAPHY Olinto, P., S.S. Morris, R. Flores, and A. Veiga. Ravallion, Martin. 2002. “Are the Poor Protected 2003. “The Impact of the Bolsa Alimen- from Budget Cuts? Evidence from Ar- tacao Program on Food Consumption.� gentina.� Journal of Applied Economics, Paper presented at Annual Meetings of Vol.V. No. 1: 95-121 the International Association of Agricul- ———. 2003. “Targeted Transfers in Poor Coun- tural Economists: Durban, South Africa. tries. Revisiting the Tradeoffs and Policy Oosterbeek, H., J. Ponce, and N. Schady. 2008. Options�. WPS3048. World Bank. Wash- “The Impact of Cash Transfers on ington DC. May. School Enrollment : Evidence from Ec- ———. 2007. “How Relevant is Targeting to the uador.� Policy Research Working Paper Success of an Antipoverty Program?� Series 4645. Washington, DC: The World WPS4385. World Bank. Washington, Bank. DC. November. Paitoonpong, Srawooth et.al 2008. “The meaning ———. 2009a. “Bailing Out the World’s Poor- of “social safety nets�. Journal of Asian est.� Challenge. 52(2):55-80. Economics. 19 (5-6): 467-473, November- December. ———. 2009b. “Do Poor Countries Have Less Capacity for Redistribution?� World Parker, S.W. and E. Skoufias. 2000. “The Impact Bank Policy Research Paper No. 5046, of PROGRESA on Work, Leisure, and Washington, DC. Allocation.� Mimeo. Washington, DC: IFPRI. ------------. 2009c. “Pro-Poor $timulus. Lessons from the Developing World.� Paxson, C. and N. Schady. 2007. “Does Money Matter ? The Effects of Cash Transfers Ravallion, M. and S. Chen. 2005. “Hidden Im- on Child Health and Development in pact: Household Saving in Response to Rural Ecuador.� Policy Research Work- a Poor-Area Development Project.� ing Paper Series 4226. Washington, DC: Journal of Public Economics, Elsevier, The World Bank. 89(11-12): 2183–2204. Pianto, D.M. and S. Soares, 2004. “Use of Survey Ravallion, M., and Datt G. 1994. “Transfer Bene- Design for The Evaluation Of Social fits from Public Works Employment: Programs: The PNAD and PETI.� Anais Evidence for Rural India.� do XXXII Encontro Nacional de Econo- Ravallion, M. and Q. Wodon. 1999. “Does Child mia [Proceedings of the 32th Brazilian Labor Displace Schooling? Evidence on Economics Meeting] 133, ANPEC— Behavioral Responses to an Enrollment Associação Nacional dos Centros de Subsidy.� Policy Research Working Pa- Pósgraduação em Economia [Brazilian per Series 2116. Washington, DC: The Association of Graduate Programs in World Bank Economics]. Ravallion, M., Datt G, and Chaudhuri S. 1993. Ponce, J. and A. S. Bedi. 2008. “The Impact of a “Does Maharashtra’s Employment Cash Transfer Program on Cognitive Guarantee Scheme Guarantee Employ- Achievement: The Bono de Desarrollo ment: Effects of the 1988 Wage In- Humano of Ecuador.� IZA Discussion crease.� Washington, DC: The World Papers 3658. Bonn, Germany: IZA Dis- Bank cussion Papers 121, Institute for the Study of Labor. Ravallion, M., E. Galasso, T. Lazo, and E. Phi- lipp. 2001. “Do Workfare Participants Quisumbing, A. R. 2003. “Food Aid and Child Recover Quickly from Retrenchment?“ Nutrition in Rural Ethiopia.� FCND dis- Policy Research Working Paper Series cussion papers 158. Washington, DC: 2672. Washington, DC: The World Bank. IFPRI. 169 BIBLIOGRAPHY Rawlings, L.B. and G.M. Rubio. 2005. “Evaluat- the Food for Education Program in Ban- ing the Impact of Conditional Cash gladesh on Schooling Outcomes and Transfer Programs.� The World Bank Re- Earnings.� Impact Assessment Discus- search Observer, 20(1). sion Paper No. 22. Washington, DC: IFPRI. Raymond, M. and E. Sadoulet. 2003. “Educational Grants Closing the Gap in Sadoulet, E., A. de Janvry, and B. Davis. 2001. Schooling Attainment between Poor and “Cash Transfer Programs with Income Non-Poor.� Working Paper Series 986, Multipliers: PROCAMPO in Mexico.� Department of Agricultural & Resource World Development, Elsevier, 29(6): Economics, University of California 1043-56. Berkeley. Scandizzo, P., R. Gaiha, and K. Imai. 2009. “Does Ribe, Helena, David Robalino, and Ian Walker. the Employment Guarantee Scheme 2010. Achieving Effective Social Protec- Stabilize Household Incomes in Rural tion for All in Latin America and the India?“ Applied Economics Letters, Tay- Caribbean: From Right to Reality.� lor and Francis Journals, 16(12): 1221-5. Washington, DC: The World Bank. Schady, N. and M. Caridad Araujo. 2006. “Cash Rivera, J. A., D. Sotres-�lvarez, J-P. Habicht, T. Transfers, Cconditions, School Enroll- Shamah, and S. Villalpando. 2004. “Im- ment, and Child Work: Evidence from a pact of the Mexican Program for Educa- Randomized Experiment in Ecuador.� tion, Health, and Nutrition Policy Research Working Paper 3930. (PROGRESA) on Rates of Growth and Washington, DC: The World Bank. Anemia in Infants and Young Children: Schady, N. and J. Rosero. 2007. “Are Cash Trans- A Randomized Effectiveness Study.� fers Made to Women Spent Like Other Journal of the American Medical Associa- Sources of Income?� Policy Research tion 291(21): 2563–70. Working Paper 4282. Washington, DC: Rodriguez-Planas, N., and J. Benus. 2006. The World Bank. “Evaluating Active Labor Market Pro- Schultz, T. Paul. 2004. “School Subsidies for the grams in Romania.� IZA Discussion Pa- Poor: Evaluating the Mexican pers 2464. Bonn, Germany: IZA Discus- PROGRESA Poverty Program.� Journal sion Papers 121, Institute for the Study of Development Economics 74(1): 199– of Labor. 250. Rubio-Codina, M. 2009. “Intra-household Time ———. 2000. “Impact of PROGRESA on School Allocation in Rural Mexico: Evidence Attendance Rates in the Sampled Popu- from a Randomized Experiment.� Lon- lation.� Washington, DC: IFPRI. don, UK: The Institute for Fiscal Studies. Skoufias, E. and V. di Maro. 2006. “Conditional Ruiz-Arranz, M., B. Davis, M. Stampini, P. Win- Cash Transfers, Adult Work Incentives, ters, and S. Handa. 2002. “More Calories and Poverty.� Policy Research Working or More Diversity? An Econometric Paper Series 3973. Washington, DC: The Evaluation of the Impact of the World Bank. PROGRESA and PROCAMPO Transfer Programmes on Food Security in Rural Skoufias, E., M. Unar, and T. Gonzalez-Cossio. Mexico.� Working Papers 02-09, Agri- 2008. “The Impacts of Cash and In-Kind cultural and Development Economics Transfers on Consumption and Labor Division of the Food and Agriculture Supply: Experimental Evidence from Organization of the United Nations Rural Mexico.� Policy Research Work- (FAO—ESA). ing Paper Series 4778. Washington, DC: The World Bank. Ryan, J. G. and X. Meng. 2004. “The Contribu- tion of IFPRI Research and the Impact of 170 BIBLIOGRAPHY Sparrow, R. 2007. “Protecting Education for the Vermeersch, C. and M. Kremer. 2004. “School Poor in Times of Crisis: An Evaluation Meals, Educational Achievement, and of a Scholarship Programme in Indone- School Competition: Evidence from a sia.� Oxford Bulletin of Economics and Randomized Evaluation.� Policy Re- Statistics 69(1): 99–122. search Working Paper 3523. Washing- ton, DC: The World Bank. Stecklov, G., P. Winters, M. Stampini, and B. Davis. 2005.� Do Conditional Cash Wagstaff, Adam. 2010. “Estimating Health In- Transfers Influence Migration? A Study surance Impacts under Unobserved He- Using Experimental Data from the Mex- terogeneity: The Case of Vietnam’s ican Progresa Program.� Demography, Health Care Fund for the Poor.� Health 42(4): 769-90. Economics 19: 189-208. Stifel, D. and H. Alderman. 2003. “The “Glass of Weigand and Grosh. 2008 “Levels and Patterns Milk� Subsidy Program and Malnutri- of Safety Net Spending in Developing tion in Peru.� World Bank Economic and Transition Countries�. Discussion Review, 20(3): 421–48. Paper 0817. Washington, DC: The World Bank. Subbarao, K. Del Ninno, C. Andrews, C. Rodríguez-Alas, C. Subbarao and oth- ———. 2010. “Designing and Implementing a ers. 2010. “The Design and Implementa- Rural Safety Net in a Low-Income Set- tion of Public Works Programs: A Tool- ting: Lessons Learned from Ethiopia’s kit for Practitioners.� Washington, DC: Productive Safety Net Program, 2005– The World Bank. 2009. World Bank. Washington DC. Tan, J-P., J. Lane, and G. Lassibille. 1999. Wodon, Quentin, and Hassan Zaman. 2009. “Student Outcomes in Philippine Ele- “Higher Food Prices in Sub-Saharan mentary Schools: An Evaluation of Four Africa: Poverty Impact and Policy Res- Experiments,� World Bank Economic Re- ponses.� World Bank Research Observer view 13(3): 493-508. November 11, 2009. Teruel, G. and B. Davis. 2000. “Final Report: An World Bank. 1990. “World Development Report, Evaluation of the Impact of PROGRESA World Bank 1990: Poverty�. World Cash Payments on Private Inter- Bank. Washington, DC. Household Transfers.� Washington, DC: ———. 2000. “World Development Report, IFPRI. World Bank 2000/01: Attacking Pover- van de Walle, Dominique, and Kimberly Nead ty�. World Bank. Washington, DC. Au- (eds.). 1995. Public Spending and the Poor: gust. Theory and Evidence. Baltimore: Johns ———. 2001. “Social Protection Sector Strategy. Hopkins University Press. From Safety Net to Springboard.� van Ours, J. C. 2000. “Do Active Labor Market World Bank. Washington DC. Policies Help Unemployed Workers to ———. 2005. Old Age Income Support in the 21st Find and Keep Regular Jobs?“ Bonn, Century: An International Perspective on Germany: IZA Discussion Papers 121, Pension Systems and Reform. Washington, Institute for the Study of Labor. DC: World Bank. Veras Soares, F., R. Perez Ribas, and G. I. Hirata. ———. 2007. "Hashemite Kingdom of Jordan: A 2008. “Achievements and Shortfalls of Note on Strategy for Modernization of Conditional Cash Transfers: Impact the Social Safety Nets." Report No. Evaluation of Paraguay’s Tekoporã Pro- 41663-JO. World Bank. Washington DC. gramme.� Publications 3, International Policy Centre for Inclusive Growth. ———. 2008a. “For Protection and Promotion: The Design and Implementation of Ef- 171 BIBLIOGRAPHY fective Safety Nets.� Social Protection Bank Human Development Network. Department. Report No. 1105. World Bank. Washing- ton DC. ———. 2008b. “Guidance for Responses from the Human Development Sector to Ris- Yamano, T., H. Alderman, and L. Christiaensen. ing Food and Fuel Prices.� World Bank. 2005. “Child Growth, Shocks, and Food Washington DC. Aid in Rural Ethiopia.� American Jour- nal of Agricultural Economics, Ameri- ———. 2009a. “Assessing the Use of Poverty can Agricultural Economics Association, and Social Impact Analysis in World 87(2): 273-88. Bank Development Policy Loans.� World Bank, Washington, DC. Yap, Y-T., G. Sedlacek, and P. Orazem. 2008. “Limiting Child Labor through Beha- ———. 2009b. “DPL Retrospective: Flexibility, vior-Based Income Transfers: An Expe- Customization, and Results.� OPCS pa- rimental Evaluation of the PETI Pro- per. gram in Rural Brazil.� In Child Labor and Education in Latin America: An Economic ———. 2009c. “Operational Response to the Cri- Perspective, eds. Orazem, P., G. Sedlacek, sis. SecM2009-0168 and Financial Crisis and Z. Tzannatos. Palgrave. Response and Use of Instruments�. World Bank and Johns Hopkins University. SecM2009-0167d. 2010. “Asset Creation, Productive Safety Nets and Poverty Reduction: Learning ———. 2010. Achieving Social Protection for All in from the “Atención a Crisis� Program in Latin America and the Caribbean. Wash- Nicaragua’s Drought Region.� Washing- ington, DC: World Bank. ton, DC: The World Bank. Mimeo. ———. 2011. “Results Readiness in Social Pro- tection and Labor Operations.� World 172 Endnotes Chapter 1 1 Recent theoretical and empirical research points to a complementarity rather than a trade-off be- tween equity and efficiency with safety nets: with the failure of credit and insurance markets to pro- tect the poor in developing countries, safety nets permit households to manage their risks more effi- ciently and make wiser investments in their futures. In addition, safety nets facilitate structural reforms to the economy by replacing inefficient redistributive elements in other programs and by protecting the poor and vulnerable who may be hurt by otherwise growth enhancing reforms (Raval- lion 2003; Alderman and Hoddinott 2007; Grosh and others 2008). 2 The definition of SSNs used most often by the Bank, and used here to assess the Bank’s work in this area, is most similar to “social assistance� or “social welfare programs.� Other donors define SSNs in various ways, as does the academic literature. Indeed, the Bank’s use of SSNs has evolved throughout the past, but for the past decade, during the period of evaluation, it has specified this definition and used it to classify its SSN lending portfolio. For a history and timeline of the Bank’s involvement in SSNs see appendix A. 3 The Bank’s social protection strategy in 2001 emphasized the risk elements (function 3 and 4) of these functions, although the Bank’s SSN work seeks to address the broader five objectives. 4 These functions are articulated, to different degrees, in the broader literature on SSNs and elabo- rated by the Social Protection Department on its Web site as well as in their flagship publication, “For Protection and Promotion: The Design and Implementation of Effective Safety Nets.� 5 For example, social protection programs include labor policy, contributory social insurance, and social care services; social risk management includes services for small farmers such as irrigation, mi- crofinance, and weather insurance; and policies to improve equity include universal education and land redistribution; and poverty reduction programs include services that facilitate the poor’s partic- ipation in growth. Safety nets can also contribute to all these objectives (Grosh and others 2008). 6 Assessing the Bank’s broader support for social protection is beyond the scope of this evaluation. Nevertheless, the evaluation does examine if the Bank’s limited definition of SSNs has led to a nar- row approach to support or whether it has been able to expand beyond the boundaries of its defini- tion and support a broader set of household risks within the Bank’s country program (see chapter 4 for Support to SSNs within Broader Social Protection Systems). 7 Eighty percent of respondents to an IEG survey said their country had been affected; see appendix B, section VII. 8 World Bank financial crisis Web page. http://www.worldbank.org/financialcrisis/ 9 Mean spending is 1.9 and median spending is 1.4 percent of GDP (Weigand and Grosh 2008). 10 In Macedonia, the Bank supported additional benefits for a CCT program for households with children ages 15-19 yeasr old and serves as a top up of the broader Social Financial Assistance benefit financed by the government. 11 Although this is the first IEG evaluation of SSNs, this work builds on a number of previous evalua- tions on related topics including: “Evaluation of the World Bank’s Support to Poverty and Social Im- pact Analysis� (IEG 2009); Using Knowledge to Improve Development Effectiveness: An Evaluation fo World Bank Economic and Sector Work and Technical Assistance (IEG 2009b); Safeguards and Sustainability Poli- 161 ENDNOTES cies in A Changing World: An Independent Evaluation of World Bank Group Experience (IEG 2010c); Cost Benefit Analysis of World Bank Projects (IEG 2010b); The World Bank Group’s Response to the Global Eco- nomic Crisis: Phase I (IEG 2011d); Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population (IEG 2009a); and Water and Development: An Evaluation of World Bank Support 1997–2007 (IEG 2010d). 12 For this evaluation, AAA includes economic and sector work (ESW), nonlending technical assis- tance, impact evaluation, and global knowledge. Other types of AAA include donor aid coordination, research services, the World Development Report, and external training. Under certain conditions, the Bank also provides AAA on a fee-for-service basis. Chapter 2 1 Commitments do not include Recipient Executed Trust Funds. 2 Countries were classified as low income countries and middle income countries according to the Bank’s definition in 2005 (the mid decade year for the portfolio review). 3 See Figure D1 and D2 in appendix D for trends in SSN lending and number of operations by coun- try income level. 4 Loan size refers to the share of the total loan directed towards SSNs. (See portfolio review metho- dology in appendix B.) 5 This is based on the World Development Indicators data and a poverty line of $2/day. Results hold after conducting sensitivity analysis using $1.25/ day and national poverty lines. 6 The results are affected by China and India, countries that are large Bank borrowers but small SSN borrowers with vast numbers of poor. When this analysis is repeated without China and India in the top 10, results remain similar, though they are not quite as dramatic (48 percent of Bank lending and 46 percent of the poor). 7 Regional and income effects are not significant. For more details on the SSN uptake analysis, see appendix B. 8 Recipient Executed Trust Funds are funds that the Bank passes on to a third party recipient for de- velopment activities, typically financing the investment and recurrent needs of service delivery, ca- pacity building, and technical assistance. The Bank normally plays an operational role for these funds, including appraisal and supervision of funded activities. 9 As of May 2010, the Catalytic Trust Fund has been distributed as follows: 48 percent to the Africa Region, 15 percent to South Asia, and 6 percent to the rest of the Regions. 10 The WBG President’s Contingency Fund funded the SSNs CCF. The CCF provides money for in- cremental activities in SSNs such as: (i) technical and policy advice leading to new and/or expanded operations; (ii) scale up existing programs; and/or (iii) new SSNs programs or benefits. 11 As of October 2010, multidonor trust fund disbursements constitute 63 percent ($37.655 million) of total amount committed. The Africa Region received 45 percent of the funds, followed by South Afri- ca (18 percent), the Middle East and North Africa (11 percent), Europe and Central Asia (9 percent), East Asia and Pacific (8 percent), and Latin America and the Caribbean (6 percent). 12 Objectives are not mutually exclusive. 13 IEG flagged whether any of the following SSNs programs were used in each project: CCTs (includ- ing education, health, and nutrition); UCTs (including income support, severance/unemployment, 162 ENDNOTES family/child allowances, non-contributory pensions, disability benefits, other), in-kind transfers (in- cluding food, basic transfers); health, and education subsidies; energy, water, and housing subsidies; and PWPs. 14 Fragile states in 2005 15 58 countries had at least one operation between FY08 and FY10. Denominator is total IBRD/IDA countries (N = 137). 16 Trust funds financed $16.8 million Bank-executed ESW and nonlending technical assistance activi- ties. 17 The other 38 percent corresponds to regional studies. 18 The percentages do not add to 100 percent because the regional studies have no income classification. Chapter 3 1 This difference is statistically different from 0 at the 5 percent significance level. 2 Project performance was tested using the full portfolio as well as those with explicit SSN objectives. Results were consistent in both cases (when N = 92 and when N = 71). Eighty-six percent for SSNs compared to 78 percent for all Bank projects were rated moderately satisfactory or higher. 3 Definition of quality of the results framework: The proportion of “good� attributes that the results framework has adopted. Good attributes are defined as: baseline, impact evaluation, linkage of out- put and outcomes to development objectives, outcome and output targets are clearly articulated, are time-bound, and specify target group. 4 Sources include the portfolio review, 30 country case studies, Project Performance Assessment Re- ports, and the staff survey on the food, fuel, and financial crises and SSNs. 5 Key issues related to the country context include poverty challenges, administrative capacity, avail- ability of complementary infrastructure and services, political acceptability of various program op- tions, other existing programs and institutions, and involvement of other donors. Key issues related to SSN design include clear identification of specific target group; eligibility criteria and targeting method; and amount, length, and timing of benefits. 6 Operations were randomly selected, stratified by region and proportioned to the the number of Bank supported PWPs per region. 7 There is no single World Bank policy on when an operational manual needs to be completed. The issue of when operational manuals should be completed has changed throughout the decade and va- ries by region. For example, in the LAC region, having a completed operational manual has been generally a condition of project effectiveness; however, in the past few years, the completion of opera- tional manuals prior to negotiations have been strongly encouraged. 8 The initial experience with this type of programs has been mainly in Latin America and the Cari- bean. Mexico introduces the first CCT program, Progresa (later Oportunidades), in 1997. The Bank first supported CCts in 2001 in Colombia, Jamaica and Turkey. However, new CCT programs are being piloted in Nigeria, Tanzania, and Lao PDR. Even developed countries, including the United States, are piloting CCTs based on the Latin America experience. 9 This finding is evident from the broader SSN literature as well as the IEG country case studies and project Project Performance Assessment Reports of Jamaica and Colombia. 163 ENDNOTES 10 Brazil and Chile also scaled up existing CCT programs in response to the crisis, but these were not supported by the Bank at the time. 11 In the last few years, however, the Bank started paying attention also to the access and sources of chronic poverty and food insecurity, with SSN dialogue starting in 2007. In 2008 the Bank’s work in Niger benefitted from the crisis window trust funds and used part of the money for developing a SSN system. The government has put in place a cash transfer pilot for 2,500 people that is fully operation- al. 12 IEG analyzed 122 randomly selected projects with SSN components, stratified by fiscal year of ap- proval. 13 Performance indicators should be SMART—specific, measurable, achievable, realistic, and time- bound—and aligned so that outputs can achieve intermediate outcomes and final outcomes. 14 For example, a CCT can encourage school attendance among poor children in the short term, but what is sought is higher learning and more opportunities for them in the longer run. This requires quality education for which improvements in the education sector are necessary. 15 The findings of IEG’s results framework analysis are also consistent with those of a few recent, widely disseminated studies. Recent IEG evaluations of the health, water, and education sectors, for example, have pointed out similar problems with the results frameworks. The Bank’s Human Devel- opment (HD) Network assessment of SSN projects, “Results Readiness in Social Protection and Labor Operations� (World Bank 2011) found that key performance indicators do not consistently measure SSN project objectives, particularly with regard to institutional development and service delivery, the most common objectives of SSN projects. This gap in the results chain is referred to as “the missing middle� and was confirmed by IEG’s assessment of intermediate outcome indicators. The report re- commends further development of indicators to measure the gap. These findings are also consistent with concerns raised by an in-depth review of monitoring and evaluation within the Human Devel- opment portfolio of Latin America and the Caribbean. This is consistent with the health sector where IEG’s evaluation of the health, nutrition, and population sector indicated that 27 percent of projects did not have baseline values. 16 Similar impacts are also identified in the impact evaluation literature for other comparable pro- grams that have not been supported by the World Bank. 17 Whereas SSNs subjected to impact evaluation appear to increase learning outcomes for young children, it is difficult to establish a pattern among the few studies that assessed the impacts on learn- ing for older children. 18 For some SSN programs, analysts note the multiple indirect effects of various policy alternatives and the technical difficulty of capturing them all. While this is true for SSNs, it is also true for most areas of public policy and should not deter the definition of explicit program objectives and a rigor- ous assessment of policy alternatives to achieve them. For example, if improving learning is the prin- cipal outcome sought, then various policies for improving test scores should be considered. If raising consumption is the primary objective, then a CCT should be compared to a UCT. In contrast, if the principal development benefit of a CCT program is reduced fertility, and the program is justified be- cause of this outcome, then various alternatives for achieving this outcome should be assessed. 19 A few recent CCTs have used cost-benefit analysis to assess rates of return to CCT projects. Relying on assumptions regarding returns to education they calculate the program investment and find them positive. These assumptions about program benefits need to be confirmed through evaluation of longer-term impacts. 164 ENDNOTES 20 IEG only began assessing risk to development outcome in 2006, so these 53 projects represent only the latter half of the decade. 21 In Macedonia, the Bank supported an additional benefit which helped top up the government’s sociall assiatance program with a CCT component for households with children ages 15-19 yeard old. Hence the Bank’s financing of the CCT as part of a broader social program financed by the govern- ment. 22 Ethiopia is a notable exception where years of uncoordinated donor emergency relief were ended when the Productive Safety Net Program was initiated in 2004 with the primary goal of coordinating donor food aid and having a more efficient and predictable form of assistance during the hungry sea- son each year. 23 The increase in the weight of the Mexican program as a share of GDP also reflects the drop in GDP and slowdown of the economy as a result of the crisis even without program expansion. 24 IEG country case study. 25 See Annex C for suggestions on types of measures that could insulate SSNs from political manipu- lation. Chapter 4 1 In evaluation terminology, this is referred to as relevance of objective and relevance of design. 2 To assess the relevance of Bank support, IEG examined case studies for the relevance of SSNs in that country to address each function/objective and the extent to which the Bank’s support to SSNs fo- cused on each function in that country. The relevance of each SSN function for that country was based on different criteria, drawn from the literature as well as discussions with SSN and poverty experts. Based on this assessment, some functions are more relevant than others within a country context. For example, some countries are more prone to natural disasters and systemic shocks than others and would receive a higher SSN country relevance ranking for Function 3. Some countries are lacking basic education and health services, so using an SSN such as a CCT to address education needs of the poor may not be as relevant as in a country with adequate supply but where stimulating demand among the poor is most important. Thus, the relevance ranking for Function 2 would differ between the two countries. The Bank’s focus on each SSN function was assessed drawing on a range of documents including CASs, country SSN or social protection strategies, project documents, lend- ing, and AAA programs. It should be noted that one SSN instrument can address more than one SSN function. This was taken into account in determining the focus of Bank’s SSN efforts. A country’s re- levance rating for each SSN function was compared to the rating for the Bank’s SSN focus on that function and the gap between the two ratings was used to determine the relevance of the Bank’s fo- cus in that country. Results were aggregated across countries to derive average relevance of the Bank’s efforts for each SSN function (see methodology in appendix B). 3 IEG country case studies and Results Framework Analysis of IEG portfolio review. 4 Evidence is drawn most confidently from countries which have conducted rigorous impact evalua- tions. However, these countries often have other features that make them more likely to succeed (such as motivated government or longer-term horizon for government and Bank efforts). 5 When looking beyond Bank-supported SSN programs, the positive evidence from impact evalua- tions of SSNs (IEG 2011c) is even more impressive and makes a compelling case for using SSN to ad- dress Function 1. 165 ENDNOTES 6 Beyond countries in the Latin America and the Caribbean Region, CCTs have been evaluated to have positive impacts on intermediary school outcomes in Turkey (Social FSSP) and Cambodia (Cambodia Education Sector Support Project Scholarship Program, CESSP) (IEG 2011c). 7 There were no positive effects for tests given to adolescents close to finish high school who were covered by the program in Mexico (Behrman and others 2005), Cambodia (Filmer and Schady 2009b) , and in Colombia (IEG 2011a). For school completion evidence of positive impacts in Mexico (Behr- man and others 2005) and Colombia (IEG 2011a) are documented. 8 It is difficult to assess impacts of idiosyncratic shocks using impact evaluations, and there have been no rigorous studies on Bank-supported SSN programs explicitly addressing idiosyncratic shocks (for example, health insurance and pensions for the poor). The limited evidence from three CCT pro- grams in Latin America and the Caribbean (Honduras, Mexico, and Nicaragua) shows that CCTs helped households keep children in school and prevented them from engaging in child labor when households were faced with individual shocks. There is no evidence available for the effectiveness of other types of SSN programs (other than CCTs) or in countries outside of the Latin America and the Caribbean Region. 9 Operational Policy 8.60 established that “the Bank determines whether specific country policies supported by the operation are likely to have significant poverty and social consequences, especially on poor people and vulnerable groups. For country policies with likely significant effects, the Bank summarizes in the Program Document relevant analytic knowledge of these effects and of the bor- rower’s systems for reducing adverse effects and enhancing positive effects associated with specific policies being supported. If there are significant gaps in the analysis or shortcomings in the borrow- er’s systems the Bank describes in the Program Document how such gaps or shortcomings would be addressed before or during program implementation, as appropriate.� 10 Effective systems and institution building becomes more complex when working in federal struc- tures (as in Argentina, Brazil, India, Nigeria, and Pakistan), but also when taking into account the separate and varied needs of central and local government agencies. Whether the Bank has worked in formally federal countries or not, it normally has addressed the issue of division of responsibilities between different levels of government in the design, implementation, management, and financing of SSN programs. 11 The PARIS21 Task Team on Improved Support to Monitoring Development Goals has produced recommendations relating to improving the estimation of key indicators, in both national and inter- national processes. 12 The Social Risk Management framework lays out how the social protection system responds to multiple sources of household and systemic risk, and their characteristics address vulnerability and prevent negative consequences on social and economic development. 13 The country reviews were based on Country Assistance Strategy documents, the lending portfolio, AAA on poverty and social protection, and relevant economic reports as well as interviews with Bank staff and management. Chapter 5 1 Several sources are used to provide insights on when and how the Bank has been most effective at influencing the development of sound SSNs. These sources include the 30 case studies, the staff sur- vey of on the SSN response to the food, fuel, and financial crises, staff and management surveys re- garding the organizational structure of the Bank, background papers on political economy, decentra- lization, review of DPLs and SSNs, review of Bank involvement with public works programs, 166 ENDNOTES interviews with participants in south-south learning events—almost all of which involved interviews with Bank staff and management and sometimes clients as well. 2 In the case of Mexico, the Progresa program changes names to Oportunidades under new political leadership. 3 In only 7 countries (23%) was there no movement toward tighter targeting. Although Chile was not one of the randomly selected case studies featured throughout the report, its case is noteworthy as it is at the forefront of development of social protection systems. It has a unique family allowance bene- fit which has expanded in coverage and broadened in scope and now reaches the bottom 40% of the population. 4 Bank identified opponents and proponents of reform in 47% of countries and the Bank gained suffi- cient or at least partial buy-in from stakeholders in 80% of country cases. There was a clear policy champion within the government in 67% of countries. This information was obtained from detailed question in country case studies. 5 In 12 countries change in political leadership has been an important circumstance for SSN reform advancement. This information was obtained from country case studies by comparing the timing of SSN reform advancement to political changes and other important events in the countries. 6 Sustained engagement was determined in the country case studies through examination of a detailed timeline of Bank involvement throughout the decade. Chapter 6 1 This is consistent with a recommendation put forward in IEG’s recent crisis response evaluation (IEG 2011d). Appendix A 1 Before the 1980s there were two important country developments that influenced the SSN agenda: (1) The Employment Guarantee Scheme in India in the 1970s, which made India the first country in the developing world to implement a PWP designed to serve as an SSN to protect the poor, and (2) Chile’s invention of the Proxy Means Test targeting methodology, widely used in CCT programs that have since spread throughout Latin America and the world. 1 SSNs Portfolio Review. 2 http://www.worldbank.org/financialcrisis/bankinitiatives.htm 3 SSNs Portfolio Review. 4 http://intranet.worldbank.org/WBSITE/INTRANET/SECTORS/SOCIALPROTECTION/ 0,,contentMDK:22438786~menuPK:6710095~pagePK:210082~piPK:254376~theSitePK:280559,00.html 5 http://www.worldbank.org/financialcrisis/bankinitiatives.htm Appendix B 1 Project task team leaders assign thematic codes to projects. The team leader may use up to five codes for each project. The guidance note for coding defines thematic code 54 as “Activities intended 167 ENDNOTES to provide social assistance (that is, in kind or cash assistance to poor and vulnerable individuals or families to help them cope with consequences of economic or other shocks). It includes: benefits in- cash to vulnerable groups; benefits in-kind to vulnerable groups; subsidies for goods to vulnerable groups; subsidies for services to vulnerable groups; workfare programs; other risk coping activities.� 2 Out of the 62 projects in the anchor’s FY07-09 dataset, 17 were Recipient-Executed Trust Funds, Spe- cial Financing, or supplemental financing, leaving a total of 45 free-standing IBRD/IDA lending op- erations. 3 IEG flagged whether any of the following SSN programs were used in each project: CCTs (including education, health, and nutrition); UCTs (including income support, severance/unemployment, fami- ly/child allowances, non-contributory pensions, disability benefits, other), in-kind transfers (includ- ing food, basic transfers); health, and education subsidies; energy, water, and housing subsidies; and PWPs. 4 Many DPLs are designed to help implement macro reforms, such as structural adjustment or sector reform. However, a great number specifically support country Poverty Reduction Strategies, which tend to support other safety net functions. 5 The list was pulled from an internal Bank database as of July 30, 2010. 6 The actual percentage was 27 percent. However, as the majority of the projects were from FY00–07, IEG consulted with its internal database analyst. The analyst informed IEG of a possible mistake in the classification of these projects as they did have a final delivery to the client date. IEG decided to include them as closed projects as the probability that they have been active for more than five years is very low. 7 The actual percentage of active projects was 44 percent. However, the majority of the projects were from FY00 to FY07. IEG consulted with the database analyst. The analyst informed IEG of a possible mistake in the classification of these projects as they had a final delivery to the client date. IEG de- cided to include them as closed projects as the probability that they have been active for more than five years is very low. 8 IEG searched in its operating system, client connection, operational portal, and the internal data- base. 9 As of FY10, trust funds are assigned thematic codes. 10 IDF is financed by IBRD money. 11 Other donors include: Belgium-Directorate General for Development Cooperation, Canadian In- ternational Development Agency, Danish Ministry of Foreign Affairs, European Union-Commission of the European Communities, France - Ministry of Foreign Affair (including Ministry of Coopera- tion, Germany-Deutsche Gesellschaft Fur Technische Zusammenarbeit, Italy—Ministry of Foreign Affairs, Netherlands Minister for Development Cooperation, Norwegian Agency for Development Cooperation (NORAD), Swedish International Development Cooperation Agency, United Kingdom - Department for International Development, Australian Agency for International Development, Fin- land - Ministry for Foreign Affairs, Iceland - Ministry of Foreign Affairs, India-Ministry of External Affairs India, Japan - Ministry of Finance, Korea - Ministry of Foreign Affairs and Trade, Kuwait - Council of Ministers Government of State of Kuwait, QATAR-Ministry of Foreign Affairs, Spain - Ministry of Economy, TR-The Undersecretariat of Treasury, United States Agency for International Development, Government of the Russian Federation, EU-Commission of the European Communi- ties. 168 ENDNOTES 12 The coefficient on borrower category dummies is sensitive to how income or prior SSN take up is defined. The coefficient on the share of total country borrowing on overall lending is statistically sig- nificant. 13 However, in regressions where the dependent variable is share of SSN commitments in total Bank lending to the country, the coefficient on crisis is not statistically significant. 14 However, in regressions where the dependent variable is share of SSN commitments in total Bank lending to the country, the coefficient on share of SSN commitments in total Bank lending to country in 1990–99 is not statistically significant. 15 The identification of SSN operations in the Bank’s portfolio is explained elsewhere in the report. 16 There are some differences between SSN projects that had an SSN objective versus those that did not. We found that SSN projects in the portfolio that did not have an explicit SSN objective are less likely to have a high performance rating, a cash transfer component and less likely to have a good quality results framework. They are also more likely to be in the East Asia and Pacific Region, com- pared with projects with an SSN objective and included in our sample 17 There are no East Asia and Pacific-based SSN operations in this sample. 18 There is a wide range of social protection instruments/interventions. These include cash and near cash transfer programs (food stamps, family allowances, non-contributory pension); in-kind transfers (supplemental feeding, emergency food distribution, quantity rations); general price subsidies (ener- gy subsidy); public works (usually labor-intensive infrastructure development projects); conditional cash transfers (targeted transfers conditioned on school attendance and preventative health care, etc); fee-waivers, exemptions and scholarships, among others (World Bank 2008b). 19 The efficacy rating is based on a four-point scale: high, substantial, modest and low/negligible. 20 The efficacy rating is based on a four-point scale: high, substantial, modest and low/negligible. 21 1 refers to a rating of highly unsatisfactory, 2 is unsatisfactory, 3 is moderately unsatisfactory, 4 is moderately satisfactory, 5 is satisfactory, and 6 is highly satisfactory. 22 The three categories are still hierarchical (ordinal) in nature. 23 Based on sample distribution, only projects achieving a rating of 5–6 could be defined as perform- ing better than average 24 We had previously defined a binary variable satisfactory/unsatisfactory (1/0), whereby operations with a rating of 4–6 were assigned a 1 (satisfactory) while those with ratings (1–3) were assigned a (unsatisfactory). However, 61 of 71 operations had a rating of 4 or higher; given the limited sample size and only 10 observations with a 0, the estimation of conditional correlations would have been problematic. For instance, under this definition, all Emergency Recovery Loans have a 1 (so 100 per- cent correlation). In general, dichotomous grouping is lumpy and does not always allow us to detect variations within a group. 25 Because duration is high correlated with instrument type, we cannot introduce instrument type in a pooled specification that includes duration. Within their own cohort, fast disbursing loans of short- er duration did not have ratings that were statistically different from longer fast disbursing loans of longer duration. This result is however sensitive to use of controls. In contrast, shorter investment loans were more likely to be associated with higher ratings than longer investment loans. 169 ENDNOTES 26 It might be that the outcomes being measured between two instruments are qualitatively different or the level of ease/certainty in achieving outcomes is significantly different between the instrument types (for for example, DPL triggers). These aspects are not accounted for in the regression analysis 27 “Results Readiness in Social Protection and Labor Operations,� Social Protection and Labor, Hu- man Development Network, April 2010. 28 The Africa Region was further stratified between Anglophone and Francophone countries. 29 At the outset, 20 DPLs were targeted. One had been cancelled, and no response was received on three others. 30 Some respondents have multiple affiliations; that is, they may be Sector Board members while also being in the SSN GET Team and/or managing a relevant project. 170