Approach Paper Adaptive Social Protection for Effective Crisis Response Independent Evaluation Group Evaluation of the World Bank’s Contribution July 24, 2023 1. Background and Context 1.1 Interconnected and often devastating covariate shocks are a threat to human development. Covariate shocks are shocks that affect large numbers of people or communities at once and can be natural, economic, or political. Occurrence and the human devastation from natural disasters has increased over the last 50 years, and the negative impacts of climate change are expected to exacerbate this trend (Bowen et al. 2020). Displacement reached a post–World War II record high of 68.5 million forcibly displaced persons in 2019 (World Bank 2019). Economic shocks are increasingly recurrent: in 2007–08, a dramatic increase in food prices sent millions into poverty (World Bank 2017); the COVID-19 pandemic had devastating effects on the lives and livelihoods of the most vulnerable; and, more recently, the Russian invasion of Ukraine has been having dire effects on the food security of the poorest and most vulnerable people in many countries (World Bank 2022b). 1.2 Poor households are particularly vulnerable to covariate shocks because they lack adequate capacity to prepare for, cope with, and adapt to shocks. Factors that hamper people’s preparedness include limited savings and assets to draw on when a shock occurs, limited or no access to public (social protection) and private (insurance) instruments, and limited access to information on their exposure and vulnerability to covariate shocks to inform action. Without adequate savings and access to social protection or private insurance, households usually resort to negative coping strategies such as removing children from school to work for extra income, selling productive assets, or taking high-interest loans (del Ninno, Coll-Black, and Fallavier 2016). The lack of savings and social protection also hinders households’ adaptation to covariate shocks, given the lack of appropriate means to adjust livelihoods and assets to reduce vulnerability. 1.3 Covariate shocks can also impoverish vulnerable households when their capacity to prepare, cope, and adapt is overwhelmed. Many households live close to the poverty line, which makes them especially vulnerable to poverty resulting from even small variances in income and consumption. An estimated 26 million people fall into poverty every year because of natural disasters, especially frequent floods and drought 1 (Hallegatte et al. 2017). In Senegal, for instance, 45 percent of poor households escaped poverty between 2006 and 2011, but 40 percent of nonpoor households fell into poverty the same period (Dang, Lanjouw, and Swinkels 2014). 1.4 Covariate shocks vary in magnitude, speed of onset, predictability, and duration, and thus these aspects should be considered when designing the most appropriate social protection response. Covariate shocks vary in magnitude and severity, which will generate different needs among affected populations and have different implications for those mandated to address them. Covariate shocks can be protracted (usually conflict); recurrent (repeated natural hazards such as droughts, floods, or crop failures); occasional; or one time. They can also be short term (earthquakes), medium term (high food prices and economic downturns), and long term (civil war; O’Brien et al. 2018a).1 Thus, the scale of a shock will determine the number of people affected. The level of urgency will determine the need for speedy action, with life-threatening events requiring immediate action to save lives. Required support may be temporary or more regular because needs may continue to change over time. Similarly, the phasing of the shock also has broad implications for the social protection sector’s role in relation to others (because needs may evolve and continue to change over time). 1.5 Moreover, the needs and challenges that vulnerable and directly affected populations face will have implications for social protection systems. Implications may vary, for instance, depending on the funding made available (for example, size, speed, duration, and the extent of conditions attached); the mix of stakeholders involved in responding and their coordination (or lack of) and the broader political economy of the sectors involved; the potential to leverage capacity and systems from different sectors (for example, early warning systems); the feasibility of providing different services (for example, extent of market disruption, destruction of service delivery infrastructure, and so on); the relevant legal and policy frameworks (for example, a refugee response will need to respect national legislation on the topic), and so on (TRANSFORM 2020). 1.6 Adaptive social protection (ASP) builds resilience by helping poor and vulnerable households prepare for, cope with, and adapt to covariate shocks. ASP protects poor and vulnerable households’ well-being by ensuring that they do not fall into poverty or become trapped in poverty because of the negative impacts of covariate shocks. It does so by bringing the social protection, disaster risk management (DRM), and climate change adaptation sectors together to build the resilience of poor and vulnerable people in relation to covariate shocks (Bowen et al. 2020). Considering all three sectors simultaneously when responding to covariate shocks can gain synergies, given the commonalities and complementarities among them (Davies et al. 2008). 2 2. Adaptive Social Protection in World Bank Support 2.1 The ASP agenda is developing rapidly and is a critical part of the World Bank Group’s Crisis Response Platform for systems to better respond to and prepare for covariate shocks. ASP is an evolving concept but not a new one. The World Bank understands ASP as a specific focus area within social protection that goes beyond relief and recovery from covariate shocks to focus on promoting household resilience. 2.2 Increasing the responsiveness of social protection programs to adapt to and meet changed needs after a shock materialized and build household resilience to covariate shocks are clear objectives of the World Bank’s work on ASP. The World Bank launched the Sahel Adaptive Social Protection Program in 2014, aiming to increase access to effective ASP systems for poor and vulnerable populations in six countries in the Sahel (Burkina Faso, Chad, Mali, Mauritania, Niger, and Senegal). The goal of supporting the development of ASP systems and programs to help individuals, households, communities, and societies build resilience, equity, and opportunities was clearly outlined in the Sahel Adaptive Social Protection Program then and has since been strengthened. The 2018 South-South Learning Forum, Building Resilience through Adaptive Social Protection, outlined an ASP framework enhancing the focus on enabling social protection to address the impacts of all type of covariate shocks on households.2 ASP was already centered around two interrelated approaches: building the resilience of the households that are most vulnerable to covariate shocks, and increasing the responsiveness of social protection programs after a shock has occurred. These approaches were later captured in the World Bank’s 2020 report Adaptive Social Protection: Building Resilience to Shocks and further operationalized in the development and application of the 2021 Stress Test Tool, which aims to conduct rapid appraisals of the adaptability of social protection systems and assess their readiness to scale up. 2.3 These objectives are also observed in the evolution of the World Bank’s strategic approach to social protection from a program-focused approach to a system-based approach aimed at building and strengthening social protection systems and equipping them better to respond to poverty and to risks and vulnerability. The evolution is evident in the framing and evolution of the past three strategies that have guided the World Bank’s work on social protection, and in the articulation of a defined framework for ASP, as described in box 2.1. 2.4 The 2012–22 Social Protection and Labor Strategy clearly outlined many of the key ASP features, even if it did not yet describe them as such, that were then articulated in the ASP framework. The strategy encourages “systemic approaches,” interventions coordinated across programs and responsive to crises and shocks (World Bank 2012), and highlights the importance of collaboration, coordination, and “policy coherence” 3 among social protection, DRM, early warning, and climate change adaptation approaches (Kuriakose et al. 2012). It also focuses on strengthening national systems in the context of the wide spread of noncontributory cash transfer programs across the world and an increased use of cash in humanitarian responses. It recognizes that although cash working groups served as coordination mechanisms between humanitarian actors and donor funded programs in many countries, the limited government involvement contributed to an increased risk of setting up parallel structures (World Bank Group 2016). Strong government leadership was increasingly recognized as fundamental to ensuring the integration and guidance of the multiple actors for an adequate emergency response. Box 2.1. Evolution of the World Bank’s Strategic Approach toward Adaptation in Social Protection 2001–11: Social Protection Sector Strategy: From Safety Net to Springboard. This strategy adopted a new social risk management framework recognizing that all individuals, households, and communities are exposed to multiple risks from different sources and identifying risk and vulnerability as major drivers of poverty (World Bank 2012). Vulnerability analysis complemented the more traditional poverty analysis, highlighting the importance of a more fluid approach — grounded in a range of instruments—to address risks. The strategy aimed for social protection programs to provide poor people with the capacity to climb out of poverty (that is, the springboard). Guided by this strategy, the World Bank support evolved from a project-focused approach that emphasized delivering social assistance benefits toward helping countries build social protection systems and institutions to respond better to poverty, risk, and vulnerability (World Bank 2011). 2012–22: Resilience, Equity, and Opportunity: The World Bank’s Social Protection and Labor Strategy 2012–2022. This strategy built on the social risk management conceptual framework and aimed to help client countries move from fragmented program approaches to more harmonized systems for social protection and labor. The strategy promoted “systemic approaches” to SMART (synchronized, measurable, affordable, responsive, transparent and accountable) social protection and labor, including programs that were synchronized; monitored, evaluated, and adapted; affordable; responsive to crises and shocks; and transparent and accountable. Drawing from the lessons from the previous strategy and the Independent Evaluation Group’s evaluation (World Bank 2011), this strategy identified four key areas for improvement: a stronger focus on solutions to build coherent and country-appropriate portfolios supporting social protection systems that protect people from multiple risks; increased engagement in lower-income countries, with a focus on helping these countries put the appropriate building blocks in place toward their social protection goals; the role of jobs and enhanced productivity as key pathways to opportunity; and the central role of evidence-driven social protection practice. 2020: Adaptive Social Protection: Building Resilience to Shocks. The World Bank’s adaptive social protection framework promotes key actions that build on investments in routine system strengthening but go beyond business as usual. The adaptive social protection framework promotes strategic investments in adaptive elements in four of the building blocks of any social protection system: programs, data and information, finance and institutional arrangements, and 4 partnerships. Over the long term, strategic investments in adaptive elements within these four building blocks can increase household resilience to covariate shocks, accelerate human capital accumulation, and reduce poverty and economic impacts from shocks (Bowen et al. 2020). 2022: Charting a Course Towards Universal Social Protection: Resilience, Equity, and Opportunity for All. Also known as the Social Protection and Jobs Compass, this strategy acknowledges how several global issues such as climate change, pandemics, fragility and conflict, and technological innovation are reshaping economies and societies. It also stresses their negative impacts on well- being and poverty and therefore the implications for how countries organize their social protection systems (World Bank 2022a). The vision of universal social protection is at the heart of the Compass. To achieve it, the Compass outlines five strategic priorities: (i) build strong foundational social protection systems; (ii) increase coverage and promote greater inclusion (coverage gap); (iii) build more resilient, adaptive, and dynamic programming (flexibility gap); (iv) increase more effective economic inclusion and labor systems (opportunity gap); and (v) create more fiscal space for universal social protection (fiscal gap). Source: Independent Evaluation Group, based on World Bank 2001, 2012, and 2022a. 2.4 Social protection systems play a fundamental role in addressing covariate shocks, but more investments in adaptation are needed for better preparedness and response. Countries whose routine social protection systems provide high coverage have an intrinsic cushion from covariate shocks, as COVID-19 has shown recently.3 A crucial starting point for any ASP system is building and strengthening strong routine social protection systems with the aim of ensuring increased coverage over time, adequacy, comprehensiveness, and sustainability across a range of programs at the policy, programmatic, and delivery chain levels. Moreover, the ability to sustain routine benefit delivery amid covariate shocks is an essential feature of an ASP system. Strategic investments in adaptive elements are needed in routine social assistance, social insurance, and labor market programs along the pre- and postshock continuum to provide much-needed income support during a crisis or its immediate aftermath. Programs need to be prepared and able to respond and expand fast enough to protect both those affected directly and the most vulnerable, which may imply flexing, for example, surge capacity, alternate delivery methods, and so on.4 It may also imply minor design tweaks to incorporate a focus on covariate shocks, complete adaptations to support better response, and investments in risk-informed targeting approaches to identify at-risk households. 2.5 Information on households’ vulnerability to covariate shocks and their capacity to cope and recover is essential for the design and implementation of effective ASP programs. For ASP to cover at-risk populations, social registries need to expand into and within high-risk areas, updating information in those areas more frequently and including variables related to household vulnerability to covariate shocks. This may imply strengthening links with information systems that are typically disconnected from the social protection sector, such as DRM information systems. More investments may 5 be needed in countries’ capacity to assess vulnerability before and after covariate shocks and to act on early warning information. 2.6 Investments in adequate risk financing and risk layering, ensuring appropriate institutional arrangements, and fostering strategic partnerships are critical to the sustainability of ASP systems. Adequate planning for risk financing requires governments to develop risk financing strategies that enable funding to flow if a covariate shock occurs and thus mobilize a faster response. Investments supporting adequate institutional arrangements and strengthening partnerships and collaboration across sectors can contribute to much-needed coordination among the myriad actors involved in an effective preparation and response to covariate shocks. The evaluation theory of change in section 4 provides more details about these adaptive elements. 2.7 Recognizing the growing impact of the multiple overlapping crises affecting the world, the Bank Group adopted a Global Crisis Response Framework in 2022 that reinforces the importance of additional investments in ASP. The framework has guided the Bank Group’s operational response since April 2022 and continues through June 2023, deploying US$170 billion to support developing countries to navigate these unprecedented crises (World Bank 2022b). Building on lessons learned from the COVID- 19 response and past food crises, the framework rests on four interconnected pillars combining support to crisis response and long-term development: • Responding to food insecurity through supporting production, facilitating trade, supporting the vulnerable, and investing in sustainable food systems; • Protecting people and preserving jobs to help mitigate the medium- to long-term impacts of crises; • Strengthening resilience by identifying and supporting paths to build long-term resilience; • Strengthening policies, institutions, and investments for rebuilding better to use long-term policies to improve development outcomes. 3. Evaluation Purpose 3.1 The purpose of this evaluation is twofold: (i) assess whether World Bank support for social protection has incorporated adaptive elements over time, and (ii) assess how effective the World Bank has been at helping client countries make their social protection systems more adaptive. 3.2 Assessing adaptiveness in social protection over time responds to the greater focus on better preparedness, inclusion of adaptive design elements, and a system 6 strengthening approach as outlined in World Bank (2022a)—the current strategy guiding the future work of the Social Protection and Jobs Global Practice (GP)—and the Global Crisis Response Framework. Given the unpredictability of crises, their dynamic nature, and the diversity of impacts across different countries, the framework demands a flexible, adaptive results approach. World Bank operations are expected to focus on upgrading systems to make them less vulnerable to crises, enhance crisis preparedness emergency response planning to save lives and livelihoods, and enable sustainable public services (World Bank 2022b). 3.3 The evaluation is intended to promote learning emerging from World Bank operations and client countries’ policy responses to recent covariate shocks, including COVID-19, focusing on the key investments in adaptive elements needed for a more effective social protection response. The evaluation aims to provide independent and evidence-based feedback on the achievements and challenges of translating the ASP framework into effective World Bank support and advice and into client countries’ policies, with special emphasis on the framework’s usefulness in different country settings. To do so, the evaluation will review experiences across a wide range of contexts on how the ASP framework is used, and it will assess whether and how the framework influences key changes in social protection systems in client countries to make them more adaptive. It will also review the usefulness of the key adaptive elements outlined in the ASP framework across a variety of covariate shocks, country settings, and social protection maturity levels. 3.4 The evaluation will contribute to the use of evidence-based approaches to maximize development impact in future crisis responses. The results will inform World Bank management and the Board of Executive Directors on the relevance and effectiveness of Social Protection and Jobs support to the most recent crises. In the current context of polycrises design ASP systems to identify ex ante vulnerable households, improve protection of the most vulnerable if a shock occurs, and strengthen the crisis response coordination of key actors for a coherent response is of utmost importance (Bastagli and Lowe 2021; Packard et al. 2019). COVID-19 exposed structural gaps and limitations in routine social protection systems, such as how to respond quickly to vulnerable populations that fell into poverty because of the crisis. Invisibility and lack of information on certain vulnerable groups (such as informal workers in urban settings) was a common barrier to adequate policy response (Bastagli and Lowe 2021). The COVID-19 response experience and experiences from other recent crises offer valuable insights to consider when designing ASP support for the way forward to ensure better preparedness for dynamic crises (Gentilini 2022). The evaluation aims to contribute to the evidence on these experiences to inform future World Bank financing operations and advisory services and country policy responses. 7 4. Theory of Change and Evaluation Questions Theory of Change 4.1 This evaluation uses a theory of change to guide its understanding of the World Bank’s contribution to making social protection systems more adaptive in client countries (figure 4.1). We developed the theory of change based on a review of the World Bank’s ASP framework (Bowen et al. 2020), relevant external literature, and consultations with social protection experts. We designed the evaluation questions and methods we propose in this Approach Paper to test many of the causal assumptions embedded in this theory of change. 4.2 Country clients’ uptake of ASP involves many factors, some of which may be influenced by World Bank support. As shown in the blue boxes in figure 4.1, country characteristics such as the existence of strong and highly institutionalized programs, the capacity and shock responsiveness of social protection infrastructure, and the nature and coverage of social protection information systems may affect the uptake of ASP. Moreover, social protection delivery systems evolve over time, do so in a nonlinear fashion, and are affected by the starting point (Lindert et al. 2020). Wider country characteristics beyond the realm of social protection—such as the recurrence of covariate shocks, the presence of comprehensive identification systems, and the availability of fiscal space—also constitute key factors affecting clients’ decision to invest in ASP elements. The World Bank uses its advisory services and analytics (ASA) in its policy dialogue and convening to raise country clients’ awareness of their vulnerability to covariate shocks and how they can mitigate their impacts through incorporating adaptive elements into their social protection systems. World Bank corporate priorities guide the response to covariate shocks (for example, the Global Crisis Response Framework), provide financing and can facilitate effective coordination across practices (for example, Social Protection and Jobs coordination with the Urban, Disaster Risk Management, Resilience, and Land GP for DRM). 4.3 To foster better preparedness, coping, and adaptation to covariate shocks, the World Bank supports clients through strategic investments in adaptive elements within the four building blocks of social protection systems. The organizing framework for ASP that guides World Bank financing and advisory services support is composed of four building blocks: programs, data and information systems, finance and institutional arrangements, and partnerships. The red boxes in figure 4.1 show the priorities and core investments in adaptive elements within each building block that support ASP design and implementation. These adaptive elements go beyond those for regular social protection systems, aiming at building household resilience to covariate shocks. 8 • Strengthened programs. Social protection programs become adaptive when they (i) adjust targeting approaches to integrate covariate shock risk and household vulnerability into eligibility criteria and beneficiary selection, fine-tuning benefits and services to enhance resilience-building outcomes among selected households; (ii) incorporate design elements to support preparedness, such as disseminating risk information within at-risk communities and promoting increased savings and financial inclusion among beneficiary households; and (iii) incorporate shock-responsive capacity through design tweaks, vertical or horizontal expansions, and supporting the use of piggybacking shock responses on already existing programs. • Increased data and information. ASP requires detailed information and analyses to improve understanding of covariate shock risk and vulnerability as a basis for program design and implementation. Critical information includes the type of covariate shocks to which a country may be vulnerable, populations and assets affected by covariate shocks, frequency and location of covariate shocks, populations most in need of assistance, and the extent of assistance required. An effective ASP system will draw on DRM analyses and integrate them with assessments of household poverty and vulnerability (which social protection and poverty teams commonly conduct) to provide an informed, need-based foundation for program design. Such integration requires effective data-sharing platforms and protocols. ASP also requires linking early warning systems and postshock assessments (usually carried out by other sectors) to social protection registries. Social registries are a key data tool to support outreach, intake, registration, and determination of potential eligibility in social protection programs. But in most countries, these registries typically operate on fixed lists and are generally updated every four to five years. For effective ASP systems, social registries need to expand into and within high-risk areas, updating information in those at-risk areas more frequently and including variables related to household vulnerability. • Ensure adequate and sustainable finance. Effective ASP systems require moving away from a reactive approach to financing (which addresses the impact of covariate shocks once they happen) to a more proactive approach that puts the required financing in place to respond to covariate shocks before they take place. A proactive approach entails increased capacity to estimate and plan for adequate financing to cover the cost of shock response. Risk and vulnerability data can be used to estimate costs and contingent liabilities of predicted covariate shocks through a social protection response. Yet estimating costs is not sufficient. Efficient ASP requires a financing strategy to cover those costs through risk 9 layering (that is, the sequencing of financial instruments that respond to risks of differing likely magnitudes and frequency). Finally, linking financing instruments to shock-responsive social protection programs is as important as securing the financing in the first place. • Ensure adequate institutional arrangements and partnerships. Effective ASP involves multiple actors, many of which may not be traditional partners of social protection, such as DRM and climate change adaptation agencies. This variety of actors adds a multiplicity of programs that must be aligned to ASP objectives, calling for strong government leadership and adequate institutional capacity to underpin a coordinated planning, management, and delivery of ASP assistance. Government leadership and institutional capacity can be reflected in (i) the inclusion of resilience-related objectives across social protection, DRM, and climate change adaptation policies and strategies; (ii) strong political commitments supported by adequate implementation capacity, financing, legislation, and accountability mechanisms; and (iii) adequate coordination mechanisms between ASP and humanitarian actors. 4.4 A timely and adequate social protection response to a shock is the main expected outcome of these investments in ASP systems. ASP can contribute to timely social protection responses with adequate population coverage and benefits and, in the long term, to increased household resilience (Bastagli and Lowe 2021; O’Brien et al. 2018a,b). The orange (bottom) and green (top) boxes under “outcomes” in figure 4.1 show these effects. Timeliness pertains to whether the response was delivered when it was needed most; coverage refers to whether it reached all those it needed to reach; and adequacy is related to whether the response met people’s needs through monetary and in-kind transfers and services. Other relevant criteria that characterize effective social protection responses but are outside the evaluation scope include cost effectiveness and value for money (whether the response makes the best use of resources), sustainability (if the benefits are likely to continue after the immediate response), accountability (including the voices of the affected individuals in the design and management of the response), and equity and inclusion (whether the response meets different people’s needs fairly; Holmes, Levine, and Shakespeare 2021). Finally, as shown in the orange box under “impact,” the contribution of ASP to increased household resilience is beyond the scope of the evaluation. 10 Figure 4.1. Evaluation Theory of Change Source: Elaborated by the Independent Evaluation Group, based on Bastagli and Lowe 2021 and Bowen et al. 2020. Note: ASP = adaptive social protection. 11 Evaluation Questions 4.5 The evaluation is focused on World Bank support to social protection and will answer two key evaluation questions, which are each supported by subquestions as follows: Question 1. To what extent has the World Bank support for ASP been relevant? a. To what extent has the World Bank supported ASP elements in countries where vulnerability to covariate shocks is higher? b. To what extent has the World Bank incorporated ASP elements into its social protection support, and to what extent are these aligned with good practice and evidence of what works? c. To what extent is the World Bank ASP framework a realistic model in different settings? Question 2. How effectively has the World Bank supported ASP outcomes (timeliness and adequacy of social protection response) in client countries? a. How effective has the World Bank’s support been for ASP practices and activities? b. What has worked to achieve successful ASP outcomes in client countries? What factors explain success, and what was the role of the World Bank? Scope 4.6 The evaluation will review the universe of World Bank support for social protection between fiscal year (FY)12 and FY22. The 10-year period was chosen to assess the evolution of the World Bank’s approach since the beginning of the Sahel Adaptive Social Protection Program, one of the first ASP programs to be labeled specifically as such. The chosen evaluation period will allow capturing evidence of effectiveness from the closed portfolio of lending projects with social protection activities. The evaluation excludes the International Finance Corporation or the Multilateral Investment Guarantee Agency because social protection is not a corporate priority for these institutions, and they do not have interventions supporting ASP elements. 4.7 The Independent Evaluation Group (IEG) identified a global lending portfolio in which Social Protection and Jobs is either a leading or contributing GP. This portfolio consists of 424 projects and US$90.3 billion in commitments approved between FY12 and FY22, and almost two-thirds of the projects are still active. Social Protection and Jobs 12 leads 200 projects (47 percent), and the remaining 224 are distributed across Macroeconomics, Trade, and Investment (17 percent); Health, Nutrition, and Population (9 percent); Education (8 percent); and other GPs (19 percent). Social Protection and Jobs is a contributing GP in these 224 projects. Two-thirds of the portfolio is investment project financing (278 projects), followed by development policy financing (117 projects) and Program-for-Results (29 projects). Just under half of projects are in International Development Association countries, more than 40 percent of projects are in Africa, and about 50 percent of approvals took place between FY20 and FY22. About 25 percent of the portfolio (105 projects) is in countries classified as fragile and conflict-affected situations (see appendix B for more details on portfolio identification methodology). 4.8 Based on the global lending portfolio, we identified a subset of projects tagged with either safety net or social protection delivery system themes and used it to conduct a preliminary review of its focus and improve the evaluation scope. We assumed that safety nets and social protection delivery systems were more likely to have adaptive elements of social protection, and we selected this focus because (i) the World Bank’s ASP framework emphasizes the importance of safety nets for ASP because of their noncontributory nature; and (ii) support to social protection delivery systems often includes investments in data and information, capacity building, strengthening institutional arrangements, and systems coordination—all actions relevant for delivering adaptive elements. The subset includes 265 projects and accounts for about US$60 billion in commitments, mostly led by Social Protection and Jobs (60 percent of both projects and volume). Project distributions across leading GPs, lending instruments, lending groups, Regions, and approval years are similar to the global portfolio. Most of the World Bank’s investment (investment project financing and Program-for-Results) focuses on social assistance programs (60 percent), mainly through financing cash transfers and public works. Support to data and information seeks to institute policy reforms to strengthen social registries and cross-sector data-sharing platforms and is done primarily through development policy financing. The World Bank’s support also focuses on ensuring adequate financing and institutional strengthening, especially through development policy financing support to budget allocation in case of covariate shocks, preplanned risk financing, risk layering for shock responses, and policies and regulations. 4.9 We also identified a preliminary ASP portfolio of World Bank ASA. We used a targeted keyword search to identify a purposive sample of ASA addressing ASP based on consultations with World Bank staff and a review of recent ASP ASA (see appendix B for more details on the identification methodology). The purposive sample consists of 115 ASA worth about US$95 million in total cumulative expenditures. Social Protection and Jobs leads most ASA (69 percent). ASA related to ASP have increased since FY12 13 but spiked in FY20 (22 annual approvals compared with an average of annual approvals between FY12 and FY19). West Africa accounts for one-third of ASA, in line with World Bank support to ASP in Sahel countries through the Sahel Adaptive Social Protection Program. We will use text analytics to identify the final ASA portfolio and conduct a manual review for identifying the final lending portfolio. The current lending and ASA portfolios are preliminary estimates, and we will take a more comprehensive approach to identifying them throughout the evaluation.5 4.10 Almost half of the preliminary ASA portfolio focuses on providing integrated support for strengthening ASP systems. The support is provided through a mix of activities, typically involving system assessments and diagnostics with follow-up capacity building; just-in-time technical assistance for design, implementation, and evaluation of ASP systems and programs; policy dialogue; and collaboration and coordination with technical and financial partners. Next is pure diagnostic and assessment work on disaster risk vulnerability and social protection systems, including social protection and climate and health stress tests, estimation of welfare impacts of covariate shocks, impact evaluations of social safety net programs, and social protection expenditure analyses, among others. These are followed by ASA that focuses support on improving specific shock-responsive elements such as design and implementation of emergency cash transfers (for example, improvement of payment delivery mechanisms after emergencies), proof of concept activities for DRM and decision support tools, financial resilience against natural disasters, better use of disaster management funds, and increased disaster insurance coverage. Few sampled ASA have an exclusive focus on dialogue, coordination, or strategies. They include ASA seeking to strengthen synergies between the Social Protection and Jobs and the DRM and health sectors within the World Bank, ASA that exclusively supports experience sharing and learning on ASP, and ASA that supports only ASP strategies. Examples of synergy strengthening are mainstreaming DRM into the social protection sector through analytic inputs and knowledge exchange activities, and investigating how Human Development delivery systems (education, health, and social protection) could be integrated and leveraged better in relation to covariate shocks and crisis situations; experience sharing and learning includes, for example, regional learning activities to share social protection experiences in building resilience to disaster and climate-related shocks. ASA focusing exclusively on strategies can include disaster and climate shock response strategies, preparedness and adaptation strategies, and policies aiming to promote and protect human capital from the effects of the COVID-19 emergency, among others (table 4.1). 4.11 Since the development of the ASP framework, several World Bank ASA country and regional reports have used it to further define and measure adaptive elements of social protection and to assess country systems’ preparedness and required investments 14 for ASP. Among the tools used for ASP country assessment, the development of a traffic light assessment by Beazley and Williams (2021) and a more standardized tool, the World Bank Stress Test Tool, stand out. The Finance, Competitiveness, and Innovation GP has also developed disaster risk finance diagnostic tools for some Regions and countries. Table 4.1. Support to Adaptive Social Protection in Advisory Services and Analytics Cumulative Activities Expenditures Share Type of Focused Support to Adaptive Social Protection (no.) Share (%) (US$, millions) (%) Integrated support for strengthening ASP systems 51 44.3 47.6 50.1 Diagnostics and assessments 26 22.6 17.1 18.0 Support for improving specific shock-responsiveness elements 24 20.9 21.2 22.3 Dialogue, coordination, or strategies 14 12.2 9.1 9.5 Total 115 100.0 95.0 100.0 Source: Independent Evaluation Group. Note: ASP = adaptive social protection. 5. Evaluation Design and Methodology 5.1 The evaluation is designed to answer two evaluation questions about the relevance and effectiveness of the World Bank’s support for ASP. We will conduct data collection and analysis at two levels: (i) the universe of social protection projects and ASA that supports building or strengthening social protection systems, and (ii) selected countries. IEG will select case studies based on four criteria: (i) significant World Bank engagement in supporting the country to adapt its social protection system to better respond to covariate shocks; (ii) countries with similar level of social protection maturity; (iii) countries that are exposed to similar types of shocks (focusing on countries exposed to slow and predictable onset versus rapid, unpredictable onset); and (iv) countries that have experienced different ASP outcomes when faced with covariate shocks. The methodology will further be refined during the evaluation process and discussed with World Bank experts to ensure its relevance. Figure 5.1 describes the design and accompanying methods. The evaluation design matrix in appendix A provides more details, and this section explains them in sequence. 15 Figure 5.1. Evaluation Design Source: Independent Evaluation Group. Note: ASP = adaptive social protection; ASPIRE = Atlas of Social Protection Indicators of Resilience and Equity. 16 Relevance 5.2 The evaluation seeks to assess three aspects of relevance regarding the World Bank’s support for ASP. 5.3 First, it asks whether the World Bank is engaging strategically in those countries where different types of covariate shocks pose or will pose serious threats. To answer this question, we will (i) use the INFORM Risk Index to rank countries according to their level of vulnerability to covariate shocks, and (ii) identify support for ASP in the World Bank’s lending and advisory portfolio. The INFORM Risk Index database provides a comprehensive assessment of three dimensions of risk useful to this evaluation: hazards and exposure, vulnerability, and lack of coping capacity dimensions. By focusing not only on the hazard and exposure dimension (as many other risk databases do) but also on identifying the vulnerability and lack of coping capacity dimensions, this database provides useful information to categorize countries’ vulnerabilities. The evaluation will compare these two data sets to map how World Bank support for adaptive elements aligns with country risk. 5.4 Second, the evaluation seeks to assess whether World Bank support for social protection has incorporated adaptive elements over time and whether this support is aligned with evidence on what works. To answer this question, we will first conduct a structured literature review to identify ASP good practices and evidence to help assess the comprehensiveness of the World Bank ASP framework and understand what works and in what contexts. Second, based on the adaptive elements already identified plus any identified through the structured literature review, we will conduct a portfolio review and analysis to identify the World Bank support to ASP as evidenced in the lending and advisory services portfolio. Finally, the evaluation will compare information from the structured literature review with the portfolio review and analysis data set to assess whether the World Bank support for ASP has evolved in line with evidence of what works. 5.5 Third, the evaluation seeks to assess whether the World Bank’s ASP framework outlines a realistic model in different settings. Realistic means (i) the extent to which the framework can be applied to different country contexts (that is, to countries with exposure to different types of covariate shocks, with different levels of social protection system maturity, and with a different fragility status); and (ii) whether the adaptive elements can be operationalized and in what order. To answer this question, we will use insights from the structured literature review conducted to answer subevaluation question 1b. To complement the literature review findings, we will conduct interviews with World Bank staff and key government officials and convene a Delphi panel of 17 global experts. We also expect the findings from the comparative case studies to provide substantive evidence. Effectiveness 5.6 The evaluation seeks to assess two key aspects of effectiveness regarding World Bank support for ASP outcomes. 5.7 First, the evaluation will assess how effective the World Bank has been in supporting ASP elements. We will identify effectiveness through the portfolio review and analysis conducted for subquestion 1b and with the information obtained in the comparative case studies. Examples of adaptive elements include support for setting up or expanding social registries, linking social protection programs to early warning systems, using risk-informed targeting approaches to identify at-risk households, and establishing data-sharing protocols among government agencies. To assess the efficacy of these activities and practices, we will conduct desk reviews of project documents and results frameworks, along with interviews to gain an in-depth understanding of factors contributing to success or failure. ASA will be assessed only at the country case level. 5.8 Second, the evaluation will assess what has worked to achieve successful ASP outcomes in client countries, delving also into the World Bank’s role, if any. To answer this question, we will undertake comparative case study analysis to uncover factors that explain successful and less successful ASP outcomes in countries that have experienced the same type of shock and have similar levels of social protection maturity. In identifying factors that explain success, we will consider the role of humanitarian actors in supporting or hindering coordination. To identify variability in ASP outcomes, we will rely on relevant literature and the advice of experts from within and outside the World Bank. World Bank experts will be identified based on the portfolio review (focusing on task team leaders of ASP projects and ASA) and through consultations with Global Leads experts in the GP. External experts will be identified through consultations with World Bank experts and our peer reviewers, as well as through the literature review. We will identify social protection maturity levels via indicators from the Atlas of Social Protection Indicators of Resilience and Equity (ASPIRE) and any other relevant databases and literature. By comparing countries with similar social protection maturity levels that have been affected by the same type of shock but that achieved different ASP outcomes, we expect to uncover the factors that explain this variation and what role the World Bank played in supporting those outcomes. Appendix A provides more details on the selection criteria for country cases. 18 Limitations 5.9 The ultimate objective of any ASP system is to strengthen household resilience to covariate shocks, but assessing the extent to which the World Bank has contributed to increased household resilience via ASP is beyond the scope of this evaluation. Data and methodological constraints prevent us from fully assessing the World Bank’s contribution to household resilience through its support to ASP. Resilience is a difficult concept to measure (information on household resilience is not easily available or comparable across countries) and requires long-term data. It is infeasible to assess a household’s ability to prepare for, cope with, and adapt to covariate shocks in a way that protects their well-being and prevents them from slipping into poverty or being trapped in it because of the impacts (Bowen et al. 2020), and do so for the countries the World Bank supports. 5.10 Identifying ASP outcomes through expert advice and using ASPIRE indicators to identify the level of maturity of countries’ social protection systems can have biases. To mitigate this challenge, we will complement this information whenever possible with the review of available literature and data sets documenting the level of maturity of social protection systems, as well as ASP outcomes for specific covariate shocks. For instance, if we were to choose COVID-19 as a shock for study, we could rely on an existing data set tracking countries’ social protection response to the COVID-19 shock (Gentilini et al. 2022). We are aware that ASPIRE indicators may provide an approximate measure of social protection systems performance. It may not be comprehensive about the universe of programs existing in a country because many household surveys that feed the database have limited information on social protection programs (that is, some surveys collect information only on program participation without including the transfer amounts). Moreover, ASPIRE performance indicators may not be fully comparable across program categories and countries. To address this challenge, we will seek expert advice, including from World Bank task team leaders, and review available literature and complementary information as needed. 5.11 Limited evidence from closed projects may pose challenges for assessing the efficacy of World Bank support for ASP elements. As noted in the Scope section, only one-third of the global portfolio identified is closed. Additionally, support for adaptive elements can be a small focus within a larger intervention. To address this challenge, we will map adaptive elements supported in projects to results framework indicators and use this to complement the effectiveness assessment. Country-level information obtained through comparative case studies will be used in addition to this information. 19 6. Quality Assurance Process 6.1 The Approach Paper and the final evaluation report will follow IEG’s standard internal quality assurance, which includes internal IEG and World Bank management review and an external peer review process. The experts who will peer review the evaluation are (i) Ralf Radermacher (head of the social protection team at the German Agency for International Cooperation), (ii) Francesca Bastagli (head of the social protection team at the Overseas Development Institute), and (iii) Louise Fox (nonresident senior fellow at the Brookings Institution). The team will benefit from the advice and guidance of a leading global expert and senior consultants with social protection expertise and will conduct the work under the guidance of Galina Sotirova (manager), Carmen Nonay (acting director), and Sabine Bernabe (Director-General, Evaluation). 7. Staffing, Resources, and Timeline 7.1 Gisela Garcia (evaluation officer) will task manage this evaluation under the guidance of Galina Sotirova (manager of the Human Development and Corporate Evaluations Unit) and Carmen Nonay (acting director of the Human Development and Economic Management Department). The team’s core members will include Eduardo Fernandez Maldonado (evaluation officer), Santiago Ramirez Rodriguez (evaluation analyst), and Stuti Sachdeva (evaluation officer). Valentina Barca (leading global expert) will provide overall guidance. Additional expert support and local consultants will be engaged to complete the team. Estelle Raimondo (senior evaluation officer) will provide methodological guidance. Yezena Yimer (program assistant) will provide administrative support. 7.2 The evaluation report will be submitted to the Committee on Development Effectiveness in mid-May 2024. The evaluation will be prepared with an estimated budget of US$725,000. 8. Expected Outputs, Audience, Outreach, and Tracking 8.1 Expected outputs. The main output will be a final evaluation report that we will deliver to the Board’s Committee on Development Effectiveness after integrating comments from World Bank management. Smaller outputs will communicate important findings and messages (for example, briefings, board statements, brown bag lunches, blogs, and so on). 8.2 Engagement. We intend to engage meaningfully and regularly with World Bank colleagues and other key stakeholders throughout the evaluation process, including through an expert panel. Regular consultations at key stages of the evaluation will 20 ensure its relevance, provide us with guidance on case study selection and feedback on preliminary findings, and create ownership of the evaluation. 8.3 Audience. The evaluation’s primary audience is the Board and World Bank management and staff working on social protection. The findings will also be relevant to a broader audience, including policy makers and stakeholders in World Bank client countries, and civil society and academia interested in or working on social protection. 8.4 Outreach and tracking. We will disseminate the evaluation’s findings to multiple audiences to stimulate discussion and encourage exchange of ideas within and outside the World Bank. We will create a blog and post relevant materials on IEG’s website and explore internal and external forums for further dissemination. Involving key external experts in different roles throughout the evaluation will broaden the approach to the topic, give the evaluation more visibility, and ensure that its findings are not World Bank–centric. Engaging one of the top global experts on the topic (who also leads an online shared space on ASP resources) as an expert adviser and team member will ensure that we reach out to policy makers, practitioners, and researchers beyond the World Bank. Whenever possible, the communications and influence strategy will include detailed indicators to track the report’s influence. 1 Slow-onset shocks can suddenly evolve, and a gradually worsening situation can turn into an emergency and thus trigger a response. Drought is the typical slow-onset shock, and earthquakes represent the typical sudden shock. Slow-onset shocks are largely predictable to some degree, though it is not always possible to predict whether a slowly worsening situation, such as rain flooding, will turn into an emergency. By contrast, rapid-onset shocks are less predictable, but increasingly, there is an indication of where those shocks can be expected (for example, the existence of geological fault lines). Shocks can be protracted (usually conflict), recurrent (repeated natural hazards such as droughts, floods, or crop failures), or occasional or one-off. They can be short-term (earthquakes), medium-term (high food prices and economic downturns), and protracted (long-term conflict; Levine and Sharp 2015; O’Brien et al. 2018b). 2For more information about the South-South Learning Forum 2018, see https://www.worldbank.org/en/events/2018/03/13/south-south-learning-forum-2018. 3 See Lowe, McCord, and Beazley (2021) for ex post evaluation and ex ante microsimulation work from the World Bank’s COVID-19 Household Monitoring Dashboard (accessed 2023). https://www.worldbank.org/en/data/interactive/2020/11/11/covid-19-high-frequency-monitoring- dashboard. 4See Barca et al. (2020) for a list of preparedness measures for social protection systems in the event of a shock. 21 5We will implement rigorous text analytics to identify a robust advisory services and analytics (ASA) portfolio and identify a list of core ASA projects related to adaptive social protection based on a final purposive sample of ASA and consultations with World Bank stakeholders. We will then use this list as a benchmark against which to compare other ASA approved within the evaluation period. For lending (given the smaller portfolio size compared with ASA), we will use a more traditional systematic manual review of the identified portfolio to extract adaptive elements. 22 Bibliography Barca, Valentina. 2018. “Setting the Scene: Building on Existing Systems for Adaptive Social Protection. ” Background Paper, Oxford Policy Management, Oxford. Barca, Valentina. 2021. “Useful COVID-19 and Social Protection Materials.” Social Protection Approaches to COVID-19 Expert Advice Service (SPACE), DAI Global UK Ltd, Hemel Hempstead. Barca, Valentina, Edward Archibald, Rodolfo Beazley, and Simon Little. 2020. “Preparing for Future Shocks: Priority Actions for Social Protection Practitioners in the Wake of COVID-19.” UK Foreign, Commonwealth and Development Office, London, and German Agency for International Cooperation, Bonn. Bastagli, Francesca, and Christy Lowe. 2021. “Social Protection Response to COVID-19 and Beyond: Emerging Evidence and Learning for Future Crises.” ODI Working Paper 614, ODI, London. https://odi.org/en/publications/social-protection-response-to-covid-19-and-beyond-emerging- evidence-and-learning-for-future-crises/. Beazley, Rodolfo, and Asha Williams. 2021. “Adaptive Social Protection in the Caribbean: Building Human Capital for Resilience.” Background Note, World Bank and the Global Facility for Disaster Reduction and Recovery, Washington, DC. https://openknowledge.worldbank.org/server/api/core/bitstreams/4b4233f7-18c4-58c0-8542- cfa435b1d31b/content. Bowen, Thomas, Carlo del Ninno, Colin Andrews, Sarah Coll-Black, Ugo Gentilini, Kelly Johnson, Yasuhiro Kawasoe, Adea Kryeziu, Barry Maher, and Asha Williams. 2020. “Overview: A Framework for Adaptive Social Protection.” In Adaptive Social Protection: Building Resilience to Shocks, 1–32. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/579641590038388922/pdf/Adaptive-Social- Protection-Building-Resilience-to-Shocks.pdf. Costella, Cecilia, Anna McCord, Maarten van Aalst, Rebecca Holmes, Jonathan Ammoun, and Valentina Barca. 2021. “Social Protection and Climate Change: Scaling Up Ambition.” Social Protection Approaches to COVID-19 Expert Advice Service (SPACE), DAI Global UK Ltd, Hemel Hempstead. Dang, Hai-Anh H., Peter F. Lanjouw, and Rob Swinkels. 2014. “Who Remained in Poverty, Who Moved Up, and Who Fell Down? An Investigation of Poverty Dynamics in Senegal in the Late 2000s. ” Policy Research Working Paper 7141, World Bank, Washington, DC. Davies, Mark, Bruce Guenther, Jennifer Leavy, Tom Mitchell, and Thomas Tanner. 2008. “Adaptive Social Protection: Synergies for Poverty Reduction.” IDS Bulletin 39 (4): 105–12. 23 Del Ninno, Carlo, Sarah Coll-Black, and Pierre Fallavier. 2016. Social Protection Programs for Africa’s Drylands. Washington, DC: World Bank. http://documents.worldbank.org/curated/en /736221471343475745/pdf/107854-PUB-PUBLIC-PUBDATE-8-9-16.pdf. Gentilini, Ugo. 2022. Cash Transfers in Pandemic Times: Evidence, Practices, and Implications from the Largest Scale-Up in History. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/099800007112236655/pdf/P17658505ca3820930a2 54018e229a30bf8.pdf. Gentilini, Ugo, Mohamed Almenfi, Hrishikesh TMM Iyengar, Yuko Okamura, John Austin Downes, Pamela Dale, Michael Weber, David Newhouse, Claudia Rodriguez Alas, Mareeha Kamran, Ingrid Veronica Mujica, Maria Belen Fontenez, Muhammad Ezzat, Sandra Asieduah, Vikesh Ramesh Mahboobani Martinez, Gonzalo Javier Reyes Hartley, Gustavo Demarco, Miglena Abels, Usama Zafar, Emilio Raul Urteaga, Giorgia Valleriani, Jimmy Vulembera Muhindo, and Sheraz Aziz 2022. “Social Protection and Jobs Responses to COVID-19: A Real-Time Review of Country Measures.” COVID-19 Living Paper version 16 (February 2), World Bank, Washington, DC. Guven, Melis, Himanshi Jain, and Clement Joubert. 2021. Social Protection for the Informal Economy: Operational Lessons for Developing Countries in Africa and Beyond. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/36584. Hallegatte, Stephane, Mook Bangalore, Laura Bonzanigo, Marianne Fay, Tamaro Kane, Ulf Narloch, Julie Rozenberg, David Treguer, and Adrien Vogt-Schilb. 2016. Shock Waves: Managing the Impacts of Climate Change on Poverty. Washington, DC: World Bank. https://openknowledge.worldbank.org/entities/publication/cc42cb9b-3f8e-5bf3-bba3- 2cce3b0793ac. Hallegatte, Stephane, Adrien Vogt-Schilb, Mook Bangalore, and Julie Rozenberg. 2017. Unbreakable: Building the Resilience of the Poor in the Face of Natural Disasters. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/512241480487839624/pdf/110618-PUB- Box396333B-PUBLIC-PUBDATE-11-24-16-UNIT-ITSKI.pdf. Holmes, Rebecca, Simon Levine, and Fleur Shakespeare. 2021. “Key Considerations for Monitoring Shock-Responsive Social Assistance: A Guiding Framework.” Social Protection Approaches to COVID-19 Expert Advice Service (SPACE), DAI Global UK Ltd, Hemel Hempstead. Kuriakose, Anne T., Rasmus Heltberg, William Wiseman, Cecilia Costella, Rachel Cipryk, and Sabine Cornelius. 2012. Climate-Responsive Social Protection—Background Paper for the World Bank 2012–2022 Social Protection and Labor Strategy. http://doi.wiley.com/10.1111/dpr.12037. Lindert, Kathy, Tina George Karippacheril, Inés Rodriguez Caillava, and Kenichi Nishikawa Chavez. 2020. Sourcebook on the Foundations of Social Protection Delivery Systems. Washington, DC: World Bank. http://hdl.handle.net/10986/34044. 24 Longhurst, Daniel, Sophie Evans, Danya Connolly, Felix Lung, Anna McCord, Stephanie Allan, and Michele Plichta. 2021. “What Are Future Financing Options for Shock-Responsive Social Protection? A Technical Primer.” Social Protection Approaches to COVID-19 Expert Advice Service (SPACE), DAI Global UK Ltd, Hemel Hempstead. Lowe, C., A. McCord, and R. Beazley. 2021. “National Cash Transfer Responses to Covid-19: Operational Lessons Learned for Social Protection System-Strengthening and Future Shocks.” ODI Working Paper 610, ODI, London. https://www.odi.org/en/publications/national-cash- transfer-responses-to-covid-19-operational-lessons-learned-for-social-protection-system- strengthening-and-future-shocks. O’Brien, Clare, Rebecca Holmes, Zoë Scott, and Valentina Barca. 2018a. “Shock-Responsive Social Protection Systems Toolkit: Appraising the Use of Social Protection in Addressing Large-Scale Shocks.” Oxford Policy Management, Oxford, UK. O’Brien, Clare, Zoë Scott, Gabrielle Smith, Valentina Barca, Andrew Kardan, Rebecca Holmes, Carol Watson, and Jenny Congrave. 2018b. “Shock-Responsive Social Protection Systems Research: Synthesis Report.” Oxford Policy Management, Oxford, UK. https://www.opml.co.uk/files/Publications/a0408-shock-responsive-social-protection- systems/srsp-synthesis-report.pdf?noredirect=1. Packard, Truman, Ugo Gentilini, Margaret Grosh, Philip O’Keefe, Robert Palacios, David Robalino, and Indhira Santos. 2019. Protecting All: Risk Sharing for a Diverse and Diversifying World of Work. Washington, DC: World Bank. Parekh, Nidhi, and Oriana Bandiera. 2020. “Poverty in the Time of COVID: The Effect of Social Assistance.” LSE Public Policy Review 1 (2): 1–11. doi: http://doi.org/10.31389/lseppr.14. Rigolini, Jamele. 2021. “Social Protection and Labor: A Key Enabler for Climate Change Adaptation and Mitigation.” Discussion Paper 2108, World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/36669. Rigolini, Jamele, Sarah Coll-Black, Rafael de Hoyos, and Ha Thi Hong Nguyen. 2023. Protecting Human Capital through Shocks and Crises: How Lessons Learned from the COVID‐19 Response across Eastern Europe and the South Caucasus Can Be Used to Build Better and More Resilient Human Development Delivery Systems. Washington, DC: World Bank. https://openknowledge.worldbank.org/server/api/core/bitstreams/8eee8ca7-631a-480e-ae80- dc6a8e478a48/content. Shepherd, Andrew, Tom Mitchell, Kirsty Lewis, Amanda Lenhardt, Lindsey Jones, Lucy Scott, and Robert Muir-Wood. 2012. The Geography of Poverty, Disasters, and Climate Extremes in 2030. London: Overseas Development Institute. 25 TRANSFORM. 2020. Shock Responsive Social Protection. n.p.: TRANSFORM. https://transformsp.org/wp- content/uploads/2020/09/SRSP-BD_singles_v12.pdf. World Bank. 2001. Social Protection Sector Strategy: From Safety Net to Springboard. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/299921468765558913/pdf/multi- page.pdf. World Bank. 2011. Social Safety Nets: An Evaluation of World Bank Support, 2000–2010. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/ssn_full_evaluation.pdf . World Bank. 2012. “Resilience, Equity, and Opportunity: The World Bank’s Social Protection and Labor Strategy 2012–2022.” World Bank, Washington, DC. https://documents1.worldbank.org/curated/en/443791468157506768/pdf/732350BR0CODE200do c0version0REVISED.pdf. World Bank. 2017. Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/reports/building-resilience.pdf. World Bank. 2019. World Bank Group Support in Situations Involving Conflict-Induced Displacement. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/Forced_Displacement.p df. World Bank. 2022a. Charting a Course Towards Universal Social Protection: Resilience, Equity, and Opportunity for All. Washington, DC: World Bank. https://openknowledge.worldbank.org/entities/publication/84ba2380-624c-553a-b929- 2882e72c7468. World Bank. 2022b. Navigating Multiple Crises, Staying the Course on Long-Term Development: The World Bank Group’s Response to the Crises Affecting Developing Countries. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/099640108012229672/pdf/IDU09002cbf10966704f a00958a0596092f2542c.pdf. World Bank Group. 2016. Cash Transfers in Humanitarian Contexts: Strategic Note. Washington, DC: World Bank Group. https://documents1.worldbank.org/curated/en/697681467995447727/pdf/106449-WP-IASC- Humanitarian-Cash-PUBLIC.pdf. 26 Appendix A. Evaluation Design Matrix Evaluation Question or Subquestion Logic Methods Data Limitations Question 1. To what extent has the World Bank support for adaptive social protection been relevant? 1a. To what extent has the To assess whether the World Country vulnerability profiles INFORM Risk Index 2023; Low country demand or the World Bank supported adaptive Bank engages strategically in Identify support for adaptive World Bank portfolio data presence of other donors elements in countries where countries with high social protection in the World may explain evidence of no vulnerability to shocks is higher? vulnerability to shocks. Bank’s lending and advisory engagement. portfolio Compare identified support for social protection in the World Bank’s lending and advisory portfolio with country risk profiles 1b. To what extent has the To assess whether World Bank Structured literature review to Academic and gray literature; Project documents may not World Bank incorporated support for social protection identify good practices and World Bank portfolio data and capture the breadth and adaptive elements into its social has incorporated adaptive evidence with respect to project documents depth of the World Bank’s protection support, and to what elements over time and adaptive social protection support for adaptive social extent are these aligned with whether this support is aligned PRA to identify the presence of protection. good practice and evidence of with evidence on what works. adaptive social protection what works? elements in the World Bank’s social protection lending and advisory portfolio Compare information from the structured literature review with the PRA data set to assess whether the World Bank support for adaptive social protection has evolved in line with evidence of what works 1c. To what extent is the World To assess whether the World Insights from the structured Academic and gray literature The structured literature Bank adaptive social protection Bank’s adaptive social literature review carried out for review may yield few insights 27 Evaluation Question or Subquestion Logic Methods Data Limitations framework a realistic model in protection framework answering subevaluation into whether the model is different settings? constitutes a realistic model in question 1b realistic. To mitigate this different settings. Interviews with World Bank staff, challenge, we will conduct key government officials, and a additional data collection Delphi panel of global experts methods such as interviews and a Delphi panel of global experts. Question 2. How effectively has the World Bank supported adaptive social protection outcomes in client countries? 2a. How effective has been the To assess how effective the Review efficacy of adaptive Project documents and results Effectiveness analysis requires World Bank’s support been for World Bank has been with social protection elements frameworks looking at closed projects key adaptive social protection respect to its support for key based on the PRA carried out for Existing evaluations (ICRs, (that is, projects that were practices and activities? adaptive social protection subquestion 1b ICRRs, PPARs) approved early in the elements. For each adaptive element, Key informant interviews for evaluation period). The conduct desk reviews of project each selected adaptive social number of closed projects for documents and interviews with protection element the chosen adaptive elements key staff to identify and assess Country strategy documents may be limited. To mitigate results and factors of this challenge, we will and project and ASA effectiveness using a saturation documents for country cases consider adding projects for method review if needed (relevant Country literature projects approved before Triangulate and corroborate findings with World Bank and FY12 that closed within FY12– external technical experts 22). Use information collected Project documents and through comparative country results frameworks may case studies to assess efficacy contain limited information on the efficacy of World Bank support for adaptive social protection elements. Information on the factors that support effectiveness may be limited in project documents. Thus, we will triangulate the information obtained from project documents with interviews with key World Bank staff and external technical experts. 28 Evaluation Question or Subquestion Logic Methods Data Limitations 2b. What has worked to achieve By choosing countries with Comparative country case Variation on adaptive social Expert advice to identify successful adaptive social similar social protection studies using a structured protection outcomes based on variation on adaptive social protection outcomes in client maturity that experienced the template to identify the factors advice from experts both protection outcomes may be countries? What factors explain same type of shock but that contributed to successful within and outside the World biased. To lessen this success, and what was role the achieved different adaptive outcomes in countries with a Bank challenge, we will triangulate of the World Bank? social protection outcomes, similar level of social protection Countries social protection expert advice with existing this question seeks to identify maturity that experienced the maturity level using ASPIRE literature and any available the factors that explain those same type of shock. database databases tracking outcomes. variations in outcomes. The Review of World Bank project Country vulnerability profiles ASPIRE indicators provide an question also seeks to identify documents, interviews with drawn from INFORM Risk approximate measure of the World Bank’s role (if any) in World Bank staff and relevant Index 2023 social protection systems achieving those outcomes. stakeholders, and review of performance. When needed, Country strategy documents social protection literature and project and ASA er will seek expert advice and relevant to the country documents review available literature to Desk review of country address this challenge and Country literature strategies and ASA and project complement information documents from the ASPIRE database. Case selection based on expert advice, relevant literature, and ASPIRE database Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; ASPIRE = Atlas of Social Protection Indicators of Resilience and Equity; FY = fiscal year; ICR = Implementation Completion and Results Report; ICRR = Implementation Completion and Results Report Review; PPAR = Project Performance Assessment Report; PRA = portfolio review and analysis. 29 Appendix B. Comparative Country Case Study Selection The comparative analysis of case studies will be carried out on 9–12 countries. Although the country choice has not been finalized at this stage, specific selection criteria have been defined that allow restriction of the number of eligible countries. The criteria used to select the country cases are detailed in this appendix. Criteria for selection of country cases are as follows: i. Significant engagement of the World Bank in supporting the country to adapt its social protection system to better respond to covariate shocks, identified through the presence of World Bank interventions focused on adaptive social protection (both projects and advisory services and analytics). ii. Countries with a similar level of social protection maturity. In the absence of a data set on the social protection maturity level, relevant literature will be reviewed, complemented by the use of ASPIRE (Atlas of Social Protection Indicators of Resilience and Equity) database and expert advice (including from World Bank task team leaders). iii. Countries that are exposed to similar types of shocks (we will focus on countries exposed to slow and predictable onset versus rapid, unpredictable onset). This typology will be developed using the literature and in consultation with World Bank and external experts. iv. Countries that have experienced different adaptive social protection outcomes when faced with covariate shocks. This list will be developed using the literature and in consultation with World Bank experts. 30 Appendix C. World Bank Adaptive Social Protection Support This appendix explains preliminary lending and advisory services and analytics (ASA) portfolio identification strategies and presents the results of portfolio reviews conducted on those identified portfolios. Portfolio Identification Lending Projects To identify relevant lending projects to include in the adaptive social protection (ASP) evaluation, we used an identification strategy based on four elements: • The evaluation’s conceptual framework that (i) defines our understanding of ASP and emphasizes the importance of safety nets for building the resilience of poor and vulnerable populations in relation to covariate shocks, and (ii) provides details on the four building blocks of ASP (programs, data and information, finance and institutional arrangements, and partnerships) for developing ASP policies and systems and thus justifies the inclusion of social protection delivery systems in addition to safety nets within the scope of the portfolio. The ASP building blocks are also important for the portfolio’s subsequent characterization. • Consultations with relevant stakeholders, including many Social Protection and Jobs Global Practice (GP) staff we interviewed during the scoping exercise. The consultations help identify important ASP projects from management’s standpoint and divide the World Bank support into two groups: projects led by Social Protection and Jobs for ASP and projects co-led by Social Protection and Jobs for ASP. • In-depth review of World Bank support in the Sahel (the World Bank has been supporting ASP in at least six countries since 2014: Burkina Faso, Chad, Mali, Mauritania, Niger, and Senegal). The review offers guidance and a possible benchmark in terms of design for the review of ASP support (World Bank 2019). • Previous Independent Evaluation Group evaluations that assessed ASP at the portfolio level, which provided valuable insights and which we used for cross- checking our identification strategy’s relevance and usefulness. Based on these criteria, the portfolio identification strategy restricted the evaluation period to approvals during the past decade (fiscal years [FY]12–22). This time horizon allowed covering the support to the Sahel, the first set of projects to articulate the term “adaptive social protection” explicitly within the World Bank. The time horizon gave us 31 two years before that to cover any additional efforts taking place at about the same time and allowed us to increase the number of closed operations that offer more information for analysis. Regarding the financing source, we considered only International Bank for Reconstruction and Development, International Development Association, and recipient-executed trust fund agreement types, restricting recipient-executed trust fund support further to projects with US$5 million or more in commitments to guarantee availability of documentation. Projects that met those criteria of period and instrument were then classified according to three more delimitation criteria: (i) having social safety net or social protection service delivery Operations Policy and Country Services theme codes (we considered projects with these two theme codes relevant regardless of their percentage share because they capture the more likely channels through which the World Bank provided support to ASP); (ii) projects with Social Protection and Jobs as the lead GP (any such project has the potential to address at least one ASP building block); and (iii) having Social Protection and Jobs as a contributing GP (this criterion allowed us to identify projects with cross-sector collaboration, an important feature when responding to shocks and building ASP systems). We then used those criteria to identify (i) a global lending portfolio of social protection composed of projects for which the Social Protection and Jobs GP is either a leading or contributing GP, and (ii) a subset of this lending portfolio reviewed for the Approach Paper. The lending portfolio is composed of three groups: (i) projects led by Social Protection and Jobs for ASP (projects with relevant theme codes and led by Social Protection and Jobs, assuming that only Social Protection and Jobs leads investments in ASP through safety nets or social protection delivery systems or both); (ii) projects co-led by Social Protection and Jobs for ASP (projects with relevant theme codes and for which Social Protection and Jobs is not the leading but a contributing GP, which reflects the fact that other sectors such as disaster risk management use social protection systems to deliver their support); and (iii) other projects relevant to ASP based on consultations with World Bank counterparts. Finally, we conducted manual verification of the subset of the lending portfolio to eliminate false positives (projects that do not support any of the four ASP building blocks). Using those criteria and the identification strategy, the global lending portfolio of social protection consists of 573 operations, of which 424 are parent projects and 149 are additional financing. The lending portfolio subset for this Approach Paper consists of 370 operations, of which 265 are parent projects and 105 are additional financing (figure B.1). 32 Figure B.1. Lending Portfolio Identification Strategy Source: Independent Evaluation Group based on (i) “All Projects” report from World Bank Data Explorer (accessed on September 29, 2022, at https://dataexplorer.worldbank.org/search/dataset/details?id=46bcd871-07f8-ec11-bb3d- 00224804dd77); and (ii) lending project database built by the Independent Evaluation Group based on “ssrds_Operations_Lending” report in Power BI (accessed on September 23, 2022, at https://app.powerbi.com/home?experience=power-bi). Note: In the delimitation and inclusion stages, the base is the 4,309 projects identified in the search stage. In the inclusion stage, criterion 6 captures projects in Social Protection and Jobs’ Food Security Operations Response Tracker that do not already meet criteria 4 or 5. Consultations with World Bank counterparts and review of the Sahel Adaptive Social Protection Program Annual Report of FY22 (Draft) also revealed specific projects supported by the Sahel program in its phases 1 and 2, 2014–22—all of which meet criteria 4 or 5. AF = additional financing; ASP = adaptive social protection; DPF = development policy financing; FY = fiscal year; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; OPCS = Operations Policy and Country Services; RETF = recipient-executed trust fund; SPJ = Social Protection and Jobs. Advisory Services and Analytics The evaluation will implement text analytics for identifying a robust ASP ASA portfolio. We will identify a list of core ASA projects related to ASP based on preliminary scoping and review of a purposive sample of ASA and on consultations with World Bank stakeholders and use this list as a benchmark against which to compare ASA from a larger pool within the evaluation period. We expect this strategy to yield a final list of ASA related to ASP that can be systematically reviewed. For this Approach Paper, we selected a preliminary purposive sample of ASA for review based on targeted keyword searches on project names, development objectives, and names of deliverables. The base for this targeted keyword search was all active and closed ASA with sign-off dates between FY12 and FY22, which totals 15,772 activities. Based on quick scanning of recent ASP ASA in the Sahel, selected keywords included “adaptive social protection,” “adaptive,” “strengthening systems,” “stress test,” “sahel,” “saspp,” “shock respons,” “shock-respons,” “disaster,” “drf,” “drm,” and “hazard.” The 33 targeted keyword search yielded a purposive sample of 145 projects for review, of which 115 were true positives after manual review (79 percent; figure B.2). Figure B.2. Advisory Services and Analytics Purposive Sample Identification Strategy Source: Independent Evaluation Group based on (i) “All Projects” report from World Bank Data Explorer (accessed on September 29, 2022, at https://dataexplorer.worldbank.org/search/dataset/details?id=46bcd871-07f8-ec11-bb3d- 00224804dd77); and (ii) ASA database built by the Independent Evaluation Group based on “ssrds_Operations_ASA” report in Power BI (accessed on September 23, 2022, at https://app.powerbi.com/home?experience=power-bi). Note: In the delimitation stage, the base for criteria 1 is all 15,772 ASA identified in the search stage; the base for criteria 2 is the 1,030 ASA from the search stage led by the Social Protection and Jobs GP. ASA = advisory services and analytics; FY = fiscal year; GP = Global Practice. Preliminary Portfolio Review and Analysis Lending Portfolio The portfolio review was applied to the lending portfolio subset reviewed for the Approach Paper and was centered around the four building blocks of the ASP conceptual framework. It identified relevant World Bank support to these building blocks, hinting at some of the adaptive elements. The building blocks are defined as “the basic elements of any social protection system… [they] refer to the core components that enable social protection systems, including, for instance, social protection policies and institutions, safety net programs, and information systems such as social registries” (World Bank 2019, 69). We organized the review as follows. First, we manually tagged each project as focused on an ASP building block, based on commitment allocation to relevant components; in the case of development policy financing (DPF), we tagged individual prior actions as focused on a building block based on their content, but we did not use commitments as 34 criteria. We classified projects further in areas of support, which provided more disaggregated and specific information on each project. Second, for investment project financing (IPF) and Program-for-Results (PforR), we also coded secondary building blocks addressed by each project and identified their corresponding areas of support through keyword searches on the relevant component text of each of the tagged building blocks (table B.1). Table B.1. Portfolio Review Framework ASP Building Block Area Process 1. Programs Social assistance: cash transfers, cashFor IPF and PforR, we manually a. Social assistance for work, food and in-kind support, coded a single focus building block b. Labor markets targeting, fee waivers, and targeted and corresponding area based on subsidies desk review of components and their c. Social insurance Labor markets: training, employment commitment amount allocations incentives and wage subsidies, Each relevant component text is unemployment subsidies linked to its corresponding tagged Social insurance: contributory old building block. For DPF, we coded age, survivor, and disability pensions building blocks and corresponding areas for individual prior actions 2. Data and information Data and information: data-sharing platforms and protocols, social For IPF and PforR, we manually 3. Finance registries, early warning systems, coded one or more secondary 4. Institutional arrangements and building blocks based on desk review partnerships postshock needs assessments of components and their Finance: preplanned risk financing commitment amount allocations and risk layering for shock response For IPF and PforR, we identified areas Institutional arrangements and related to secondary building blocks partnerships: institutional capacity, through targeted keyword searches policies, and regulations on the corresponding building block’s relevant component text, where each area and related terms in the conceptual framework are used as specific keywords For each building block, keywords used include the following: Social assistance: “cash transfer,” “public work,” “food,” “in kind,” “in- kind,” “targeting,” “fee waiver,” “subsidy” Labor market programs: “training,” “active labor market program,” “almp,” “wage subsid,” “unemployment benefit,” “subsistence allowance,” “unemployment assistance” Social insurance: “pension,” “social security” Data and information: “management information system,” “social registr,” “early warning system,” “post- disaster,” “post disaster” 35 ASP Building Block Area Process Finance: “risk layering,” “early financ,” “risk financ”; Institutional Arrangements and partnerships: “surge capacity,” “coordination,” “strategy,” “partner,” “accountability,” “humanitarian” Source: Independent Evaluation Group. Note: Component text relevant to each tagged building block was manually retrieved from Project Appraisal Documents for all parent IPFs and from Project Appraisal Document results areas and disbursement-linked indicators for all parent PforR. N = 265 projects. For DPF, prior action text was retrieved from the OPCS DPF Prior Actions database. ASP = adaptive social protection; DPF = development policy financing; IPF = investment project financing; OPCS = Operations Policy and Country Services; PforR = Program-for-Results. Advisory Services and Analytics We classified the purposive sample of 115 ASA according to type of focused support, based on development objectives and activity description text available in the Operations Portal. The final taxonomy of focused support included four categories: integrated support for strengthening ASP systems; diagnostics and assessments; support for improving specific shock-responsiveness elements; and dialogue, coordination, and strategies. We explain each of these categories in the results section for ASA. Results Lending Portfolio The global lending portfolio of social protection consists of 424 parent projects and US$90.3 billion in commitments, with almost two-thirds still active. Social Protection and Jobs is the lead in 200 projects (47 percent), with the remaining 224 distributed across Macroeconomics, Trade, and Investment (17 percent); Health, Nutrition, and Population (9 percent); Education (8 percent); and other GPs (19 percent). Social Protection and Jobs is a contributing GP in these 224 projects. Two-thirds of the portfolio is IPF (278 projects), followed by DPF (117 projects) and PforR (29 projects). Just under half of projects are in International Development Association countries, more than 40 percent of projects are in Africa, and about 50 percent of approvals took place between FY20 and FY22. About 25 percent of the portfolio (105 projects) is in countries classified as fragile and conflict-affected situations. The lending portfolio subset reviewed for the Approach Paper is composed of 265 projects amounting to about US$60 billion in commitments, mostly led by Social Protection and Jobs (60 percent of both projects and volume). IPF is the main financing instrument in about 65 percent of projects, followed by DPF with 30 percent (led by the Macroeconomics, Trade, and Investment GP). The Health, Nutrition, and Population GP contributes with 10 percent of IPF; other GPs generally do not lead projects in the 36 portfolio. Most of the portfolio is active (159 projects or 60 percent),1 with approvals picking up starting in FY16 and more than 50 percent of them taking place between FY20 and FY22, consistent with the World Bank’s response to the COVID-19 crisis (table B.2). The portfolio is distributed evenly between International Development Association and International Bank for Reconstruction and Development and blend countries, with about half of projects in each group. Geographically, half of the projects are in Africa, and the other half are distributed evenly across other Regions. IPF is important across Regions, and DPF stands out in Latin America and the Caribbean (figure B.3). Only about 25 percent of the portfolio (67 projects) is in countries classified as fragile and conflict-affected situations. 37 Table B.2. Lending Portfolio by Global Practice and Lending Instrument Global Portfolio of Social Protection Lending Portfolio Subset Reviewed for AP IPF DPF PforR Total Share IPF DPF PforR Total Share Lead Global Practice Measure (%) (%) Social Protection and Jobs Projects (number) 167 22 11 200 47.2 136 17 10 163 61.5 Commitment amount (US$, 32,726 4,445 4,596 41,767 46.2 26,591 3,580 4,576 34,748 58.4 millions) Macroeconomics, Trade, and Projects (number) — 73 — 73 17.2 — 48 — 48 18.1 Investment Commitment amount (US$, — 20,903 — 20,903 23.1 — 14,797 — 14,797 24.9 millions) Health, Nutrition, and Population Projects (number) 32 3 2 37 8.7 17 2 1 20 7.5 Commitment amount (US$, 3,394 800 570 4,764 5.3 1,953 700 420 3,073 5.2 millions) Education Projects (number) 26 3 5 34 8.0 4 3 1 8 3.0 Commitment amount (US$, 2,228 950 1,424 4,602 5.1 169 950 400 1,519 2.6 millions) Social Sustainability and Projects (number) 11 1 — 12 2.8 5 1 — 6 2.3 Inclusion Commitment amount (US$, 1,540 526 — 2,066 2.3 533 526 — 1,059 1.8 millions) Urban, Disaster Risk Projects (number) 6 3 1 10 2.4 2 2 1 5 1.9 Management, Resilience, and Commitment amount (US$, 1,335 1,070 125 2,530 2.8 742 570 125 1,437 2.4 Land millions) Governance Projects (number) 5 2 3 10 2.4 2 2 — 4 1.5 Commitment amount (US$, 447 149 1,925 2,522 2.8 112 149 — 261 0.4 millions) Agriculture and Food Projects (number) 9 — 1 10 2.4 2 2 — 4 1.5 Commitment amount (US$, 1,280 — 320 1,600 1.8 891 — — 891 1.5 millions) Finance, Competitiveness, and Projects (number) 12 4 4 20 4.7 — 2 — 2 0.8 Innovation Commitment amount (US$, 2,228 1,005 1,357 4,590 5.1 — 105 — 105 0.2 millions) Energy and Extractives Projects (number) 1 3 1 5 1.2 1 1 — 2 0.8 38 Global Portfolio of Social Protection Lending Portfolio Subset Reviewed for AP IPF DPF PforR Total Share IPF DPF PforR Total Share Lead Global Practice Measure (%) (%) Commitment amount (US$, 50 1,500 750 2,300 2.5 50 400 — 450 0.8 millions) Transport Projects (number) 4 — — 4 0.9 1 — — 1 0.4 Commitment amount (US$, 964 — — 964 1.1 473 — — 473 0.8 millions) Poverty and Equity Projects (number) — 2 — 2 0.5 — 1 — 1 0.4 Commitment amount (US$, — 510 — 510 0.6 — 500 — 500 0.8 millions) Environment, Natural Resources, Projects (number) 3 1 1 5 1.2 — 1 — 1 0.4 and the Blue Economy Commitment amount (US$, 209 200 500 909 1.0 — 1 — 1 0.4 millions) Water Projects (number) 1 — — 1 0.2 — — — — — Commitment amount (US$, 200 — — 200 0.2 — — — — — millions) Digital Development Projects (number) 1 — — 1 0.2 — — — — — Commitment amount (US$, 115 — — 115 0.1 — — — — — millions) Total Projects (number) 278 117 29 424 100.0 172 80 13 265 100.0 Commitment amount (US$, 46,716 32,058 11,567 90,342 100.0 31,514 22,478 5,521 59,513 100.0 millions) Source: Independent Evaluation Group. Note: AP = Approach Paper; DPF = development policy financing; IPF = investment project financing; PforR = Program-for-Results. 39 Figure B.3. Lending Portfolio Subset Reviewed for Approach Paper, by Lending Group and Region a. By Lending Group b. By Region Source: Independent Evaluation Group. Note: In panel a, N = 257 projects, excluding 5 regional projects and 3 projects in The Bahamas and St. Maarten, which are unclassified by income group. In panel b, N = 264 projects, excluding 1 regional project for the whole Africa region. DPF = development policy financing; EAP = East Asia and Pacific; ECA = Europe and Central Asia; ESA = Eastern and Southern Africa; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IPF = investment project financing; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; PforR = Program-for-Results; SAR = South Asia; WCA = Western and Central Africa. Most of the World Bank’s investment focuses on social assistance (60 percent) and is mainly through financing cash transfers and public works with IPF. World Bank support also focuses on institutional strengthening, especially through DPF support to policy and regulations. World Bank support to data and information seeks mostly to strengthen social registries and cross-sector data-sharing platforms, primarily through DPF, whereas support to labor market programs focuses mainly on educational and vocational training through IPF and PforR. IPF is the preferred instrument for World Bank support to social assistance and labor market programs, and DPF is the main instrument used when supporting the other building blocks of data and information, finance and institutional arrangements, and partnerships. Important areas for DPF support include budget allocation to shock responses, preplanned risk financing, and risk layering for shock responses (table B.3). Table B.3. Adaptive Social Protection Building Blocks by Lending Instrument Main Building Block IPF DPF PforR Total Share (%) Social assistance programs 110 43 6 159 60 Institutional arrangements 19 34 5 58 22 Data and information 13 33 1 47 18 40 Main Building Block IPF DPF PforR Total Share (%) Labor market programs 29 10 — 39 15 Finance — 13 1 14 5 Social insurance programs 1 1 — 2 1 Total 172 80 13 265 100 Source: Independent Evaluation Group. Note: Table shows project distribution by main building block and lending instrument, based on results from implementing the portfolio review framework to the lending portfolio subset. DPF = development policy financing; IPF = investment project financing; PforR = Program-for-Results. The portfolio review offers a preliminary assessment of how the World Bank is combining its support to ASP building blocks. Table B.4 shows the intersection of main and secondary building blocks and areas of support for IPF and PforR projects. Projects that focus their support on social assistance through cash transfers, for instance, often also support food and in-kind interventions, targeting, educational and vocational training, strengthening social registries, and institutional arrangements for social protection. For example (reading the table from left to right), among the 74 projects focused on financing cash transfers, 44 (59 percent) also include food and in-kind support (for example, nutrition interventions).1 Projects that focus on cash for work show a similar pattern but with fewer intersecting projects. 41 Table B.4. Key and Secondary Building Blocks and Areas in Investment Project Financing and PforR Secondary Building Block Social Institutional Labor Market Insurance Data and Arrangemen Social Assistance Programs Programs Programs Information Finance ts Secondary Area Fee Employ Preplanned No Waive ment Une Contributo Data- Risk Polici Sec Cas Food rs and Incentiv mplo ry Old Age, Sharing Soci Early Financing Instit es on h and In- Target es and ymen Survivor, Platform al Warn and Risk ution and dar Focus Cash for kind Targ ed Trai Wage t and s and Regi ing Layering al Regu y Building Trans Wor Suppor etin Subsi nin Subsidie Subsi Disability Protocol strie Syste for Shock Capac lation Are Block Focus Area fers k t g dies g s dies Pensions s s ms Response ity s a Social Cash transfers 12 44 49 2 34 1 1 8 12 24 1 31 25 6 assistance (n = 74) programs Cash for work 18 13 13 20 1 2 7 2 1 2 10 10 1 (n = 26) Food and in- 1 2 2 5 1 1 1 1 kind support (n = 6) Targeting (n = 5) 1 1 1 1 3 1 Fee waivers and 1 1 1 1 1 1 2 targeted subsidies (n = 5) Labor Training (n = 14) 3 2 4 5 1 3 1 2 4 5 1 market Employment 4 1 2 5 1 7 1 1 2 2 1 programs incentives and wage subsidies (n = 10) Unemployment 3 1 1 1 2 2 1 subsidies (n = 5) 42 Secondary Building Block Social Institutional Labor Market Insurance Data and Arrangemen Social Assistance Programs Programs Programs Information Finance ts Secondary Area Social Contributory old 1 1 1 1 1 insurance age, survivor, programs and disability pensions (n = 1) Data and Data-sharing 3 1 1 3 2 2 3 4 information platforms and protocols (n = 10) Social registries 2 1 1 (n = 3) Early warning 1 1 1 systems (n = 1) Finance Preplanned risk 1 1 financing and risk layering for shock response (n = 1) Institutional Institutional 3 3 3 6 2 2 6 11 arrangemen capacity (n = 21) ts Policies and 2 1 1 1 1 1 1 regulations (n = 3) Source: Independent Evaluation Group. Note: Table reports number of projects. Projects can be double counted across columns, since a single project can be tagged with multiple secondary areas. The “Focus Area” column shows the number of unique projects for each focus area through the “n” in parentheses, and thus the table should be read from left to right by individual row. For example, the cash transfers focus area row shows that there are 74 unique projects with a cash transfers focus area, of which 12 also address cash for work (16 percent), 44 also address food and in-kind transfers (59 percent), and so on. Blue cells darken in color with the number of projects. N = 185 projects: 172 investment project financing and 13 PforR. PforR = Program-for-Results. 43 The lending portfolio is composed mostly of projects led by Social Protection and Jobs (61 percent), mainly through IPF. Projects co-led by Social Protection and Jobs (37 percent) are mainly DPF and IPF (table B.5). Table B.5. Type of Adaptive Social Protection Support by Lending Instrument Share Type of ASP Support IPF DPF PforR Total (%) Led by SPJ for ASP 135 17 10 162 61 Co-led by SPJ for ASP 34 62 3 99 37 Other 3 1 - 4 2 Total 172 80 13 265 100 Source: Independent Evaluation Group. Note: Table shows project distribution by type of ASP support and lending instrument, based on results from implementing the portfolio review framework to the identified lending portfolio. ASP = adaptive social protection; DPF = development policy financing; IPF = investment project financing; PforR = Program-for-Results; SPJ = Social Protection and Jobs. Most of the projects led by Social Protection and Jobs for ASP are shock-responsive and support coping mechanisms through horizontal or vertical expansion of cash transfers and through interventions to support graduation from poverty and improved service delivery mechanisms, such as payment systems to vulnerable populations. Institutional capacity building and training are also important areas of these projects. Similarly, projects co-led by Social Protection and Jobs for ASP include horizontal and vertical expansion of cash transfers combined with institutional strengthening and training; however, they emphasize emergency response interventions like fee waivers and targeted subsidies and food and in-kind support as part of their focus and also emphasize World Bank emergency response instruments like catastrophe deferred drawdown options. Box B.1 describes one project led by Social Protection and Jobs for ASP and one project co-led by Social Protection and Jobs for ASP. Box B.1. Showcasing of Projects Led and Co-Led by Social Protection and Jobs for Adaptive Social Protection P177814: “Pathway to Sustainable Livelihoods,” Lesotho, FY22, IPF, led by Social Protection and Jobs, active. The project seeks to improve the efficiency and equity of selected social assistance programs and strengthen the livelihoods of poor and vulnerable households. It (i) supports the introduction of an economic inclusion package designed to strengthen beneficiary livelihoods, aligned with the graduation and “cash plus” successful approaches to economic inclusion evidenced in the literature; (ii) pre-positions contingent finance to enable timely delivery of shock-responsive cash transfers using existing delivery systems for more efficient shock responses; and (iii) continues to strengthen core delivery systems to improve the efficiency and equity of the selected social assistance programs. This includes enhancing the ability of programs to target poor people and improving efficiency by reducing administrative costs, nontransparent and costly processes, and leaks. It also aimed to increase the Ministry of Social 44 Development’s preparedness to respond to shocks by (i) supporting wider adoption of digital payments that can facilitate faster delivery of postshock cash transfers, and (ii) strengthening the social registry and management information systems as the primary operational tools used when increasing coverage in response to shocks. P165056: “Malawi Disaster Risk Management Development Policy Financing with CAT DDO [catastrophe deferred drawdown option],” Malawi, FY19, DPF, led by Urban, Disaster Risk Management, Resilience, and Land, co-led by Social Protection and Jobs, closed. The project sought to strengthen the institutional and financial capacity of the government of Malawi for multisectoral disaster and climate risk management. It addressed three pillars: pillar A, strengthening the institutional framework and coordination mechanisms for the implementation of the national disaster and climate resilience agenda; pillar B, increasing climate and disaster resilience in physical developments and infrastructure; and pillar C, strengthening adaptive social protection mechanisms and government financial capacity to respond to disasters. Pillar C sought approval and adoption of (i) the Malawi National Social Support Program II (2018 –23) and Implementation Plan that prioritizes development of a shock-sensitive social protection system; and (ii) the Disaster Risk Financing Strategy and Implementation Plan, outlining the government’s strategic objectives to strengthen financial preparedness for effective and timely disaster response. Source: Independent Evaluation Group. The lending subset also includes projects from the Independent Evaluation Group’s Reducing Disaster Risks from Natural Hazards: An Evaluation of the World Bank’s Support, Fiscal Years 2010–20, which identified at least five parent lending projects as part of ASP, all of which are included in the portfolio group led by Social Protection and Jobs for ASP.1 Advisory Services and Analytics The purposive sample consists of 115 ASA worth about US$95 million in total cumulative expenditures. Social Protection and Jobs leads most ASA (69 percent), followed distantly by Finance, Competitiveness, and Innovation and Poverty and Equity (4 percent each). ASP ASA approvals have been increasing since FY14 but spiked in FY20, with 22 approvals compared with an average of 8 annual approvals between FY12 and FY19, consistent with the World Bank’s response to COVID-19 (figure B.4). West Africa accounts for one-third of ASA and cumulative expenditures in the sample, in line with World Bank support to ASP in Sahel countries through the Sahel Adaptive Social Protection Program, funded by a multidonor trust fund (table B.6). Table B.6. Advisory Services and Analytics Purposive Sample by Region ASA Share Cumulative Expenditures Share Region (no.) (%) (US$, millions) (%) WCA 37 32 33 35 LAC 18 16 5 5 45 ASA Share Cumulative Expenditures Share Region (no.) (%) (US$, millions) (%) ESA 14 12 8 8 OTH 14 12 12 13 EAP 11 10 9 10 SAR 9 8 14 14 AFR 8 7 12 13 ECA 4 3 2 2 Total 115 100 95 100 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; ESA = Eastern and Southern Africa; LAC = Latin America and the Caribbean; OTH =Other; SAR = South Asia; WCA = Western and Central Africa. Figure B.4. Advisory Services and Analytics Purposive Sample by Sign-Off Year Source: Independent Evaluation Group. Almost half of the purposive ASA sample focuses on providing integrated support for strengthening ASP systems through a mix of activities, typically involving system assessments and diagnostics with follow-up capacity building; just-in-time technical assistance for design, implementation, and evaluation of ASP systems and programs; policy dialogue; and collaboration and coordination with technical and financial partners. Sometimes, this integrated support is tailored to vulnerable groups, such as in the support to gender-smart ASP. Next, there is pure diagnostic and assessment work on disaster risk vulnerability and social protection systems, including social protection and climate and health stress tests, analyses of interrelated stresses, estimation of welfare impacts of shocks, impact evaluations of social safety net programs, and social protection expenditure analyses, among others. These are followed by ASA that focuses support on improving specific shock-responsive elements such as design and implementation of emergency cash transfers (for example, improvement of payment delivery mechanisms after emergencies), proof of concept activities for disaster risk management and decision support tools, financial resilience against natural disasters, 46 better use of disaster management funds, and increased disaster insurance coverage. Few sampled ASA have an exclusive focus on dialogue, coordination, or strategies. They include ASA seeking to strengthen synergies among the Social Protection and Jobs and Disaster Risk Management and Health sectors within the World Bank, ASA that exclusively supports experience sharing and learning on ASP, and ASA that supports only ASP strategies. Examples of synergy strengthening are the mainstreaming of disaster risk management into the social protection sector through analytic inputs and knowledge exchange activities, and investigating how Human Development delivery systems (education, health, and social protection) could be integrated and leveraged better in relation to shocks and crisis situations; experience sharing and learning includes, for example, regional learning activities to share social protection experiences in building resilience to disaster and climate-related shocks. ASA focusing exclusively on strategies can include disaster and climate shock response strategies, preparedness and adaptation strategies, and policies aiming to promote and protect human capital from the effects of the COVID-19 emergency, among others (table B.7 and box B.2). Table B.7. Focused Support to Adaptive Social Protection in Advisory Services and Analytics Cumulative Projects Share Expenditures Type of Focused Support to ASP (no.) (%) (US$, millions) Share (%) Integrated support for strengthening ASP systems 51 44.3 47.6 50.1 Diagnostics and assessments 26 22.6 17.1 18.0 Support for improving specific shock-responsiveness 24 20.9 21.2 22.3 elements Dialogue, coordination, or strategies 14 12.2 9.1 9.5 Total 115 100.0 95.0 100.0 Source: Independent Evaluation Group. Note: ASP =adaptive social protection. Box B.2. Four Advisory Services and Analytics, Each with a Different Type of Focused Support Integrated support for ASP systems: P156093, “Design of Adaptative Social Protection Interventions,” Niger, fiscal year FY15, advisory, led by Social Protection and Jobs, closed. The project sought to improve the evidence base and foster policy dialogue on adaptive social protection systems aiming to improve the resilience of poor and vulnerable households in Niger. The activities were grouped under four main pillars: Pillar 1: Poverty and vulnerability analysis. Involved data collection on vulnerability, entailing support of the collection of 2014 national household survey data and the preparation of an updated and expanded analysis of poverty, consumption, income, food security, resilience, and 47 nutrition; analysis of the links between weather and other shocks on welfare and household resilience, including a review of risks and vulnerabilities such as climatic risks and food security vulnerability, to define vulnerability and resilience among poor households; and support for data collection to complement information available from the latest population census and household surveys. Pillar 2: Design of adaptive tools and instruments for resilience. The design of adaptive cash-for- work programs: Cash-for-work projects and microprojects implemented in the country reviewed and lessons learned are informing the elaboration of a menu of innovative and effective microprojects that contribute to climate resilience and disaster preparedness. Accompanying measures and promotion of livelihoods: cash-based interventions are increasingly implemented together with accompanying measures. Two sets of such measures expanded (i) productive accompanying measures to promote resilience and address the structural causes of food insecurity by helping households diversify their livelihoods and income portfolios, and (ii) behavioral accompanying measures seeking to change nutrition and parenting practices. Pillar 3: Learning from innovative approaches. A retrospective and a prospective targeting assessment. The retrospective assessment reviewing the effectiveness of targeting mechanisms used in Niger (proxy-means targeting and participatory approaches such as Household Economy Analysis) with the objective to establish the most effective ways to target households as part of an adaptive social protection system. The prospective assessment evaluated the effectiveness of different targeting methods: (i) community-based Household Economy Analysis, (ii) proxy-means testing, and (iii) mixed proxy-means testing and community-based approaches. It collected benchmark data and complementary data to assess their relative effectiveness regarding reaching the economically poor and the legitimacy of the outcomes. Diagnostics and assessments: P167768, “Evidence for Building Madagascar’s Social Safety Net,” Madagascar, FY18, analytical, led by Social Protection and Jobs, closed. This advisory services and analytics (ASA) sought to generate evidence to help harmonization, coordination, and scalability of Madagascar’s social safety net system. It had three main activities: i. Social Protection Expenditure Analysis. The objective of component 1 was to carry out a Public Expenditure Review of social protection in Madagascar, mapping all resources to the social safety net, both government budget allocations and expenditures, direct donor contributions off budget, private sector and community contributions, and nongovernmental organizations. The ASA would bring key national actors and development partners together toward a common understanding of the financing needs and gaps through roundtable meetings, workshops, and South-South learning. The main deliverable for this component will be the Social Protection Public Expenditure Review. ii. Impact Evaluations of the Main Safety Net Programs. This component aimed at increasing the understanding of national actors and local communities of the performance of the Human Development Cash Transfer, Productive Safety Net Program, and Fiavota program regarding their respective objectives. The comparative analysis would be used to improve the implementation of the programs and help the government in defining the future landscape of safety net programs. iii. Promoting Evidence-Based Policy Dialogue in Social Protection. This activity consisted of several high-level events, roundtable and working group meetings, and publication of key analytical documents to promote dialogue on the harmonization of donor approaches and the future direction of social protection in the country. 48 Support for improving specific shock-responsiveness elements: P176473, “Developing an Evidence-Based Adaptive Safety Net in Haiti,” FY21, analytical, led by Social Protection and Jobs, active. This ASA seeks to build an evidence base to inform the design and improve the delivery of a shock-responsive safety net targeting female beneficiaries in households with small children and persons with disabilities in Haiti (including COVID-19 and natural disasters). This will be done through (i) analyzing the impact of COVID-19 on vulnerable households, particularly on women and persons with disabilities (and the intersection of the two); (ii) producing evidence on the impact of a safety net targeting female beneficiaries in households with small children and persons with disabilities; (iii) gathering evidence on the constraints and opportunities from the demand (beneficiary) side for digital payment modalities and identifying recommendations for the accompanying measures; and (iv) building capacity and disseminating knowledge among social protection stakeholders. Dialogue, coordination, or strategies: P155074, “Social Risk Management and Disaster Risk Management Program,” FY15, analytical, led by Urban, Disaster Risk Management, Resilience, and Land, closed. This ASA sought to mainstream disaster risk management (DRM) into the social protection sector. In doing so, it supported two lines of action: i. Country program to mainstream DRM into World Bank operations: analytical outputs have contributed to mainstreaming DRM into social protection operations, providing frameworks for social protection task team leaders to better understand how to integrate DRM into country dialogue, project design, and implementation. ii. Facilitated the exchange of knowledge, expertise, and technology between Japan and developing countries: facilitated the dissemination of previously lesser-known Japanese expertise on mainstreaming DRM into social protection, bringing a Japanese expert to a large international forum on adaptive social protection to present this experience and learning and producing a detailed case study for wider dissemination. Source: Independent Evaluation Group. Reference World Bank. 2019. Sahel Adaptive Social Protection Program: Annual Report 2019. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/680361585895594749/pdf/Sahel-Adaptive- Social-Protection-Program-Annual-Report-2019.pdf. Appendix B, Part 1 1Of the closed projects (106), 56 percent are development policy financing, 42 percent are investment project financing, and only 2 percent are Program-for-Results. In fact, 74 percent of development policy financing projects are closed compared with only 26 percent of investment project financing and 15 percent of Program-for-Results projects. Appendix B, Part 2 49 1Note that this is a preliminary assessment using keywords searches, and we will confirm the data during the evaluation. Appendix B, Part 3 1This list includes the following projects: Niger Adaptive Social Safety Nets Project (P155846), Senegal Social Safety Net (P156160), Senegal Additional Financing to the Social Safety Net Project (P162354), Niger Adaptive Safety Net Project 2 (P166602), and Mauritania Social Safety Net System Project II (P171125). 50 Appendix D. Lessons from Recent Independent Evaluation Group Evaluations Social Safety Nets: An Evaluation of World Bank Support, 2000–2010, 2013. The evaluation assessed the World Bank’s support for social safety nets (SSNs) during 2000–10. Main findings relevant to this evaluation include the following: • The World Bank delivered most of its support to SSNs during the evaluation period in 2009–10 responding to food, fuel, and financial crises. • The food, fuel, and financial crises pointed out weaknesses in SSNs. For example, poverty-targeted SSNs in many middle-income countries were not flexible enough to increase coverage or benefits as needed, and low-income countries lacked poverty data and systems to target and deliver benefits. These weaknesses, in turn, limited the use of social protection in the event of a shock and highlighted the necessity of investing in flexible SSNs and enhancing country preparedness and responsiveness. • The World Bank support evolved in a positive direction, moving from a project- focused approach that emphasized delivery of social assistance benefits toward an approach that focused on helping countries build SSN systems and institutions to respond better to poverty, risk, and vulnerability. • The World Bank support achieved its stated short-term objectives and helped countries achieve immediate impacts. Yet achieving the longer-term goal of developing country SSNs requires a long-haul effort that could not be achieved without strong poverty data to ensure adequate reach of target groups. • The evaluation recommended the following to improve SSNs: (i) Engage during stable times to build SSNs that can help countries respond effectively to shocks; (ii) support the development of SSN institutions and systems; (iii) increase SSN engagement in low-income countries; (iv) improve the results frameworks of World Bank–supported SSN projects; and (v) improve the internal coordination of World Bank support for SSNs. Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations, 2017. This synthesis drew lessons from Independent Evaluation Group evaluations of World Bank Group support for strengthening client country response and resilience to systemic shocks. The synthesis covered response to systemic shocks—that is, shocks that affect highly significant numbers of economic agents in at least one country, possibly several— and their impact that unfolds during a relatively short time. Systemic shocks included the sharp increases in food prices in 2007–08, the 2008–09 global financial crisis, natural 51 disasters, and pandemics. Main findings relevant to this evaluation include the following: • SSNs to address shocks received relatively little attention during the decade that preceded the food, fuel, and financial crises. During this precrisis decade, SSNs focused on addressing the needs of chronically poor or vulnerable people and developing the human capital of poor people. Although these areas of support were relevant and important, the World Bank and its borrowers did not focus on developing flexible SSNs appropriate for responding to systemic shocks. • The World Bank was most effective in helping countries design effective SSNs and provide targeted social transfers where it had steady engagement through lending, analytic and advisory work, and dialogue for an extended period before the crisis. Such long-term engagement—evident in Brazil, Colombia, Ethiopia, and Moldova, for example—enabled countries to develop well-functioning SSN institutions and the World Bank to develop a deeper understanding of country dynamics. • Many middle-income countries found that when the food, fuel, and financial crises hit, poverty-targeted SSNs were not flexible enough to increase coverage or benefits as needed, and low-income countries lacked poverty data and systems to target and deliver benefits. But countries that had developed safety net programs or institutions during stable times could scale up better than those that had not, and the World Bank was able to help them better. This experience underlines the importance of engaging during stable times to build SSNs that can help countries respond effectively to shocks. • The two most common constraints for World Bank support for shock-responsive safety nets were weak country institutions and inadequate data. The lack of adequate SSN programs in many countries led the World Bank to support instruments that were not designed for crisis response. • The World Bank’s focus on systemic shocks has accelerated since the 2008 crises, but designing safety nets (combined with other relevant programs) that address systemic shocks adequately needs greater attention. Because shocks are transitory in nature, an important characteristic is the ability to expand and contract to reach different population groups as needed. Access to reliable poverty data, crisis monitoring systems, and flexible targeting systems are important elements in developing appropriate SSNs. Reducing Disaster Risks from Natural Hazards: An Evaluation of the World Bank’s Support, Fiscal Years 2010–20, 2022. The evaluation focused on World Bank support for reducing 52 disaster risks caused by natural hazards. The report assessed World Bank support for risk identification; risk reduction activities; the integration of disaster reduction and recovery (DRR) into institutions, policy, and planning; preparedness activities, including support for early warning systems; and disaster risk finance. The evaluation found that the World Bank is significantly expanding its support to strengthen disaster preparedness by mainstreaming DRR considerations in its adaptive social protection programs. The share of Social Protection and Jobs Global Practice DRR- mainstreamed lending projects rose from 7 percent to 20 percent between fiscal year (FY)10 and FY15 and between FY16 and FY20, with new projects in 26 countries, mostly in Africa. The World Bank also tripled the amount of Social Protection and Jobs DRR country analytical work between FY10 and FY20. Philippines—Social Welfare and Development Reform Project (Project Performance Assessment Report 2019). The project had two objectives: (i) strengthen the effectiveness of the Department of Social Welfare and Development to efficiently implement the Pantawid Pamilyang Pilipino Program (a conditional cash transfer [CCT] program known as Pantawid); and (ii) strengthen the effectiveness of the Department of Social Welfare and Development to expand an efficient and functional National Household Targeting System of social protection programs. Relevant lessons derived from this experience include the following: • The success of a large, nationwide social protection program like Pantawid lies in creating and strengthening the operational and institutional systems needed to support it. The Social Welfare and Development Reform Project supported not only the CCT program (funding the grants) but primarily the creation of the poverty registry—a critical tool for targeting and providing credibility to the program—and the capacity of the government to manage and sustain such a large program. Thanks to its solid institutional base, Pantawid expanded much faster than originally anticipated—faster than similar programs in any other country globally—and became the third-largest CCT program in the world in population coverage. Adopting a systemic approach was one of the keys to the project’s success. • Strong government ownership is critical to establishing and sustaining ambitious programs like Pantawid. Pantawid is a very good example of many elements coming together successfully: country conditions were conducive to introducing a major social protection program, the World Bank had solid expertise in establishing CCT programs, and the government’s buy-in was strong. This last element, especially, ensured that a new targeting mechanism 53 could replace the traditional patronage system and protected the program from the initial general skepticism. • The World Bank’s ability to bring global knowledge to bear and skillfully deploy a full technical engagement was critical to success. The World Bank’s critical contribution was not the financial part—the government eventually provided 10 times the amount it borrowed. The World Bank’s key contribution was technical assistance (expertise, training, knowledge) mobilized through the project and through trust-funded activities before and after the start of the project, and this became crucial for preparatory work and continued technical support throughout implementation. The World Bank’s support was especially critical in designing and implementing Listahanan (the national household targeting system), the proxy-means testing, and the conditionalities and in carrying out the impact evaluations. • Continuous monitoring and evaluation are essential to maintaining CCT programs like Pantawid and ensuring their constant evolution. A fundamental World Bank contribution was the creation of a highly sophisticated management information system and the introduction of an evaluation culture to support the program. The management information system regularly ensures that Pantawid beneficiaries receive the appropriate grant based on their degree of compliance with conditionalities; it is constantly updated to regulate complex interdependent processes. At the same time, evaluation of processes and results is needed to inform the government of the necessary changes to keep the program effective. • The quality of education and health, not just service use, is critical to achieving the expected gains in human capital. Like all CCT programs, Pantawid acts on the demand side. The project’s conditionalities, such as the family development sessions, have been fundamental to bringing about behavioral change. The program was quite effective in changing attitudes and behaviors (increasing school attendance and supporting regular visits to health clinics) but not as much in affecting development outcomes such as learning, stunted growth of children, or maternal mortality. These outcomes, however, also depend crucially on the quality of services provided. Supply-side conditions need to operate together with incentives on the demand side (such as Pantawid conditionalities) to achieve gains in human capital. • A graduation strategy is essential to ensure that the program delivers on longer-term benefits and acts as a stepping-stone into more stable livelihoods (as with all CCT programs). Pantawid had no significant impact on household 54 expenditure. To the extent that poverty is one of the root causes of school dropouts, the grant amount needs to be adjusted for the income effect to continue to operate. This adjustment needs to occur together with efforts to reduce exclusion errors and ensure that the right beneficiaries are served by the program. Lessons from Health and Nutrition-Related Interventions during Crisis to Inform Future Responses, April 2021. This crisis response note reviewed closed nutrition projects to identify interventions addressing crisis situations. Special attention was given to the underlying factors explaining projects’ results. Success and challenge factors were extracted from the projects’ Implementation Completion and Results Reports, Implementation Completion and Results Report Reviews, and Project Performance Assessment Reports. Aggregate lessons were identified across projects to help inform future crisis responses. Lessons relevant for social protection include the following: • Expanding SSNs has helped protect households with young children in terms of access to caregiving resources and nutritious food. Emergency interventions in the Kyrgyz Republic, Madagascar, Senegal, and Togo used safety nets to enhance food availability in households and reach vulnerable children. In Senegal, the nutrition project used the existing community-based nutrition program and its monitoring system to track the benefits of safety nets to young children. In Madagascar, the social protection program enhanced the availability of nutritious food, support to income generation and assets for the family, and advice to households with children from nutrition agents. The social protection program had the advantage of a large network of trained staff with a constant presence in communities and mechanisms to reach vulnerable households. In El Salvador, during the financial crisis, social protection was important in safeguarding the health, nutritional, cognitive, and social outcomes of young children living in the poor and violent urban slums. Lessons from the Review of Health and Social Innovations in the Coronavirus (COVID-19) Pandemic Response, March 2022. This crisis response note identified promising new approaches and practices reported by World Bank operational teams supporting COVID-19 responses in client countries. Main findings relevant to social protection include the following: • Partnerships with private sector actors enabled rapid delivery of SSN payments to beneficiaries. In Belize, cash transfers were delivered with the national bank and telecommunications company. Beneficiaries were notified of the funds by SMS message, and they collected their payment through a network of agents. In Bolivia, SÍNTESIS, a payment platform that interoperates with 65 55 financial institutions, allowed the government to add 400 payment points during the COVID-19 emergency, including through mobile banks for isolated areas. In Togo, the government partnered with the University of California and nonprofit GiveDirectly to build a digital mass payment platform, and model for direct cash transfers, allowing beneficiaries to enroll and receive payments within 120 seconds via basic mobile phones—without internet. Despite the absence of a national social registry, Togo managed to distribute US$34 million across the two phases to one-quarter of its adult population. Findings for COVID-19 from the World Bank’s Support to Address Ebola Outbreaks, July 2021. This note presented findings from a rapid review of the World Bank’s support for Ebola outbreaks between 2014 and 2019 with the objective of informing the COVID-19 pandemic response and future crisis responses. Main findings relevant for social protection include the following: • Strengthening national social protection delivery systems was linked to efforts to rapidly expand income, in-kind, or cash transfer support to protect vulnerable groups. Flexible grants and cash transfers helped support vulnerable households and geographical areas affected by Ebola, such as funds for safety nets, orphanage care, female-headed households, reopening of schools, and seed and fertilizer supply for farmers. • These efforts built on existing social protection support that had improved the food security, health, education, and economic conditions of beneficiary households and produced multiplier effects for the community. World Bank Engagement in Situations of Conflict: An Evaluation of FY10–20 Experience, November 2021. The evaluation aimed to draw lessons to inform early implementation of the World Bank’s fragility, conflict, and violence strategy. The evaluation analyzed how the World Bank works differently in conflict-affected situations by assessing four key aspects of engagement: (i) the extent to which the World Bank identified and addressed conflict drivers and risks at the strategy and country levels; (ii) how these drivers and risks are integrated into operations; (iii) the ways in which the World Bank has adapted its engagement by working with clients during situations of political instability, partnering with the United Nations and the International Committee of the Red Cross, leveraging corporate security, and adjusting its portfolio instruments; and (iv) how the World Bank has contributed to project-level results and higher-level outcomes of peace and stability. Main findings relevant for social protection included: • Forty percent of the social protection projects addressed gender-based violence risks. 56 • The evaluation found examples of countries whose uptake of a conflict lens for SSN projects was challenging. Although conflict-specific projects were naturally conflict sensitive, a conflict lens was not present in non–social development projects. • The evaluation found several good examples of SSN projects that incorporated a conflict lens adequately. World Bank Support to Reducing Child Undernutrition: An Independent Evaluation, October 2021. This evaluation assessed the World Bank’s contributions to improving nutrition determinants and outcomes for children through its interventions during FY08–19. Main findings relevant for social protection include the following: • The health; social protection; agriculture; and water supply, sanitation, and hygiene sectors can deliver effective interventions. In the social protection sector, CCT programs are the only intervention with consistent and positive evidence of improving household access to nutrient-rich food, schooling, and knowledge and attitudes. By contrast, provision of daycare services and facilitation of access to microfinance, credit, and banking have mixed results. • The World Bank aligns with evidence on what works mainly by focusing on countries’ cash transfer programs. Addressing Country-Level Fiscal and Financial Sector Vulnerabilities: An Evaluation of the Bank Group’s Contributions, July 2021. The evaluation assessed Bank Group support to client countries to build resilience to exogenous shocks through the systematic identification of fiscal and financial sector vulnerabilities and through efforts to support the reduction of these vulnerabilities. Given the importance of protecting the most vulnerable from shocks, this evaluation also looked at the extent to which the Bank Group has helped client countries adapt their SSNs so that they can be scaled up effectively in a crisis. Main findings relevant for social protection include the following: • The World Bank’s direct support for SSNs increased during the evaluation period, especially in low-income countries, but it tended to focus more on expanding access to chronically poor people than on building adaptable systems to respond to cyclical or more severe downturns. • The evaluation still found that the World Bank is increasingly working with clients to incorporate an adaptive social protection approach to reduce the vulnerability of poor and near-poor populations to shocks by building household resilience and enhancing safety net preparedness through flexible and scalable program designs and dynamic delivery systems. 57 • Important challenges remain, despite progress to strengthen social protection delivery systems and SSN programs. Coverage of SSNs is limited, especially in low-income countries, where automatic stabilizers tend to be limited because they do not generally extend to the informal sector. Financing and institutional issues constrain the intake of beneficiaries, registration, and targeting. The Natural Resource Degradation and Vulnerability Nexus: An Evaluation of the World Bank’s Support for Sustainable and Inclusive Natural Resource Management (2009–19), April 2021. This evaluation assessed how well the World Bank has addressed natural resource degradation to reduce the vulnerabilities of resource-dependent people. Main findings relevant to social protection include the following: • One-sixth of the projects evaluated were mapped to Social Protection and Jobs, with 80 percent of them located in Sub-Saharan Africa. This Social Protection and Jobs portfolio consisted of projects providing cash or food in some cases for work to resource users to restore degraded land and resources, often as part of an SSN program that provided financial transfers to poor households. • Public works often included integrated community-based watershed management activities such as soil and water conservation measures, rangeland management (in pastoral areas), and the development of community assets such as roads, water infrastructure, schools, and clinics. The objective of public works intervention was to improve livelihoods (through increased availability of natural resources such as water and cultivatable land, soil fertility, increased agricultural production, and improved market access), strengthen disaster risk management and climate resilience, and enhance nutrition. • None of the closed within this pool of Social Protection and Jobs projects adequately provided attributable evidence of resource restoration. The evaluation recommended that Social Protection and Jobs measure, assess, and report the attributable resource- and vulnerability-related outcomes of their different sustainable land and resource management approaches. For enhancing coordination, Social Protection and Jobs could share lessons on targeting vulnerable groups and measuring vulnerability-reducing effects. Managing Urban Spatial Growth: World Bank Support to Land Administration, Planning, and Development, July 2021. The focus of this evaluation was the Bank Group’s support to clients in building urban resilience to shocks and chronic stresses—to cope, recover, adapt, and transform. The evaluation’s main objective was to draw lessons from the Bank Group’s evolving experience to inform future efforts at urban resilience building. Main findings relevant for social protection include the following: 58 • Responses involving multiple Global Practices offer the potential for urban resilience multiplier effects. In some of the cities studied, the World Bank is linking its social protection programs to disaster response. In Can Tho, Vietnam, the World Bank helped adapt the city’s existing social protection system to become disaster responsive in a parallel operation (Systems Strengthening Project) for households affected by flooding. In Addis Ababa, the Productive Safety Net Program is tackling unemployment by linking it to urban resilience investments. The program provides opportunities for youth in occupations that promote green growth activities in cities (urban agriculture, park maintenance, and other nature-based solutions to protect the city). Other employment activities are linked to drainage and sanitation maintenance. 59