RESILIENCE RATING SYSTEM A Methodology for Building and Tracking Resilience to Climate Change Synthesizing Key Lessons from IDA19 Piloting MARCH 2024 II Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 1 Contents Acknowledgments 3 Summary 4 1. Introduction 7 2. What is the RRS? 9 3. Piloting the RRS 13 Overview 13 Breakdown of results 16 Applying RRS in fragile and conflict-affected settings 17 4. Lessons learned from piloting the RRS 19 Lesson 1. Capturing resilience impacts is a valuable complement to input metrics 20 Lesson 2. Timing, flexibility, and good communication are crucial for successful RRS applications 23 Lesson 3. Having a generalized methodology, with sector-specific spin-offs, ensures consistency and easy tracking 25 Lesson 4. Embedding climate expertise into project preparation is vital 29 Lesson 5. Project decision-making requires robust climate and disaster risk data and analytical tools that communicate uncertainty 31 Lesson 6. Robust climate risk stress testing requires a quality baseline 34 5. Summary of revisions to RRS Methodology 37 6. Applying RRS beyond World Bank operations 39 7. Moving forward 41 Appendix A. Full list of IDA19 pilot projects 43 Acronyms & abbreviations 47 Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 2 Contents Boxes Box ES1. Summary of lessons learned from piloting the RRS 6 Box 1. Advancing the adaptation and resilience agenda 10 Box 2. RRS knowledge products: a timeline 11 Box 3. RRS pilot success stories 23 Box 4. Sector-specific guidance for incorporating climate risk and resilience considerations in project appraisal and design 27 Box 5. Data challenges for SIDS 32 Box 6. The importance of high-quality baseline EFAs 35 Figures Figure 1. RRS pilot countries and sectors (2021–22) 14 Figure 2. RRS scorecard of IDA19 pilot projects 16 Figure 3. Breakdown of pilot project resilience ratings, by dimension 17 Figure 4. Historical data and future estimates for Dili 32 Figure 5. An overview of RRS criteria 38 Tables Table 1. RRS IDA19 pilot projects (FY 2021/22) 15 Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 3 Acknowledgments The preparation of this report was led by Veronique Morin Floissac and Jia Li, with lead authors Sundus Naeem Siddiqi and Elham Shirin Shabahat, and contri- butions from Farayi Onias Madziwa, Keiko Ashida, Ammara Shariq, and Esther Naikal. The author team wishes to recognize the work of colleagues in the broader Resilience Rating System team who contributed to the IDA19 piloting exercise and provided valu- able comments and contributions, including Ammara Shariq, Camilla Knudsen, Esther G Naikal, Nian Sadiq, Francisna C L Fernando, Tania Jaye Alicia Abraham, Keiko Ashida, April Frake, Rubaina Anjum, Sohee Gu, and David Groves. Neven Fučkar of the University of Oxford and University of St. Andrews and MacKenzie Dove provided valuable advice and input, producing and interpreting climate data and scenarios. The team would like to recognize the valuable support of the World Bank sector focal points who supported the IDA19 piloting process, including Celine Ramstein, Ioannis Vasileiou, Jamele Rigolini, Nalin Kishor, Nathan Engle, Oceane Keou, Paolo Avner, Sachin Shahria, Stephen Dorey, and Zoe Elena Trohanis. The author team also benefited from the invaluable insights of Stéphane Hallegatte, Senior Climate Change Advisor, who led the development of the Resilience Rating System and Climate and Disaster Risk Stress Testing methodology, and guidance from Rahul Kitchlu, Practice Manager of Climate Change Group Operation- alization Unit, and Hania Dawood, Practice Manager of Climate Change Group Climate Finance & Economics Unit. Lucy Southwood provided editorial support and Fiorella Gil provided design support. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 4 Summary In response to the growing recognition that measuring inputs, such as climate finance, is not enough to capture the impacts of investments, the World Bank Group developed the Resilience Rating System (RRS).1 Developed over a two-year, multisectoral consultative process through close collaboration with internal and external actors, the RRS methodology aims to guide investment decisions and improve climate resilience in project design and outcomes. The methodology report is publicly available. 2 The RRS evaluates and rates investment projects from C to A+, based on their resilience attributes in two complementary dimensions. The resilience of rating considers a project’s design, reflecting the confidence that it will achieve its expected objectives and maximize development benefits in the face of climate and disaster risks. The resilience through rating considers a project’s outcomes and reflects its contribution to improving climate resilience in the broader community, sector and systems, and to driving transformational adaptation. Combining the two dimension ratings provides an overall project rating, from CC to A+A+. Projects with an A rating in the resilience of dimension incorporate climate and disaster risk stress testing to the project against plausible disaster and climate scenarios.3 This helps provide evidence that the economic viability of the project is not threatened by current and future climate risks. The Risk Stress Test (RiST) tool, developed to support these estimates, is publicly available.4 The RRS does not impose uniform performance standards on all projects, be- cause appropriate levels of resilience are project- and context-specific; nor does it intend to replace a project’s economic and financial analysis (EFA) or engineering analysis. Instead, it complements existing project appraisal procedures by ensuring that the EFA or engineering analysis—which remain the key tools for determining a project’s physical viability and economical and financial desirability—properly capture current and future disaster and climate risks. Further, the RRS focuses on assessing how climate risks affect project viability and desirability. 1 World Bank. 2021. ‘What You Need to Know About the Climate Change Resilience Rating System.’ Fea- ture story, January 25. https://www.worldbank.org/en/news/feature/2021/01/25/what-you-need-to-know- about-the-climate-change-resilience-rating-system. 2  World Bank Group. 2021. Resilience Rating System: A Methodology for Building and Tracking Resilience to Climate Change. Washington, DC: World Bank. http://hdl.handle.net/10986/35039. 3  World Bank. 2021. ‘Is Your Project Robust to the Impacts of Climate Change and Disasters?’ Feature story, August 12. https://www.worldbank.org/en/news/feature/2021/08/12/is-your-project-robust-to-the- impacts-of-climate-change-and-disasters. 4  https://www.worldbank.org/en/topic/climatechange/brief/risk-stress-test-tool. Based on an Excel spreadsheet, the RiST is being transformed into an online tool to enhance the analytical features and better connect with climate data. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change SUMMARY 5 Over the last two years, a World Bank team piloted the RRS on 21 investment projects financed under the 19th Replenishment of International Development Association (IDA19) across 21 countries. These pilot projects, with a total investment of $2.92 billion, were in multiple sectors: energy, transport, urban, human development (health, education, social protection, and jobs), agriculture, water, and environment. Fig- ure ES1 presents the overall RRS ratings of the IDA19 pilot projects. Figure ES1. RRS scorecard for IDA19 pilots: overall ratings and breakdown by dimension 0 0 NR/A 1 BC 1 1 1 9 Resilience 7 regions 6 11 of RRS Rating Component: 14 A 21 countries B C Rating RRS BB AA Component: NR 21 projects 5 Overall 8 A 0 B RRS scorecard C 8 sectors 1 NR $2.92 billion 6 Resilience 14 AB investments 1 BA 5 through RRS Rating Component: A B C NR Note: the numbers in the figures represent the count of projects receiving a certain rating. Capturing resilience impacts is a valuable complement to input metrics to mea- sure the quality and expected outcomes of investments. Both RRS dimensions are important, to encourage investments to achieve the best outcomes and contribute to building wider resilience for beneficiaries. The RRS piloting makes it clear that a project’s broader climate resilience contributions are not always captured by climate co-benefits, and that resilience impacts are not proportionate to the amount of climate finance in- vested. For example, 10 percent of the investment in a World Bank water supply project in Dili, Timor is for institutional strengthening and preparing a disaster risk management program. But these components will contribute to system resilience impacts beyond the project’s immediate boundaries, as it mainstreams disaster risk management and climate change adaptation considerations into strategic, operational, and investment plans in the sector. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 6 SUMMARY As a guidance and reporting tool, the RRS allows project developers and inves- tors to shift public and private investments toward more resilient projects and programs. Using the guidance will help them design and label their projects in a way that attracts more financing. By translating complex and project-specific information into simple ratings, the RRS allows decision-makers, investors, and project developers to evaluate projects and thus prioritize and incentivize more resilient projects, helping redi- rect private investment flows toward more resilience. To support its corporate commitment to develop new climate results metrics and meet the increasing demand for more and better adaptation and resilience, the World Bank continues to apply RRS to IDA20 and other World Bank opera- tions and is working with external stakeholders to develop and refine resilience metrics, data, and tools. The goal is twofold: to strengthen operational capacity to systematically integrate adaptation and resilience in project development; and to work with external partners, private sector actors, standard-setting bodies, and credit rating agencies to advance the development of rating systems and standards to drive public and private investments towards climate resilience and scale up proven lessons across sectors and countries.  Box ES1. Summary of lessons learned from piloting the RRS 1. Capturing resilience impacts is a valuable complement to input 4. Climate expertise is necessary for embedding climate adaptation metrics. RRS pilots make it clear resilience outcomes are not into project design. The RRS team facilitated access to proportionate to the amount of climate finance invested. A climate change specialists, scientists and economists, which broader definition of what constitutes climate adaptation and helped pilot projects embed climate and disaster resilience resilience allows project teams to tell more comprehensive considerations into project preparation, enabling them to get stories about resilience building and helps development the highest possible rating. projects operations strive for more and better impacts from its investments and interventions. 5. Project decision-making requires robust climate and disaster risk data and analytical tools that can manage and communicate 2. Timing, flexibility, and good communication are crucial for uncertainty. Applying the RiST tool ensures a project’s cost- successful RRS applications. Applying the RRS methodology benefit analysis identifies plausible climate and disaster risks early in the project design and development stage increases and impacts, considers potential adaptation and resilience success in RRS pilots, drives project teams to obtain the measures to address these risks, and ensures the project is highest rating possible, and embeds adaptation and resilience viable and can deliver its intended development goals in the considerations into the project design and monitoring and face of climate change and uncertainty of its impacts. evaluation (M&E) plan, allowing teams to track results. 6. Robust climate risk stress testing needs a quality baseline 3. A generalized methodology is required, with sector-specific spin- economic analysis. It is important to: improve the EFA baseline offs. While the RRS and RiST tool provide the overall and quality, develop some standardization across sector and consistent framework for evaluating and tracking projects’ activity types where appropriate, and ensure that EFAs are resilience performance, more detailed sector-specific guidance prepared upfront and in an integrated manner with technical and activity-level information are required to increase the experts, to influence and optimize project design conditions. useability and consistency of RRS application. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 7 1. Introduction Climate change and weather-related nat- Resilience is the capacity to prepare for ural disasters increasingly pose serious these types of disruption, recover from threats to human well-being and the health shocks, and grow from a disruptive experi- of the planet. Among other manifesta- ence.7 Climate adaptation aims to enhance tions, drought, floods, heavy precipitation, adaptive capacity, strengthen resilience, increased temperatures, and extreme cli- and reduce vulnerability; and boosting re- mate events place a heavy burden on the silience and adaptation is both urgent and capacity of people, assets, institutions, integral to sustainable development and and services to cope with—and recover poverty reduction. Investing in resilience from—shocks and adapt to change. Cli- and adaptation helps safeguard past de- mate change and disasters disproportion- velopment gains, accelerate poverty re- ately affect poor and vulnerable countries duction, and ensure long-term sustainable and communities and could reverse de- development, increasing people’s and cades of development gains by pushing as communities’ resilience to natural disasters many as 132 million people into poverty by and a changing climate. As such, success- 20305 and causing 216 million people to ful adaptation and successful development migrate internally by 2050.6 go hand in hand.8 5  Jafino, B, Walsh, B, Rozenberg, J and Hallegatte, S. 2020. Revised Estimates of the Impact of Climate Change on Extreme Poverty by 2030. Washington, DC: World Bank. http://hdl.handle.net/10986/34555. 6  Clement, V, Rigaud, K, de Sherbinin, A, Jones, B, Adamo, S, Schewe, J, Sadiq, N and Shabahat, E. 2021. Groundswell Part 2: Acting on Internal Climate Migration. Washington, DC: World Bank. http://hdl.handle. net/10986/36248. 7 Adapted from the Intergovernmental Panel on Climate Change (IPCC) definition of resilience. IPCC, 2012: Summary for Policymakers. In: Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation [Field, C.B., V. Barros, T.F. Stocker, D. Qin, D.J. Dokken, K.L. Ebi, M.D. Mastrandrea, K.J. Mach, G.-K. Plattner, S.K. Allen, M. Tignor, and P.M. Midgley (eds.)]. A Special Report of Working Groups I and II of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cam- bridge, UK, and New York, NY, USA, pp. 1-19. 8  Adaptation was brought to the fore under the United Nations Framework Convention on Climate Change, with the adoption of the Paris Agreement in 2015. Aiming to strengthen the global climate change response by increasing the ability of all to adapt to adverse impacts of climate change, the agreement defines a glob- al goal on adaptation and requires all Parties to engage in adaptation planning and implementation, and provide information related to climate change impacts and adaptation. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 8 Introduction Public and private actors alike are increas- help decision-makers and project devel- ingly developing climate risk screening and opers integrate climate resilience consid- resilience metrics and measurement frame- erations into their investments.9 works to identify climate risks and evaluate This paper highlights experiences from pi- resilience performance and the impacts loting the RRS in 21 World Bank investment of investments and policy interventions projects during fiscal years 2021 and 2022 at international, national, and subnational (FY21 and FY22). It captures and shares levels. With varied objectives, approaches, lessons learned and consequent revisions scopes, and outputs, these burgeoning to the original RRS methodology through efforts range from reporting on climate ad- a transparent and iterative process that aptation finance to disclosing climate risks, aims to facilitate continued development evaluating the resilience performance of and improvement of resilience rating meth- portfolios and systems, and measuring ods. This, in turn, will help planners and progress toward adaptation objectives. practitioners—whether from governments, This includes developing metrics, disclo- the private sector, credit rating agencies, sure standards, and labels that assess the development organizations, or multilateral resilience attributes of investment activities development banks (MDBs)—mainstream (for example, for infrastructure develop- climate resilience in project development ment) to ensure investment decisions con- and investment decisions. sider climate risks and to mobilize capital toward resilient investment. Achieving re- The paper is organized as follows: Section silience at system, country, and global lev- 2 describes the RRS methodology; Sec- els requires all investment decisions to be tion 3 presents the results of applying RRS resilient to current and future climate risks, to 21 pilot projects; Section 4 synthesiz- and all investments to contribute to broad- es the key lessons learned, illustrated with er resilience building and transformative experiences from selected pilots; Section change. 5 summarizes the revisions to the original RRS methodology that resulted from the In the context of the global climate change piloting experience; Section 6 explores adaptation landscape, the World Bank the usefulness of the RRS beyond World Group has developed the Resilience Rating Bank operations; and Section 7 discusses System (RRS). This methodology and set next steps for applying RRS to the broader of resilience metrics, developed through community. an extensive consultation process, aims to 9  World Bank Group. 2021. Resilience Rating System: A Methodology for Building and Tracking Resilience to Climate Change. Washington, DC: World Bank. http://hdl.handle.net/10986/35039. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 9 2. WHAT IS THE RRS? In response to the growing recognition that The RRS evaluates and measures a proj- measuring inputs, such as climate finance, ect’s resilience attributes along two com- is not enough to capture the impacts of plementary dimensions: investments, the World Bank Group de- • Resilience of a project’s design, which veloped the RRS to guide investment de- rates the confidence that expected de- cisions and improve climate resilience in velopment objectives and investment project design and outcomes (Box 1). Pub- outcomes will be achieved, based on lished in February 2021, the RRS method- whether a project has considered cli- ology10 was developed over a two-year, mate and disaster risks in its design; multisectoral consultative process that in- and volved close collaboration with internal and • Resilience through a project’s out- external actors across multiple sectors and comes, which rates its contribution institutions. This extensive engagement to increasing climate resilience in the and consultation within and outside of broader community, sector, and sys- the World Bank Group aimed to make the tem, and to driving transformational RRS methodology relevant and applicable adaptation. to a wide range of operations and project activities, from human and sustainable de- Combining the two dimension ratings pro- velopment to infrastructure, and equitable vides an overall project rating, from CC to growth, finance and institutions projects. A+A+. To achieve an A rating in both di- The team then piloted the system across mensions, projects must demonstrate that 21 IDA projects in select operations during they are designed to be resilient and eco- FY21 and FY22, revising and updating the nomically viable in the face of current and methodology (Figure 5) as a result of the future climate and disaster risks (resilience findings and feedback received. Although of) and that they improve resilience be- developed and piloted within the World yond their own boundaries, with impacts Bank, the tool can be used by all private beyond direct outputs through improved and public sector project developers. institutions, policies, incentives, technolo-   RRS approach did not lead to project selection; rather, it was applied to existing World Bank projects that 10 were addressing identified risks and vulnerabilities in the sectors and locations they work in. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 10 What is the RSS? Box 1. Advancing the adaptation and resilience agenda Developed as part of its 19th Replenishment of International quantifying the share of World Bank lending that contributes to Development Association (IDA19) policy commitments, the RRS climate change response), the World Bank has made progress methodology brings together the World Bank Group’s corporate toward using additional metrics to further incentivize effective commitments on adaptation—climate and disaster risk screening, climate adaptation actions and better capture climate impacts. climate (adaptation) co-benefits, and climate (adaptation) Developing resilience metrics to increase incentives for more indicators—under one umbrella. Building on the World Bank’s effective climate adaptation actions and piloting these in 21 IDA19 climate and disaster risk screening commitment, which mandates operations was a policy commitment under the IDA19 package.b that all projects be screened for short- and long-term climate The IDA20 Results Measurement Systemc increases this ambition, and disaster risks, it offers guidance on conducting deeper risk encouraging at least 10 IDA operations to achieve an AA rating, assessments and provides qualitative and quantitative estimates and the World Bank Group Climate Change Action Plan 2021–25d for levels of climate risk. It goes beyond Paris alignment by identifies the RRS as an important way of measuring the resilience encouraging users to integrate appropriate adaptation measures of operations to physical climate shocks. to support the resilience of project design and strengthen resilience The RRS is an ex ante metric with incentives for including climate through project outcomes for investments that follow the three indicators for continuous monitoring and tracking, and the World steps outlined in the MDBs’ Joint Methodology for Tracking Climate Bank is increasingly focused on measuring results and outcomes. Change Adaptation Finance.a Finally, it encourages the use of The bank has updated its institutional vision and focus to “create climate indicators to measure outputs or outcomes of adaptation a world free of poverty on a livable planet” and is developing an interventions and monitor and track the progress of climate results. integrated climate results framework as part of a broader effort The RRS builds on existing commitments and advances a to support this vision and strengthen an outcome orientation corporate mandate to use new metrics to better capture climate to measure, report and monitor its climate action.e The RRS adaptation and resilience. As well as delivering close to 49 methodology and its applications a inform the development of percent in adaptation climate finance in FY22 (an input metric resilience metrics in this framework. https://thedocs.worldbank.org/en/doc/20cd787e947dbf44598741469538a4ab-0020012022/original/20220242-mdbs-joint-methodology-cli- a mate-change-adaptation-finance-en.pdf. b IDA. 2020. Additions to IDA Resources: Nineteenth Replenishment IDA19: Ten Years to 2030: Growth, People, Resilience. Report from the Executive Directors of the International Development Association to the Board of Governors. https://documents1.worldbank.org/curated/en/459531582153485508/ pdf/Additions-to-IDA-Resources-Nineteenth-Replenishment-Ten-Years-to-2030-Growth-People-Resilience.pdf. c World Bank Group. 2021. The IDA20 Results Measurement System. Washington, DC: World Bank Group. http://documents.worldbank.org/curated/ en/498181625066308834/The-IDA20-Results-Measurement-System. d World Bank Group. 2021. World Bank Group Climate Change Action Plan 2021–2025: Supporting Green, Resilient, and Inclusive Development. Wash- inton, DC: World Bank. http://hdl.handle.net/10986/35799. e World Bank. 2023. “Remarks by World Bank Group President Ajay Banga at the 2023 Annual Meetings Plenary.” Speeches and transcripts, October 13. https://www.worldbank.org/en/news/speech/2023/10/13/remarks-by-world-bank-group-president-ajay-banga-at-the-2023-annual-meetings-plenary. gies, or capacities (resilience through). Fig- and the Excel-based Risk Stress Testing ure 5 in section 5 provides an overview of (RiST) tool11 for climate and disaster risk the rating criteria. stress testing in project economic and fi- nancial analysis (EFA). A relatively simple To help projects achieve an A rating in the but novel approach to integrating climate resilience of dimension, the RRS team de- considerations into project EFAs, they help veloped an accompanying methodology ensure projects are economically viable   https://www.worldbank.org/en/topic/climatechange/brief/risk-stress-test-tool. 11 Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change What is the RSS? 11 in the face of current and future impacts of climate change and climate extremes Box 2. RRS knowledge products: a timeline and can deliver intended outcomes while accounting for climate uncertainty. By making visible the costs and benefits of January 2021: A feature story and interview with World Bank Climate Change Lead Economist Stéphane Hallegatte that highlights the history behind the RRS projected climate change impacts and re- and the problem it is trying to resolve. silience options, the RiST tool helps sup- port robust decision-making and can aid • World Bank. January 25, 2021. “What You Need to Know About the Climate negotiations around allocating resourc- Change Resilience Rating System”. Feature story. https://www.worldbank. es for resilience-building measures. The org/en/news/feature/2021/01/25/what-you-need-to-know-about-the-cli- RiST tool complements other World Bank mate-change-resilience-rating-system. tools, such as the hydrometeorological risk February 2021: The original RRS methodology. stress test tool for water projects, and sup- ports stress testing. • World Bank Group. 2021. Resilience Rating System: A Methodology for Building and Tracking Resilience to Climate Change. Washington, DC: World In summary, the RRS is both a methodol- Bank. http://hdl.handle.net/10986/35039. ogy to support better project design and June 2021: A guidance note that advises on how to add a stress test for a rating or label to monitor and report the climate change and natural disasters to the economic analysis of a project, with quality by which a project design considers accompanying excel-based RiST tool and tutorial videos. This report and package adaptation and resilience. Using the RRS: provide support to help teams get an A rating for resilience of their project. • Provides a standardized methodology • Hallegatte, S, Anjum, R, Avner, P, Shariq, A, Winglee, M and Knudsen, C. for evaluating a project’s resilience at- 2021. Integrating Climate Change and Natural Disasters in the Economic tributes; Analysis of Projects: A Disaster and Climate Risk Stress Test Methodology. • Informs decision-makers, investors, Washington, DC: World Bank. http://hdl.handle.net/10986/35751. and other stakeholders on the resilience • World Bank. June 20, 2021. “Risk Stress Test Tool” (introduction and tuto- of projects and investments, translating rial videos). https://www.worldbank.org/en/topic/climatechange/brief/risk- complex project design details—such stress-test-tool. as climate models, engineering design, August 2021: A feature story that highlights the risk stress testing methodology and EFA—into a simple rating; and tool. • Creates incentives for more wide- spread and effective climate adaptation • World Bank. August 12, 2021. “Is Your Project Robust to the Impacts of Cli- through enhanced transparency and mate Change and Disasters?” Feature story. https://www.worldbank.org/en/ simpler disclosure; news/feature/2021/08/12/is-your-project-robust-to-the-impacts-of-climate- change-and-disasters • Informs project developers on ways to manage risk and improve project quali- ty, while allowing for flexibility in sectors and countries; Because appropriate levels of resilience • Ensures that the EFA—which remains are context-specific, the RRS does not im- the key tool for determining a project’s pose uniform performance standards on economical and financial viability and all projects or compare project resilience desirability—properly captures disaster with alternative baseline projects. Instead, and climate risks; and it focuses on how climate risks affect proj- • Identifies best practices to allow proven ect viability and desirability. A comparison lessons on resilience to be scaled up with a less resilient baseline project would across sectors and countries. be easy to manipulate and depend on Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 12 What is the RSS? arbitrary assumptions about the other proj- The RRS, on the other hand, would focus ect. For example, assessing how much a on whether the resilience of the well-de- well-designed bridge improves resilience signed bridge makes it a viable and desir- requires ad hoc assumptions about the able project.12 performance of a poorly designed bridge.   This is distinguished from the climate and disaster risk stress test that the project economic analysis is 12 performed by comparing ‘with’ and ‘without’ project scenarios. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 13 3. PILOTING THE RRS The team piloted the RRS on 21 IDA19 its implementation; and third, to system- investment projects as part of the IDA19 atically integrate adaptation and resilience policy commitment to improve monitoring considerations into the design of the pilot and reporting on adaptation and resilience. projects. This was to ensure that projects The purpose of the piloting was threefold: maximize development benefits in the face first, to test and refine the methodology; of climate risks and help improve climate second, to support the development of outcomes, where possible. the additional data and tools necessary for Overview Throughout 2021 and 2022, the RRS team economist to support the RRS application applied the methodology to 21 IDA19 proj- throughout the project preparation phase. ects in 21 countries, across eight sectors While project development objectives were and seven regions (figure 1 and table 1) already established at the time of engage- in a broad range of sectors, with activities ment, the climate experts were able to help aimed at creating the physical and social teams think through: infrastructure necessary to reduce pover- • Climate and disaster risks that could ty and create sustainable development. impact project operations (including Where possible, the RRS team targeted pi- sourcing and translating complex cli- lot projects that were in the early phases of mate projections); development, providing the opportunity to • Adaptation measures that could support and influence project design early strengthen project viability; on. It is important to note that there may • Areas to further strengthen and incor- have been some sample bias in the project porate climate resilience-building activ- selection, as many already had strong resil- ities; and ience-building objectives. • Opportunities for integrating climate adap- Most of the project teams had an embed- tation indicators into project results frame- ded climate change specialist and climate works to better track climate impacts. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 14 Piloting the RRS Figure 1. RRS pilot countries and sectors (2021–22) 7 21 21 8 $2.92b regions countries projects sectors investments As well as directly supporting the project The RRS team’s engagement with and teams, the RRS team maintained frequent support to the pilot project teams in FY21 communication and coordination with sec- and FY22 had a budget of approximate- toral focal points, who acted as climate ly $500,000. Compared to the $2.92 bil- champions. As well as helping fast-track lion in total project investments, the cost the RRS implementation, which resulted of providing high-quality technical climate in stronger buy-in and better engagement expertise with new frontier science was from projects, the focal points generated marginally small; but it had significant and and exchanged sector-specific technical impactful returns.13 knowledge and linked RRS methodology efforts with other initiatives on climate re- silience.   Although costs varied depending on the components and complexity of a project, they were more sig- 13 nificant when a stress test was carried out. Costs will go down due to data and tool development, scaling up and ongoing learning. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Piloting the RRS 15 Table 1. RRS IDA19 pilot projects (FY 2021/22) Country/ies Project name RRS rating AGRICULTURE AND FOOD Gambia, The Gambia Inclusive and Resilient Agricultural Value Chain Development Project (GIRAV) BA Honduras Innovation for Rural Competitiveness Project - COMRURAL III BA Pakistan Punjab Resilient and Inclusive Agriculture Transformation AA ENERGY AND EXTRACTIVES Liberia Liberia Electricity Sector Strengthening and Access Project (LESSAP) BC Somalia Somali Electricity Sector Recovery Project AB Côte d’Ivoire, Mali, Niger, Regional Electricity Access and Battery Energy Storage Technology (BEST) Project AA Mauritania, Senegal ENVIRONMENT, NATURAL RESOURCES, AND THE BLUE ECONOMY Lao PDR Lao Landscapes and Livelihoods Project BA Tajikistan RESILAND CA+ Program: Tajikistan Resilient Landscape Restoration Project AA Uzbekistan RESILAND CA+ Program: Uzbekistan Resilient Landscapes Restoration Project AA HEALTH, NUTRITION, AND POPULATION Niger Niger, Improving Women’s and Girls’ Access to Improved Health and Nutrition BB Services in the Priority Areas Project (LAFIA-IYALI) SOCIAL PROTECTION AND JOBS Afghanistan Early Warning, Early Finance and Early Action Project NR/A Sierra Leone Productive Social Safety Nets and Youth Employment BB TRANSPORT Nepal Accelerating Transport and Trade Connectivity in Eastern South Asia – Nepal Phase AA 1 Project Yemen, Rep. Emergency Lifeline Connectivity Project BB URBAN, RESILIENCE, DISASTER MANAGEMENT, AND LAND Grenada Grenada Resilience Improvement Project AA Niger Niger Integrated Urban Development and Multi-sectoral Resilience Project AA Pakistan Sindh Resilience Project Additional Financing BA Tonga Tonga Safe and Resilient Schools Project BB WATER Ghana Ghana AF for Greater Accra Metropolitan Area Sanitation and Water Project BB Niger Niger Integrated Water Security Platform Project (Niger-IWSP Project) AA Timor-Leste Dili Water Supply Project BA Notes: See Appendix A for more details on each project. NR = not rated. In situations where projects could be exposed to climate change and disaster risks, but insufficient information, data, or tools are available to assess these risks, they receive an NR rating for the resilience of dimension. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 16 Piloting the RRS Breakdown of results Thirty-eight (38) percent of the pilots—one ed A for resilience of, and 67 percent (14 energy, one agriculture, two resilient land- projects) were rated A rating for resilience scape, two urban resilience, one transport, through. While there may have been some and one water project—received an AA selection bias in terms of nominating pilots rating (figure 2). Scoring A in both resil- that already included resilience objectives, ience of and resilience through the project it is encouraging that more than half of the means that they have a resilient design pilot projects demonstrate strong climate and show resilience in expected perfor- impacts, strengthening resilience that tran- mance, given identified climate risks, and scends boundaries beyond their direct will contribute to building wider resilience outputs and timescales. In fact, the pilot- for beneficiaries. Five projects received a ing process demonstrated that incorporat- BA rating; one project was rated AB; five ing A-rated resilience through activities of- projects received a BB rating; one project ten required little climate input finance and was rated BC; and one project was rated could easily be integrated as a small (often NR/A, with NR for resilience of and A in soft) element of overall project design. resilience through. Achieving an A rating in the resilience of Breaking down the ratings into the two di- dimension proved to be a much more mensions (figure 3) shows that 43 percent complex and larger undertaking—in terms of projects (9 out of 21 projects) were rat- of time, resources, and data—for the teams. Specifically, incorporating climate risk stress testing was more challenging Figure 2. RRS results of IDA19 pilot projects for several reasons (discussed in section 4). These included: the timing of project NR/A EFA preparation, as these are often con- BC 1 ducted in the later project appraisal stage, 7 regions 1 leaving little time for stress testing, which is both time and resource-intensive; limit- 21 countries ed climate projection and climate impact data; varying levels of both quality and de- 21 projects BB AA tail in the baseline EFA; and varying ability 5 8 to integrate such information with the RiST 8 sectors Overall tool. RRS scorecard $2.92 billion All but one pilot project had an overall rat- investments ing of BB or higher. The only project that received a C rating for resilience through AB BA 1 was too advanced in the preparation 5 phase to be supported with additional cli- mate expertise. Note: the numbers in the figures represent the count of projects receiving The ability to aggregate RRS rating results a certain rating. demonstrates the value of the simple rat- ing system to both evaluate and rate proj- Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Piloting the RRS 17 Figure 3. Breakdown of pilot project resilience ratings, by dimension 0 0 1 1 9 6 BA Resilience Resilience 14 5 of through 11 RRS Rating Component: A B C NR Note: the numbers in the figures represent the count of projects receiving a certain rating. ects’ resilience performance and track, and with varying component, climate risks, aggregate and report projects’ resilience and geography, at portfolio, sector, and performance of projects across sectors country levels. Applying RRS in fragile and conflict-affected settings Conflict and climate change both present signed to withstand the impacts of climate immense challenges for poverty reduc- change while also explicitly contributing to tion, and these are exacerbated when they peace and building the resilience of people overlap. Climate change can create ma- and communities in fragile settings. The jor strains on society, especially in fragile RRS is designed to do exactly this. Unlike settings where governments have limited many of the tools the financial industry is resources to manage crises and help the developing on exposure to climate haz- population adapt. The poorest and most ards—which can miss opportunities to vulnerable communities feel the impacts of incentivize resilience investments in vulner- climate change most intensely, especially able countries or communities—the RRS those living in fragile and conflict-affected incentivizes both good design and resil- settings (FCS). As a result, it is even more ience building. important that resource-scarce countries prioritize investments that are truly de- Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 18 Piloting the RRS The World Bank has successfully applied tion—in FCS projects. It also highlights that the RRS methodology in FCS. Eight of the the RRS can be applied to a diverse set of pilot projects were in FCS countries: Af- countries with vast and varying underlying ghanistan, Côte d’Ivoire, Mali, Mauritania, causes of climate vulnerability. Notably, the Niger, the Republic of Yemen, Senegal, RRS pilot in the Republic of Yemen was Somalia, and Timor-Leste. This shows that an emergency transport and connectivity there is demand for the RRS framework project, highlighting that the RRS approach to strengthen adaptation and resilience can be impactful even in projects that are considerations—and improve their integra- prepared with rapid and urgent timelines Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 19 4. LESSONS LEARNED FROM PILOTING THE RRS The RRS pilots in FY21 and FY22 pro- these lessons are twofold: to facilitate vided initial insights into its feasibility and further improvements and ongoing de- impacts across a range of sectors and re- velopment of World Bank resilience rat- gions. Supporting 21 projects with a min- ing metrics and their applications; and to imal budget, the small RRS team yielded encourage others to develop resilience positive impacts on project outcomes and metrics that enable more and better adap- built knowledge and capacity for main- tation and build resilience. The RRS team streaming climate resilience considerations distilled the key lessons outlined here from in project development. The simplicity of desktop research, as well as interviews the RRS metric—and its ability to compare with and feedback from their own team and aggregate project resilience attributes members, project team members, and across investments and sectors—makes it sector focal points. The desktop research a tangible, easy-to-understand approach consisted of document analysis of project for evaluating the resilience performance appraisal documents, climate risk screen- and quality of individual investments and ing reports, RRS methodology, technical investment portfolios. guidance documents, World Bank internal reports, and other climate-relevant publi- The experiences and key lessons outlined cations such as Intergovernmental Panel here have relevance for all development on Climate Change reports. and investments. The goals of sharing Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 20 Lessons learned from piloting the RRS LESSON 1 As a novel approach that considers re- or climate co-benefit approaches may be silience impacts and outcomes, the RRS biased toward more costly projects, es- Capturing resilience methodology is a valuable complement to pecially physical infrastructure-heavy proj- impacts is a valuable adaptation climate co-benefits. The MDBs’ ects, where the finance numbers are much complement to input Joint Methodology for Tracking Climate larger. But they do not always adequately metrics Change Adaptation Finance14 captures the recognize the high-quality impacts of low-, finance directed at adaptation activities that zero-, or negative-cost project activities, are carried out in response to experienced which build resilience. and anticipated climate change impacts, The RRS methodology expands existing measuring the volume of additional finance metrics and provides a broader evaluation for adaptation and resilience activities in of resilience outcomes that adaptation fi- MDB projects. But, although an important nance or climate co-benefits metrics do metric for measuring progress and support not always capture. The RRS measures for adaptation and resilience, volume of the resilience of a project’s design, its ex- climate adaptation finance provides an in- pected performance given identified cli- complete picture of effectiveness for—and mate risks, and its contribution to build- impacts on—resilience building. ing wider resilience for beneficiaries. To The RRS pilots make it increasingly clear achieve an A rating in the resilience of di- that resilience impacts and outcomes mension, a project needs to demonstrate are not proportionally dependent on the its economic viability and robustness for amount of climate finance invested or the achieving expected project outcomes in climate co-benefits measured. For exam- the face of current and potential future cli- ple, World Bank investment operations mate change and climate extremes. With are often made up of more than one ac- the resilience through dimension, the met- tivity, coupling physical infrastructure de- ric framework goes further, evaluating a velopment with soft infrastructure, such project’s contribution to broader resilience as institutional capacity building or ser- building. To achieve an A rating for the re- vice delivery. The pilots show that some silience through dimension, a project must activities—especially those that address demonstrate that it influences resilience or and build systemwide resilience through adaptation beyond its direct outputs and institutional systems—can have minimal timescale, to reduce or remove obstacles costs compared to the overall financing and underlying causes of vulnerability and of the operation. In fact, such impactful build resilience. activities can be such a small proportion Resilience performance can vary across of overall financing that they are not cap- sectors and domains, with one dimen- tured as an integral element of a project sion more prominent in certain sectors, development objective. But they can depending on the project’s scope. For make significant contributions toward re- example, human development and social silience-building and result in an A rating inclusion projects—such as those in the for resilience through the project. Although health, education, jobs, and social protec- they capture the financial inputs that sup- tion sectors—are likely contribute more to port adaptation, the adaptation finance 14   https://thedocs.worldbank.org/en/doc/20cd787e947dbf44598741469538a4ab-0020012022/origi- nal/20220242-mdbs-joint-methodology-climate-change-adaptation-finance-en.pdf. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 21 building resilience through the project. a method to measure a project’s quality LESSON 1 This dimension of the RRS captures how and expected outcomes from an adapta- a project builds resilience for communities, tion or resilience perspective. The pilots households, and populations, increasing show that the RRS captures a project’s their ability to build their adaptive capacity broader climate resilience contributions, to be more resilient to future shocks. So, which are not always captured by climate as work focused on adaptive social pro- co-benefits. Having a broader definition of tection systems will score well on the re- what constitutes climate adaptation and silience through dimension, for human de- resilience allows teams to tell more com- velopment sectors, the resilience through prehensive stories about resilience build- rating offers an opportunity to measure ing and helps organizations strive for more resilience outcomes more comprehensive- and better impacts from their investments ly than through climate finance-based cli- and interventions. The RRS also incentiv- mate co-benefits alone. izes the use of climate indicators, which The RRS complements the tracking of ad- help projects monitor and track their de- aptation co-benefits, which measures the tailed climate results by measuring outputs quantity of adaptation finance, by offering or outcomes of adaptation interventions. Pilot Project Box 1. Dili Water Supply Project Sector: Water Country: Timor-Leste (East Asia and Pacific Region) Project code: P176687 RRS rating: BA The Dili Water Supply Project shows how a project with relatively heavy precipitation and drought on service delivery. With 10 lower climate co-benefits can ensure that its design strives to be percent of the investment going to institutional strengthening, resilient to future extremes, while also building transboundary the project has a system resilience impact beyond its immediate resilience. With a total investment of $125.5 million, it aims boundaries, as it strengthens institutional capacity and provides to increase access to safe drinking water and improve the incentives to improve the sustainability and resilience of the water operational performance of the water utility in Dili, a small supply infrastructure financed under the project. By preparing and geographic area in Timor Leste with complex climatic conditions, implementing a disaster management and resilience program, it such as precipitation trends that differ drastically from even the strengthens the government’s capacity to manage disaster and neighboring island. The project, estimated to have 22 percent climate-related risks and mainstream disaster risk management climate co-benefits, received a BA rating. and climate change adaptation considerations into strategic, operational, and investment plans. The project also undertook a The RRS team engaged closely with and supported the project water resources assessment to identify and evaluate long-term team, processing high-resolution historical climate data for supply alternatives to traditional water sources in the Comoro Dili, estimating seasonal precipitation patterns and changes Basin—including groundwater, artificial recharge and seasonal in return periods under future climate change, and evaluating storage, and alternative non-groundwater sources—to meet climate impacts on the project cost-benefit analysis. This climate demands in a changing climate. Although they constitute a small information served as a direct input into the project engineering proportion of the overall project finance, these activities are analysis and design considerations, and showed that the project crucial to longer-term sustainability and resilience building. is economically viable, even when considering the impacts of Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 22 Lessons learned from piloting the RRS LESSON 1 Applying the RRS methodology helps proj- must be accompanied by objective, stan- ect teams think more comprehensively dard guidance that applies across sectors about integrating climate change during (see lesson 3). This will also help ensure the project development phase. Even the application of ratings is not subjective when projects did not get an A rating, us- across sectors and assessors. ing the methodology provided gave teams a clear and objective method for assessing While the RRS has some limitations as a climate risk, developing resilience mea- metric—for example, it does not provide sures at a granular level, integrating climate specific project results or outcomes—its monitoring and evaluation (M&E), and em- simplicity and applicability across sectors phasizing climate change actions into their makes it a good decision-making and investments. The RRS process sets a min- portfolio monitoring tool, both in terms of imum standard for disclosing climate risk a project’s viability in the face of climate while also improving the quality of projects, and disaster impacts and its broader resil- including during implementation. ience impacts. The RRS aims to transform very complex project design information, At the same time, it is important to balance the complexity of a metric with the right which often requires deep engineering incentives. And while both dimensions knowledge, into a simple rating that deci- of the RRS—resilience of and resilience sion-makers, policy makers, and investors through—are useful for capturing distinct can use to select projects, even if they do types of activity, there can be some confu- not have deep climate and disaster techni- sion around how to classify different activ- cal expertise. In this sense, the RRS pro- ities in these dimensions. For the uptake vides a foundational step in the move to- of this methodology to be successful, it ward capturing resilience impacts. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 23 Evaluating climate risks in the earliest stag- arriving at an ideal adaptation solution is LESSON 2 es of project development and incorporat- important for building resilience in a proj- ing appropriate adaptation and resilience ect’s design. On the other hand, involving Timing, flexibility, and options in the project design phase leads the RRS method at a late stage of the proj- good communication are to the most cost-effective resilience mea- ect—after design decisions are made or crucial for successful sures that typically only marginally increase the project EFA is complete—fails to max- RRS applications project investment costs. Considering imize its potential for flagging climate risks climate risks and adaptation in the early and resilience considerations. stages of project development and ap- Embedding the RRS method early in the praisal will give teams the time they need project preparation cycle is not without to consider alternatives and select the best its challenges. Climate risk stress testing and most cost-effective option. is a time-intensive exercise that requires Applying the RRS methodology early in-depth assessment of climate data, re- during design and development, when search on climate impacts, and back-and- teams are screening for climate risk, in- forth communication with project econo- creases the chances of successful RRS mists, but project teams often work under pilots, drives teams to obtain the highest a tight timeline, with limited resources. In possible rating, and embeds adaptation some of the pilots, the project teams had and resilience considerations into project held decision meetings before the RRS M&E plans, allowing result tracking. One team started working with them, and it of the most significant impacts of applying was too late to incorporate any meaning- the RRS methodology is that it encourages ful updates. It was particularly difficult to teams to consider alternative adaptation embed climate risks into project EFAs in options when they identify climate risks. a timely manner, as these had often been The assessment and thought process of prepared quickly in the late stages of the Box 3. RRS pilot success stories Responding to real-time demand for support during the project • Emergency Lifeline Connectivity Project in the Republic preparation stage, the RRS team was able to integrate climate of Yemen, an emergency transport connectivity project that and disaster risk information and embed climate resilience into explicitly integrated building resilience to climate and disas- project appraisals and designs, adding significant value to the ter risks in its development objective and built-in related ad- following projects: aptation components; • Niger Integrated Urban Development and Multi-sectoral • Productive Social Safety Nets and Youth Employment in Resilience Project (pilot project box 5), an urban resilience Sierra Leone (pilot project box 4), which integrated climate project that expanded the analysis to consider potential resilience measures in public works based on poor house- impacts from extreme heat, which had previously been ne- holds’ exposure to climate risks and interlinkages between glected; and climate and food security; • Regional Electricity Access and BEST Project in Côte • Grenada Resilience Improvement Project, a critical infra- d’Ivoire, Mali, Mauritania, Niger, and Senegal, an energy structure project that included capacity building and institu- project in which the analysis considered options for con- tional strengthening to boost the country’s climate resilience struction materials and site selection to reduce the risk of capacity; system failure from extreme heat and wildfire. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 24 Lessons learned from piloting the RRS LESSON 2 project appraisal stage, just before the de- pandemic was not easy, but the team con- cision review (see lesson 6). tinued its mission by employing the values of diplomacy, respect, and flexible time- The more successful RRS and RiST ap- lines in the face of uncertainty, mitigating plications involved pilots that had strong risks through frequent communication with project teams with supportive leadership, project teams and aligning the RRS appli- where there was good and effective com- cation process with project deadlines and munication between the RRS team, the milestones. For example, in a project that project team leader, and all team members. has already been approved by the board, RRS application was more impactful in the the team decided to pilot the RiST tool teams where engineers, economists, cli- retroactively, to ensure learning could con- mate specialists and other technical experts tinue. In some cases, the RiST findings in- cross-coordinated in their findings, which in formed the project during implementation turn, led to optimized project design. or before detailed design had taken place, Flexibility is also crucial. The RRS team and in others, where pilot operations had employed an open and flexible approach been postponed or delayed, or operations from the start, tailoring support to meet had to change their approach in response unique project objectives while respecting to changing country environments and sit- sectoral priorities. Piloting a new method- uational complexities, the team was able ology through the constraints of a global to identify new pilot projects. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 25 Designed to apply to all sectors, the gener- climate risk stress testing to sector needs, LESSON 3 alized RRS methodology and metrics allow building in flexibility to reflect RRS applica- for consistency and tracking across multi- tions in different sectors and projects with Having a generalized ple investments, comparing and aggregat- varying levels of complexity and risk. methodology, with ing resilience performance across projects, sector-specific spin-offs, It is therefore important to develop the activities, and portfolios to evaluate their ensures consistency and methodology within a consistent RRS overall quality and performance. It is also easy tracking framework that addresses sector-specific benchmarked against the development of features. Although both RRS dimensions the World Bank Group’s Paris Alignment are important for telling resilience stories, methodology to ensure RRS encourages the pilots show that some projects operate more and better adaptation and resilience largely in one dimension. For example, in efforts and outcomes. the case of human development projects Due to the multidimensional nature of re- with no physical assets, teams expressed silience, refinements of the RRS and RiST concerns that the RRS rating for resilience methodologies and applications need to of a project can be arbitrary and unfair take stock of lessons learned from the pi- and could bias ratings toward infrastruc- lot phase to incorporate different thought ture-heavy projects. At the same time, processes and approaches for climate not all projects will aim to build resilience risks, stress testing, and resilience mea- through their activities. Such is the case, surements in different sectors and con- for example, with projects that address texts. This reinforces the need to develop emergency needs in pandemic or conflict sector-specific guidance for RRS and tailor situations. Pilot Project Box 2. Early Warning, Early Finance and Early Action Project Sector: Social protection and jobs Country: Afghanistan (South Asia Region) Project code: P173387 RRS rating: NR/A This project disburses cash transfers and cash-for-work payments, • Monitoring and evaluating project implementation and based on incidence of drought and identified through early strengthening institutions. warning systems. Its development objective is “to increase the The project received 100 percent climate change adaptation co- food and nutrition security of the most vulnerable households benefits, as its primary objective is to support climate adaptation living in drought-prone rural areas and to build systems for early to increasing drought conditions. It got an A rating for resilience warning and response with pre-arranged financing.” Investments through the project, as it influences resilience beyond its direct financed through the project include: outputs and timescale and reduces underlying causes of • Strengthening drought early warning decision support, im- vulnerability. But it has no physical assets, and at the time of the proving hydrometeorological services, and increasing com- piloting, the data and tools available within the RRS and RiST munity resilience; environment did not allow for “stress testing” on soft investments. • Establishing a shock-responsive delivery mechanism to As a result, it received an NR rating for resilience of the project—a build resilience; clear indication of the need to pilot and develop tools for soft • Establishing procedures for early financing to support pre- investments. agreed early actions and rapid responses; and Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 26 Lessons learned from piloting the RRS LESSON 3 So, for projects where the scope is limited the main task for the RRS team is to un- to a single dimension, the methodology has derstand the differences between the RRS been expanded to explicitly include a not methodology and the sectoral assessment applicable (NA) rating. In situations where and metrics and promote consistency projects could be exposed to climate through discussion with the sectoral focal change and disaster risks, but not enough points. Following a review of the tools and information, data, or tools are available to approaches outlined in box 4, the RRS assess these risks, they receive an NR rat- team has determined that these are con- ing for the resilience of dimension. These sistent with the RiST principles, and if the two categories allow assessors to cap- sectoral teams are committed to applying ture the contribution of projects that may them in their project appraisal processes, only operate within a single RRS dimen- they can yield an A rating for the resilience sion. Despite this, all projects can benefit of dimension. This includes multi-phased from being screened for the resilience of programmatic operations of which proj- dimension. This is the current practice in ect components will be decided at a later the World Bank with climate and disaster stage of project design. risk screening even without conducting a Some focal points expressed concern that, stress test, especially for follow-on proj- as their sectoral methodology for climate ects where a country system has a full risk analysis is more rigorous than required stress test and/or the project’s activities in the RRS, projects in other sectors could have been screened previously. The RRS receive an A rating without undergoing the team will develop more guidance and clar- same level of scrutiny. Acknowledging that ification in sectoral applications to address some teams go the extra mile, one of the the applicability of NA and NR. RRS’s main goals at its current stage of More sector-specific guidance is required development is to ensure that all projects for RRS implementation. While the RRS meet the same baseline criteria. Once proj- and RiST tool provide an overall frame- ects across all sectors better incorporate work for the approach, more detailed, sec- climate resilience considerations, it will be tor-specific, activity-level data and infor- possible to further develop metrics to rate mation would increase the useability and and reward even better projects. Where consistency of RRS application. Several teams and sectors are only just starting to World Bank sectoral teams have devel- consider climate risks and resilience mea- oped their own guidance, methodologies sures in their project design and practice, and tools—either on their own initiative or RRS team support—in the form of climate as a result of the RRS partnership—for data, the RiST tool, and communications incorporating climate risk and resilience on RRS methodology and ratings—pro- considerations in project appraisal and de- vides an important toolkit to help them sign with different stages of application in develop their climate capacity and embed sectoral operations (box 4). In such cases, climate considerations in their operations. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 27 LESSON 3 Box 4. Sector-specific guidance for incorporating climate risk and resilience considerations in project appraisal and design Agriculture sector • ESMAP. 2022. Economic Analysis of Power Proj- During the RRS piloting process, the World Bank collaborated ects: Integration of Climate Change and Disaster Resil- with the Food and Agriculture Organization of the United ience. Washington, DC: World Bank. http://documents. Nations (FAO) to identify available data sources and literature for w o r l d b a n k . o rg / c u r a t e d / e n / 0 9 9 2 2 0 0 1 2 1 5 2 2 1 9 9 5 5 / assessing the impacts of climate change and extreme events on P1661220fe68eb082081bf04c6a2c6cb97e. agricultural production and prices. This collaboration between Environment sector (forests) FAO economists, who were familiar with the literature on climate Developed as part of the World Bank’s Building Climate impacts on agricultural production, and a World Bank climate Resilience in Landscapes approach and in coordination with the economist, who was familiar with climate risk stress testing, led to International Finance Corporation, the climate resilience tool for the fastest RiST application to date, with stress testing completed forestry projects incorporates hazard exposure assessments in two weeks. As a result of this pilot, FAO developed a guidance using climate scenario data for the project area and conducts an note for improving RRS in the agriculture sector. Further work impact assessment on the project EFA. Applied during project is needed on the impacts of climate change on downstream preparation and implementation, it supports the selection of activities such as transportation, processing, distribution, and suitable species variations and site locations, given that risks and consumption. resilience measures can be very site-specific, and has unique • FAO. 2022. Recommendations on Improving the Resilience features, including mitigation actions and costs. The RRS team Rating System (RRS) in the Agriculture Sector. provided feedback and engaged in productive discussions with the sector focal points while the tool was under development. Energy sector The World Bank has developed sector-specific resources for • World Bank. 2021. Resilience Tool for Forestry Projects. scaling up climate adaptation action across its energy sector Social protection and jobs sector operations. These include guidance documentation on how to Piloting the RRS in non-infrastructure sectors has also proven integrate climate resilience measures into the design of energy fruitful. For example, working with the Productive Social Safety systems, with studies in Benin, Cabo Verde, and the Democratic Nets and Youth Employment Project in Sierra Leone (pilot project Republic of Congo on building the climate resilience of their energy box 4) led to exploratory conversations with specialist economists infrastructure, and a technical note on steps and considerations on how best to incorporate climate risk stress testing in this for incorporating climate and disaster risk and resilience design sector. This is an area of innovation and further development for measures in power project EFAs, which has similar steps to RiST. the RRS methodology. The Regional Electricity Access and BEST Project pilot in Côte d’Ivoire, Mali, Mauritania, Niger, Senegal developed a technical Transport sector guidance note for the West African energy sector on how to The Nepal Regional Transport Program applied the RRS incorporate climate and disaster risks into energy sector project methodology to a major highway project and tested the feasibility EFAs and how to manage uncertainty in climate projections when of applying it to geophysical hazards, such as earthquakes. planning investments that span longer periods of time. There is opportunity to continue the collaboration and integrate climate and disaster risk stress testing directly into the planned • Schweikert, A, Ramstein, C and Nicolas, C. 2022. Power- upgrade of the Highway Development and Management Model, ing through the Storm: Climate Resilience for Energy Sys- which is the standardized economic analysis tool for the sector. tems. Washington, DC: World Bank. http://hdl.handle. Developed by the World Bank, the HDM model helps in planning net/10986/37999 Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 28 Lessons learned from piloting the RRS Box 4. (cont.) and prioritizing road construction and rehabilitation projects by Change and Other Threats: A Road Map. Washington, DC: evaluating economic viability and optimizing resource allocation World Bank. http://documents.worldbank.org/curated/ en/425871546231664745/Building-the-Resilience-of-WSS- Water sector Utilities-to-Climate-Change-and-Other-Threats-A-Road- Considering natural climate variances and disaster planning has Map. long been an integral part of the design process for water utility • Ray, P and Brown, C. 2015. Confronting Climate Uncertainty planners and engineers. To address the uncertainty surrounding in Water Resources Planning and Project Design: The Deci- future climate conditions and impacts, the World Bank has sion Tree Framework. Washington, DC: World Bank. http:// developed extensive guidance, data, and tools that evaluate hdl.handle.net/10986/22544. climate risks and resilience design and support decision-making • World Bank. 2020. Resilient Water Infrastructure Design under deep uncertainty for water projects. These include: Brief. Washington, DC: World Bank. http://hdl.handle. • Bonzanigo, L, Rozenberg, J, Felter, G, Lempert, R and Reed, net/10986/34448 P. 2018. Building the Resilience of WSS Utilities to Climate LESSON 3 Applying RiST in non-infrastructure sectors uating—or can be meaningfully applied or is an area that requires further develop- adapted to evaluate—the effects of climate ment. Some World Bank teams have their change and disasters in health and other own sectoral approaches and methodol- human development projects, the RRS ogies for integrating climate shocks. For team is looking to expand the next round example, the Climate and Health Vulner- of piloting to more human development in- ability Assessment tool integrates climate vestment projects and those in the equita- stress testing into economic analysis at the ble growth, finance, and institutions sector. aggregate (country) level, and has a meth- Although the RRS team continues to work odology for estimating climate change im- to expand the scope of RiST and develop pacts on health endpoints that could be approaches to stress test climate risks in useful at project level; and the Social Pro- more sectors, it is important to note that tection Stress Test tool brings together so- RRS—particularly applying climate stress cial protection, disaster risk management, testing through RiST—is currently only and climate change adaptation sectors suited to investment projects and not to to leverage their respective contributions those financed through program-for-re- in reducing household vulnerability and sults or development policy financing. As- building household resilience. As well as sessing how to apply RRS to these project coordinating and collaborating with these types is another area of future work. sector teams to ensure that the RRS meth- odology and RiST tool are suitable for eval- Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 29 The RRS pilots have highlighted how ex- climate and disaster resilience consid- LESSON 4 pertise in climate science, adaptation and erations, enabling them to get the best resilience, and climate economics are nec- possible rating. The RRS team facilitated Climate expertise essary for effective project development access to climate economists and scien- is necessary for support. The RRS team comprises climate tists, enhancing capacities and therefore systematically change specialists, climate scientists, and enabling more effective implementation of embedding climate climate economists with broad sectoral the RRS methodology and faster applica- adaptation into project expertise, experience, and knowledge of tion of the RiST tool. design climate science, impacts, and economics, The team’s ability to communicate and corporate commitments, and operations. build trust enabled constructive discus- This has allowed the RRS team to engage sions that helped project teams improve with World Bank Group teams across a their assessments and outcomes and variety of sectors and regions to develop enabled some to stress test climate risks and socialize the standards for the RRS in their EFA to achieve an A rating in the methodology. resilience of dimension. The main take- The RRS team helped curate climate data, away from personalized support during conduct or deepen climate risk identifica- piloting was realizing that, far from being tion and assessments, and think through an “add-on”, climate adaptation needs to adaptation options, while supporting a fo- be systematically embedded within project cus on climate M&E through indicators to design to achieve the greatest impact of ensure the regular monitoring and tracking development objectives. of progress. On request, RRS team mem- Throughout the RRS development and bers also joined certain project missions to piloting processes, sectoral World Bank support client dialogue on climate adapta- teams appreciated the high technical in- tion and resilience. One of the primary ob- tegrity of the RRS team, who drew on a jectives of piloting the RRS methodology wide range of literature on climate impacts in operations was to help projects embed and new frontier science. Both in its written Pilot Project Box 3. Gambia Inclusive and Resilient Agricultural Value Chain Development Project Sector: Agriculture and food Country: The Gambia (Africa West Region) Project code: P173070 RRS rating: BA The RRS team was asked to join a virtual mission with The Gambian in project design. Although it was not possible to complete climate government, which was very interested in the RRS methodology stress testing through the RiST tool, its initial application helped and strengthening this project’s resilience across both dimensions: improve the quality of the project’s baseline EFA. After testing resilience of and resilience through the project. The project team the tool in this pilot, the RRS team was able to further refine and welcomed support on climate and disaster risk data, designing simplify RiST, streamlining it for future applications. The RRS team resilience measures, and climate indicators; and having a climate communicated clearly with the project team as it identified areas specialist involved in client interactions and engaging with the for improvement to enhance consideration of climate risks and project team enhanced the integration of climate considerations vulnerability or to clarify assumptions made in the EFA. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 30 Lessons learned from piloting the RRS LESSON 4 work and when communicating with oth- comes, the World Bank Group is build- er teams and external partners, the RRS ing more climate expertise and capacity team was explicit about the uncertainties in regions and strengthening its ability to around climate modeling and data limita- provide climate support—for example, tions. Where teams or country coordina- through trainings on climate data, such tors had sector-specific questions, having as the Climate Change Knowledge Portal an experienced sectoral climate specialist (CCKP),15 and adaptation and resilience clarify the RRS objectives and explain how analytical tools, and by decentralizing Cli- the RRS methodology is applied to proj- mate Change Group staff to provide cli- ects proved an effective way of securing mate expertise to other teams. buy-in for the methodology and process. It is worth noting that sectoral and region- The RRS team’s support increased the al climate champions played crucial roles ambitions of the pilot projects around in- as “translators”, supporting RRS with their tegrating climate change, without placing technical understanding of climate issues additional burden on the project teams. and sector-specific concerns and priori- But this piloting model relied heavily on ties. They also helped make connections the World Bank Climate Change Group’s with sectoral methods and tools, stream- resources and individualized target sup- line the RRS application process, and se- port. To mainstream climate change and cure new pilot projects when needed. improve operations and development out Pilot Project Box 4. Productive Social Safety Nets and Youth Employment Sector: Social protection and jobs Country: Sierra Leone (Africa West Region) Project code: P176789 RRS rating: BB The RRS methodology influenced the climate-resilient design a detailed analysis of poor households’ exposure to climate risks of this project, which was approved with 27 percent climate co- and interlinkages between climate change and food security as benefits. With support from climate experts on the RRS team, a climate annex in the project documentation. The RRS team the project team strengthened the project design against climate is exploring with sectoral economists how to best incorporate and disaster risks by: integrating climate resilience measures into climate risk stress testing in social protection and jobs projects. public works; prioritizing business plans that strengthen urban This is an area of innovation and further development for the RRS resilience; and including activities to disseminate information, host methodology. events, and deliver trainings on climate change. It also included 15   https://climateknowledgeportal.worldbank.org/. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 31 Climate science, data, and analytics have quality of analysis performed during project LESSON 5 played a central role in RRS ratings and preparation and documented in the project supporting sectoral World Bank teams to appraisal documents. A project’s design Project decision-making evaluate climate and disaster risks and in- can only be as good as the assumptions requires robust climate corporate climate resilience considerations it is based on; so, any new infrastructure and disaster risk data in projects. The RRS team has developed or investments will only be as robust as the and analytical tools and shared climate risk overviews using risks it has foreseen for the relevant times- that communicate the best available climate data and projec- cales. Uncertainty is a given when it comes uncertainty tions for countries or project areas, such to climate futures, but data limitations ex- as the CCKP and ThinkHazard!.16 In sev- acerbate this problem. eral projects (including the one outlined in The RRS team found great variation in the pilot project box 5), such climate risk in- climate and disaster risk screening and cli- formation led teams to expand their con- mate risk assessments across the pilots, sideration of climate risk factors in project indicating that the quality of risk screening development and design. also varies greatly across World Bank proj- Whether the RRS makes a project more re- ects. This may be due to a lack of climate silient depends on the quality of data used. change capacity, or of climate data and Within the World Bank project preparation scenarios at project-relevant scale, with cycle, RRS ratings are expected to be ap- many project appraisals using country-lev- plied ex ante at the board approval stage, el climate risks.17 The RRS team provided and as such, the rating will depend on the multiple sources of climate and disaster Pilot Project Box 5. Niger Integrated Urban Development and Multi-sectoral Resilience Project Sector: Urban, resilience, and land Country: Niger (Africa West Region) Project code: P175857 RRS rating: AA With the development objective to “increase resilience to floods extreme heat was included as a key climate hazard to consider and improve urban management and access to basic services during the detailed design of the flood protection works. in selected municipalities in Niger”, this project largely finances The impact of this RRS engagement expanded beyond the pilot municipal infrastructure and flood risk reduction infrastructure, to an urban development analytical and advisory project in Sierra including nature-based solutions. Given this focus on flooding, Leone. Following a request from the project team leader for more the project team’s primary concern was to incorporate climate granular, location- and sector-relevant climate information that risks such as increasing rainfall into the project design. But connects with future climate change scenarios to support climate the RRS team’s detailed country climate risk and vulnerability risk assessments, the RRS team developed a similar climate risk overview showed that extreme heat is also a key climate hazard overview with enhanced information on climate extremes. This in Niger’s urban areas, which could impact the effectiveness suggests enhanced understanding and proactive considerations and sustainability of some of the proposed flood risk reduction of multiple climate risks at the onset of a new project. measures, particularly nature-based solutions. As a result,   https://climateknowledgeportal.worldbank.org/; https://www.thinkhazard.org/en/. 16   This is further complicated by some projects did selecting project sites until a later stage. 17 Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 32 Lessons learned from piloting the RRS LESSON 5 data and information, and supported oth- Given the uncertain nature of future socio- er teams’ capacity to produce, interpret, economic development and climate sys- and act on information. Although the avail- tem responses, the requirement for explic- ability of climate data was a key challenge itly considering uncertainty is an important in some pilots, particularly in small island feature in both the RRS rating criteria and development states (SIDS), these projects climate risk stress testing. Assessing the have made progress, with support from resilience of a project requires alternative the RRS team and further data develop- climate scenarios for each rating, from C ment (box 5). This shows promise for fu- to A. Likewise, incorporating a range of cli- ture RRS and RiST applications. mate futures and potential impacts, using Box 5. Data challenges for SIDS Given the limited scientific and monitoring capacity of many Despite these challenges, the RRS successfully advanced low- and middle-income countries, historical climate data are not climate data and scientific support to such projects. For example, always available in a project area. This challenge is particularly for the Dili Water Supply Project (pilot project box 1), the RRS acute in SIDS, where climate vulnerability is high and both climate team worked with a climate scientist at the University of Oxford risk assessments and the effectiveness of resilience-building to process high-resolution historical observational precipitation interventions are constrained by a lack of data. Projections of data to inform annual and seasonal precipitation profiles for Dili, future climate change and climate extremes for SIDS are also and estimates of future changes in heavy precipitation events lacking or face limitations when downscaling from global climate and droughts, considering climate change scenarios (figure 4). models. Several pilot projects were implemented in SIDS— This information was then used for a more detailed engineering including Grenada, Timor-Leste, and Tonga—and the RRS team study to ensure robustness of project design. faced significant challenges obtaining reliable climate data to undertake climate risk stress testing for these projects. Figure 4. Historical precipitation data and future estimates for Dili a) Annual precipitation (2000–18) b) Precipitation estimates, by return period Meanwhile, the CCKP—a key source of climatology and future on future scenarios and return periods of climate extremes to projection information used by the RRS team—is advancing support RRS and RiST applications. rapidly with downscaled data products, expanding information Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 33 a simplified approach of decision-making The RRS team has since streamlined and LESSON 5 under deep uncertainty to facilitate the simplified several features of the RiST tool inclusion of climate uncertainty, is a key to facilitate analysis, and is developing a component of RiST. The RiST application prototype online tool that helps automate ensures a project’s cost-benefit analysis some steps in the analysis—such as cli- identifies plausible climate and disaster mate data and scenarios and climate im- risks and impacts, considers potential ad- pacts (“damage functions”) with sufficient aptation and resilience measures to ad- resolutions—and enhance data analytics dress these risks, and ensures the project capability, such as uncertainty analysis and is viable and can deliver its intended devel- characterization. Given the feedback from opment goals in the face of the uncertainty users and the increasing interest in climate of climate change and its impacts. risk stress testing, an online tool can facili- tate learning, data development, and con- The pilots revealed the need to further re- necting with other tools. With an approach fine the RiST tool, including through a more that is scalable and easier to implement, automated, user-friendly interface that can the uptake of climate risk stress testing more readily and effectively facilitate a dia- both within and outside of the World Bank logue with project teams and better incor- is likely to increase. porate climate resilience in project design. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 34 Lessons learned from piloting the RRS LESSON 6 An EFA helps inform decisions about proj- type. This lack of standardized approach ect selection and design in three ways. and outputs within and across sectors led Robust climate risk First, it identifies where scarce resources to challenges in applying the RiST tool. In stress testing requires can have the most impact. Second, it en- some cases, the underlying assumptions a quality baseline sures appropriate fiscal impact and finan- and methods for calculating costs and economic analysi cial viability. And third, it ensures benefits benefits were unclear or the reported cost are accessible for the poor or other target- and benefit information was not sufficient- ed beneficiaries. All World Bank investment ly disaggregated to connect with climate operations are required to carry out an EFA impacts, making it difficult to incorpo- to inform decision-making on undertaking rate climate risk stress testing. In others, a project. In an increasingly complex de- the availability of data and information for cision-making environment with compet- stress testing was a challenge, as some ing needs for limited resources, climate EFAs did not contain comprehensive infor- risk-informed project investment decisions mation upfront. To more readily apply the require renewed attention on EFA quality RiST tool, there is a need to raise aware- and role in decision-making. ness of the minimum data and information Applying the RRS offers an opportunity to requirements in the baseline EFA. further strengthen EFAs by ensuring proj- Some EFAs were conducted in a discon- ects remain robust to the impacts of cli- nected way, and their timing made it dif- mate change and disasters. For several ficult to incorporate climate and disaster of the pilots that undertook a climate risk risk considerations so they would have stress test, the analysis showed that the a meaningful impact on project design. benefits of investing in resilience measures Many were developed and shared with significantly outweigh the small incremen- the RRS team in the later project apprais- tal cost of such investment. Achieving an al stage—that is, after project design was A rating for the resilience of dimension re- completed—leaving little time to carry out quires the project EFA to be stress tested and complete climate risk stress testing for climate and disaster risks. The RiST to inform project decisions. In some cas- methodology provides a relatively simple, es, the project economist completed the yet novel, approach for connecting project EFA almost in a silo, without properly con- EFAs with climate and disaster information, necting with the wider project team. This climate and disaster impact estimates, and approach raises questions around the ef- uncertainty considerations. But applying fectiveness of the EFA (and RiST) as a tool the RiST tool requires a robust baseline to modify, tweak, and improve project de- EFA. sign. The pilots show that, to influence and The pilot EFAs varied widely in their under- optimize project design conditions, there is lying approach and overall quality. While a need to improve the quality of baseline different sectors are bound to approach EFAs (box 6), standardize across sector/ EFAs differently, there was a lack of con- activity types where appropriate, and en- sistency and comparability, even within sure EFAs are prepared upfront and with all the same sector and investment activity relevant technical experts. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Lessons learned from piloting the RRS 35 Box 6. The importance of high-quality baseline EFAs “The main purpose of project economic analysis is to help design public sector rationale or comparing projects against alternatives; and select projects that contribute to the welfare of a country. and because they are usually prepared after the decision to Economic analysis is most useful when used early in the project proceed with the project has been made, they are of limited use cycle, to catch bad projects and bad project components.”a for decision-making. The Independent Evaluation Group’s study of cost-benefit The RRS team identified issues—including errors in estimates analyses in World Bank projects from the 1970s to the early and assumptions—in several of the pilot EFAs, and these were 2000s finds the percentage of projects justified by cost-benefit addressed, improving the analyses. In some cases, applying the analysis has been declining for several decades, owing to a RiST tool helped identify flaws and errors in the EFAs, improving decline in adherence to standards.b Many cost-benefit analyses the quality of project appraisal beyond climate risk and resilience fail to pay attention to fundamental analytical issues, such as the analysis. a World Bank. 1998. Handbook on Economic Analysis of Investment Operations. Washington DC: World Bank Group. http://documents.worldbank.org/ curated/en/749061468740206498/Handbook-on-economic-analysis-of-investment-operations. b Independent Evaluation Group. 2010. Cost-benefit Analysis in World Bank Projects. Washington DC: World Bank. http://hdl.handle.net/10986/2561. For agriculture sector pilots, the RRS team • Present “with” and “without” project LESSON 6 worked closely with FAO, which carried scenarios instead of aggregated calcu- out the RiST application. This resulted in lations; and the following recommendations to create • Define and present optimistic and pes- EFA standards for agricultural projects and simistic project scenarios in the analysis. best practices, which can be applied more Early results from the pilots demonstrated broadly: that incorporating a climate and disaster • Adopt FAO’s flexible, appropriate, risk stress test is far from straightforward, structured and transparent (FAST) stan- and as a result, achieving an A rating for dard18 for building the EFA model; resilience of a project requires significant • Clearly document input assumptions, effort in terms of both time and technical such as yield or productivity, area of capacity. Conducting climate risk stress production, commodity prices, and testing was the most resource-intensive changes over time; step in applying the RRS, and teams did • Disaggregate the categories of costs not always have the time and technical ca- and benefits linked to agricultural ac- pacity to carry out RiST analysis. Connect- tivities—for example, by crop, capital ing the project EFA with RiST is not always costs, operating costs, production, up- straightforward either, given the different stream vs. downstream benefits; methodologies and practices available • Disaggregate nonmonetary values— for conducting EFAs and the RiST tool’s such as the value of greenhouse gas need for transparent and disaggregated sequestration—from financial benefits; information on project costs and benefits. 18  FAO. 2022. Food and Agriculture for Sustainable Transformation Initiative – FAST. https://www.fao.org/ documents/card/en?details=cc2186en. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 36 Lessons learned from piloting the RRS LESSON 6 The different units of analysis and lack of ties to incorporate climate and disaster risk empirical information on climate impacts analysis in existing sectoral models and for some project activities make applying tools—for example, in the transport sec- RiST more challenging, preventing some tor’s project assessment model, HDM5. projects from achieving an A rating for the The extent to which stress testing is ap- resilience of dimension. And, as already plied will depend on investment size and discussed under lesson 5, the RiST appli- type and the proportionality of climate risks. cation requires a quality baseline EFA. For example, a small rural road need not undergo the same level of rigorous, cost- Despite these challenges, several pilots ly, and time-intensive resilient design and successfully integrated climate and disas- stress testing as a large hydropower dam. ter risks into their EFAs, and the outcomes At the same time, it will not be in the scope of these analyses helped shape project de- of all projects to address all the risks posed sign or raise awareness on the potential for by climate change and disasters. As such, climate and disaster risks to threaten the teams will need to prioritize the risks to ad- project’s overall robustness. For example, dress and the extent to which they address the RiST application findings for the Re- them, focusing their efforts on the hazards gional Electricity Access and BEST Project that pose the most risk to the project de- in five West African countries demonstrat- velopment outcomes. The RRS team will ed the implications of choosing different review developing more nuanced guidance construction materials and selecting the and criteria for rating projects that consider right sites to reduce the risk of system the level of risk and magnitude of impacts failure from extreme heat and wildfire. In to the project. the transport connectivity project in Ne- pal, stress testing demonstrated and gave To date, the RiST tool has largely focused confidence that, despite being situated in a on assets and infrastructure. As outlined highly vulnerable area, the project remained in lesson 3, this presents a challenge robust to climate and disaster impacts be- when applying it to other types of invest- cause it integrated robust adaptation and ment project, such as those relating to resilience measures into its design. It also cash transfers and jobs (pilot project box demonstrated that spending approximate- 4), which will have a different way of es- ly 3 percent of project costs on climate timating costs and benefits. Ongoing and resilience measures would create substan- planned activities to address this include: tial savings in terms of potential economic • Systematically reviewing World Bank and financial losses in the event of extreme and sectoral guidance and tools for flooding in high-impact scenarios. EFA and considering climate and disas- To mainstream climate risk stress testing ter risk; within the World Bank system, it will need • Evaluating opportunities for consider- to become an integral part of the EFA and ing climate and disaster risks in current project appraisal process, with project sectoral approaches and linking with economists integrating climate impacts as RiST tool use; and part of their sensitivity analysis. As well as • Developing sectoral guidance and the developing climate data and the RiST tool, RiST tool to help teams integrate cli- the RRS team is working with World Bank mate and disaster risk considerations in sectoral focal points to identify opportuni- their projects’ design and assessment. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 37 5. SUMMARY OF REVISIONS TO RRS METHODOLOGY Building on the lessons learned from the metric from the B rating for resilience IDA19 piloting, the RRS team has refined through; and revised the original methodology (Fig- • Including results monitoring as a man- ure 5), addressing issues experienced datory requirement for an A rating in during the pilot phase, and with the inten- both resilience of and resilience through tion of simplifying the assessment, stan- dimensions, so that higher ratings dardizing application across multiple sec- emphasize, among other project attri- tors, and providing the best incentives. butes, a solid M&E system to provide confidence that a resilience-related Details of the RRS methodology and con- project will deliver on its intended out- siderations for each rating are presented in comes and results; the RRS methodology (World Bank 2021). Significant revisions include: • Introducing “transcending boundaries” as a requirement for an A rating for re- • Clarifying the NR and NA ratings to silience through, to reflect projects that make room for operations that primar- build systems-level resilience beyond ily operate in one dimension, lack the their own boundaries and timescales; necessary data or information for a risk and assessment to be conducted, or do • Positioning a transformational project not report on their development out- as an A+ rating for resilience through, comes—so, for example, projects with with a revised definition of transforma- no physical components do not need to tion as, “a project that sets the wid- be rated for the resilience of dimension; er system on a resilient development • Bundling all risk screening-related re- pathway by fundamentally altering the quirements under the C rating for the current system and having a transfor- resilience of dimension, simplifying the mational impact.” assessment of climate and disaster risks that affect a project; The RRS team will continue to work with • Adopting adaptation measures across World Bank sectoral teams to review and the B rating for both dimensions to en- refine the methodology to ensure it cap- sure congruence and standardization, tures both best practice and best practical and decoupling the climate co-benefit applicability in different contexts. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 38 Summary of revisions to RRS methodology Figure 5. An overview of RRS methodology Resilience of a project Resilience through a project C Project developers report identified threats based C In most circumstances, resilience to climate on a qualitative estimate of climate/disaster change is enhanced by good development, with risk. The main goal is for project developers to higher and more stable incomes, lower poverty, understand the project’s short and long-term better access to infrastructure and financial exposure to climate change and disasters, as well services, and stronger social protection and health as the potential impacts of this exposure, and care systems. Projects with development benefits to prioritize which risks need to be addressed are assigned a C rating. through the project’s design. B The project addresses its vulnerabilities to climate/ B The project influences adaptation and resilience disaster risks by including appropriate adaptation beyond its immediate boundaries, outputs and measures to make the project more resilient and timescale by removing or significantly reducing reduce its residual risk, such that it can still achieve the underlying causes of vulnerability, barriers for its main development objectives. adaptation and resilience, and building resilience. A The project incorporates a climate and disaster risk The project influences adaptation and resilience A stress test that considers a range of climate and beyond its immediate boundaries, outputs and disaster impacts (for example, in its EFA or other timescale by removing or significantly reducing project appraisal analysis) and ensures that, after the underlying causes of vulnerability, barriers for risk reduction measures are included, residual adaptation and resilience, and building resilience. risks do not make the project economically or It also monitors and tracks the progress of activities financially unviable or unable to achieve its building resilience through the project via at least intended development outcomes for any likely one climate adaptation indicator. or probable climate scenarios. The project also monitors and tracks the progress of activities building resilience of the project via at least one climate adaptation indicator. + The project conducts a more systematic exploration of the risks to the project and + The project sets the wider system on a resilient development pathway by fundamentally altering undertakes contingent planning in case of the current system and having a transformational unexpected situations that were not considered impact. in the project design. Projects can be rated A+ or B+ if they include the appropriate criteria for contingency planning. NR The project is possibly exposed to climate change NR The project does not report on its contribution and disaster risks, but no information is available, to development, growth, poverty reduction, or or the risks are unmanageable and threaten the resilience. project’s economic viability. NA The project is not exposed to climate change risks in a material way, or a resilience rating is not relevant, based on the nature of project activities or types of outcome. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 39 6. APPLYING RRS BEYOND WORLD BANK OPERATIONS The RRS was conceived and developed learned from the pilots at events—such as as a rating system that could be used not a workshop organized by Brazil’s Ministry just within the World Bank Group but by of Economy, a masterclass organized by governments, private sector actors, devel- the Global Center on Adaptation, the Arab opment partners, and others to evaluate Conference for Cooperation on Climate investments and development projects. Change, the World Resources Institute When developing the system, the RRS Adaptation and Resilience Mainstreaming team consulted and included several ex- Program’s course on adaptation finance, ternal stakeholders and institutions, such and the International Renewable Ener- as private sector actors and other MDBs. gy Agency’s Adaptation Metrics Working Many partners continue to express interest Group—and offering technical assistance in further understanding and applying the to the city of Ekurhuleni through the Cities RRS methodology, and the RRS method Support Program led by the government development has inspired external ac- of South Africa.19 These external engage- tors—such as the Climate Bonds Initia- ments have spurred others to adopt simi- tive—to develop their own resilience guid- lar approaches. For example, the Brazilian ance. The team also worked closely with Ministry of Economy has developed guid- FAO’s Economic and Policy Analysis of Cli- ance for incorporating climate risks in the mate Change team to tailor the risk stress economic analysis of infrastructure proj- testing methodology to specific sectors. ects that followed the climate and disaster risk stress test methodology, and the RRS With increasing demand and efforts team has supported European and Cen- around resilience evaluation among ex- tral Asian countries to mainstream climate ternal stakeholders, the RRS team has risk screening and climate-informed eco- supported outreach, engagement, and nomic analysis for public investment man- dialogue with several partners. This in- agement. RRS is also contributing to the cludes sharing the RRS and climate risk development of a resilience classification stress testing methodologies and lessons   Avner, P, Shariq, A, Shoaib, A and Koh, I. 2023. “Integrating Disaster and Climate Risks into Capital 19 Project Appraisal- Application to Projects and Programs in the Ekurhuleni Metropolitan Municipality South Africa.” Washington, DC: World Bank. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 40 Applying RRS beyond World Bank operations system led by the Climate Bond Initiative investments, development projects, and to spur adaptation and resilience finance. sectoral and national policies, as well as global progress on adaptation. The World Helping to cement the World Bank’s role Bank will continue to engage with exter- as innovator and thought leader in climate nal partners and stakeholders to advance adaptation and resilience, the RRS has the development of resilience assessment the potential to become a standard and metrics and encourage consistency be- contribute to measuring and tracking the tween the RRS method and the standards adaptation and resilience performance of and resilience metrics they develop. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 41 7. MOVING FORWARD Within and outside the World Bank, there is achieve at least 10 projects with an AA increasing interest in—and demand for— rating over the FY23–25 period. better metrics to evaluate the impacts and The goal is not to move the entire World outcomes of development activities and Bank portfolio toward an AA rating, as do- mobilize private finance to support climate ing so could exclude necessary develop- adaptation and resilience. In response to ment projects that do not focus on trans- these needs, the RRS team is focused on formational climate adaptation impacts. several streams of work, which include: Rather, it is to accurately reflect and com- a second round of RRS applications with municate the degree of resilience build- IDA20 operations; updating the method- ing embedded in our projects and where ology and developing sectoral guidance; appropriate, seek opportunities to further developing the web-based RiST tool; us- embed activities that build systemwide ing RRS foundational work and insights resilience. Regional and sectoral coun- from testing to develop metrics that can terparts have indicated strong interest in be linked with results-based adaptation and requests for applying RRS to IDA20 finance; and dissemination, outreach and projects. Doing so will further strengthen external engagement. capacity to systematically consider ad- aptation and resilience in project design, With the first round of pilot projects under provide a baseline, and track the quality of IDA19 outlined in this paper, the RRS team adaptation and resilience building across was able to demonstrate that the method- the World Bank portfolio.  ology and rating system can be useful for The RRS team will also strive to align the further integrating adaptation in the World World Bank’s adaptation and resilience Bank portfolio and to help track the ambi- methodologies and corporate commit- tion of World Bank projects. The team will ments—for example, on climate and disas- continue to support sectoral World Bank ter risk screening, Paris-aligned adaptation, teams to apply the methodology and inte- and climate indicators—and its emerging grate resilience measures in select opera- Integrated Climate Results Framework, to tions nominated by regions. At minimum, streamline assessment and reporting for under IDA20, there is a commitment to World Bank teams and support the evo- Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change 42 Moving Forward lution toward measuring the impacts and scenarios—for example, through CCKP— outcomes of World Bank operations. The climate risk stress testing, and analytics. team will undertake deeper work with sec- Other activities include a systematic review toral teams, particularly to adapt the RRS of World Bank and sectoral guidance and methodology for human development tools for EFAs and identify opportunities projects and explore opportunities for re- to better incorporate climate and disaster sults-based financing and policy lending. risks for climate-informed project eco- To disseminate knowledge and share les- nomic analysis early in project preparation. sons learned, the RRS team aims to pres- Meanwhile, the review evaluates oppor- ent and discuss the revised methodology, tunities for incorporating such risks into case studies, and EFA improvements that current approaches, linking them with the integrate climate and disaster risk consid- RiST tool, developing further sectoral guid- erations with World Bank staff. ance, and tailoring the RiST tool to support The RRS team will continue to revise and the integration of climate and disaster risk update the methodology, based on feed- considerations into project designs and back and lessons learned, ensuring a con- assessments for different sectors. sistent and standardized approach across Finally, given the immense interest and sectors, while supporting the development need for metrics to monitor and track the of sector-specific guidance where nec- resilience attributes of investment activities essary. Specifically, the team will focus and drive investments toward more resilient on identifying and sourcing relevant sec- projects, the RRS team will continue to en- tor-specific climate projection and impact gage with internal and external partners— data to better integrate them into RRS and including research institutes, think tanks, RiST, improving their usability, and link with private sector actors, standard-setting other sectoral tools—such as the World bodies, and credit rating agencies—to ad- Bank’s hydroclimatic stress test tool—to vance the development of methodologies, provide the best information to encourage rating systems, metrics, and standards for and support adaptation in projects. The evaluating and labeling the resilience attri- team will also continue to develop rele- butes of investment projects. The team will vant data, tools, and approaches to make also continue to link these with metrics at the RRS more widely applicable. Working different scales, such as country-level ad- with internal counterparts and the scientific aptation and resilience readiness diagnos- community, it is leading an effort to build tics, to inform progress toward the global a web-based RiST tool that can facilitate goal on adaptation and support the scaling access to climate data, risk thresholds and up of adaptation finance. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Appendix A. Full list of IDA19 pilot projects Country/ies Project Approval Financing Project name RRS Notes on engagement number fiscal year rating AGRICULTURE AND FOOD Gambia, The P173070 FY22 $40 million Gambia Inclusive and BA The RRS team provided feedback on the project appraisal Resilient Agricultural documents (PADs), including: strengthening the vulnerability context Value Chain and using climate indices as part of the narrative, and suggesting Development Project both additional climate adaptation and mitigation measures to (GIRAV) include in the project design, and options for climate indicators that could be included in the results framework. The team also engaged in virtual missions for this project, provided feedback on enhancing climate considerations, and drafted inputs for the project document. Due to the operation’s complexity, limited information on assumptions used for the EFA, and short timeline to board approval, the RiST was not finalized for this project. The RRS team continued to engage with FAO and the World Bank’s agriculture and food team to compile climate and disaster risk data and developed a case study on this project as part of an ongoing sectoral RIST note. Honduras P174328 FY21 $100 million Innovation for Rural BA The project team adopted and incorporated almost all the RRS team Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Competitiveness Project suggestions on strengthening climate considerations as part of - COMRURAL III project design. The RRS team’s initial feedback on the PAD included strengthening the vulnerability context and using climate indices as part of the narrative, adding climate adaptation and mitigation measures to the project design, and including options for climate indicators in the results framework. For the RiST analysis, the project had a set of subcomponents that posed challenges for integration into the RiST tool. The EFA had been based on “possible” incremental net revenue flow from potential subproject implementation. Selected subprojects for the analysis were operations financed under COMRURAL II. Although the RRS and FAO teams worked to incorporate climate considerations in the EFA, the short turnaround time for project approval and the complexity of the EFA made it impossible to complete the analysis before board approval. 43 Country/ies Project Approval Financing Project name RRS Notes on engagement 44 number fiscal year rating Pakistan P176786 FY2320 $200 million Punjab Resilient and AA The RRS team provided feedback on the PAD, including a detailed, Inclusive Agriculture subnational vulnerability context and how to strengthen climate Transformation considerations and indicators in project design. With the support from FAO economists, this project was able to complete its RiST analysis. ENERGY AND EXTRACTIVES Liberia P173416 FY21 $59 million Liberia Electricity BC The RRS team was not able to engage in this project as the piloting Sector Strengthening started when the project was very close to board approval stage, so and Access Project it was rated retroactively. (LESSAP) Somalia P173088 FY22 $150 million Somali Electricity Sector AB Climate risk stress testing for this project was included in the EFA, Recovery Project so the RRS team did not apply the RiST. With support from the RRS team, the project team included a detailed climate annex outlining climate risks and potential adaptation options. Côte P167569 FY21 $465 million Regional Electricity AA With RRS team support, the project team included a detailed climate d’Ivoire, Mali, Access and Battery annex outlining climate risks and potential adaptation options. Mauritania, Energy Storage Although the RiST analysis was finalized with active engagement Niger, Senegal Technology (BEST) from the project team, there was not enough time to include it in the Project final PAD. ENVIRONMENT, NATURAL RESOURCES, AND THE BLUE ECONOMY Lao PDR P170559 FY21 $34 million Lao Landscapes and BA The World bank environment focal points took a proactive approach, Livelihoods Project with ownership over the resilience rating and stress testing process. They created and are piloting an approach to build climate resilience in landscapes projects, with a stress-testing tool that is similar to RiST. Tajikistan P171524 FY22 $45 million RESILAND CA+ AA The stress-testing tool developed by the environmental focal points Program: Tajikistan was applied to this project, resulting in an A rating for resilience of the Resilient Landscape project. Restoration Project Uzbekistan P174135 FY22 $142 million RESILAND CA+ AA The stress-testing tool developed by the environmental focal points Program: Uzbekistan was applied to this project, resulting in an A rating for resilience of Resilient Landscapes the project. Restoration Project 20   The Pakistan Panjab Punjab Resilient and Inclusive Agriculture Transformation Project was originally a FY22 project but its approval was deferred to FY23 due to bunching at Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Appendix A. Full list of IDA19 pilot projects the end of FY22. Country/ies Project Approval Financing Project name RRS Notes on engagement number fiscal year rating HEALTH, NUTRITION, AND POPULATION Niger P171767 FY22 $100 million Niger, Improving BB The RRS team provided comments on the PAD, targeted toward Women’s and Girls’ enhancing resilience. Although the team started to apply the Access to Improved RiST, this was paused to explore working with the Johns Hopkins Health and Nutrition Bloomberg School of Public Health on incorporating climate Services in the Priority considerations in the Lives Saved Tool (www.livessavedtool.org/). Areas Project (LAFIA- This is an area for future exploration with the World Bank health IYALI) team. Appendix A. Full list of IDA19 pilot projects SOCIAL PROTECTION AND JOBS Afghanistan P173387 FY21 $97.5 million Early Warning, Early NR/A The project had no physical assets and the available data and tools Finance and Early Action within the RRS and RiST environments do not allow for “stress Project testing” on soft investments. The project therefore received an NR rating in the resilience of dimension. Developing the tools needed to pilot RRS for soft investments is an area for future work. Sierra Leone P176789 FY22 $40 million Productive Social BB The project team incorporated the RRS team’s feedback into the Safety Nets and Youth project design and PAD and developed a detailed climate annex. Employment RRS engagement helped to further strengthen and integrate climate resilience measures in public works, prioritizing business plans that strengthen urban resilience, disseminating information about climate change, and hosting climate-related events and trainings. The PAD annex further highlighted poor households’ exposure to climate risks and interlinkages between climate change and food security. For RiST, the RRS team held exploratory conversations with World Bank social protection and jobs economists on how to best incorporate stress testing in projects in this sector. This is an area of further Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change development for RRS. TRANSPORT Nepal P177902 FY22 $275 million Accelerating Transport AA The RRS team provided detailed climate data to the project team, and Trade Connectivity which described the climate vulnerability context for the project area in Eastern South Asia – using the latest CCKP data. The team also applied the RiST to this Nepal Phase 1 Project project, which was rated A for both dimensions. Yemen, Rep. P177053 FY22 $50 million Emergency Lifeline BB The RRS team provided feedback on the PAD, including suggestions Connectivity Project on how the project team could strengthen the climate vulnerability context, and highlighted measures to build the climate resilience of project beneficiaries. Although the RRS team started stress testing, this was not completed before the project went to board for approval. URBAN, RESILIENCE, DISASTER MANAGEMENT, AND LAND Grenada P175720 FY22 $15 million Grenada Resilience AA The project team conducted stress testing using Monte Carlo Improvement Project simulations. The RRS team shared climate data and scenarios to provide additional support to the stress testing. 45 Country/ies Project Approval Financing Project name RRS Notes on engagement 46 number fiscal year rating Niger P175857 FY22 $250 million Niger Integrated Urban AA The RRS team provided written feedback on the project’s PAD, Development and including a detailed climate vulnerability context, which helped Multi-sectoral Resilience inform project design. As a result, the project team added extreme Project heat as a key climate risk consideration in urban infrastructure improvements, climate resilience investment planning and implementation, and risk assessments. The RRS team also completed a RiST analysis for this project. Pakistan P173087 FY21 $200 million Sindh Resilience Project BA This project became a pilot too late in the project cycle for the RRS Additional Financing team to be able assist in a significant manner. Tonga P174434 FY22 $15 million Tonga Safe and Resilient BB The project was board-approved, and additional funding was Schools Project prepared due to a recent volcanic eruption in Tonga. The RRS team provided comments and feedback on project design, strengthening the vulnerability context, and identifying possible climate-resilient design options. The team also explored ways to overcome challenges in data availability and identify additional and innovative climate data sources for future RiST implementation. WATER Ghana P171620 FY21 $125 million Ghana Additional BB This project joined the pilot too late in the project cycle for the Financing for Greater RRS team to be able assist in a significant manner. Approved in Accra Metropolitan Area September 2020, the project was rated retroactively. Sanitation and Water Project Niger P174414 FY22 $400 million Niger Integrated Water AA The RRS team made comments on the PAD to influence project Security Platform Project design and completed the RiST analysis for this project, which (Niger-IWSP Project) received an A rating in both dimensions. Timor-Leste P176687 FY22 $121 million Dili Water Supply Project BA The RRS team closely supported the project team, processing high-resolution historical climate data for Dili, estimating seasonal precipitation patterns and changes in return periods under future climate change, and evaluating climate impacts on the project cost-benefit analysis. This climate information served as direct input into the project engineering analysis and design considerations, and showed that the project was economically viable, even when considering the impacts of heavy precipitation and drought on service delivery. The RRS team also provided comments to various iterations of the project documents, including concept notes and draft Project Appraisal Document. Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change Appendix A. Full list of IDA19 pilot projects 47 Acronyms & abbreviations BEST battery energy storage technology CCKP Climate Change Knowledge Portal EFA economic and financial analysis FAO Food and Agriculture Organization of the United Nations FAST Flexible, Appropriate, Structured and Transparent (FAO standard) FCS fragile and conflict-affected settings FY financial year IDA19 19th Replenishment of International Development Association IDA20 20th Replenishment of International Development Association M&E monitoring and evaluation MDB multilateral development bank NA not applicable NR not rated PAD project appraisal document RiST risk stress test (tool) RRS Resilience Rating System SIDS Small Island Developing States All dollar ($) amounts are US dollars.  Resilience Rating System A Methodology for Building and Tracking Resilience to Climate Change