Independent Evaluation Group Validation of the Management Action Record 2024 © 2024 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org ATTRIBUTION Please cite the report as: World Bank. 2024. Independent Evaluation Group Validation of the Management Action Record 2024. Independent Evaluation Group. Washington, DC: World Bank. COVER PHOTO Shutterstock/Fahroni EDITING AND PRODUCTION Amanda O’Brien GRAPHIC DESIGN Rafaela Sarinho This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclu- sions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denomina- tions, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Independent Evaluation Group Validation of the Management Action Record 2024 October 29, 2024 Contents Abbreviations .................................................................................................................................................... vi Acknowledgments ..........................................................................................................................................vii Overview ............................................................................................................................................................. ix Report to the Board from the Committee on Development Effectiveness .............................xiv 1. Introduction ...................................................................................................................................................1 2. Validation Framework ................................................................................................................................3 3. Progress toward Achieving Intended Outcomes of Recommendations .................................7 Quality of Evidence .....................................................................................................................................8 Progress of Recommendations ...............................................................................................................9 Retire .............................................................................................................................................................. 10 Progress toward Outcomes .................................................................................................................... 13 Stay on Track ............................................................................................................................................... 13 Additional Follow-Up................................................................................................................................ 14 4. Progress on the Implementation of Recommendations, by Evaluation............................... 17 People ............................................................................................................................................................ 17 Planet ............................................................................................................................................................. 19 Infrastructure .............................................................................................................................................. 25 Digital ............................................................................................................................................................ 30 Prosperity..................................................................................................................................................... 33 Cross-Cutting .............................................................................................................................................. 41 Corporate .................................................................................................................................................... 47 5. Conclusions and Suggestions for Further Enhancement of the Management Action Record ................................................................................................................................................................ 52 References ........................................................................................................................................................ 54 Boxes Box 1.1. The Management Action Record and the Better Bank Initiatives .......................................2 Box 3.1. Pathways for Implementing Recommendations ......................................................................8 Box 3.2. Enhancements in Urban Development Operations Linked to Recommendations .....11 Box 3.3. Rapid Progress on Public Financial and Debt Management............................................. 14 iii Contents Figure Figure 3.1. Progress of Recommendations toward Outcomes .......................................................... 10 Tables Table O.1. Recommendations Proposed for Retirement ....................................................................... x Table 2.1. Five-Step Process for Conducting and Validating the Management Action Record, Fiscal Year 2024 ..................................................................................................................................................4 Table 2.2. Management Action Record Assessment Framework, Fiscal Year 2024 ......................5 Table 3.1. Independent Evaluation Group Assessment of Implementation Progress of Recommendations, by Fiscal Year of CODE Discussion ........................................................................ 7 Table 3.2. Recommendations Proposed for Retirement: Management Action Record 2024 .... 12 Table 3.3. Cross-Cutting Recommendations without Implementation Pathways and with Institution-Wide Implications....................................................................................................................... 16 Table 4.1. Undernutrition, 2022–26 ............................................................................................................ 17 Table 4.2. COVID-19 Health and Social Response, 2023–27 .............................................................. 18 Table 4.3. Natural Resource Degradation and Vulnerability, 2021–25 ............................................ 19 Table 4.4. Municipal Solid Waste Management, 2022–26.................................................................. 21 Table 4.5. Disaster Risk Reduction, 2023–27 .......................................................................................... 22 Table 4.6. Agrifood Economics, 2023–27 ................................................................................................ 23 Table 4.7. Urban Resilience, 2020–24 ....................................................................................................... 25 Table 4.8. Renewable Energy, 2021–25 .................................................................................................... 26 Table 4.9. Urban Spatial Growth, 2022–26 ............................................................................................. 27 Table 4.10. Energy Efficiency, 2023–27 ..................................................................................................... 28 Table 4.11. Mobilizing Technology for Development, 2021–25 ........................................................ 30 Table 4.12. Regional Integration, 2019–24 .............................................................................................. 33 Table 4.13. State-Owned Enterprises, 2021–25 ...................................................................................... 33 Table 4.14. Public Financial and Debt Management, 2021–25 ......................................................... 34 Table 4.15. Doing Business Report and Business Ready, 2022–26 ................................................... 35 Table 4.16. Sustainable Development Finance Policy, 2022–26 ....................................................... 36 Table 4.17. Low-Income Country Debt Sustainability Framework, 2023–27 ............................... 37 Table 4.18. Domestic Revenue Mobilization, 2023–27 ........................................................................ 38 Table 4.19. COVID-19 Economic Response, 2023–27 .......................................................................... 40 Table 4.20. Private Capital Mobilization, 2021–25 ................................................................................. 41 Table 4.21. World Bank Engagement in Situations of Conflict, 2022–26 ...................................... 42 Table 4.22. Private Investment in Fragile and Conflict-Affected Situations, 2022–26 .............. 43 iv Contents Table 4.23. International Finance Corporation Additionality in Middle-Income Countries, 2023–27 ............................................................................................................................................................. 44 Table 4.24. Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence, 2023–27 ............................................................................................................................................................. 46 Table 4.25. Citizen Engagement, 2019–23 .............................................................................................. 47 Table 4.26. Convening Power, 2020–24 ................................................................................................... 48 Table 4.27. Outcome Orientation at the Country Level, 2021–25 ................................................... 49 Table 4.28. Global Footprint Effectiveness, 2022–26 ........................................................................... 50 Appendix Appendix A. Progress on Recommendations ..................................................................................... 59 v Abbreviations AI artificial intelligence B-READY Business Ready CCDR Country Climate and Development Report CODE Committee on Development Effectiveness CPF Country Partnership Framework DRR disaster risk reduction DSA debt sustainability analysis DSEE demand-side energy efficiency DTT disruptive and transformative technology FCS fragile and conflict-affected situation FCV fragility, conflict, and violence FY fiscal year GEMS Geo-Enabling Initiative for Monitoring and Supervision GP Global Practice IBRD International Bank for Reconstruction and Development IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation MAR Management Action Record MIGA Multilateral Investment Guarantee Agency PPA performance and policy action SME small and medium enterprise SOE state-owned enterprise All dollar amounts are US dollars unless otherwise indicated. vi Acknowledgments This report was led by Stephen Porter under the supervision of Lily L. Chu. Arjun Kaushik and Rocio Manchado coproduced the report. Sabine Bernabè provided overall direction. Independent Evaluation Group contributors who provided critical inputs to this report include Anahit Aghumian, Ana Belen Barbeito, Elena Bardasi, Soniya Carvalho, Jenny Gold, Mitko Grigorov, Rasmus Heltberg, Patrick Hettinger, William Hurlbut, Stephen Hutton, Ramachandra Jammi, Jennifer Keller, Lauren Kelly, Marialisa Motta, Maria-Elena Pinglo, Estelle Raimondo, Heinz Rudolph, Andrew Stone, Mercedes Vellez, and Stephan Wegner. Zorayda Opana provided administrative support. The collaborative approach between the Independent Evaluation Group and World Bank Group management led to rich discussions and concrete proposals to enhance the quality of the Management Action Record process and product. The Independent Evaluation Group is grateful to Operations Policy and Country Services, International Finance Corporation, and Multilateral Investment Guarantee Agency colleagues, including Espen Bernton, Ellinore Carroll, Atul B. Deshpande, Matteo Goffredo, Ariya Hagh, Hiroyuki Hatashima, Liane Asta Lohde, Stephanie Lukins, Tanushri Majumdar, Eduardo Maldonado, Lisandro Martin, Maurya West Meiers, Yeukai Mudzi, Ayesha Muzaffar, Vlajko Senic, Mili Varughese, and Peixuan Zhou, for their close collaboration. We also thank the more than 150 technical focal points and respective managers for their commitment to providing the expanded evidence base and enhanced engagement process. vii Overview This Independent Evaluation Group (IEG) report validates the World Bank Group management’s report Better Results through Learning and Adaptation for a Better World Bank Group: The FY24 Management Action Record for the period July 2023 to June 2024. The purpose of the Management Action Record (MAR) system is to support accountability, learning, and adaptation for the implementation of recommendations made by IEG evaluations and previously reviewed by the Board of Executive Directors’ Committee on Development Effectiveness. This validation assesses the evidence for all 28 IEG evaluations included in the MAR—that is, all evaluations reviewed by the Committee on Development Effectiveness between fiscal year (FY)19 and FY23. The MAR validation finds that the Bank Group made swift and significant progress on some recommendations and closed out older ones, and in some key areas relevant to Better Bank initiatives, focused follow-up is needed. The FY24 MAR report tracks progress on 77 recommendations and assesses progress toward the achievement of their intended outcomes. Of these 77 recommendations, IEG proposes to retire 18 new and old recommendations, almost all of which demonstrate a change of direction. Another 24 recommendations show emerging evidence of progress and could move toward retirement in coming years. A group of 27 recommendations have defined pathways for implementation and are generally within the first two years of implementation with limited evidence of progress. Movement toward achieving their outcomes is likely if focus is maintained. Progress is limited or constrained for 8 recommendations for which pathways of follow-up are unclear. This is concerning because 3 of these require senior management engagement and 5 others are linked to Better Bank initiatives. Retire IEG proposes to retire 18 recommendations. Of these, IEG assessed 17 recommendations as change of direction, with 1 recommendation retiring with emerging evidence after being tracked for four cycles. For 2 recommendations, IEG proposes retirement even though management did not, as progress is assessed as being higher than reported. Compared with the 2023 MAR, the Bank Group is more consistently bridging the gap between delivering outputs and achieving the expected outcomes of the recommendations. Overwhelmingly, the recommendations proposed for retirement demonstrate a change of direction (table O.1). Seventeen of the 18 retiring recommendations have demonstrated a change of direction. Of these, 9 recommendations are in their first or second year of MAR tracking, showing that rapid progress is possible. Eight recommendations are in their third to fifth year of tracking. The remaining ix Overview recommendation that did not achieve a change of direction is from The World’s Bank: An Evaluation of the World Bank Group’s Global Convening (World Bank 2020d) and is validated for retirement because of elapsed time. The recommendation relates to internal systems to support major convening initiatives and is likely to continue to progress given demonstrated changes linked to the evolution agenda of the Bank Group. IEG disagrees with management’s proposal to retire two recommendations. Management proposes to retire the recommendation on crime and violence in urban contexts (recommendation 3) from Building Urban Resilience: An Evaluation of the World Bank Group’s Evolving Experience (2007–17) (World Bank 2019a) based on elapsed time, although progress is constrained. IEG requests that management continues to provide reporting related to the outcomes of the recommendation based on its follow-up to Mid- Term Review of the World Bank Group Strategy for Fragility, Conflict, and Violence (2020–25) (World Bank 2023h). IEG also disagrees with the retirement of the recommendation on expanding private capital mobilization platforms (recommendation 2) from World Bank Group Approaches to Mobilize Private Capital for Development (World Bank 2020e). Management presented only emerging evidence of change in its third year of MAR tracking, and additional reporting is needed. IEG agrees with the extension of one recommendation. IEG agrees with management’s proposal to extend by a year the tracking of one recommendation on citizen engagement. This is its second year of extended tracking given insufficient follow-up, and action on this recommendation is expected through the Operational Dashboard linked to the Corporate Scorecard. Table O.1. Recommendations Proposed for Retirement Pillar IEG Evaluation Recommendations to Retire (n = 18) Planet Transitioning to a Circular Economy: An Evaluation Recommendations 1, 2, and 3a of the World Bank Group’s Support for Municipal IEG and management agree CD for all. Solid Waste Management (2010–20) Reducing Disaster Risks from Natural Hazards: An Recommendation 1a Evaluation of the World Bank’s Support, Fiscal Years IEG and management agree CD. 2010–20 Infrastructure Building Urban Resilience: An Evaluation of the Recommendation 2 World Bank Group’s Evolving Experience (2007–17) IEG and management agree CD. Renewable Energy: Evaluation of the World Bank Recommendation 2a Group’s Support for Electricity Supply from IEG and management agree CD. Renewable Energy Resources, 2000–2017 Managing Urban Spatial Growth: World Bank Recommendation 1a Support to Land Administration, Planning, and IEG proposes CD. Development x Overview Pillar IEG Evaluation Recommendations to Retire (n = 18) Prosperity Two to Tango: An Evaluation of World Bank Group Recommendation 5 Support to Fostering Regional Integration IEG and management agree CD. World Bank Support for Public Financial and Debt Recommendations 1 and 2a Management in IDA-Eligible Countries IEG and management agree CD. The International Development Association’s Recommendations 1 and 3a Sustainable Development Finance Policy: An Early- IEG and management agree CD. Stage Evaluation Cross-cutting World Bank Group Approaches to Mobilize Private Recommendation 3a Capital for Development IEG and management agree CD. World Bank Engagement in Situations of Conflict: Recommendations 1 and 2a An Evaluation of FY10–20 Experience Recommendation 1: IEG and management agree CD. Recommendation 2: IEG proposes CD. Corporate The World’s Bank: An Evaluation of the World Bank Recommendations 1, 2, and 3 Group’s Global Convening Recommendations 1 and 3: IEG and management agree CD. Recommendation 2: IEG and management agree with retirement based on time. Sources: Independent Evaluation Group; World Bank 2019a, 2019b, 2020a, 2020d, 2020e, 2021b, 2021c, 2021f, 2021g, 2022e, 2022g. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; FY = fiscal year; IDA = International Development Association; IEG = Independent Evaluation Group. a. Retire early. Progress toward Outcomes Twenty-four recommendations are moving toward achieving a change of direction. All these recommendations show emerging evidence of progress. This means that there is evidence of outputs being delivered of sufficient quality, and pathways for change are established for each recommendation. These outputs and pathways enable consistent dialogue with IEG, which helps move recommendations toward retirement. Approximately half of the recommendations with good progress are in their first year of implementation, showing that swift progress can be made. Stay on Track Continued focus is needed for 27 recommendations with defined pathways for implementation that mainly have limited evidence of progress. Of these, 18 are in their first year of tracking and demonstrate limited evidence of progress but can show progress toward desired outcomes in years to come. For these recommendations, there is evidence that initial activities are being undertaken with a process in place. The remaining 9 recommendations in this group are in their second and third years of tracking and either need to provide improved evidence or have yet to demonstrate the delivery of outputs. Among these, a single recommendation has progress constrained xi Overview but has a pathway to a change of direction with the planned publication of the follow-up report to Doing Business. Additional Follow-Up Additional follow-up is needed for eight important recommendations. The level of progress on five recommendations in the later years of tracking has declined compared with 2023. Meanwhile, three recommendations that have institution-wide implications and are in their second year of reporting have yet to define pathways. This means they may lag in implementation in the coming years. Five recommendations demonstrating lower levels of progress in the FY24 MAR are relevant to Better Bank initiatives. The World Bank’s follow-up on recommendations focused on inclusion and vulnerability is fragile. Although the World Bank agreed to act on recommendations on monitoring citizen engagement, addressing urban crime and violence, and addressing natural resource degradation and vulnerability, it did not show progress in follow-up in 2024. It is unclear how two recommendations will achieve a change of direction focused on the Bank Group’s outcome orientation at the country level, and management’s report provided limited evidence. Consequently, these recommendations moved from emerging evidence to either limited evidence or progress constrained. These five recommendations are highly relevant to Better Bank initiatives, such as the Corporate Scorecard and Global Challenge Programs. Three recommendations with institution-wide implications are at risk of lagging and would benefit from increased senior management direction. Implementation of recommendations can stall when aspects of them need coordination among multiple vice presidential units or Bank Group institutions. Pathways need to be defined that integrate the efforts of multiple units. Senior management support can be helpful in such situations by weighing in on resources and priorities. Issues are arising for three World Bank–focused recommendations: • For Enhancing the Effectiveness of the World Bank’s Global Footprint (World Bank 2022c), recommendation 1 shows limited evidence of progress as actions have not been defined to measure the outcomes that the staff decentralization process aims to achieve. • For Managing Urban Spatial Growth: World Bank Support to Land Administration, Planning, and Development (World Bank 2021c), recommendation 3 shows limited evidence of progress. No evidence has been provided on how broadly the World Bank will strengthen its data collection protocols on geospatial information and increase the use of technologies capturing land value. The definition of this approach is beyond the responsibility of the Urban, Disaster Risk Management, xii Overview Resilience, and Land Global Practice and the Geo-Enabling Initiative for Monitoring and Supervision. • For World Bank Engagement in Situations of Conflict: An Evaluation of FY10–20 Experience (World Bank 2021f), recommendation 3 shows limited progress. Evidence has yet to be provided on how the World Bank will address factors that dissuade engagement in conflict-affected areas specifically, rather than fragility, conflict, and violence in general. Enhancement of the Management Action Record System Maintaining recent positive changes in the implementation of the MAR is important. Defining and then implementing pathways that reconcile IEG’s findings and management’s capability to make changes has enabled swift implementation and closing out of older recommendations. Progress can stall when pathways are not defined or when they need to be adapted so that change does not tail off between activities and the consolidation of outputs. Over the past two MAR cycles, progress has also improved in areas where IEG and management invested in dialogue on possibilities for implementing recommendations. As IEG recommendations are relevant to Better Bank initiatives, maintaining these improvements in the MAR system needs to be prioritized. IEG suggests further efforts to appoint champions who can lead progress on recommendations that are lagging. The Bank Group has made progress on identifying and working with champions. The appointment of a management champion can assist with setting out pathways for recommendation implementation, engaging senior management, convening teams on follow-up, and reporting their progress. Engaging champions in this way would be a continuation of their current role in reviewing progress on recommendations and approving MAR submissions. IEG is available to jointly review pathways to help accelerate implementation with champions and the associated teams. xiii Report to the Board from the Committee on Development Effectiveness The Committee on Development Effectiveness met to consider the report Better Results through Learning and Adaptation for a Better World Bank Group: The FY24 Management Action Record and Independent Evaluation Group Validation of the Management Action Record 2024. The committee welcomed the World Bank Group Management Action Record (MAR) report and the validation report of the Independent Evaluation Group (IEG) for FY 2024. Members appreciated IEG and management’s strong collaboration, acknowledged the MAR’s enhanced quality, and commended management for the significant progress it had made in implementing IEG recommendations. They were pleased to note that management and IEG assessments brought about similar results in evaluating progress on the recommendations. Members highlighted the importance of the MAR in support of the Bank Group Evolution Roadmap and the Global Challenge Programs. While acknowledging management’s concerns about the increasing accumulation of recommendations, members called on management to further strengthen implementation of IEG’s recommendations to bring better outcomes and impacts. Members agreed with IEG’s observation that sustaining recent positive changes, such as defining and implementing pathways that reconcile IEG findings and management’s capability to make change in the implementation of the MAR, is important. Members noted the slow progress and the lag on recommendations connected to outcome orientation at the country level. They highlighted the need to better define ex ante high-level outcomes and assure client country’s ownership with respect to the progress toward these outcomes. The committee inquired about the automatic retirement principle and the fate of automatically retired recommendations that had not achieved a change of direction (CD) level. Members emphasized the need for clearer criteria and processes for prioritization and sharpening of the implementation focus. Management highlighted that they continue to act on recommendations even after they have been retired. IEG added that they plan to engage with management separately over the next year in reviewing the criteria for automatic retirement. Members expressed support for management’s initiative to further emphasize the appointment of “champions” to ensure effective and efficient implementation of IEG recommendations introduced in FY23. They encouraged management to take on IEG’s suggestion to appoint even more manager- and director-level champions, particularly xiv Report to the Board from the Committee on Development Effectiveness for recommendations that are lagging. They agreed to retire 18 recommendations and to continue monitoring the recommendation on citizen engagement (extended already last year) for an additional year given insufficient follow-up. Members were also pleased to hear that management had agreed with IEG’s recommendation to not retire 2 recommendations that management had originally recommended to retire given that implementation progress was assessed by IEG as lower, as well as to retire 2 recommendations that management did not propose originally because their implementation progress was assessed by IEG as higher than reported. xv 1. Introduction 1.1 This Independent Evaluation Group (IEG) report validates the findings of the World Bank Group’s report Better Results through Learning and Adaptation for a Better World Bank Group: The FY24 Management Action Record for the period July 2023 to June 2024. The purpose of the Management Action Record (MAR) is to support accountability, learning, and adaptation for the Bank Group’s implementation of recommendations made by IEG evaluations and previously reviewed by the Board of Executive Directors’ Committee on Development Effectiveness (CODE). As such, this report is not an evaluation and does not include IEG’s evidence and perspectives on the development issues covered in the evaluations considered by the MAR. 1.2 CODE is the principal audience for this report. In its validation, IEG adopts a forward-looking approach to assessing progress against the intended outcomes of evaluations, as expressed by their recommendations. This document provides CODE with an assessment of the Bank Group’s progress on implementing the agreed recommendations and IEG’s position on the evidence of their implementation, advancement, and retirement. In doing this, the validation informs CODE of clusters of recommendations that are advancing, lagging, or not being followed up. This year, the MAR provides information relevant to the Bank Group’s Better Bank initiatives (box 1.1). The teams and managers in the Bank Group who implement changes informed by IEG recommendations can also use this validation to help maintain or adjust the implementation of recommendations. 1.3 This validation is structured as follows: (i) the framework for the MAR and validation; (ii) progress toward intended outcomes of recommendations; (iii) the record of agreements and disagreements on progress for recommendations within the five new operating verticals—People, Planet, Infrastructure, Digital, and Prosperity—and cross- cutting and corporate areas; and (iv) conclusions and suggestions on how to continue to improve MAR effectiveness. 1 Chapter 1 Introduction Box 1.1. The Management Action Record and the Better Bank Initiatives The Management Action Record (MAR) contributes to tracking of the World Bank Group’s evolution toward becoming a Better Bank. Through the Better Bank initiatives, the Bank Group implements changes in how it operates. The process was initiated by the Board of Governors at the Annual Meetings in 2022 as part of the evolution agenda with the aim to “assist countries to navigate intertwined crises, tackle global challenges, and achieve the vision of a world free of poverty on a livable planet.” Independent Evaluation Group (IEG) evaluations have been cited in documents that define aspects of the initiatives, and all recommendations can be linked directly or indirectly to them. For example, in addition to the areas discussed in this box, recommendations are relevant to initiatives related to the guarantee platform, operational efficiency and effectiveness, and the Environmental and Social Framework. Management’s MAR report often cites Better Bank initiatives in relation to the implementation of recommendations. Hence, the MAR 2024 offers insights into how the Bank Group, informed by IEG evidence, is learning and adapting in four areas: • The Bank Group’s Knowledge Compact seeks to “empower all…clients—public and private—by systematically making the latest development knowledge available to respond more effectively to increasingly complex development challenges” (World Bank 2024c, iv). Recommendations from The World’s Bank: An Evaluation of the World Bank Group’s Global Convening on knowledge partnerships and Enhancing the Effectiveness of the World Bank’s Global Footprint evaluation on knowledge flows are linked by management to the implementation of the Knowledge Compact. Progress on many IEG evaluations is relevant to clients’ knowledge on public finance, debt sustainability, municipal solid waste management, and private capital mobilization, among others. • The World Bank Group Scorecard is a strategic management tool that seeks to drive action for results. In the Board paper on the Scorecard, Bank Group management cites multiple IEG products, such as Results and Performance of the World Bank Group 2021 and recommendations from The World Bank Group Outcome Orientation at the Country Level. In evidence provided by management in the MAR, the Scorecard is cited as relevant to the follow-up of recommendations in 10 evaluations. • The Crisis Preparedness and Response Toolkit seeks to fill gaps based on lessons learned from previous crisis responses and expand the tools available to countries to ensure protection in times of crisis. IEG evaluations provide recommendations relevant to ongoing development and implementation of the tool kit; for example, from The World Bank’s Early Support to Addressing COVID-19: Health and Social Response—An Early-Stage Evaluation; The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response, April 2020–June 2021: An Early-Stage Evaluation; and Reducing Disaster Risks from Natural Hazards: An Evaluation of the World Bank’s Support, Fiscal Years 2010–20. In evidence provided by management, the Crisis Preparedness and Response Toolkit is cited as relevant to the follow-up of recommendations in five evaluations. • The Global Challenge Programs are intended to provide clients with additional support to address development challenges with greater speed, scale, and impact. Approach Papers have been defined for six such programs. IEG recommendations from various evaluations are relevant for common issues these programs need to address, such as vulnerability and inclusion. In evidence provided by management, the Global Challenge Programs are cited as relevant to the follow-up of recommendations in 10 evaluations. Sources: Independent Evaluation Group; World Bank 2023f, 2024a, 2024c, 2024d, 2024e. Note: See appendix A for a full list of evaluations and recommendations covered by the Management Action Record. 2 2. Validation Framework 2.1 IEG and Bank Group management continued to implement the MAR assessment framework developed over the past four years, with some additions. The framework uses transparent criteria and definitions to enable comparison of progress among evaluations and over time. The MAR framework has three elements: • A five-step process for conducting and validating the MAR (table 2.1); • Categories and criteria applied in the review of evidence to assess progress toward the outcomes of recommendations; and • Recommendations for proposed retirements based on set criteria. 2.2 Management and IEG jointly adapted the MAR framework in fiscal year (FY)24 to prioritize engagement on recommendations, better define pathways for their implementation, and further engage management champions. Applying these three elements in reference to prioritized recommendations helps produce a discussion on overall progress against recommendations. 3 Chapter 2 Validation Framework Table 2.1. Five-Step Process for Conducting and Validating the Management Action Record, Fiscal Year 2024 Process Step Key Activities (i) Upstream • IEG provided descriptions for levels of progress for each recommendation. engagement • IEG and management undertook initial discussion of progress toward implementing recommendations among 22 IEG evaluators and more than 150 focal points— technical staff from operations and corporate units in charge of providing implementation evidence in their areas of work. • IEG and management jointly discussed and prioritized engagement on recommendations. IEG focused on those that were close to retirement, lagging in implementation, or linked to Better Bank initiatives, and those where agreement needed to be refined on the outcomes of implementation. (ii) Evidence collection by • Focal points undertook new analysis or collated evidence on implementation using technical focal points a template defined by management. For each evaluation, clearances ranged from managers up to vice presidents, depending on the evaluation. • IEG and management discussed technical evidence at the working level. For each recommendation, IEG and management reviewed whether a pathway was defined for achieving the outcome of recommendations. (iii) MAR report • Management produced the MAR report by assessing progress and proposing the retirement of recommendations based on the inputs they received from the technical focal points. • IEG provided informal feedback on the content and the technical evidence for all recommendations. (iv) IEG MAR validation • IEG validated management’s MAR report through a structured review of previous report and CODE review MAR submissions and progress reported by management this year against the intended outcomes of recommendations. • Validation was shared with CODE for discussion in conjunction with the report. (v) After-action review • Management and IEG reflect on the previous cycle and define areas for improvement. Source: Independent Evaluation Group. Note: CODE = Committee on Development Effectiveness; IEG = Independent Evaluation Group; MAR = Management Action Record. 2.3 As was the case in 2023, progress was assessed against four categories with defined criteria (table 2.2). Results chain thinking was applied to these categories to better assess progress toward the intended outcomes of IEG recommendations. The “progress constrained” category was applied judiciously to identify recommendations that require intensified support and to alert management and CODE to implementation delays. 4 Chapter 2 Validation Framework Table 2.2. Management Action Record Assessment Framework, Fiscal Year 2024 Category Criteria Progress constrained • No progress. • Valid evidence on progress is unavailable or inadequate. • Reporting in years three or four continues to show insufficient evidence of progress. Limited evidence: activities • Activities delivered and knowledge generated are linked to the recommendation. conducted • Skills developed are linked to the recommendation. • Limited new evidence of progress since the previous Management Action Record. Emerging evidence: • Evidence of developed capacities or application or use of outputs—such as processes, demonstrating application information technology systems, and guidance implemented—is supported by limited of outputs  examples. • Early evidence of changes in behavior. Change of direction: • Meaningful change in behavior in the intended outcome of the recommendation that is demonstrating systematic likely to be sustained. behavior changes • Implemented systems changes—for example, incentives, financing mechanisms, processing, and new standards being applied across the relevant portfolio. Source: Independent Evaluation Group. 2.4 IEG and management applied four criteria to propose recommendations for retirement. Retirements are always subject to CODE’s guidance. Such guidance is especially important when IEG and management disagree on whether to retire or continue tracking a recommendation. Recommendations can be retired, or exited, from MAR reporting when they meet any one of the four criteria: • Time. All recommendations are considered for retirement in their fifth FY. Reporting usually takes place over four years, starting in the fiscal year after an IEG evaluation was discussed by CODE. Management can propose the extension of recommendations for additional years of reporting. IEG can propose additional analytic exercises or additional reporting within the MAR. • Change of direction. Management has achieved either a change in behavior or systemic change for the intended outcome of the recommendation. • Change in external context. The outcome of the recommendation is no longer considered relevant—for example, through force majeure or a refinement of global development priorities requiring a change in Bank Group strategy. • Change in internal context. Management can make no further progress on, or no longer agrees with, the intended outcome of a recommendation, or IEG has conducted a fresh evaluation that supersedes the earlier recommendation. 2.5 Bringing together three elements helps in considering the overall progress of recommendations. To provide an accessible view of the broad progress on 5 Chapter 2 Validation Framework recommendations and those needing attention, we cluster recommendations based on (i) their level of progress, (ii) their year of tracking, and (iii) whether there is a pathway in place to achieve the outcome. These clusters mainly align with the level of progress. In this year’s MAR, we discuss four such clusters, from those being retired to those where there are constraints—namely, retire, progress toward outcomes, stay on track, and additional follow-up. 2.6 The MAR’s approach has some limitations. The consistent application of the jointly agreed elements outlined in this chapter over a period of years has helped make the MAR efficient as both management and IEG understand the process. However, some challenges emerge when implementing the MAR framework: • Coordination on recommendations across units. Most recommendations require coordination across the Bank Group to instigate change. If these units do not work together regularly or there is no defined lead unit, it can take time to organize roles and responsibilities. Further refinement of the approach to champions at the management level for cross-cutting recommendations may help coordinate the implementation of recommendations and subsequent reporting of evidence. Focal points can be determined during the evaluation as an evaluation moves from its conclusion to the MAR phase. IEG and management will continue to identify any gaps in focal point and champion coverage. • Use of mainly secondary evidence. Technical focal points collate and summarize existing evidence and may conduct fresh analysis, such as portfolio reviews in the MAR. This sometimes limits the depth of evidence but enables timely completion of the MAR. • Turnover of technical focal points. The MAR generally provides better-quality evidence when focal points have experience with the process, have authority to make decisions based on overall authorization, and understand the requirements of recommendations. Focal points, however, can change between years; consequently, knowledge of the process and progress on recommendations need to be rebuilt. This was less of an issue in 2024 and is mitigated by maintaining the knowledge base on recommendations from year to year. 6 3. Progress toward Achieving Intended Outcomes of Recommendations 3.1 The Bank Group made swift and significant progress on some recommendations and closed out older ones, and in some key areas relevant to Better Bank initiatives, focused follow-up is needed. The FY24 MAR tracks progress on 77 recommendations and provides an assessment of progress toward the achievement of their intended outcomes. Progress on these recommendations can be summarized in four clusters. Of the 77 recommendations, IEG proposes to retire 18 both new and old recommendations, with 17 demonstrating a change of direction and 1 having emerging evidence. Another 24 recommendations show emerging evidence of progress and could move toward retirement in the coming years. A group of 27 recommendations have defined pathways for implementation and are generally within the first two years of implementation with limited evidence of progress. Movement toward their outcomes is likely if they are kept on track. Progress is limited or constrained for 8 recommendations for which pathways of follow-up are unclear. This is concerning as 3 of these require senior management engagement and 5 others are linked to Better Bank initiatives. 3.2 This year, the validation provides assurance (based on the levels of progress) that a wide range of recommendations show progress toward a change of direction. Of the 77 recommendations, 42 demonstrate emerging evidence or a change of direction (table 3.1). Importantly, 9 recommendations are proposed for retirement in their first or second year of MAR tracking, showing that rapid progress is possible. Of the 7 older recommendations from FY19 and from FY20, 4 recommendations show a change of direction. Of the 32 recommendations new to the MAR this year, 14 show emerging evidence of a change of direction, whereas 18 have limited evidence of progress but have initiated activities, with pathways for implementation in place to some degree. Table 3.1. Independent Evaluation Group Assessment of Implementation Progress of Recommendations, by Fiscal Year of CODE Discussion FY Discussed by Change of Emerging Limited Progress Total CODE Direction Evidence Evidence Constrained FY19 1 — — 1 2 FY20 3 1 — 1 5 FY21 4 7 3 — 14 FY22 8 4 11 1 24 FY23 1 13 18 — 32 Total 17 25 32 3 77 Source: Independent Evaluation Group. Note: CODE = Committee on Development Effectiveness; FY = fiscal year; — = no recommendations show this level of progress. 7 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations Quality of Evidence 3.3 Management’s MAR reporting for FY24 builds on the FY23 MAR report, generally offering a good standard of evidence on almost all recommendations and candid discussion of progress. Management has continued to refine portfolio review techniques to identify changes linked to recommendations. For example, the reporting on nutrition used an artificial intelligence (AI)-augmented portfolio review tool to track progress on recommendations. Management also used corporate information systems to quantify progress for many recommendations. Many teams preserved consistency in their MAR reporting against defined pathways (box 3.1). Reporting on evolving process changes over several years helps provide assurance that progress is more likely to be sustained. In submissions to IEG, management increasingly provides electronic links to knowledge products to show evidence of progress. In addition, where there are limitations in following up on recommendations, management often candidly acknowledges and names those constraints that are outside the control of the responsible units. Box 3.1. Pathways for Implementing Recommendations Pathways for implementing recommendations help provide good evidence and support swift implementation toward retirement. Management outlines the following five pathways that describe how teams typically progress. We observe elements of different pathways in recommendations showing progress toward a change of direction. • Leadership direction and momentum. Top leadership defines the direction of change and provides associated resources that assist with the implementation of recommendations. For example, rapid progress has been made toward two recommendations that arose from the Independent Evaluation Group’s early-stage evaluations of the COVID-19 response. The recommendations for the COVID-19 health and social response (recommendation 4; World Bank 2022h) and the COVID-19 economic response (recommendation 1; World Bank 2023i) have shown emerging evidence of progress in their first year of tracking. • Step-by-step. This pathway involves predictable steps that show incremental progress each year, such as preparing guidance, applying guidance, and showing quality application of guidance. The implementation of regional integration (recommendation 2; World Bank 2019b) followed a step-by-step process and incrementally showed the incorporation of indicators to assess spillover effects. This recommendation is set to be retired based on a change of direction. • Ride the wave. This pathway occurs when recommendations meet considerable institutional momentum behind an agenda. Two recommendations from the evaluation on World Bank engagement in situations of conflict (recommendations 1 and 2; World Bank 2021f) show elements of riding a wave, relating to conflict analysis. Conflict analysis is an area where the World Bank Group’s strategy for fragility, conflict, and violence sought change. Both recommendations are identified for early retirement as a change of direction. 8 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations • Being opportunistic. In this pathway, teams hook themselves onto a moving agenda to make progress on the recommendation. The progress on the evaluation of incorporating disaster risk reduction activities in regions and sectors with coverage gaps (recommendation 1; World Bank 2022e) looks to have elements of taking available opportunities in new operations. This recommendation shows a change of direction in the first year of tracking. • Cross-unit collaboration. Units work together across organization boundaries to achieve progress on a recommendation. Progress on cross–Practice Group (the World Bank) and cross–industry group (the International Finance Corporation) approaches for the evaluation on demand-side energy efficiency (recommendation 4; World Bank 2023j) provides an example of collaboration in operations driving progress. This recommendation shows emerging evidence of progress in the first year of tracking. In contrast, a lack of a defined pathway can be linked to lower-quality or more limited evidence of progress. For example, for the evaluation on building urban resilience (recommendation 3; World Bank 2019a), no process was established to respond to crime and violence risks. In addition, in responding to the evaluation on engaging citizens for better development results (recommendation 3; World Bank 2018), the pathway for strengthening of monitoring has yet to be followed through. Sources: Independent Evaluation Group; World Bank 2018, 2019a, 2019b, 2021f, 2022e, 2022h, 2023i, 2023j. 3.4 IEG and management have similar assessments of progress on the recommendations. For 82 percent of the recommendations, IEG and management assess progress at the same level (which is in line with last year). In its reporting, management often candidly recognizes issues with progress. For example, management assessed progress as limited or constrained for 36 percent of recommendations in FY24. This is higher than in 2023 (which was 24 percent), in part because of the large number of recommendations entering the MAR this year; most new recommendations are expected to show limited progress. For 18 percent of the recommendations, IEG and management assessed progress differently. For 14 percent of the recommendations, IEG determined evidence of a lower level of progress. For 4 percent of the recommendations, IEG saw evidence of more progress than assessed by management. Progress of Recommendations 3.5 This section summarizes the progress being made in four clusters of the recommendations. After reviewing the 77 recommendations, we sorted them into four clusters considering their level of progress, the year of their implementation, and the clarity of their pathway (figure 3.1). These clusters—retire, progress toward outcomes, stay on track, and additional follow-up—mainly link to one of the four levels of progress represented in table 3.1, with some exceptions noted. The retire cluster includes all recommendations IEG validates for retirement. The progress toward outcomes includes all recommendations that demonstrate outputs and have defined pathways to achieving their outcome. The stay on track category includes all recommendations that have yet to 9 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations demonstrate outputs but have defined a pathway toward achieving the outcome. The additional follow-up category includes recommendations that need CODE and management attention because they do not show outputs, are late in their MAR cycle, or have no defined pathway to achieve outcomes. Figure 3.1. Progress of Recommendations toward Outcomes Additional follow-up, 8 Retire, 18 Stay on track, 27 Progress toward outcomes, 24 Source: Independent Evaluation Group. Note: This figure separates the progress on recommendations (N = 77) into four clusters: retire (n =18; 17 change of direction, 1 emerging evidence); progress toward outcomes (n = 24; 24 emerging evidence); stay on track (n = 27; 26 limited evidence, 1 progress constrained); and additional follow-up (n = 8; 6 limited evidence, 2 progress constrained). Retire 3.6 The recommendations proposed for retirement overwhelmingly demonstrate a change of direction. IEG proposes retiring 18 recommendations (table 3.2). Among these, 17 demonstrated a change of direction. This year, 12 advanced from emerging evidence to change of direction. Four recommendations skipped over levels of progress: 2 from limited evidence to change of direction, 1 from progress constrained to change of direction, and 1 recommendation reached a change of direction in its first year. Rapid progress is shown with 9 recommendations demonstrating a change of direction in their first or second year of MAR tracking. Of the 17 proposed retirements, 2 are additional to management’s suggestions as IEG assessed more progress to have been achieved on urban spatial growth (recommendation 1; World Bank 2021c) and engagement in situations of conflict (recommendation 2; World Bank 2021f). Compared with the FY23 MAR, this progress shows that the Bank Group is more consistently bridging the gap between delivering outputs and reaching the outcomes of recommendations. The bridging of the gap is shown in urban development, where 5 of the 8 recommendations in this area showed a change of direction this year (box 3.2). 10 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations Box 3.2. Enhancements in Urban Development Operations Linked to Recommendations The World Bank Group has shown consolidated change in its approach to urban development through five recommendations in three evaluations on enhancing practices in municipal solid waste (World Bank 2022g), the incorporation of resilience characteristics in urban projects (World Bank 2019a), and urban spatial growth (World Bank 2021c). Each recommendation made progress along a pathway where improvements in diagnostics or partnerships supported enhanced urban operations. For example: • Reinforcing actions were undertaken for three recommendations on municipal solid waste that helped them to progress simultaneously. Constraints in demand were identified, engagement in international partnerships and platforms was enhanced, and the portfolio of operations increased in size. As a result of these actions, all three recommendations for the evaluation are proposed for retirement. • Incremental and increasing adoption of resilience characteristics in operations was shown over three rounds of Management Action Record reporting. Over three years, resilience characteristics have been strengthened in operations through enhanced design standards, risk-informed cost-benefit analysis, and city and interjurisdictional coordination. • An updated approach to applying urban spatial growth frameworks supports a change in the design and delivery of relevant lending and analytic work. This year, three existing frameworks have together informed the design of operations —a report to the Group of 20 on inclusive cities that was endorsed in its 2023 communiqué and Country Climate and Development Reports that inform country engagements. Sources: Independent Evaluation Group; World Bank 2019a, 2021c, 2022g, 2023e. 3.7 One recommendation from The World’s Bank: An Evaluation of the World Bank Group’s Global Convening (World Bank 2020d) is validated for retirement due to time. The recommendation (recommendation 2) relates to internal systems to support major convening initiatives and is likely to continue to progress given demonstrated changes linked to the evolution agenda in the Bank Group. 3.8 IEG agrees with the extension of one recommendation. IEG agrees with management’s proposal to extend by a year the tracking of one recommendation on citizen engagement. This is its second year of extended tracking given insufficient follow-up on defined actions and expected action on this indicator through the Operational Dashboard linked to the Corporate Scorecard. 3.9 IEG does not agree with management’s proposal to retire two recommendations. Management proposes to retire the recommendation on crime and violence in urban contexts from Building Urban Resilience: An Evaluation of the World Bank Group’s Evolving Experience (2007–17) (World Bank 2019a) based on time, although progress is constrained. IEG requests that management continues to provide reporting related to the outcomes of the recommendation based on its follow-up to Mid-Term Review of the World Bank Group Strategy for Fragility, Conflict, and Violence (2020–25) (World Bank 2023h). IEG also disagrees 11 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations with the retirement of the recommendation on expanding private capital mobilization platforms because the evidence presented provides only emerging evidence of change for approaches to guarantees and certain disaster risk management products. Table 3.2. Recommendations Proposed for Retirement: Management Action Record 2024 Pillar IEG Evaluation Recommendations to Retire (n = 18) Planet Transitioning to a Circular Economy: An Evaluation of Recommendations 1, 2, and 3a the World Bank Group’s Support for Municipal Solid IEG and management agree CD for all. Waste Management (2010–20) Reducing Disaster Risks from Natural Hazards: An Recommendation 1a Evaluation of the World Bank’s Support, Fiscal Years IEG and management agree CD. 2010–20 Infrastructure Building Urban Resilience: An Evaluation of the World Recommendation 2 Bank Group’s Evolving Experience (2007–17) IEG and management agree CD. Renewable Energy: Evaluation of the World Bank Recommendation 2a Group’s Support for Electricity Supply from Renewable IEG and management agree CD. Energy Resources, 2000–2017 Managing Urban Spatial Growth: World Bank Support Recommendation 1a to Land Administration, Planning, and Development IEG proposes CD. Prosperity Two to Tango: An Evaluation of World Bank Group Recommendation 5 Support to Fostering Regional Integration IEG and management agree CD. World Bank Support for Public Financial and Debt Recommendations 1 and 2a Management in IDA-Eligible Countries IEG and management agree CD. The International Development Association’s Recommendations 1 and 3a Sustainable Development Finance Policy: An Early- IEG and management agree CD. Stage Evaluation Cross-cutting World Bank Group Approaches to Mobilize Private Recommendation 3a Capital for Development IEG and management agree CD. World Bank Engagement in Situations of Conflict: An Recommendations 1 and 2a Evaluation of FY10–20 Experience Recommendation 1: IEG and management agree CD. Recommendation 2: IEG proposes CD. Corporate The World’s Bank: An Evaluation of the World Bank Recommendations 1, 2, and 3 Group’s Global Convening Recommendations 1 and 3: IEG and management agree CD. Recommendation 2: IEG and management agree with retirement based on time. Sources: Independent Evaluation Group; World Bank 2019a, 2019b, 2020a, 2020d, 2020e, 2021b, 2021c, 2021f, 2021g, 2022e, 2022g. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; FY = fiscal year; IDA = International Development Association; IEG = Independent Evaluation Group. a. Retire early. 12 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations Progress toward Outcomes 3.10 Many recommendations show good progress this year. Of the 77 recommendations, just under one-third (24 recommendations) demonstrate emerging evidence of progress—a sign that outputs are delivered with sufficient quality for progress toward outcomes of recommendations. Of the 24 recommendations, 17 are in the first or second year of implementation. Identification by management of the recommendations that are closest to demonstrating outcomes could support their retirement next year. For example, in relation to The World Bank’s Role in and Use of the Low-Income Country Debt Sustainability Framework (World Bank 2023k), outputs have been delivered on improving debt data quality in its first year of MAR tracking (recommendation 2). With the continued implementation of the initiatives of the debt transparency heat map and updates based on review of the low-income country debt sustainability framework, progress toward a change in direction could be demonstrated. 3.11 Evidence has shown progress on the Better Bank initiatives on the Knowledge Compact and crisis preparedness. The definition and implementation of the Knowledge Compact have helped demonstrate progress on recommendations from The World’s Bank: An Evaluation of the World Bank Group’s Global Convening (World Bank 2020d) and Enhancing the Effectiveness of the World Bank’s Global Footprint (World Bank 2022c). A change of direction is shown on how the Bank Group scopes engagements and contributions to major global convening, with the Knowledge Compact building on earlier experience to set out an approach that is more selective regarding global partnership activities. Moreover, the Knowledge Compact and the definition of the new World Bank Group Academy contribute to evidence of improved knowledge flows globally. Stay on Track 3.12 Continued focus is needed for recommendations that demonstrate limited evidence of progress and, to some extent, have defined pathways for implementation. This group consists of 27 recommendations. Of these, 18 recommendations are in their first year of tracking and show limited evidence of progress but are likely to show progress in the coming years. For these recommendations, evidence has been provided of initial activities with supportive processes in place. The remaining 9 recommendations in this group are in their second or third year of tracking and either need to show enhanced evidence of progress or have yet to demonstrate the delivery of outputs. Among these, a single recommendation (1) has progress constrained because the new follow-up to the Doing Business report, Business Ready (B-READY), had yet to be published during FY24. A jump in progress from these showing limited evidence to a change of direction is 13 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations possible; where this has happened, it can inform efforts to advance the implementation of recommendations (box 3.3). Box 3.3. Rapid Progress on Public Financial and Debt Management Rapid progress was demonstrated on clusters of recommendations related to public financial and debt management and on many of those related to urban development. All the recommendations from The International Development Association’s Sustainable Development Finance Policy: An Early-Stage Evaluation and World Bank Support for Public Financial and Debt Management in IDA-Eligible Countries advanced this year. Three of the five recommendations from these evaluations jumped levels of progress—moving from either constrained or limited progress to a change of direction. The remaining two show emerging evidence of progress. This rapid movement was facilitated by intensive deliberation between management and the Independent Evaluation Group on pathways for achieving the outcomes of recommendations, the application of tools, and the implementation of systematic processes. Sources: Independent Evaluation Group; World Bank 2021b, 2021g. Note: IDA = International Development Association. Additional Follow-Up 3.13 Additional follow-up is needed on recommendations lagging in implementation, particularly those relevant to Better Bank initiatives and cross-cutting ones without pathways. Eight recommendations are in this category. Of these, five are linked to issues of inclusion and vulnerability and outcome orientation at the country level that are relevant to Better Bank initiatives. Three recommendations have institution-wide implications and have yet to define pathways and may be likely to lag in implementation. 3.14 Follow-up on recommendations focused on enhancing the World Bank’s approach to inclusion and vulnerability appears fragile. Recommendations on monitoring citizen engagement, addressing urban crime and violence, and addressing natural resource degradation and associated vulnerability did not continue earlier progress and moved from emerging evidence to either limited evidence or progress constrained. Internal constraints appear to have restrained progress. For example, the citizen engagement monitoring system was to be active by now but is awaiting implementation as the Bank Group’s Operational Dashboard linked to the Corporate Scorecard is to be defined. 3.15 Two recommendations focused on the Bank Group’s outcome orientation at the country level have also had their level of progress reduced. A recommendation from The World Bank Group Outcome Orientation at the Country Level (World Bank 2020f)—to reform the country-level results system to ensure that it accurately captures the Bank Group 14 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations contribution to country outcomes usefully—slipped back. Based on reporting provided in 2023, IEG would have anticipated evidence on how country programs were being managed to better achieve outcomes based on the guidance developed and training undertaken. Furthermore, no evidence has been provided on how the previous reforms to the country engagement model have improved the utility of the country-level results system—another key finding of the evaluation. World Bank Engagement in Situations of Conflict: An Evaluation of FY10–20 Experience recommends rethinking success to reflect contributions to higher-level outcomes (recommendation 4; World Bank 2021f). This year’s reporting focused on the release of new and updated guidance. IEG would have anticipated that previous changes described in earlier MARs would have been followed up on this year. For example, no evidence was provided on the benefit of the previously highlighted guidance note How to Improve Results in Situations of Fragility, Conflict & Violence: 12 Recommendations (Garrido et al. 2022). 3.16 It is concerning that follow-up has been difficult to maintain on recommendations connected to inclusion and vulnerability and outcome orientation at the country level. The evidence underlying these recommendations is relevant to the implementation of Better Bank initiatives. For example, the evidence from The Natural Resource Degradation and Vulnerability Nexus: An Evaluation of the World Bank’s Support for Sustainable and Inclusive Natural Resource Management (2009–19) shows that the vulnerability-related outcomes for sustainable land and resource management approaches should be measured and operations should ensure synergies when they are operating in the same geographic area (World Bank 2021e). This is relevant to the Forests for Development, Climate, and Biodiversity Global Challenge Program; its Approach Paper states that improved inclusion is key to managing forest capital (World Bank, forthcoming). Evidence has not been provided on the improved utility of country program results frameworks—a key issue raised in The World Bank Group Outcome Orientation at the Country Level (World Bank 2020f). Evidence on the utility of results frameworks from this evaluation is relevant to the Corporate Scorecard because management is considering ways to cascade the Corporate Scorecard into results frameworks across Country Partnership Frameworks (CPFs; World Bank 2024d). The ongoing update to the country engagement model may provide an opportunity for issues on country-level outcome orientation to be further addressed. 3.17 World Bank senior management needs to be involved in recommendations that have institution-wide implications and do not yet have defined pathways for implementation. For recommendations that cut across multiple institutional units, having a defined pathway to follow is important for making progress. MAR experience has shown that individual units cannot move forward with recommendations without elevating decisions to senior management who make decisions on priorities and 15 Chapter 3 Progress toward Achieving Intended Outcomes of Recommendations resources across units. Recommendations without pathways in their second year and showing limited evidence of progress can lag in later years. Table 3.3 highlights recommendations that are outside of the remit of individual units (but within the control of the Bank Group) and do not yet have clear pathways for their implementation. Table 3.3. Cross-Cutting Recommendations without Implementation Pathways and with Institution-Wide Implications Evaluation and Recommendation Management IEG Gap Enhancing the Effectiveness of the World LE LE No evidence has been provided about the Bank’s Global Footprint outcomes that the decentralization process aims Recommendation 1: The World Bank to achieve. The expected outcomes or benefits of should refine its current approach to expanding the World Bank’s country presence managing its staffing global footprint by need to be better articulated by senior clearly specifying decentralization’s management. expected outcomes and adopting principles to guide and adjust decentralization decision-making based on evidence. Managing Urban Spatial Growth: World LE LE No corporate approach has been defined across Bank Support to Land Administration, the World Bank that fully reflects the international Planning, and Development consensus on geospatial information. It was Recommendation 3: Strengthen and recommended that the World Bank should ensure implementation of the World strengthen its protocols for recording and Bank’s protocol to identify and record reporting geospatial data on the precise locations precise project locations and collect land of World Bank–financed investments. This is market data necessary to support clients central to managing urban spatial growth and is with managing urban spatial growth. included in SDG 11 (Make cities and human settlements inclusive, safe, resilient, and sustainable). World Bank Engagement in Situations of EE LE The focus of the recommendation is specifically on Conflict: An Evaluation of FY10–20 conflict-affected areas; evidence has been Experience provided only on the broader designation of Recommendation 3: Address factors that fragility, conflict, and violence. Furthermore, dissuade World Bank engagement in similar to last year, evidence is focused on conflict-affected areas. guidance and intent without describing how measures come together to improve engagement in conflict-affected areas. Sources: Independent Evaluation Group; World Bank 2021c, 2021f, 2022c. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; SDG = Sustainable Development Goal. 16 4. Progress on the Implementation of Recommendations, by Evaluation 4.1 This chapter contains IEG’s validation of implementation progress for all 28 evaluations and 77 recommendations in the FY24 MAR (tables 4.1 through 4.28). People Table 4.1. Undernutrition, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: Adjust nutrition programming in country LE LE EE EE portfolios to (i) give more priority to institutional strengthening for coordination and implementation of multisectoral nutrition interventions and (ii) increase focus on subnational targeting of interventions to reflect areas of greatest disadvantage and persistency of need. Recommendation 2: Strengthen nutrition support in GPs to (i) LE LE EE LE rebalance investments to have greater emphasis on nutrition- specific interventions and (ii) increase focus on social norms, interventions, and behavior changes, with more attention to tracking expected achievements to improve nutrition determinants. Sources: Independent Evaluation Group; World Bank 2021h. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; GP = Global Practice; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.2 The World Bank has made progress in developing pathways and adjusting nutrition programming in country portfolios, emphasizing institutional strengthening and subnational targeting (recommendations 1 and 2). The nutrition global team undertook a portfolio review, which showed that the amount of nutrition support by the World Bank more than doubled commitments from $866 million across 15 projects in FY19 to $2,077 million across 44 projects in FY23. Furthermore, 32 countries with nutrition projects prioritized institutional strengthening at various levels, including policy, financing, service delivery, and stakeholder engagement. Task teams sometimes undertake analytic work using trust fund grants to facilitate subnational targeting. For example, in the Philippines, an upcoming grant will address the limited capacity for strategic planning in local government units. The support for nutrition-specific activities doubled from 11 to 22 projects in FY23, and the inclusion of nutrition-specific activities in “nutrition-sensitive” projects increased from 1 project in FY19 to 10 in FY23, indicating a shift toward using nutrition-sensitive platforms to increase high-impact nutrition-specific interventions. Attention has also increased on social norms and behavior change interventions, with more than 50 percent of Health, Nutrition, and Population–led projects and more than one-quarter of projects led by other Global 17 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Practices (GPs) including such interventions. In the next MAR cycle, it would be useful to continue tracking change in the portfolio and provide further details on the processes to develop attention to subnational targeting, the extent to which interventions address needs, and improvements in measurement in the relevant Global Challenge Program. Table 4.2. COVID-19 Health and Social Response, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: Use the World Bank’s crisis recovery efforts to strengthen the EE LE resilience of essential health and education services to ensure that human capital is protected in a crisis. Recommendation 2: Apply a gender equality lens to health and social crisis response LE LE actions across sectors. Recommendation 3: Help countries strengthen regional cooperation and crisis response LE LE capacities for public health preparedness. Recommendation 4: Build on the COVID-19 experience to strengthen the World Bank’s EE EE internal crisis preparedness so that it has the tools and procedures ready to respond in future emergencies. Sources: Independent Evaluation Group; World Bank 2022h. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.3 The evidence management provided indicates that pathways have been established to strengthen the resilience of health and education in a crisis, apply a gender equality lens, and strengthen regional cooperation and capacities (recommendations 1, 2, and 3). The World Bank is working to integrate health security into health systems projects to help create more resilient health systems. Evidence documents the implementation of innovations in telehealth and digital technology to maintain health services and demonstrates that operations in Burkina Faso, the Democratic Republic of Congo, and Haiti have restructured their COVID-19 responses to integrate routine services. In addition, the Education GP is prepared for future crisis response by drawing lessons from its reports on COVID-19. Regarding gender equality, the Gender in Preparedness and Response Toolkit provides guidance, indicators, and tools that align with the International Health Regulations framework. The Health Emergency Prevention, Preparedness, and Response Global Challenge Program will seek to develop collective action at regional levels, while recognizing that countries find it difficult to incentivize investments in public goods. Furthermore, the Pandemic Fund is now operational and will be an important platform for strengthening regional engagement and improving stakeholder engagement, including with civil society groups. Future reporting will continue to describe how these elements are maturing and starting to deliver results. 18 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.4 The development and approval of the Crisis Preparedness and Response Toolkit, drawing on lessons from COVID-19, represents an important step in strengthening the internal crisis preparedness of the World Bank. The tool kit provides a range of tools, such as a rapid response option for borrowers to access up to 10 percent of undisbursed balances and contingent emergency response projects. Human Development operations are to use the expanded provisions of the new tool kit as it is being rolled out. For example, the COVID-19 Multiphase Programmatic Approach, which used hands-on expanded implementation support in procurement to help clients during the crisis, will continue in fragile and conflict-affected situation (FCS) countries. However, the tool kit did not reflect some important aspects highlighted in the evaluation, such as instruments that link to community responses, hands-on support for the Environmental and Social Framework, and strengthening of the monitoring and use of data on World Bank portfolios. Future reporting could review the application of the current tool kit and any adaptations that take place. Planet Table 4.3. Natural Resource Degradation and Vulnerability, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: The World Bank should identify and EE EE EE EE analyze natural resource degradation and vulnerability nexus issues and leverage this knowledge in Systematic Country Diagnostics and in country engagements where such issues matter for achieving sustainable poverty reduction and shared prosperity. Recommendation 2: World Bank operations that address natural EE EE EE EE resource degradation should direct attention to resource governance challenges and use a mix of resource management practices and financial incentives appropriate for the relevant socioecological systems. Recommendation 3: World Bank Global Practices involved in LE EE LE LE addressing natural resource degradation and associated vulnerability should share knowledge, improve measurement, and enhance coordination in the design and implementation of their projects to optimize development effectiveness. Sources: Independent Evaluation Group; World Bank 2021e. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.5 The World Bank made progress in incorporating the natural resource degradation and vulnerability nexus issues into country programs and operations (recommendations 1 and 2). These issues are routinely incorporated into Country Climate and Development Reports (CCDRs), which inform World Bank country engagements and country program frameworks. All the CCDRs published between 19 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation April 2023 and March 2024 recognized and included in their narratives the nexus of natural resource degradation, climate change, and vulnerability or poverty. In FY24, four development policy operations were approved that have incentives for improved natural resource management. Natural resource management governance has also been included in seven investment project financing operations and included at least one element supporting and providing incentives, training, tenure security, and user rights. Project design is augmented by guidance; for example, the report Detox Development: Repurposing Environmentally Harmful Subsidies (Damania et al. 2023) supports repurposing fertilizer support programs in multiple countries. For the FY25 MAR, reporting could reflect on how Better Bank initiative indicators that relate to natural resource degradation and vulnerability issues, such as the Corporate Scorecard, have informed country engagements and the design of operations, along with continued evidence on the use of CCDRs. 4.6 Coordination and collaboration have been developed on natural resource issues, although there is limited evidence of the incorporation of vulnerability issues in social protection (recommendation 3). Joint analytic work, operations, and communities of practice are reported to be improving the sharing of data and knowledge, and measurement and coordination on natural resource management issues. Examples include the following: (i) joint analytic work and tool kit development in the Blue Social Protection advisory services; (ii) operations that incentivize incorporation into projects of nature-based solutions; (iii) a Community of Practice that provides a forum for sharing among GPs. Evidence is limited on how the Social Protection and Jobs GP is enhancing work on vulnerability in operations that integrate or operate in the same geographic area as natural resource management operations (which is the focus of the recommendation). Further evidence of changes in the Social Protection and Jobs GP regarding outcomes related to resources and vulnerability, beyond the blue economy examples, would be useful next year. 20 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Table 4.4. Municipal Solid Waste Management, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: To achieve more sustainable and scalable EE EE CD CD outcomes in municipal waste management, World Bank Group technical and financial support to clients should give clear priority to the adoption and implementation of waste hierarchy practices, in line with client needs and capabilities for MSWM. Recommendation 2: To support the low-income countries EE EE CD CD where municipal solid waste is growing most rapidly, the Bank Group should identify constraints on demand and investments and leverage external partnerships to implement context- specific MSWM solutions. Recommendation 3: To bring prominence to and spur action on EE EE CD CD the global municipal solid waste agenda, the Bank Group should take up a clear leadership position, collaborating and convening with developmental partners. Sources: Independent Evaluation Group; World Bank 2022g. Note: CD = evidence substantiates a change of direction of travel; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; MSWM = municipal solid waste management. 4.7 The Bank Group demonstrated swift follow-up to Transitioning to a Circular Economy: An Evaluation of the World Bank Group’s Support for Municipal Solid Waste Management (2010–20) by emphasizing waste hierarchy principles, identifying constraints on demand, and showing leadership on the solid waste agenda. 4.8 The Bank Group has demonstrated that it prioritizes solid waste management in projects and platforms and through incorporating knowledge to support waste hierarchy principles (recommendation 1). Since the IEG portfolio assessment in FY21, the World Bank now has 46 projects with solid waste management activities, of which 12 are stand- alone solid waste management projects (active and pipeline). In FY24, the International Finance Corporation (IFC) processed 16 waste and circularity investment projects. The use of relevant platforms has expanded—for example, IFC’s Circularity Plus platform and the Global Methane Reduction Platform for Development. The work by the Multilateral Investment Guarantee Agency (MIGA) on guarantees for municipal solid waste will be informed by Bridging the Gap in Solid Waste Management: Governance Requirements for Results (World Bank 2021a). 4.9 The Bank Group has identified constraints on demand in low-income countries through analytic products and has used partnerships to help respond to them (recommendation 2). The World Bank has produced over 80 analytic products since FY21, covering 36 countries—23 low-income countries and lower-middle-income countries and 8 countries affected by fragility, conflict, and violence (FCV). This shows an increasing pace in conducting relevant analysis. Analytic efforts are aligned with climate action; for example, 41 CCDRs underscore the interconnectedness of addressing 21 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation solid waste to mitigate greenhouse gas emissions. Surveys have identified constraints on demand and investments in low-income countries, and a study examined behavior change in the solid waste management sector. The World Bank, in partnership with the Japan International Cooperation Agency, organized an intensive capacity-building event on solid waste management in Abidjan in May 2024 to highlight effective local practices and foster South-South cooperation. IFC evidence shows that it continues to support viable and scalable investment and advisory opportunities in waste management in Nigeria and Papua New Guinea. 4.10 The Bank Group has taken significant steps to provide leadership in global municipal solid waste management (recommendation 3). The Bank Group has launched the Global Methane Reduction Platform for Development and a $100 million plastic waste reduction–linked bond. These initiatives participate in key international events and produce global analytic work which demonstrates commitment to collaboration and knowledge sharing. IFC continues to develop its collaboration with the International Solid Waste Association, organizing panels and roundtables with key industry players. Table 4.5. Disaster Risk Reduction, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: Incorporate DRR activities in regions and sectors and for hazards CD CD that exhibit significant coverage gaps. Recommendation 2: Identify and measure the effects of DRR activities on exposure and EE EE vulnerability to strengthen the development case to clients facing serious disaster risks. Recommendation 3: Integrate the needs of populations that are disproportionately EE EE vulnerable to disasters caused by natural hazards into DRR project targeting and design, implementation, and results reporting. Recommendation 4: In countries affected by serious natural hazards and fragility and EE EE conflict risks, identify and assess the ways in which hazards and conflict interrelate, and use this knowledge to inform country engagement and project design. Sources: Independent Evaluation Group; World Bank 2022e. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; DRR = disaster risk reduction; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group. 4.11 The World Bank has made rapid progress toward addressing coverage gaps and measuring disaster risk reduction (DRR) activities. Over the past two FYs, the World Bank has shown a change of direction in its disaster disk reduction activities in areas that exhibit significant coverage gaps (recommendation 1). A portfolio review outlined increases in commitments in the energy and agriculture sectors and in Europe and Central Asia and the Middle East and North Africa across multiple years. Increasingly, DRR issues are being considered in CCDRs through analysis on nature-based solutions and urban climate risk. Progress has been made in identifying and measuring the effects of DRR activities on exposure and vulnerability to strengthen the development case for DRR to clients 22 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation (recommendation 2). In FY23, the World Bank introduced a comprehensive methodology and guideline to evaluate the benefits and costs of nature-based solutions for DRR and climate resilience. Future reporting on recommendation 2 could include continued evidence that tools continue to be applied to measure the effects of DRR along with examples of how the development case to clients has been strengthened. 4.12 Evidence shows that pathways are being followed to integrate the needs of vulnerable populations into operations and use assessments of hazards and conflict to inform country engagement and operations’ design (recommendations 3 and 4). DRR operations are seeking to close gender gaps, with 95 percent of FY23 DRR operations receiving a gender tag—up from 75 percent in FY22. For disability inclusion, the World Bank promotes inclusive policy reforms, early-warning systems, and resilient infrastructure, with six International Development Association (IDA) projects and two catastrophe deferred drawdown option operations in FY23, including disability actions. The Unbreakable microsimulation model helps assess disaster impacts on vulnerable populations, supporting policy analysis globally. Over the past two FYs, the World Bank has focused on integrating FCV and DRR, led by the Global Facility for Disaster Reduction and Recovery–FCV Nexus thematic area. For example, a portfolio review in FY23 analyzed disaster risk management financing trends in FCV countries from 2012 to 2022, providing insights, lessons learned, and strategies for better integration. Future reporting could include evidence of ongoing collaboration among DRR, FCV, and social inclusion teams. Moreover, additional evidence to show how vulnerable groups are considered in the implementation of operations will be important, as the gender tag is an ex ante tool that assesses intent, rather than results delivered. Table 4.6. Agrifood Economics, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: To enhance its effectiveness in developing agrifood systems, the EE EE Bank Group’s efforts to support production technologies should be complemented by efforts to improve market access, especially in LICs and in countries at the traditional stage of agrifood system development. Recommendation 2: To achieve more sustainable agrifood systems, where conditions LE LE permit, the Bank Group should support production diversification to meet the growing demand for undersupplied, high-value-added nutritious products while ensuring that smallholder farmers and SMEs benefit from the diversification. Recommendation 3: To enhance the contribution of IFC support for agrifood system LE LE development, IFC should pilot and adopt more effective ways to support clients to better meet E&S Performance Standards, especially in LICs. Sources: Independent Evaluation Group; World Bank 2022f. Note: EE = emerging evidence of a change in the direction of travel; E&S = environmental and social; FY = fiscal year; IEG = Independent Evaluation Group; IFC = International Finance Corporation; LE = limited evidence of progress; LIC = low- income country; SME = small and medium enterprise. 23 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.13 Progress has been made in enhancing market access and commercializing the agriculture sector in low-income countries and countries at the traditional stage of agrifood system development in support of production technologies (recommendation 1). In 2023, 24 of 33 operations included components and activities supporting agricultural markets, commercialization, and agribusiness development. Of these projects, 17 supported agricultural markets in low-income countries. This marks an increase from 2022, when fewer projects had components or targeted low-income countries. MIGA is backing projects in low-income and conflict-affected regions by offering innovative financial solutions for climate-related investments. IFC is enhancing market access for farmers by providing investments and advisory services, focusing on capital constraints, value chain links, climate risk assessments, and gender inclusion. Future reporting could continue to address the number of operations contributing to this recommendation and highlight how initiatives benefit small and medium enterprises (SMEs) and smallholder farmers, ensuring market access and sustainable production diversification. 4.14 The Bank Group has identified pathways to support the diversification of agrifood production into high-value, nutritious products and support clients in meeting Environmental and Social Performance Standards (recommendations 2 and 3). The Food and Nutrition Security Global Challenge Program is seeking to engage the private sector to invest in the food value chain, nutrition products, and climate-resilient agricultural innovations. MIGA has supported strengthening food systems, fostered innovation, and promoted climate-smart agriculture, and IFC has invested $2.45 billion in the food value chain and has supported projects encouraging diet diversification and access to animal protein products. IFC supports clients in meeting Environmental and Social Performance Standards, especially in low-income countries, focusing on capacity building and detailed action plans. IFC also has developed training programs to support low-capacity clients in addressing gaps in environmental and social management and is updating its agribusiness sector knowledge products and the overarching Environmental Health and Safety Guidelines. Future reporting could follow up on these areas and show how they have progressed. 24 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Infrastructure Table 4.7. Urban Resilience, 2020–24 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 2: The design and implementation of World EE EE CD CD Bank projects that build urban resilience should systematically incorporate resilience characteristics and articulate their application throughout the project cycle. These should include the following: (i) design standards in line with resilience risks, (ii) cost-benefit analysis in line with resilience risks, (iii) city and interjurisdictional coordination, and (iv) inclusive approaches for vulnerable people. Recommendation 3: In urban areas where the client has PC EE PC PC identified crime and violence as a resilience risk, the World Bank’s support should be based on a localized typology of crime and violence that is informed by relevant analytic work. This approach should be supported by an assessment of the mechanisms most effective at reducing crime and violence within operations. Sources: Independent Evaluation Group; World Bank 2019a. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; PC = progress constrained. 4.15 This is the final standard year of reporting for Building Urban Resilience: An Evaluation of the World Bank Group’s Evolving Experience (2007–17) in the MAR. The World Bank has systematically followed up on strengthening the quality of engagement to address urban resilience. For recommendation 2, a pathway of change has been maintained over three years. For example, city scans were prepared for 36 cities, directly informing projects and at least seven analytic and advisory activities. Task teams incorporate resilience into urban infrastructure planning and preliminary cost-benefit analysis, and examples of city and interjurisdictional coordination are provided. Urban resilience project designs now include gender and disability concerns. Twenty-one approved urban resilience projects received the gender tag, and six new urban resilience projects included disability inclusion and universal accessibility elements. An area of potential IEG follow-up is interjurisdictional coordination where management recognizes that more could be done. 4.16 Conversely, the World Bank did not develop a pathway to support clients who identified crime and violence as a resilience risk based on a localized typology (recommendation 3). Management confirmed that there is no global effort within the World Bank to collect, support, and manage work on issues related to criminal and urban violence; this is at odds with its management response, which agreed to address these issues where it is working with the client to address crime and violence as a priority risk. As noted in the FY23 MAR validation, outside of Latin America and the 25 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Caribbean, in Bangladesh and Tajikistan, and in West Africa, crime and violence issues have also been addressed. Furthermore, the FCV strategy recognizes that crime is part of multidimensional crises, of the kind the Better Bank initiatives seek to address, and that the World Bank will “explore ways to strengthen the justice and rule-of-law dimensions of operational and analytical work to help countries…manage crime” (World Bank 2020g, 37). The lack of action on this recommendation is concerning because it is important for addressing the underlying inclusion and vulnerability issues in urban areas. IEG requests that management continues to provide reporting on progress related to the outcomes of recommendation 3. Future reporting could draw on the follow-up to the Mid-Term Review of the FCV strategy. Table 4.8. Renewable Energy, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 2: The Bank Group to support renewable EE EE CD CD energy scale-up through comprehensive, long-term country engagements, with coordinated Bank Group solutions, based on the comparative advantages of each institution, to address barriers, aided by robust upstream diagnostics. Sources: Independent Evaluation Group; World Bank 2020a. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group. 4.17 The Bank Group uses its combined expertise and resources to support long-term country engagements in renewable energy (recommendation 2). The World Bank has supported the design of low-carbon energy sector development strategies and policies in at least 20 countries (including at least 8 FCS). Trust fund resources to support renewable energy scale-up have increased, with close to $1 billion approved in FY24 (quarters 1–2). In the past 5 years, the World Bank committed approximately $14 billion to enable close to 130 gigawatts of additional power through direct, indirect, and enabling policy support. In FY24 (quarters 1–2), the Energy and Extractives GP’s approved lending was close to $2.41 billion to support renewable deployment, with an additional 10 projects in the pipeline expected to be approved in quarter 4. Furthermore, the joint Energy Sector Management Assistance Program–IFC Offshore Wind Development Program expanded its engagement to 11 countries. IFC has committed close to $2.1 billion in finance (which includes IFC own account and mobilization) through 18 projects. These projects include financing stand-alone projects through project finance and portfolios of renewable energy projects and decarbonization projects through corporate finance, as well as collaborations in the upstream and advisory space. IFC continues actively working on programs such as Scaling Solar, Scaling Wind, and Scaling Mini-Grid to finance more projects in this growing segment. MIGA continues to develop the Renewable Energy Catalyst Trust Fund, which is designed to support renewable energy projects. MIGA used $5.18 million 26 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation from this trust fund to support three active guarantee projects, amounting to $60.04 million in gross new issuance. Building on this experience as part of the Evolution process, the World Bank, IFC, and MIGA have developed the Energy Access and Transition Global Challenge Program, which includes renewable energy and network integration as one of three focus areas. Table 4.9. Urban Spatial Growth, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: Adopt a framework that links the EE EE EE CD determinants of urban expansion to pathways for managing urban spatial growth and that contributes to the achievement of SDGs 1 and 11. Recommendation 2: Support World Bank clients with LE LE EE LE anticipating and preparing for urban spatial growth using preventive approaches, not just curative ones. Recommendation 3: Strengthen and ensure implementation of LE LE LE LE the World Bank’s protocol to identify and record precise project locations and collect land market data necessary to support clients with managing urban spatial growth. Sources: Independent Evaluation Group; World Bank 2021c. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; SDG = Sustainable Development Goal. 4.18 The World Bank demonstrated a change of direction by adopting an approach to its frameworks that links the determinants of urban expansion to pathways for managing urban spatial growth (recommendation 1). This year, three existing frameworks have informed the design of operations, a report to the Group of 20 on inclusive cities that was endorsed in its 2023 communiqué, and CCDRs have been completed with urban spatial growth considerations that inform country engagements. The Planning, Connecting, and Financing Framework; the Housing Strategy/Framework; and the Land Governance Assessment Framework have been mainstreamed in the World Bank’s analytic and lending work. These frameworks are also strongly reflected in the joint World Bank report Enablers of Inclusive Cities: Enhancing Access to Services and Opportunities (World Bank 2023e), which was officially endorsed by the Group of 20 in the final communiqué of the Group of 20 Finance Ministers and Central Bank Governors meeting. These frameworks are also used to support CCDRs and influence clients to ingrain the notion that compact cities tend to be more productive, livable, and resilient. 4.19 Clients have been supported in preparing for urban spatial growth using preventive approaches (recommendation 2). Initiatives such as the City Planning Labs and the City Climate Finance Gap Fund have increased support to help project teams in preparing for urban spatial growth through preventive measures. New projects that are 27 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation preventive in their efforts were approved, came into effect, or entered the pipeline in FY24, including, for example, the pipeline Indonesia Integrated Land Administration and Spatial Planning Project and the Tanzania Cities Transforming Infrastructure and Competitiveness Project, which is under preparation. IFC and the World Bank have developed the joint Affordable Housing Program concept, which focuses on enhancing green affordable housing in India, Kenya, and the Philippines. The recent flagship World Bank Land Conference and technical deep dives have further reinforced the importance of preventive approaches. A systematic shift in project implementation toward preventive measures is developing, and future reporting could demonstrate the shift through a portfolio review. 4.20 Although the Geo-Enabling Initiative for Monitoring and Supervision (GEMS) portfolio mapping is being used in some urban areas, thus far, no corporate cross- institutional pathway has been defined to strengthen geodata collection protocols (recommendation 3). The number of projects using GEMS portfolio mapping has increased, providing granular spatial data that supports urban spatial growth management. As of June 2024, GEMS covered 100 projects in 45 countries as part of the Urban, Disaster Risk Management, Resilience, and Land portfolio. The World Bank’s new Knowledge Compact also aims to further leverage digital tools, including AI, under its Systems to Transform Productivity pillar. GEMS is the main systematic approach to collect granular spatial project data in the World Bank. As such, a pathway for comprehensive improvements in data collection protocols and their integration into all relevant project documents remains to be fully realized (which is beyond the remit of the Global Practice for Urban, Disaster Risk Management, Resilience, and Land and the Urban Spatial Team). Consolidated progress on this recommendation across the institution may need further support, potentially from senior management setting the overall direction. Table 4.10. Energy Efficiency, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: Intensify DSEE support to MICs for decarbonization and wider EE EE socioeconomic benefits. (Bank Group) Recommendation 2: Develop energy efficiency sector-specific approaches in a select LE LE group of LMICs that seek productivity gains alongside or via DSEE, even if energy efficiency policy reforms are in early stages. (World Bank and IFC) Recommendation 3: Expand DSEE approaches by incorporating the reduction of indirect LE LE emissions (Scope 3), including embodied and operational carbon, in DSEE project design. (World Bank and IFC) Recommendation 4: Exploit untapped DSEE opportunities and help clients leapfrog by EE EE exploring cross–Practice Group (World Bank) and cross–industry group (IFC) approaches. (World Bank and IFC) Sources: Independent Evaluation Group; World Bank 2023j. Note: DSEE = demand-side energy efficiency; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; IFC = International Finance Corporation; LE = limited evidence of progress; LMIC = lower-middle-income country; MIC = middle-income country. 28 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.21 Emerging evidence from the Bank Group shows progress in supporting middle- income countries in increasing demand-side energy efficiency (DSEE; recommendation 1). Recent efforts include a significant increase in lending for energy efficiency, which went from $730 million in FY20–21 to $1,468 million in FY22–23. For example, as the most energy-intensive Region globally, Europe and Central Asia is proposing a $1.5 billion Scaling-Up Energy Efficiency in Europe and Central Asia Multiphase Programmatic Approach. The Energy Access and Transition Global Challenge Program includes energy efficiency as a focus area. IFC seeks to promote decarbonization of hard-to-abate manufacturing—for example, it undertook a sustainability-linked loan in the cement sector in Brazil. 4.22 The World Bank has shown limited evidence of progress in developing sector- specific approaches for DSEE in lower-middle-income countries (recommendation 2). The approval of about $140 million in new lending for DSEE in IDA countries, along with the ongoing global DSEE advisory services and analytics activities updating the energy efficiency database, demonstrates pathways in place to realize the outcomes of this recommendation. IFC has developed decarbonization tools that are being rolled out and piloted. For example, a cement decarbonization tool is being applied to support cement companies in emerging markets—such as two IFC clients in Brazil and Iraq. 4.23 There is limited evidence of progress in the reduction of indirect emissions being incorporated (scope 3) in DSEE project design (recommendation 3). Overall, the Bank Group remains in a pilot phase for how it can address these emissions. It did explore including embodied carbon analysis in a few operations (such as Türkiye Second Energy Efficiency in Public Buildings) but found that, in typical buildings, about 80 percent of the embedded carbon is in iron and cement, neither of which are used for building renovations. IFC’s current approach is to work with key multinational clients that have made commitments to reduce their scope 3 emissions. Specifically, IFC has been providing advisory and finance products to help incentivize participating suppliers to accelerate their adoption of lower-carbon alternatives.  Future reporting could discuss the results of the pilots and reflect on the practicality of following up on this recommendation. 4.24 The Bank Group has shown strong emerging evidence in its exploration of cross– Practice Group and cross–industry group collaboration for untapped DSEE opportunities. The Energy GP’s collaboration with other GPs has supported the development of new operations. For example, a new green industries operation in Türkiye is jointly managed by Equitable Growth, Finance, and Institutions and Finance, Competitiveness, and Innovation. Several development policy operations that include DSEE actions in Albania, Kazakhstan, Romania, and other countries were developed together with Equitable Growth, Finance, and Institutions. IFC’s Disruptive Technologies 29 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation and Funds industry group has also led efforts to explore digital solutions and cross- industry collaborations, supporting innovative projects. Furthermore, the Disruptive Technologies and Funds industry group is forming cross-industry teams with each of the other departments within the institution (IFC’s Infrastructure and Natural Resources; Manufacturing, Agribusiness, and Services; and Financial Institutions Group) to bring together digital solutions and existing industry expertise.1 Market interventions are typical ways to promote digital measures to encourage an efficient economy. The Energy Global Challenge Program will develop a partner network with digital platforms to share experiences, lessons, and documents (such as terms of reference, contracts, templates, and guides). These initiatives demonstrate a commitment to exploring DSEE within the World Bank and IFC. Future reporting could show that these kinds of initiatives have been maintained. Digital Table 4.11. Mobilizing Technology for Development, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: Where DTT offer opportunities to make EE LE EE LE progress on the twin goals more effectively or efficiently, ensure that the World Bank Group avails itself of those opportunities and addresses, in particular, the risks posed by DTT. Recommendation 2: Build a Bank Group workforce with the EE LE EE EE skills required to harness DTT opportunities and mitigate DTT risks by identifying DTT-relevant skills, determining gaps in these skills, and filling these gaps. Recommendation 3: Improve the effectiveness and efficiency of EE EE EE EE World Bank procurement for complex technology projects. (World Bank only) Sources: Independent Evaluation Group; World Bank 2021d. Note: DTT = disruptive and transformative technology; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.25 The Bank Group has made gradual progress on the three recommendations in Mobilizing Technology for Development: An Assessment of World Bank Group Preparedness by enhancing preparedness to mobilize disruptive and transformative technologies (DTTs), improve its human resource skills to harness DTT opportunities and mitigate DTT risks, and improve complex technology procurement. The evidence provided shows that pathways have been established or reinforced for all three recommendations, with progress to be described in future MAR reports. 4.26 The new Digital Vice Presidency and the Digital Global Challenge Program seek to amplify the Bank Group’s knowledge and investment through cross-sectoral and 30 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation institutional links and synergies to promote digital inclusion and transformation (recommendation 1). The development of the vice presidency (World Bank) and the Global Challenge Program (Bank Group) has sought to enhance support to digitalization and to leverage DTT. Bank Group management highlights two examples of collaborative work: (i) IFC–International Bank for Reconstruction and Development (IBRD) collaboration on operations, such as upstream engagement to support digital start-ups in Angola, Ghana, and Morocco through collaboration on IBRD operations to crowd in private capital, and (ii) the development of a hybrid public-private partnership using IDA financing to finance an IFC-developed advanced digital skills delivery with Madagascar’s Ministère du Développement Numérique, de la Transformation Digitale, des Postes et des Télécommunications. It is too early to assess how and to what extent the vice presidency will coordinate the digital agenda across institutions and practices. The Digital Economy Country Diagnostics, which cover 42 countries (extending to 53), can be used to shape a digital transformation strategy within CPFs and to inform consideration of DTT in review meetings. More specific evidence on the effects of these diagnostics has yet to be provided. The Bank Group sees these diagnostics supporting data-driven approaches and informing operational strategies and projects. IFC has conducted market mappings and developed investment strategies for technology sectors across its industries and cross-cutting themes to ensure that digital transformation expertise informs its investment approach and collaboration with the World Bank. The digital flag in iPortal could enhance management reporting and tracking of DTT commitments. IFC took steps to streamline processes, improve productivity, and ensure data accuracy. Future evidence could quantify the extent to which the Bank Group has seized DTT-related opportunities and mitigated risks in its country strategies and operations, how support to DTT has enhanced effectiveness and efficiency in addressing the twin goals, and the extent of links and synergies between technological and sectoral issues, including through enhanced collaboration and coordination. 4.27 The digital development workforce has grown from 86 staff and consultants at the end of FY20 to 128, predominantly in grades GF and GG (recommendation 2). This growth has been supported by learning through summits, learning weeks, and specialized workshops across the Bank Group. IFC has adopted a new approach to building a digitally competent workforce by upskilling staff at various levels with AI skills and promoting organizational adoption of AI through the AI Working Group and AI Advocates Network. During the formalization of the Global Challenge Programs and the new vice presidency, colleagues worked together to develop new structures and organizational arrangements. For example, in developing the methodology of the Corporate Scorecard, working groups made up of various GPs, along with IFC and MIGA, came together to define complementarities and a standard definition of the Scorecard indicator for digitally enabled services. In health technology, sector leads and 31 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation industry specialists from IFC and the World Bank convened to exchange project pipelines; collaborate on the development of flagship reports, such as the World Bank report Digital-in-Health: Unlocking the Value for Everyone; and take courses related to the subject matter (World Bank 2023d). The extent to which staffing up in the digital workforce followed a systematic assessment of skills gaps needs further evidence. Future reporting could focus on the extent to which skills gaps have been effectively filled by (i) systematically taking stock of those gaps; (ii) the efficient use of the skills of specialized staff; (iii) improved bridging of technology expertise with that of sector specialists and task team leaders across units, regions, and sectors; and (iv) evidence of a growth mind-set prevailing among staff. 4.28 Through the Identification for Development initiative, the World Bank continues to support task teams and facilitate cross-support related to available and appropriate international standards in procurement for more than 40 operations, including identification and civil registration activities (recommendation 3). Several procurement notes and resources have been created for task teams and clients focusing on particular procurement areas. Under the Digital Economy for Africa initiative, investment projects were also complemented by development policy financing operations, with 25 operations in Africa during FY23–24, which included measures such as mandating e- procurement systems for public procurement, establishing principles of universal access and infrastructure sharing, strengthening sector regulation oversight, and creating legal frameworks for e-transactions and e-commerce. The GovTech team continued its efforts to procure cloud solutions, providing technical assistance to project implementation units to support information technology procurement and contract execution. To support client teams, project implementation units have hired specialized information technology procurement advisers, alongside procurement specialists. Future evidence could focus on the following: • World Bank staff surveys showing that staff have adequate guidance and incentives for effective and efficient procurement in complex technology projects; • Innovative approaches being adopted in complex technology projects while protecting the World Bank and the borrower from procurement-related reputational risks; • An update on the roster of leading experts on the procurement of complex technology projects; and • Market consultations on technical requirements in the preparation of bidding documents occurring as relevant, including evidence on technology procurement outcomes. 32 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Prosperity Table 4.12. Regional Integration, 2019–24 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 5: Strengthen the design of IDA’s Regional CD CD CD CD Policy Window–supported projects to improve the assessment of spillover effects and generate evidence based on robust indicators. Sources: Independent Evaluation Group; World Bank 2019b. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; FY = fiscal year; IDA = International Development Association; IEG = Independent Evaluation Group. 4.29 IDA Regional Window support continued to assess spillover effects with appropriate indicators in place. All 20th Replenishment of IDA Regional Window projects include at least one indicator for spillover effects. This requirement has been consistently met since FY23. In addition, frameworks such as the Africa Regional Integration Strategy and the South Asia Regional Integration, Cooperation, and Engagement approach are in place to help measure spillover benefits (World Bank 2020c). The added reporting year confirms the system has informed the conceptualization of the IBRD Framework for Financial Incentives. Future IEG products outside the MAR assessing regionalization could include analysis of the application of the indicators and the assessment frameworks that responded to this recommendation. Table 4.13. State-Owned Enterprises, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: The World Bank Group should apply a EE EE EE EE selectivity framework for SOE reform support that considers country governance conditions, control of corruption, and sector- and enterprise-level competition. Recommendation 2: The Bank Group should apply the EE EE EE EE Maximizing Finance for Development and its embedded Cascade approach for SOE reform. Sources: Independent Evaluation Group; World Bank 2020b. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; SOE = state-owned enterprise. 4.30 The Bank Group has made progress in applying a selectivity framework in state- owned enterprise (SOE) reform (recommendation 1). The Integrated SOE Framework continues to be used across regions, and 14 Integrated SOE Framework country assessments have been completed, with several others under preparation to inform a more selective and targeted approach to SOE reforms. SOE engagement in IDA countries continues to focus on foundational aspects of corporate governance, reflecting the need for selectivity, institutional capacities, and governance context. In IBRD countries, engagement has increased to further strengthen the SOE ownership function, boards of 33 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation directors, and other governance features, including the emerging agenda of sustainability and climate reporting. A range of new analyses are being undertaken to better inform operations. Areas where progress has been less pronounced include control of corruption and country governance. For IFC, Country Private Sector Diagnostics have considered SOEs systematically in a sector context focusing on opportunities to promote private sector participation. Future reporting needs to focus more on the processes put in place to control corruption and engagement in country governance reforms linked to SOEs. 4.31 The Bank Group is undertaking SOE reforms that emphasize internal coordination and mobilize private financing (recommendation 2). The World Bank is funding national development banks for demonstration effects and providing funding to public and private debt and equity funds. Operations are also supporting restructuring and privatization of SOEs in Cabo Verde and Cameroon. The consistent application of the Cascade approach has been informed by technical guidance notes and support that incorporates structuring private capital mobilization transactions, including all guarantee operations. The Bank Group has enhanced cooperation across its institutions to foster a more integrated approach to SOE reform, focusing on corporate governance, competition, and fiscal risks. Future reporting could further describe the One World Bank Group approach, including how the Global Challenge Program is building on past approaches and enabling the Bank Group to further implement interventions. Table 4.14. Public Financial and Debt Management, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: The World Bank should regularly monitor EE EE CD CD the quality of the key pillars of PFDM for each IDA-eligible country, possibly through a centralized country-specific PFDM assessment. Recommendation 2: Actively use the previously described LE LE CD CD assessment (recommendation 1) to prioritize and sequence World Bank support for PFDM capacity building and reform in IDA-eligible countries. Sources: Independent Evaluation Group; World Bank 2021g. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IDA = International Development Association; IEG = Independent Evaluation Group; LE = limited evidence of progress; PFDM = public financial and debt management. 4.32 The World Bank conducts regular assessments to monitor essential elements of public financial and debt management to inform the sequence and priority of reforms. The systematic use of public financial and debt management diagnostics has improved data availability and use. The World Bank conducts 20 –30 country Public Expenditure and Financial Accountability assessments annually and has completed the third Global Report on Public Financial Management (focusing on budget credibility). The Public Finance Review core diagnostic has been revamped, and the Resource Center has launched the 34 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation approach and is piloting it in several IDA countries. Public finance management reforms are occurring based on up-to-date assessments—for example, seven out of eight IDA- eligible countries in the Latin America and the Caribbean Region are drawing from public financial and debt management diagnostics. In these seven countries, diagnostics- informed reform efforts were supported as prior actions in development policy loans. Furthermore, Ethiopia and Nepal are updating their public finance management reform strategies based on recent Public Expenditure and Financial Accountability outcomes. The programmatic debt reform plans in the Republic of Congo, the Democratic Republic of Congo, and Madagascar were prepared after Debt Management Performance Assessments and in the Republic of Congo and Madagascar after consultations with stakeholders. Table 4.15. Doing Business Report and Business Ready, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Lesson 1: Recognizing the powerful motivational effect of LE LE LE LE reform indicators, especially those that facilitate country rankings, this evaluation notes the limitations in the coverage and guidance offered by any single indicator set on its own and advocates integrating them with complementary analytic tools and indicators. Lesson 2: Recognizing the granularity and specificity of PC PC PC PC individual reforms in any given country context, the findings from this evaluation suggest that it is better to avoid using business regulatory or similar global indicators as explicit reform objectives or monitoring indicators in World Bank Group projects and country strategies focused on improving the business environment. This does not preclude the use of primary data to agreed targets that track and measure critical Bank Group institutional commitments. Lesson 3: Global indicators coverage and specifications are LE LE LE LE improved if, at regular and predictable intervals, they are updated to reflect learning from research and field experience to (i) improve links to important development outcomes; (ii) strengthen relevance to the experience of the subject of coverage; and (iii) adapt to technological changes in the areas covered by the indicators. Lesson 4: The Doing Business experience indicates the need PC PC LE LE for mechanisms and safeguards to ensure the accuracy and validity of Bank Group global indicator-based reports and related communications, using robust and transparent standards of evidence. Sources: Independent Evaluation Group; World Bank 2022b. Note: FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; PC = progress constrained. 35 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.33 The lessons from IEG’s Doing Business report continue to be considered in the development of the new B-READY report.2 The B-READY project emphasizes transparency and data integrity to ensure the public availability of collected granular data and the replicability of results. Processes for maintaining standards have been incorporated into the Business Ready (B-READY): Manual and Guide and the Business Ready: Methodology Handbook (World Bank 2023b, 2023c). These documents contain information to support the accuracy and validity of B-READY as a global report based on indicators. The B-READY team intends to update these documents regularly and to integrate B-READY data into various Bank Group diagnostic and strategic products, supporting policy dialogue and engagement on private sector development issues. Progress against lesson 2 is assessed as progress constrained because the report was not published during FY24. Future evidence would benefit from continued reporting on progress, coverage, and adaptations of the mechanisms in place in the B-READY Concept Note and the use of the first report (World Bank 2022a). Table 4.16. Sustainable Development Finance Policy, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: Consideration should be given to EE PC CD CD expanding the countries covered by the Debt Sustainability Enhancement Program beyond those at moderate or high levels of debt distress or in debt distress. A low level of debt distress alone should not be sufficient for exclusion from the Debt Sustainability Enhancement Program, and IEG recommends applying an additional filter. Recommendation 2: PPAs should emanate from an up-to-date LE LE EE EE assessment of country-specific debt stress and be set explicitly within a longer-term reform agenda. Recommendation 3: Where PPAs support actions that need to LE LE CD CD be taken regularly (for example, debt reporting to parliament), they should aim for long-lasting institutional reforms rather than relying on one-time actions. PPAs should seek to institutionalize good practice in fiscal and debt management by supporting the establishment of statutory requirements, the existence of which can help depoliticize future decisions. Sources: Independent Evaluation Group; World Bank 2021b. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; PC = progress constrained; PPA = performance and policy action. 4.34 The Development Finance team working on the Sustainable Development Finance Policy has completed the changes within its mandate (recommendation 1). It has updated its checklist to review debt data coverage and is checking the data status of IDA countries using the Debtor Reporting System status report. All relevant countries are up to date with their reporting, and the coverage and quality of reported data by compliant countries are good, although there is room for improvement regarding coverage of external debt 36 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation data beyond the central government. Further enhancements to debt data quality will be tracked in the follow-up to IEG’s The World Bank’s Role in and Use of the Low-Income Country Debt Sustainability Framework (World Bank 2023k) under recommendation 2 and will seek confirmation that these process changes are maintained. 4.35 Management has made progress in aligning performance and policy action (PPA) with long-term reform agendas by incorporating diagnostics to increase the focus on fiscal sustainability (recommendation 2). As more PPAs focused on fiscal sustainability, the share of PPAs underpinned by Public Expenditure Reviews, the Tax Administration Diagnostic Assessment Tool, or SOE reports increased from 23 percent in FY21 to 30 percent in FY23. Future reporting on this recommendation could draw on evidence provided by country teams of how relevant assessments are working with theories of change to address debt stress. 4.36 Management has improved how PPAs link to institutional reforms (recommendation 3). Since the Sustainable Development Finance Policy went into effect in FY21, approximately half of all PPAs have focused on consistent institutionalization measures through various actions (ministerial order, decree, official notification, or circular; approval of legislation or a bill; or revision of existing legislation, tax code, or regulations). The number of institutionalized PPAs and their share relative to all PPAs have increased, driven by a rise in the number of countries with at least one institutionalized PPA. Over 95 percent of institutionalized PPAs have continued to be implemented since the actions were taken. Table 4.17. Low-Income Country Debt Sustainability Framework, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: Expectations of the World Bank in taking the lead on long-term LE LE growth prospects should be clarified. Given the World Bank’s development mandate, the current guidance is appropriate but comes with the expectation that the World Bank systematically take the lead in highlighting the country-specific factors that influence long-term growth, which is not currently the case. Recommendation 2: The recently increased attention to debt data coverage should be EE EE sustained and extended; greater attention is needed to assess data quality. Recommendation 3: The DSA should be more directly and consistently used to inform LE EE priorities for the identification of fiscally oriented prior actions in development policy operations and SDFP performance and policy actions. Recommendation 4: The World Bank should continue to give increasing attention in the LIC-DSF LE LE to the long-term implications of climate change, in terms of both growth and fiscal requirements of adaptation and mitigation. Sources: Independent Evaluation Group; World Bank 2023k. Note: DSA = debt sustainability analysis; DSF = debt sustainability framework; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; LIC = low- income country; SDFP = Sustainable Development Finance Policy. 37 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.37 The World Bank is developing its capacity to lead on long-term growth projections in debt sustainability analysis (DSA; recommendation 1). These efforts include three core analytic products: Jobs and Growth, the Macro-Fiscal Model, and the Long-Term Growth Model. Future reporting will provide an update on these efforts. 4.38 Progress has been made in enhancing debt data coverage in DSAs and the use of fiscally oriented prior actions (recommendations 2 and 3). Between 2020 and 2023, 40 countries using the low-income country debt sustainability framework have increased their debt transparency average scores regarding accessibility, completeness, timeliness of public debt reporting, and transparency. Some countries have improved debt transparency because of the implementation of PPAs addressing limitations in debt coverage identified by DSAs. For example, the Republic of Congo, The Gambia, Tajikistan, and Zambia implemented PPAs aimed at the issuance of regulatory frameworks and guidelines for publishing comprehensive debt data, enhancing the frequency of debt data publication, expanding coverage of domestic and external debts, and improving the composition of debt data. The use of DSAs to inform fiscally oriented reforms in development policy operations and PPAs has increased. Between 2021 and 2023 of countries that were in, or at high risk of, distress and were required to prepare PPAs under the Sustainable Development Finance Policy, 91 percent used low-income country DSAs as an analytic underpinning for defining their debt ceiling PPAs. 4.39 Attention to climate change in the low-income country debt sustainability framework is growing, and pathways are being developed (recommendation 4). The World Bank is preparing supplementary guidance on incorporating climate change implications into DSAs. This includes using tools such as CCDRs. Further reporting would provide evidence of a consistent and thorough assessment of data quality and concrete plans to address data shortcomings. Future reporting can update on these efforts. Table 4.18. Domestic Revenue Mobilization, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: On a country-by-country basis, regularly take stock of the findings EE EE of the broad range of tax diagnostics tools and instruments to (i) identify knowledge gaps and (ii) more systematically inform priority setting for country-level policy dialogue, capacity building, and operations to improve DRM. Rigorous analysis and diagnostics are needed to inform country-specific DRM strategy and operational priorities, particularly in IDA-eligible countries. Recommendation 2: Given the potentially large and regressive fiscal impact of tax EE EE exemptions, the World Bank should regularly assess the effectiveness and efficiency of tax exemptions in achieving country-specific policy objectives, with an eye to more actively supporting the sustainable reduction of regressive tax exemptions through policy advice and prior actions in DPOs. 38 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Management IEG Recommendation FY24 FY24 Recommendation 3: The frequency with which tax policy reforms are reversed calls for LE LE strengthening incentives for sustaining reforms and making reversal more challenging. Recommendation 4: Provide clearer guidance to staff on the choice of results indicators LE LE to measure the impact of DRM support, facilitate learning from experience, improve monitoring of progress toward DRM-related objectives, and promote an outcome orientation in the World Bank’s support for DRM. Sources: Independent Evaluation Group; World Bank 2023l. Note: DPO = development policy operation; DRM = domestic revenue mobilization; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IDA = International Development Association; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.40 The World Bank has revamped Public Finance Reviews and advanced analytics for assessing the effectiveness and efficiency of tax expenditure (recommendations 1 and 2). The World Bank has rolled out Public Finance Review pilots and analytic support in 15 countries. Plans include a substantive revenue chapter within Public Finance Reviews. The availability of tax diagnostic tools has significantly advanced: the Tax Administration Diagnostic Assessment Tool and the Climate Policy Assessment Tool are systematically upgraded, and country coverage has increased dramatically. Country support on assessing tax expenditure has included analyzing and measuring tax incentives and reforms, implementing the Medium-Term Revenue Strategy, developing a corporate income tax microsimulation model, rewriting the tax code and reforming tax incentives, and supporting cost-benefit analysis of tax incentives. The World Bank has developed comprehensive databases, such as high-level corporate tax incentives, global effective tax rates, and marginal effective tax rates, as well as user-friendly data-visualization tools. Future reporting could show how these reforms have been consolidated and continue to be applied across all relevant countries and quantify their benefits—for example, the impact of the reforms on tax collections. 4.41 Efforts to rationalize tax expenditures and enhance transparency in tax incentives have been prioritized in several development policy financing operations and supported through capacity development efforts (recommendation 3). Among the development policy financing operations with domestic revenue mobilization-themed prior policy actions, 31 percent relate explicitly to reducing the proliferation of or adding transparency to tax incentives. However, sustaining these reforms and preventing reversals can be a challenge. The integration of cost-benefit analysis into country-specific strategies is a positive step, and further evidence could indicate the extent to which this is used. 4.42 Management recognizes that there has been limited progress in providing guidance to staff on results indicators for domestic revenue mobilization (recommendation 4). Currently, no known comprehensive database collects project-level 39 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation domestic revenue mobilization indicators. Interrelated streams of domestic revenue mobilization work programs and activities initiated in FY24 are expected to help progress on this recommendation. The new Public Finance Review guidelines will provide the framework for structured analysis of domestic revenue mobilization issues, including identifying the key reforms, and inform the engagement in CPFs, with special attention to countries with tax collection (including social security contributions) at or below 15 percent of GDP. Table 4.19. COVID-19 Economic Response, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: To effectively address future crises, codify a global crisis response EE EE playbook, ideally developed jointly with the IMF. Recommendation 2: To respond effectively during the recovery phase of the crisis, EE EE explore increasing use of structured finance solutions (such as partial credit guarantees, subordinated debt, and quasi-equity instruments) with a view to supporting small- and medium-size firms. Sources: Independent Evaluation Group; World Bank 2023i. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; IMF = International Monetary Fund. 4.43 The World Bank Group has made progress in broadening and deepening its crisis response capabilities (recommendation 1). Enhancements to the crisis response tool kit, regular crisis monitoring, and updated core diagnostics indicate a structured approach to crisis preparedness and response. The expansion of the Bank Group’s guarantee offerings is important for risk management and crisis preparedness. Although some evidence indicates that implementation has been systematic, there are no evidence is provided of a framework from the World Bank and other multilateral development banks on responding to economic crises. 4.44 The Bank Group has progressed in its use of structured finance solutions to support SMEs during the recovery phase of crises (recommendation 2). This includes the development of structured finance solutions and new products for SMEs. The use of structured finance solutions has increased, with a lending pipeline of over $5.3 billion. The main products include risk-sharing facilities under the Small Loan Guarantee Program, structured finance debt facilities, and credit infrastructure advisory work creating legal and regulatory foundations for secured transactions and asset-based lending. The updates to the guarantee business are important to provide political risk insurance. IFC also presents evidence that it uses quasi-equity and quasi-loan instruments, for example, to address valuation risks in early-stage companies. Future reporting could provide further evidence of the changes and evidence that these products are helping SMEs to recover from the effects of crises. 40 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Cross-Cutting Table 4.20. Private Capital Mobilization, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 2: Expand private capital mobilization EE EE CD EE platforms, guarantees, and disaster risk management products commensurate with project pipeline development (for the World Bank Group). Recommendation 3: Develop new products and improve EE EE CD CD product alignment with the needs of new investor groups and partners (for IFC and MIGA). Sources: Independent Evaluation Group; World Bank 2020e. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; IFC = International Finance Corporation; MIGA = Multilateral Investment Guarantee Agency. 4.45 The Bank Group has made progress in expanding private capital mobilization platforms and guarantees and enhancing domestic revenue mobilization products, although some important changes remain at an early stage (recommendation 2). A variety of platforms have helped increase private capital mobilization across the Bank Group. IBRD and IDA have launched a range of new products, such as public sector bond issuances and transaction advisory services. The expansion provides additional support to policy reforms, disaster risk financing, and leveraging treasury capability. Enhancements to the IFC Core Mobilization Procedure were undertaken to better reflect mobilization from new and expanding product areas. In early FY25, the Bank Group is also set to launch its new guarantee platform to coordinate all of its guarantee offerings to member countries and private sector clients. In the future, it would be useful to understand two things: (i) how the guarantee platform is functioning and benefiting clients through the ability to mobilize private capital, and (ii) the extent to which treasury contributions on domestic revenue mobilization are replicating other efforts, such as bond financing from Chile and Jamaica. 4.46 IFC and MIGA have demonstrated a change of direction on developing new products and aligning them with the needs of emerging investor groups (recommendation 3). Notable efforts include creating the Build-Back-Better Emerging Markets Sustainable Transaction Bond Fund and the Avatar Credit Fund, both targeting sustainable and blue financing. Implementing synthetic risk transfers has enabled banks to expand lending activities, whereas new private equity mobilization frameworks have increased capital for growth stage firms. IFC has expanded its mobilization framework to include private equity funds. In its first year under the expanded framework, IFC recorded $700 million in Core Mobilization—a ninefold increase from less than $70 million mobilized through private equity funds in all of FY22. MIGA has 41 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation implemented new strategies, including MIGA-supported cofinancing, private refinancing of member development bank loans, and the Conflict-Affected and Fragile Economies Facility. New applications focus on expanding SOE business for energy transition and exploring new markets. Table 4.21. World Bank Engagement in Situations of Conflict, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: To enhance the conflict sensitivity of World EE EE CD CD Bank engagement, ensure that politically sensitive, confidential analysis is generated, retained, and managed so that it can be used by select future staff working on that country. Recommendation 2: Ensure that country engagements are EE EE EE CD informed by timely analyses of conflict dynamics and risks. Recommendation 3: Address factors that dissuade World Bank EE LE EE LE engagement in conflict-affected areas. Recommendation 4: In conflict-affected countries, rethink what LE EE EE LE success looks like. Sources: Independent Evaluation Group; World Bank 2021f. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.47 The World Bank has made progress in enhancing its conflict-sensitive practices by developing monitoring tools and ensuring that country engagements are informed by timely analysis (recommendations 1 and 2). Risk and Resilience Assessments are conducted for all IDA FCS countries, and their findings are increasing the sensitivity of the Bank Group’s country engagement documents. All Risk and Resilience Assessments are accessible on the World Bank intranet, ensuring that staff have access to comprehensive conflict analysis. Various World Bank analytic tools help staff increase their knowledge of conflict sensitivity and apply FCV-sensitive analyses to the design and implementation of the operations, including FCV-sensitive portfolio analysis, political economy analysis, peace and inclusion lenses, and conflict filters. In addition, Recovery and Peacebuilding Assessments—a joint approach of the United Nations, the World Bank, and the European Union—help develop a shared understanding of the context of conflict, crisis, and instability. The World Bank has deployed various tools at the country, regional, and global levels, such as the Compound Risk Monitor and Crisis Preparedness Gap Analysis, to enhance conflict risk monitoring and analytics, ensure timely analyses, and inform country engagements. 4.48 The World Bank has yet to provide evidence on engagement and assessing progress in conflict-affected areas that draws on broader evidence of work in FCV (recommendations 3 and 4). The World Bank has introduced several measures and tools to address operational challenges in FCV-affected areas, including new flexibility in project 42 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation cycles and improved staffing policies, specifically for procurement, environmental and social safeguards, and financial management. However, some 2023 guidance has not progressed; for example, the guidance on preparation and implementation of World Bank projects in insecure contexts is reported to be on hold. The evidence presented on rethinking success focused more on the release of new and updated guidance than on demonstrating how previous changes in assessing outcomes had helped develop an understanding of longer-term success in conflict-affected countries. Future reporting could further specify how the pathways established for enhancing World Bank work in FCV contexts are relevant to conflict contexts (which is the focus of this recommendation). Table 4.22. Private Investment in Fragile and Conflict-Affected Situations, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: IFC and MIGA should continue to review EE LE LE LE their financial risk, make more explicit the implications of IFC’s portfolio approach for FCS, and enhance capabilities to address nonfinancial risks to ensure they align with achieving business growth targets and impacts in FCS. Recommendation 2: To focus on the development of bankable EE LE EE LE projects, IFC and MIGA should further recalibrate their business models, client engagements, and instruments to continuously adapt them to the needs and circumstances of FCS and put in place mechanisms to track their effectiveness for real-time learning. Sources: Independent Evaluation Group; World Bank 2022d. Note: EE = emerging evidence of a change in the direction of travel; FCS = fragile and conflict-affected situation; FY = fiscal year; IEG = Independent Evaluation Group; IFC = International Finance Corporation; LE = limited evidence of progress; MIGA = Multilateral Investment Guarantee Agency. 4.49 IFC and MIGA provide evidence of continued development of their approaches to manage risks, portfolios, and capabilities to address nonfinancial risks; however, it is recognized that initiating changes aligned to this recommendation is a multiyear process, and limited evidence of progress in the second year of tracking is anticipated (recommendation 1). IFC undertakes regular assessments and continuously engages with the Board on issues related to financial risk, the cost of doing business, and capital. Furthermore, the use of blended finance seeks to reduce the risks of investments in FCS countries. As of June 2024, an estimated $2 billion of IDA Private Sector Window funds (approximately 40 percent of total Private Sector Window funds since the Private Sector Window inception) have been used in FCS countries, supporting over 120 IFC and MIGA investments in 26 FCS countries. IFC has also conducted mandatory environmental and social assessments and is advancing integrity due-diligence processes in FCS states. Recipients of the training on these processes have already reported their use. MIGA manages financial risk and uses the risk capital metric as a capital adequacy metric. MIGA also notes the use of a stress-testing tool, with stress scenarios representing 43 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation project-level and country-level risks, systemic macroeconomic scenarios, and systemic event-driven scenarios. Future reporting could show how these changes work together to help align growth targets, financial sustainability, and impacts in FCS. For example, it could show how IFC is managing trade-offs to encourage the significant increase in FCS investments foreseen by the Capital Increase Package. Initiating changes aligned to this recommendation will be a multiyear process; thus, limited evidence of progress in the second year of tracking is anticipated. 4.50 IFC and MIGA state that they continue to develop pathways to recalibrate their business models, but evidence of the results of these initiatives is still developing. IFC has focused upstream effort on bankable projects and has continued to pilot the Local Champions Initiative and the joint IFC–United Nations High Commissioner for Refugees initiative to build a sustainable pipeline of investable transactions in FCS. At the global level in IFC, the IDA and FCS team promotes knowledge sharing through FCS network calls and quarterly updates to support cross-regional knowledge sharing. Some initial results are emerging, such as the inclusion of a fragility lens in two pipeline investment and advisory projects in the Democratic Republic of Congo and a proposal for a conflict mitigation plan for an advisory services project in Mozambique, but these have yet to be rolled out. To enhance IFC’s ability to deliver on the capital increase commitments, the position of chief investment officer and head of Fragility, Africa, was created. MIGA is implementing tools in FCS for technical assistance, risk sharing, and working with local investors. Future evidence could include discussion of the results of these initiatives in recalibration of business models and continued evidence of how feedback loops and evidence of real-time learning mechanisms are developing and have helped adjust work in FCS. Table 4.23. International Finance Corporation Additionality in Middle-Income Countries, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: To enhance institutional accountability, learning, and transparency, LE LE address gaps in internal systems related to monitoring, supervision, and reporting of additionality at the project and portfolio level. Recommendation 2: To enhance commitment to and fulfillment of IFC’s strategic LE LE objectives, IFC should bring its strategy for additionality in MICs and its pattern of activity in MICs into closer alignment. Recommendation 3: To enhance its strategic approach to proactive creation of markets LE LE and mobilization of private capital to provide a critical contribution to the Sustainable Development Goals, IFC should incorporate its additionality approach into its country strategies and sector deep dives. Sources: Independent Evaluation Group; World Bank 2023g. Note: FY = fiscal year; IEG = Independent Evaluation Group; IFC = International Finance Corporation; LE = limited evidence of progress; MIC = middle-income country. 44 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.51 IFC has initiated changes in internal systems related to monitoring, supervision, and reporting of additionality at the project and portfolio levels, with systems in place for FY24 (recommendation 1). The introduction of the Anticipated Impact Measurement and Monitoring Navigator consolidates various data collection streams into one platform, allowing tracking and reporting of project-level additionality. The data in the Anticipated Impact Measurement and Monitoring Navigator are structured to capture categories of additionality as highlighted in the revised additionality framework. Training and guidance have been provided to staff, and a data backfill exercise is in progress. The Anticipated Impact Measurement and Monitoring Navigator seeks to support the inclusion of additionality reporting in the Development Impact Department’s annual report. These actions lay out a pathway to contributing to institutional accountability, learning, and transparency. Future reporting would benefit from specific data on the percentage of active projects with the additionality section filled out and the percentage of active projects that have updated the additionality section during implementation as required. Furthermore, it would be helpful to see evidence of reporting of additionality at the corporate level to the Board. 4.52 IFC is following the revised additionality framework to systematically guide its work on additionality in middle-income countries (recommendation 2). For example, the revised additionality framework, which captures both financial and nonfinancial attributes, has been applied in recent projects in Brazil and Mexico. IFC has focused on knowledge sharing and capacity building, supported by advisory services to enable clients to implement new approaches. Evidence provided in future MARs could benefit from demonstrating how the revised additionality framework is applied across the portfolio of middle-income countries, reporting the placement of resources to improve design and implementation, and enhancing nonfinancial additionalities. 4.53 IFC incorporates an additionality approach in its country strategies through Country Private Sector Diagnostics (recommendation 3). The evidence highlights examples from the Colombia and Panama CPFs showing that the Country Private Sector Diagnostics and their additionality analyses inform the CPFs and sector engagements in those countries. These CPFs also include IFC’s nonfinancial additionality through capacity building, innovation, knowledge sharing, and investments. Future reporting could usefully demonstrate that additionality continues to be considered in all country strategies and that sector deep dives beyond single countries are informed by relevant diagnostics. 45 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Table 4.24. Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence, 2023–27 Management IEG Recommendation FY24 FY24 Recommendation 1: Make priorities regarding gender equality (including on WGEE and LE LE GBV) more explicit in country strategies, based on strong analytics (primarily Systematic Country Diagnostics and the World Bank Risk and Resilience Assessments) and in collaboration with key stakeholders. Recommendation 2: Foster engagements with communities, civil society, women’s LE LE organizations, local authorities, and other key stakeholders to define gender equality objectives and the actions to achieve them. Recommendation 3: Ensure that gender expertise tailored to the context is available for LE LE FCV-affected countries to support projects, as well as the country engagement. Recommendation 4: Coordinate and collaborate with relevant international stakeholders LE LE engaged in gender equality in the country, including humanitarian actors. This stronger coordination and collaboration should leverage each actor’s comparative advantage to achieve common goals. Sources: Independent Evaluation Group; World Bank 2023a. Note: FCV = fragility, conflict, and violence; FY = fiscal year; GBV = gender-based violence; IEG = Independent Evaluation Group; LE = limited evidence of progress; WGEE = women’s and girls’ economic empowerment. 4.54 Pathways are outlined for all recommendations within the following documents: A Development Approach to Advancing Gender Engagement and Addressing Gender Inequalities in Fragile, Conflict, and Violent Situations (World Bank 2024b) and World Bank Group Gender Strategy 2024–2030: Accelerate Gender Equality to End Poverty on a Livable Planet (World Bank Group 2024). 4.55 In FCV contexts, CPFs are expected to track gender outcomes (recommendation 1). Progress has been made on updating the World Bank’s core diagnostic to integrate gender analysis and incorporating gender into country strategies. IFC’s engagement in Papua New Guinea is a useful illustration of Bank Group teams coordinating in an FCV context and drawing on relevant analytic work. Future reporting could demonstrate how the changes initiated by the gender strategy will be engaged within country strategies for FCV settings. 4.56 IFC and the World Bank provide useful examples of organizational processes that foster engagement with key gender interests in FCV contexts (recommendation 2). IFC shows that it has fostered engagements with key interests to define gender equality objectives in West Bank and Gaza and Papua New Guinea. IFC also describes how its civil society organization engagement operational support function will implement a framework for high-risk projects in FY24. The World Bank highlights that in alignment with the new gender strategy, country teams will be encouraged to engage with a wide variety of interests on gender. In the World Bank, FCV settings have implemented 46 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation community-driven processes that could support the definition of gender equality objectives. 4.57 The Bank Group provided evidence that there are intended pathways for gender expertise to be tailored to FCV contexts (recommendation 3). A Development Approach to Advancing Gender Engagement and Addressing Gender Inequalities in Fragile, Conflict, and Violent Situations recommends that gender expertise tailored to the context should be made available for FCV-affected countries to support project and country engagement, particularly through training local human resources (World Bank 2024b). 4.58 The Bank Group has in place elements of a pathway on how it will engage partners at the country level (recommendation 4). The World Bank undertakes engagement on FCV with a range of organizations at the corporate level. There are also examples of engagements at the country level. It is not yet clear how these current practices will be incorporated systematically into country engagements. Corporate Table 4.25. Citizen Engagement, 2019–23 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 3: The World Bank should strengthen the EE EE LE PC monitoring of its citizen engagement activities by systematically adopting results framework indicators that are results oriented. Sources: Independent Evaluation Group; World Bank 2018. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; PC = progress constrained. 4.59 The World Bank has yet to follow up on a new composite citizen engagement and social accountability monitoring indicator as described in FY23 MAR reporting. A quality of citizen engagement indicator, proposed for inclusion in the new Operational Dashboard but not yet implemented, measures how meaningfully citizens are engaged and to what extent their voices are reflected in development decision-making. Although the draft template of the Organizational Effectiveness and Efficiency Dashboard includes a placeholder for this indicator, more clarity is needed about its approval and implementation. We appreciate management’s suggestion to track this recommendation for an additional year to see through the changes proposed earlier. 47 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation Table 4.26. Convening Power, 2020–24 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: Scope engagements and contributions to EE EE CD CD major global convening initiatives more deliberatively. Recommendation 2: Enhance how the World Bank and IFC’s EE EE CD EE internal systems and processes support managing major convening initiatives over their life cycle. Recommendation 3: Improve links between the World Bank’s EE EE CD CD global and country work. Sources: Independent Evaluation Group; World Bank 2020d. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; IFC = International Finance Corporation. 4.60 The Bank Group has been more deliberative in selecting scope engagements and contributions to major global convening initiatives and has improved the links between its global and country work (recommendations 1 and 3). Three new initiatives provide a broader reach and more expansive goals, improving targeting and clarity on priorities both at the corporate and country levels. These include the Evolution and Better Bank Initiatives, the Knowledge Compact for Action, and various internal guidance documents and messages from senior management on topics such as the Global Challenge Programs. In addition, the Bank Group adopted the Partnership Charter and established the Partnership Council to recognize the critical role of partnerships in achieving the Bank Group’s mission. The World Bank’s trust fund portfolio has continued to become more streamlined. As of the end of June 2024, the number of stand-alone trust funds had been reduced from a baseline of 486 in 2019 to 229 (53 percent reduction), whereas 67 umbrella trust funds received 70 percent of all signed contributions over the prior 12 months. Building on last year, the Health Strategic Preparedness and Response Program, the Bank Group’s Paris Alignment commitment and approach, and CCDRs continue to support how global priorities inform country work. 4.61 The Bank Group continued strengthening how it manages global convening initiatives (recommendation 2). Efforts to enhance trust fund life cycle management include ongoing trust fund portal updates, semiannual consultations with donors, a focus on evaluations with support from a dedicated monitoring and evaluation specialist, and regular monitoring and reporting to ensure proper management and timely closure. The new World Bank Group Scorecard for FY24–30 was conceived as a strategic management tool to help management and the Board translate the new Bank Group vision into action, facilitate business planning and incentives for achieving results, communicate results at scale, and provide opportunities for feedback and learning, prompting course corrections as needed (World Bank 2024d). The GPs 48 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation continue to lead in convening engagements, supporting their implementation and monitoring, and working with other units (such as Operations Policy and Country Services and Development Finance). IFC’s new Partnerships and Blended Finance Department has bolstered thematic partnerships, organized events with partners such as the International Labour Organization, and cohosted a high-level workshop on labor standards. In addition, the Communications and Outreach Department within the Corporate Support Vice Presidency has taken steps to support operations better, ensuring the production of high-quality reports and publications that facilitate or support key gatherings. Table 4.27. Outcome Orientation at the Country Level, 2021–25 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: Reform the country-level results system to EE EE EE LE ensure that it accurately captures the Bank Group’s contribution to country outcomes and usefully informs decision-making on country engagements. The Bank Group should keep a country engagement model that articulates clear outcome-level objectives and lays out the pathways that will be pursued to achieve them, conducts periodic reviews to take stock of progress, and includes an end-of-cycle review of evidence and learning. These reviews need not be a Board deliverable but should be carried out in time to meaningfully inform course correction and the next strategy. It should discontinue the reliance on results frameworks in its country strategies and midterm and terminal reviews. And it should adopt MEL plans for its country engagements. Sources: Independent Evaluation Group; World Bank 2020f. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress; MEL = monitoring, evaluation, and learning. 4.62 This year, progress has not been shown on follow-up to the recommendation on outcome orientation at the country level, with no examples of changed practices that move toward enhanced learning from country-level results systems. Initial results of the High-Level Outcome Initiative have yet to be reported, with the fourth MAR to track the implementation of this initiative. All CPFs now contain high-level outcome statements, although no evidence is presented on how these changes have advanced efforts to accurately captu re the Bank Group’s contribution to country outcomes. With the development challenges compounded by multiple crises, it would be useful to understand if country outcome measurement and learning had improved before Completion and Learning Reviews, which can occur many years after a CPF’s definition.3 Previously reported training activities also continue, but no evidence is provided of their value in enhancing country outcome orientation. 49 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation 4.63 Initial evidence is presented on how the new Corporate Scorecard and results narratives are expected to enhance the current situation, but it is uncertain how issues identified by IEG have been resolved. The Bank Group is adopting and applying a new Corporate Scorecard that informs country results measurement. Yet it remains unclear how the Corporate Scorecard is helping to address the important findings identified in the evaluation. Future reporting could be presented on how the underlying issues identified in the evaluation are being tackled within country-level results systems on (i) accountability being equated with metrics and ratings, (ii) a tool kit that is reliant on results frameworks, and (iii) incentives that are not aligned with the pursuit of outcomes. 4.64 Furthermore, it would be helpful to describe the results of advancements that were previously discussed and the commitments made in the management response. For example, it would be useful to understand how previously revised country engagement guidance and other initiatives enable IFC and MIGA to better embed their perspectives in Bank Group documents. In addition, it would be useful to understand how the Performance and Learning Reviews and Completion and Learning Reviews have supported adaptive management and other examples of how changed practice on high-level outcome indicators supported enhanced learning from country-level results systems. Finally, the evidence provided has yet to discuss management’s commitment to pilot monitoring, evaluation, and learning plans. Table 4.28. Global Footprint Effectiveness, 2022–26 Management IEG Management IEG Recommendation FY23 FY23 FY24 FY24 Recommendation 1: The World Bank should refine its current EE LE LE LE approach to managing its staffing global footprint by clearly specifying decentralization’s expected outcomes and adopting principles to guide and adjust decentralization decision-making based on evidence. Recommendation 2: The World Bank should mitigate the risks EE EE EE EE to knowledge flow brought about by decentralization and put in place safeguards to avoid developing country and regional silos. Recommendation 3: The World Bank should establish clear and EE EE EE EE structured paths to systematically promote locally recruited staff’s professional and career growth within its overall approach to improving the effectiveness of its global footprint. Sources: Independent Evaluation Group; World Bank 2022c. Note: EE = emerging evidence of a change in the direction of travel; FY = fiscal year; IEG = Independent Evaluation Group; LE = limited evidence of progress. 4.65 Although the World Bank has increased staff presence in country offices, no evidence or pathway has been provided on the outcomes sought from doing so (recommendation 1). Since the fourth quarter of FY20, the number of staff in locations 50 Chapter 4 Progress on the Implementation of Recommendations, by Evaluation outside of the United States has increased by nearly 4 percent, with 47.9 percent now based outside of US locations. However, no framework has been provided for achieving the expected outcomes of decentralization. Further data collection and analysis are necessary to demonstrate that the decentralization decisions effectively support country programs and align with strategic objectives. The development of the outcome of the global footprint has institution-wide implications, and the lack of a pathway in the second year of MAR tracking implies there could be a role for World Bank senior management to set a direction to respond to this recommendation. 4.66 Efforts to mitigate risks to and enhance knowledge flows and avoid silos across the World Bank have advanced (recommendation 2). The Knowledge Compact for Action is intended to drive how the Bank Group enables knowledge flow. The Knowledge Compact builds on 20 years of effort to form a knowledge-driven institution and advances progress made by the 2021 Strategic Framework for Knowledge. However, evidence on the systematic application of processes for better flow of knowledge and reducing silos is only beginning to emerge. The initiatives need continuous monitoring and refinement to effectively address the risks of decentralization. Although significant steps have been taken, including implementing the Knowledge Compact, Global Challenge Programs, Knowledge Hubs, and a new World Bank Group Academy and enhanced mobility programs, ongoing efforts and evidence of their impact are required to confirm a sustained change in knowledge management and flow. 4.67 The World Bank has shown a positive trend in promoting locally recruited staff, with a higher promotion rate than headquarters staff (recommendation 3). Initiatives such as the renewed Career Development Plan, FCV Mentoring and Championship programs, and the introduction of the global footprint mobility packages have enhanced locally recruited staff career development. Establishing a working group focused on compensation and careers for country office staff indicates a solid commitment to the professional growth of locally recruited staff. Introducing new programs and frameworks shows progress, but continuous monitoring and data collection are essential to ensure that these measures lead to sustained and systematic professional growth for locally recruited staff. It would also be beneficial to assess whether local staff are being offered more leadership opportunities, such as taking on project task team leader roles with administrative responsibilities. 51 5. Conclusions and Suggestions for Further Enhancement of the Management Action Record 5.1 The MAR validation finds that the Bank Group made swift and significant progress on some recommendations and closed out older ones, and in some key areas relevant to Better Bank initiatives, focused follow-up is needed. The MAR report for FY24 tracks progress on 77 recommendations and assesses progress toward the achievement of their intended outcomes. Of these 77 recommendations, IEG proposes to retire 18 new and old recommendations, almost all of which demonstrate a change of direction. Another 24 recommendations show emerging evidence of progress and could move toward retirement in coming years. A group of 27 recommendations have defined pathways for implementation and are generally within the first two years of implementation with limited evidence of progress. Movement toward achieving their outcomes is likely if focus is maintained. Progress is limited or constrained for 8 recommendations for which pathways of follow-up are unclear. This is concerning because 3 of these require senior management engagement and 5 others are linked to Better Bank initiatives. 5.2 Maintaining recent positive changes in the implementation of the MAR is important. Defining and then implementing pathways that reconcile IEG findings and management’s capability to make change has enabled swift implementation and the closing out of older recommendations. Progress can stall when pathways are not defined, or when they need to be adapted so change does not tail off between activities and the consolidation of outputs. Over the past two MAR cycles, progress has also improved in areas where IEG and management have invested in dialogue on possibilities for implementing recommendations. As IEG recommendations are relevant to the Better Bank Group, maintaining these improvements in the MAR system needs to be prioritized. 5.3 IEG suggests that further efforts are made in appointing champions who can lead progress on recommendations that are lagging. Focused action and attention are needed on these recommendations. The appointment of a management champion for these recommendations can assist with setting out pathways for their implementation, engaging senior management, convening teams on their follow-up, and providing an account of their progress. Engaging champions in this way would be a continuation of their current role in reviewing progress on recommendations and approving MAR submissions. IEG is available to jointly review with champions, and the associated teams, pathways to help accelerate implementation. 52 Notes Chapter 4 1The International Finance Corporation’s Disruptive Technologies and Funds Department was newly formed under the Disruptive Technologies, Services, and Funds Industry Group as of July 1, 2024. 2 The Business Ready (B-READY) 2024 report was launched in October 2024. 3 For example, the Completion and Learning Review for Peru Country Partnership Framework, fiscal years 2017–21, was presented to the Board in fiscal year 2024. 53 References Damania, Richard, Esteban Balseca, Charlotte de Fontaubert, Joshua Gill, Kichan Kim, Jun Rentschler, Jason Russ, and Esha Zaveri. 2023. Detox Development: Repurposing Environmentally Harmful Subsidies. Washington, DC: World Bank. Garrido, Diego, Kamal Siblini, Sarah Craig, Peter Isabirye, Macha Kemperman, Wolfgang Koehling, and Brett Libresco. 2022. How to Improve Results in Situations of Fragility, Conflict & Violence: 12 Recommendations. Washington, DC: World Bank. World Bank. 2018. Engaging Citizens for Better Development Results. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2019a. Building Urban Resilience: An Evaluation of the World Bank Group’s Evolving Experience (2007–17). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2019b. Two to Tango: An Evaluation of World Bank Group Support to Fostering Regional Integration. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020a. Renewable Energy: Evaluation of the World Bank Group’s Support for Electricity Supply from Renewable Energy Resources, 2000–2017. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020b. State Your Business! An Evaluation of World Bank Group Support to the Reform of State-Owned Enterprises, FY08–18. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020c. World Bank's Approach to South Asia Regional Integration, Cooperation and Engagement (SA RICE) 2020–2023. Washington, DC: World Bank. World Bank. 2020d. The World’s Bank: An Evaluation of the World Bank Group’s Global Convening . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020e. World Bank Group Approaches to Mobilize Private Capital for Development . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020f. The World Bank Group Outcome Orientation at the Country Level . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020g. World Bank Group Strategy for Fragility, Conflict, and Violence 2020 –2025. Washington, DC: World Bank. World Bank. 2021a. Bridging the Gap in Solid Waste Management: Governance Requirements for Results. Washington, DC: World Bank. World Bank. 2021b. The International Development Association’s Sustainable Development Finance Policy: An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. 54 References World Bank. 2021c. Managing Urban Spatial Growth: World Bank Support to Land Administration, Planning, and Development. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021d. Mobilizing Technology for Development: An Assessment of World Bank Group Preparedness. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021e. The Natural Resource Degradation and Vulnerability Nexus: An Evaluation of the World Bank’s Support for Sustainable and Inclusive Natural Resource Management (2009–19). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021f. World Bank Engagement in Situations of Conflict: An Evaluation of FY10 –20 Experience. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021g. World Bank Support for Public Financial and Debt Management in IDA-Eligible Countries. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021h. World Bank Support to Reducing Child Undernutrition. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022a. Business Enabling Environment: Concept Note. Washington, DC: World Bank. World Bank. 2022b. The Development Effectiveness of the Use of Doing Business Indicators, Fiscal Years 2010–20. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022c. Enhancing the Effectiveness of the World Bank’s Global Footprint. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022d. The International Finance Corporation’s and Multilateral Investment Guarantee Agency’s Support for Private Investment in Fragile and Conflict-Affected Situations, Fiscal Years 2010–21. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022e. Reducing Disaster Risks from Natural Hazards: An Evaluation of the World Bank’s Support, Fiscal Years 2010–20. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022f. Toward Productive, Inclusive, and Sustainable Farms and Agribusiness Firms: An Evaluation of the World Bank Group’s Support for the Development of Agrifood Economies (2010–20). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022g. Transitioning to a Circular Economy: An Evaluation of the World Bank Group’s Support for Municipal Solid Waste Management (2010–20). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022h. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response—An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. 55 References World Bank. 2023a. Addressing Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence: An Evaluation of the World Bank Group’s Support. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023b. Business Ready (B-READY): Manual and Guide. Washington, DC: World Bank. World Bank. 2023c. Business Ready: Methodology Handbook. Washington, DC: World Bank. World Bank. 2023d. Digital-in-Health: Unlocking the Value for Everyone. Washington, DC: World Bank. World Bank. 2023e. Enablers of Inclusive Cities: Enhancing Access to Services and Opportunities. Washington, DC: World Bank. World Bank. 2023f. Ending Poverty on a Livable Planet: Report to Governors on World Bank Evolution. Washington, DC: World Bank. World Bank. 2023g. International Finance Corporation Additionality in Middle-Income Countries. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023h. Mid-Term Review of the World Bank Group Strategy for Fragility, Conflict, and Violence (2020–25). Washington, DC: World Bank. World Bank. 2023i. The World Bank Group’s Early Support to Addressing the COVID -19 Economic Response, April 2020–June 2021: An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023j. World Bank Group Support to Demand-Side Energy Efficiency. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023k. The World Bank’s Role in and Use of the Low-Income Country Debt Sustainability Framework. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023l. World Bank Support for Domestic Revenue Mobilization. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2024a. “Becoming a Better Bank.” World Bank, Washington, DC. World Bank. 2024b. A Development Approach to Advancing Gender Engagement and Addressing Gender Inequalities in Fragile, Conflict, and Violent Situations. Washington, DC: World Bank. World Bank. 2024c. The Knowledge Compact for Action: Transforming Ideas into Development Impact — For a World Free of Poverty on a Livable Planet. Washington, DC: World Bank. World Bank. 2024d. New World Bank Group Scorecard FY24–FY30: Driving Action, Measuring Results. Washington, DC: World Bank. World Bank. 2024e. “World Bank Group Expands Its Crisis Toolkit to Empower Countries amid Intertwined Crises.” Factsheet, World Bank, Washington, DC. 56 References World Bank. Forthcoming. Global Challenge Program: Forests for Development, Climate, and Biodiversity (GCP-F). Approach Paper. Washington, DC: World Bank. World Bank Group. 2024. World Bank Group Gender Strategy 2024–2030: Accelerate Gender Equality to End Poverty on a Livable Planet. Washington, DC: World Bank Group. 57 Appendix A. Progress on Recommendations This appendix captures the details of the progress made on Independent Evaluation Group recommendations for fiscal years 2021 through 2024 (table A.1). The evaluations and their corresponding recommendations are organized according to the World Bank Groups’ pillars—People, Planet, Infrastructure, Digital, and Prosperity. Two additional categories—cross-cutting and corporate—are also used. Refer to the relevant sections of chapter 4 that provide a full discussion for each evaluation. Two sets of coding are used. One shows the verified progress on the recommendation against four standard categories (LE, EE, CD, and PC), color coded for easy visual review. The other uses a “traffic light” system indicating the level of follow-up required. LE Limited evidence of progress EE Emerging evidence of a change in the direction of travel CD Evidence substantiates a change of direction of travel PC Progress constrained A Recommendation that needs additional follow-up S Recommendation on track of implementation P Recommendation that showed progress toward outcome R Recommendation retired 59 Appendix A Progress on Recommendations Table A.1. Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations People Undernutrition R 1: Adjust nutrition programming in country portfolios to (i) give more priority to institutional — — LE EE strengthening for coordination and implementation of multisectoral nutrition interventions and (ii) P increase focus on subnational targeting of interventions to reflect areas of greatest disadvantage and persistency of need. R 2: Strengthen nutrition support in GPs to (i) rebalance investments to have greater emphasis on — — LE LE nutrition-specific interventions and (ii) increase focus on social norms, interventions, and behavior S changes, with more attention to tracking expected achievements to improve nutrition determinants. COVID-19 Health and Social Response R 1: Use the World Bank’s crisis recovery efforts to strengthen the resilience of essential health and — — — LE S education services to ensure that human capital is protected in a crisis. R 2: Apply a gender equality lens to health and social crisis response actions across sectors. — — — LE S R 3: Help countries strengthen regional cooperation and crisis response capacities for public health — — — LE S preparedness. R 4: Build on the COVID-19 experience to strengthen the World Bank’s internal crisis preparedness — — — EE P so that it has the tools and procedures ready to respond to future emergencies. Planet Natural Resource Degradation and Vulnerability R 1: The World Bank should identify and analyze natural resource degradation and vulnerability — LE EE EE nexus issues and leverage this knowledge in Systematic Country Diagnostics and in country P engagements where such issues matter for achieving sustainable poverty reduction and shared prosperity. R 2: World Bank operations that address natural resource degradation should direct attention to — LE EE EE resource governance challenges and use a mix of resource management practices and financial P incentives appropriate for the relevant socioecological systems. 60 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations R 3: World Bank Global Practices involved in addressing natural resource degradation and — EE EE LE associated vulnerability should share knowledge, improve measurement, and enhance coordination A in the design and implementation of their projects to optimize development effectiveness. Municipal Solid Waste Management R 1: To achieve more sustainable and scalable outcomes in municipal waste management, World — — EE CD Bank Group technical and financial support to clients should give clear priority to the adoption and R implementation of waste hierarchy practices, in line with client needs and capabilities for MSWM. R 2: To support the low-income countries where municipal solid waste is growing most rapidly, the — — EE CD Bank Group should identify constraints on demand and investments and leverage external R partnerships to implement context-specific MSWM solutions. R 3: To bring prominence to and spur action on the global municipal solid waste agenda, the Bank — — EE CD Group should take up a clear leadership position, collaborating and convening with developmental R partners. Disaster Risk Reduction R 1: Incorporate DRR activities in regions and sectors and for hazards that exhibit significant — — — CD R coverage gaps. R 2: Identify and measure the effects of DRR activities on exposure and vulnerability to strengthen — — — EE P the development case to clients facing serious disaster risks. R 3: Integrate the needs of populations that are disproportionately vulnerable to disasters caused by — — — EE P natural hazards into DRR project targeting and design, implementation, and results reporting. R 4: In countries affected by serious natural hazards and fragility and conflict risks, identify and — — — EE assess the ways in which hazards and conflict interrelate, and use this knowledge to inform country P engagement and project design. Agrifood Economics R 1: To enhance its effectiveness in developing agrifood systems, the Bank Group’s efforts to — — — EE support production technologies should be complemented by efforts to improve market access, P especially in LICs and in countries at the traditional stage of agrifood system development. R 2: To achieve more sustainable agrifood systems, where conditions permit, the Bank Group should — — — LE support production diversification to meet the growing demand for undersupplied, high-value- S added nutritious products while ensuring that smallholder farmers and SMEs benefit from the diversification. 61 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations R 3: To enhance the contribution of IFC support for agrifood system development, IFC should pilot — — — LE and adopt more effective ways to support clients to better meet E&S Performance Standards, S especially in LICs. Infrastructure Urban Resilience R 2: The design and implementation of World Bank projects that build urban resilience should LE LE EE CD systematically incorporate resilience characteristics and articulate their application throughout the project cycle. These should include the following: (i) design standards in line with resilience risks, (ii) R cost-benefit analysis in line with resilience risks, (iii) city and interjurisdictional coordination, and (iv) inclusive approaches for vulnerable people. R 3: In urban areas where the client has identified crime and violence as a resilience risk, the World LE LE EE PC Bank’s support should be based on a localized typology of crime and violence that is informed by A relevant analytic work. This approach should be supported by an assessment of the mechanisms most effective at reducing crime and violence within operations. Renewable Energy R 2: The Bank Group to support renewable energy scale-up through comprehensive, long-term — EE EE CD country engagements, with coordinated Bank Group solutions, based on the comparative R advantages of each institution, to address barriers, aided by robust upstream diagnostics. Urban Spatial Growth R 1: Adopt a framework that links the determinants of urban expansion to pathways for managing — — EE CD R urban spatial growth and that contributes to the achievement of SDGs 1 and 11. R 2: Support World Bank clients with anticipating and preparing for urban spatial growth using — — LE LE S preventive approaches, not just curative ones. R 3: Strengthen and ensure implementation of the World Bank’s protocol to identify and record — — LE LE precise project locations and collect land market data necessary to support clients with managing A urban spatial growth. Energy Efficiency R 1: Intensify DSEE support to MICs for decarbonization and wider socioeconomic benefits. (Bank — — — EE P Group) 62 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations R 2: Develop energy efficiency sector-specific approaches in a select group of LMICs that seek — — — LE productivity gains alongside or via DSEE, even if energy efficiency policy reforms are in early stages. S (World Bank and IFC) R 3: Expand DSEE approaches by incorporating the reduction of indirect emissions (Scope 3), — — — LE S including embodied and operational carbon, in DSEE project design. (World Bank and IFC) R 4: Exploit untapped DSEE opportunities and help clients leapfrog by exploring cross–Practice — — — EE P Group (World Bank) and cross–Industry Group (IFC) approaches. (World Bank and IFC) Digital Mobilizing Technology for Development R 1: Where DTT offers opportunities to make progress on the twin goals more effectively or — EE LE LE efficiently, ensure that the World Bank Group avails itself of those opportunities and addresses, in S particular, the risks posed by DTT. R 2: Build a Bank Group workforce with the skills required to harness DTT opportunities and mitigate — LE LE EE P DTT risks by identifying DTT-relevant skills, determining gaps in these skills, and filling these gaps. R 3: Improve the effectiveness and efficiency of World Bank procurement for complex technology — EE EE EE P projects. (World Bank only) Prosperity Regional Integration R 5: Strengthen the design of IDA’s Regional Policy Window–supported projects to improve the LE LE CD CD R assessment of spillover effects and generate evidence based on robust indicators. State-Owned Enterprises R 1: The World Bank Group should apply a selectivity framework for SOE reform support that — EE EE EE considers country governance conditions, control of corruption, and sector- and enterprise-level P competition. R 2: The Bank Group should apply the Maximizing Finance for Development and its embedded — EE EE EE P Cascade approach for SOE reform. Public Financial and Debt Management R 1: The World Bank should regularly monitor the quality of the key pillars of PFDM for each IDA- — EE EE CD R eligible country, possibly through a centralized country-specific PFDM assessment. 63 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations R 2: Actively use the previously described assessment (R 1) to prioritize and sequence World Bank — LE LE CD R support for PFDM capacity building and reform in IDA-eligible countries. Doing Business Report and Business Ready Lesson 1: Recognizing the powerful motivational effect of reform indicators, especially those that — — LE LE facilitate country rankings, this evaluation notes the limitations in the coverage and guidance S offered by any single indicator set on its own and advocates integrating them with complementary analytic tools and indicators. Lesson 2: Recognizing the granularity and specificity of individual reforms in any given country — — PC PC context, the findings from this evaluation suggest that it is better to avoid using business regulatory or similar global indicators as explicit reform objectives or monitoring indicators in World Bank S Group projects and country strategies focused on improving the business environment. This does not preclude the use of primary data to agreed targets that track and measure critical Bank Group institutional commitments. Lesson 3: Global indicators coverage and specifications are improved if, at regular and predictable — — LE LE intervals, they are updated to reflect learning from research and field experience to (i) improve links S to important development outcomes; (ii) strengthen relevance to the experience of the subject of coverage; and (iii) adapt to technological changes in the areas covered by the indicators. Lesson 4: The Doing Business experience indicates the need for mechanisms and safeguards to — — PC LE ensure the accuracy and validity of Bank Group global indicator-based reports and related S communications, using robust and transparent standards of evidence. Sustainable Development Finance Policy R 1: Consideration should be given to expanding the countries covered by the Debt Sustainability — — PC CD Enhancement Program beyond those at moderate or high levels of debt distress or in debt distress. R A low level of debt distress alone should not be sufficient for exclusion from the Debt Sustainability Enhancement Program, and IEG recommends applying an additional filter. R 2: PPAs should emanate from an up-to-date assessment of country-specific debt stress and be set — — LE EE P explicitly within a longer-term reform agenda. R 3: Where PPAs support actions that need to be taken regularly (for example, debt reporting to — — LE CD parliament), they should aim for long-lasting institutional reforms rather than relying on one-time actions. PPAs should seek to institutionalize good practice in fiscal and debt management by R supporting the establishment of statutory requirements, the existence of which can help depoliticize future decisions. 64 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations Low-Income Country Debt Sustainability Framework R 1: Expectations of the World Bank in taking the lead on long-term growth prospects should be — — — LE clarified. Given the World Bank’s development mandate, the current guidance is appropriate but S comes with the expectation that the World Bank systematically take the lead in highlighting the country-specific factors that influence long-term growth, which is not currently the case. R 2: The recently increased attention to debt data coverage should be sustained and extended; — — — EE P greater attention is needed to assess data quality. R 3: The DSA should be more directly and consistently used to inform priorities for the identification — — — EE of fiscally oriented prior actions in development policy operations and SDFP performance and policy P actions. R 4: The World Bank should continue to give increasing attention in the LIC-DSF to the long-term — — — LE implications of climate change, in terms of both growth and fiscal requirements of adaptation and S mitigation. Domestic Revenue Mobilization R 1: On a country-by-country basis, regularly take stock of the findings of the broad range of tax — — — EE diagnostics tools and instruments to (i) identify knowledge gaps and (ii) more systematically inform priority setting for country-level policy dialogue, capacity building, and operations to improve DRM. P Rigorous analysis and diagnostics are needed to inform country-specific DRM strategy and operational priorities, particularly in IDA-eligible countries. R 2: Given the potentially large and regressive fiscal impact of tax exemptions, the World Bank — — — EE should regularly assess the effectiveness and efficiency of tax exemptions in achieving country- P specific policy objectives, with an eye to more actively supporting the sustainable reduction of regressive tax exemptions through policy advice and prior actions in DPOs. R 3: The frequency with which tax policy reforms are reversed calls for strengthening incentives for — — — LE S sustaining reforms and making reversal more challenging. R 4: Provide clearer guidance to staff on the choice of results indicators to measure the impact of — — — LE DRM support, facilitate learning from experience, improve monitoring of progress toward DRM- S related objectives, and promote an outcome orientation in the World Bank’s support for DRM. COVID-19 Economic Response R 1: To effectively address future crises, codify a global crisis response playbook, ideally developed — — — EE P jointly with the IMF. 65 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations R 2: To respond effectively during the recovery phase of the crisis, explore increasing use of — — — EE structured finance solutions (such as partial credit guarantees, subordinated debt, and quasi-equity P instruments) with a view to supporting small- and medium-size firms. Cross-Cutting Private Capital Mobilization R 2: Expand private capital mobilization platforms, guarantees, and disaster risk management — LE EE EE P products commensurate with project pipeline development (for the World Bank Group). R 3: Develop new products and improve product alignment with the needs of new investor groups — EE EE CD R and partners (for IFC and MIGA). World Bank Engagement in Situations of Conflict R 1: To enhance the conflict sensitivity of World Bank engagement, ensure that politically sensitive, — — EE CD confidential analysis is generated, retained, and managed so that it can be used by select future staff R working on that country. R 2: Ensure that country engagements are informed by timely analyses of conflict dynamics and — — EE CD R risks. R 3: Address factors that dissuade World Bank engagement in conflict-affected areas. — — LE LE A R 4: In conflict-affected countries, rethink what success looks like. — — EE LE A Private Investment in Fragile and Conflict-Affected Situations R 1: IFC and MIGA should continue to review their financial risk, make more explicit the implications — — LE LE of IFC’s portfolio approach for FCS, and enhance capabilities to address nonfinancial risks to ensure S they align with achieving business growth targets and impacts in FCS. R 2: To focus on the development of bankable projects, IFC and MIGA should further recalibrate — — LE LE their business models, client engagements, and instruments to continuously adapt them to the S needs and circumstances of FCS and put in place mechanisms to track their effectiveness for real- time learning. 66 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations IFC Additionality in Middle-Income Countries R 1: To enhance institutional accountability, learning, and transparency, address gaps in internal — — — LE systems related to monitoring, supervision, and reporting of additionality at the project and S portfolio level. R 2: To enhance commitment to and fulfillment of IFC’s strategic objectives, IFC should bring its — — — LE S strategy for additionality in MICs and its pattern of activity in MICs into closer alignment. R 3: To enhance its strategic approach to proactive creation of markets and mobilization of private — — — LE capital to provide a critical contribution to the Sustainable Development Goals, IFC should S incorporate its additionality approach into its country strategies and sector deep dives. Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence R 1: Make priorities regarding gender equality (including on WGEE and GBV) more explicit in country — — — LE strategies, based on strong analytics (primarily Systematic Country Diagnostics and the World Bank S Risk and Resilience Assessments) and in collaboration with key stakeholders. R 2: Foster engagements with communities, civil society, women’s organizations, local authorities, — — — LE S and other key stakeholders to define gender equality objectives and the actions to achieve them. R 3: Ensure that gender expertise tailored to the context is available for FCV-affected countries to — — — LE S support projects, as well as the country engagement. R 4: Coordinate and collaborate with relevant international stakeholders engaged in gender equality — — — LE in the country, including humanitarian actors. This stronger coordination and collaboration should S leverage each actor’s comparative advantage to achieve common goals. Corporate Citizen Engagement R 3: The World Bank should strengthen the monitoring of its citizen engagement activities by LE LE EE PC A systematically adopting results framework indicators that are results oriented. Convening Power R 1: Scope engagements and contributions to major global convening initiatives more LE EE EE CD R deliberatively. R 2: Enhance how the World Bank and IFC’s internal systems and processes support managing LE EE EE EE R major convening initiatives over their life cycle. 67 Appendix A Progress on Recommendations Progress on Recommendation FY21 FY22 FY23 FY24 Recommendations R 3: Improve links between the World Bank’s global and country work. LE EE EE CD R Outcome Orientation R 1: Reform the country-level results system to ensure that it accurately captures the Bank Group’s — EE EE LE contribution to country outcomes and usefully informs decision-making on country engagements. The Bank Group should keep a country engagement model that articulates clear outcome-level objectives and lays out the pathways that will be pursued to achieve them, conducts periodic reviews to take stock of progress, and includes an end-of-cycle review of evidence and learning. A These reviews need not be a Board deliverable but should be carried out in time to meaningfully inform course correction and the next strategy. It should discontinue the reliance on results frameworks in its country strategies and midterm and terminal reviews. And it should adopt MEL plans for its country engagements. Global Footprint Effectiveness R 1: The World Bank should refine its current approach to managing its staffing global footprint by — — LE LE clearly specifying decentralization’s expected outcomes and adopting principles to guide and adjust A decentralization decision-making based on evidence. R 2: The World Bank should mitigate the risks to knowledge flow brought about by decentralization — — EE EE P and put in place safeguards to avoid developing country and regional silos. R 3: The World Bank should establish clear and structured paths to systematically promote locally — — EE EE P recruited staff’s professional and career growth within its overall approach to improving the effectiveness of its global footprint. Sources: Independent Evaluation Group; World Bank 2018, 2019a, 2019b, 2020a, 2020b, 2020c, 2020d, 2020e, 2021a, 2021b, 2021c, 2021d, 2021e, 2021f, 2021g, 2022a, 2022b, 2022c, 2022d, 2022e, 2022f, 2022g, 2023a, 2023b, 2023c, 2023d, 2023e, 2023f. Note: CD = evidence substantiates a change of direction of travel, with reversals unlikely; DPO = development policy operation; DRM = domestic revenue mobilization; DRR = disaster risk reduction; DSA = debt sustainability analysis; DSEE = demand-side energy efficiency; DSF = debt sustainability framework; DTT = disruptive and transformative technology; EE = emerging evidence of a change in the direction of travel; E&S = environmental and social; FCS = fragile and conflict-affected situation; FCV = fragility, conflict, and violence; FY = fiscal year; GBV = gender-based violence; IDA = International Development Association; IEG = Independent Evaluation Group; IFC = International Finance Corporation; IMF = International Monetary Fund; LE = limited evidence of progress; LIC = low-income country; LMIC = lower-middle-income country; MEL = monitoring, evaluation, and learning; MIC = middle-income country; MIGA = Multilateral Investment Guarantee Agency; MSWM = municipal solid waste management; PC = progress constrained; PFDM = public financial and debt management; PPA = performance and policy action; R = recommendation; SDFP = Sustainable Development Finance Policy; SDG = Sustainable Development Goal; SME = small and medium enterprise; SOE = state-owned enterprise; WGEE = women’s and girls’ economic empowerment. 68 Appendix A Progress on Recommendations References World Bank. 2018. Engaging Citizens for Better Development Results. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2019a. Building Urban Resilience: An Evaluation of the World Bank Group’s Evolving Experience (2007–17). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2019b. Two to Tango: An Evaluation of World Bank Group Support to Fostering Regional Integration. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020a. Renewable Energy: Evaluation of the World Bank Group’s Support for Electricity Supply from Renewable Energy Resources, 2000–2017. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020b. State Your Business! An Evaluation of World Bank Group Support to the Reform of State-Owned Enterprises, FY08–18. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020c. The World’s Bank: An Evaluation of the World Bank Group’s Global Convening. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020d. World Bank Group Approaches to Mobilize Private Capital for Development . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020e. The World Bank Group Outcome Orientation at the Country Level . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021a. The International Development Association’s Sustainable Development Finance Policy: An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021b. Managing Urban Spatial Growth: World Bank Support to Land Administration, Planning, and Development. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021c. Mobilizing Technology for Development: An Assessment of World Bank Group Preparedness. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021d. The Natural Resource Degradation and Vulnerability Nexus: An Evaluation of the World Bank’s Support for Sustainable and Inclusive Natural Resource Management (2009–19). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021e. World Bank Engagement in Situations of Conflict: An Evaluation of FY10 –20 Experience. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021f. World Bank Support for Public Financial and Debt Management in IDA-Eligible Countries. Independent Evaluation Group. Washington, DC: World Bank. 69 Appendix A Progress on Recommendations World Bank. 2021g. World Bank Support to Reducing Child Undernutrition. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022a. The Development Effectiveness of the Use of Doing Business Indicators, Fiscal Years 2010–20. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022b. Enhancing the Effectiveness of the World Bank’s Global Footprint. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022c. The International Finance Corporation’s and Multilateral Investment Guarantee Agency’s Support for Private Investment in Fragile and Conflict-Affected Situations, Fiscal Years 2010–21. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022d. Reducing Disaster Risks from Natural Hazards: An Evaluation of the World Bank’s Support, Fiscal Years 2010–20. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022e. Toward Productive, Inclusive, and Sustainable Farms and Agribusiness Firms: An Evaluation of the World Bank Group’s Support for the Development of Agrifood Economies (2010–20). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022f. Transitioning to a Circular Economy: An Evaluation of the World Bank Group’s Support for Municipal Solid Waste Management (2010–20). Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022g. The World Bank’s Early Support to Addressing COVID -19: Health and Social Response—An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023a. Addressing Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence: An Evaluation of the World Bank Group’s Support . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023b. International Finance Corporation Additionality in Middle-Income Countries. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023c. The World Bank Group’s Early Support to Addressing the COVID -19 Economic Response, April 2020–June 2021: An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023d. World Bank Group Support to Demand-Side Energy Efficiency. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023e. The World Bank’s Role in and Use of the Low-Income Country Debt Sustainability Framework. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023f. World Bank Support for Domestic Revenue Mobilization. Independent Evaluation Group. Washington, DC: World Bank. 70 The World Bank 1818 H Street NW Washington, DC 20433