The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response April 2020—June 2021 An Early-Stage Evaluation © 2023 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org ATTRIBUTION Please cite the report as: World Bank. 2023.The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response. April 2020—June 2021. An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. COVER PHOTO Adapted from shutterstock / Aleutie EDITING AND PRODUCTION Amanda O’Brien GRAPHIC DESIGN Luísa Ulhoa This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The bound- aries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissem- ination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response April 2020—June 2021 An Early-Stage Evaluation January 13, 2023 Contents Abbreviations v Acknowledgments vi Overview viii Management Response xxiv 1. Background and Context�����������������������������������������������������������������������������������������1 The Economic Impact of the COVID-19 Pandemic 3 The World Bank Group Response 8 Objective and Scope of the Evaluation 10 Evaluation Questions, Methods, Limitations, and Links with Other Evaluations 11 Agile Corporate Response to Address the Economic Implications of the 2.  COVID-19 Crisis�������������������������������������������������������������������������������������������������������18 The World Bank Group’s Corporate Actions to Respond to the COVID-19 Crisis 20 Portfolio of World Bank Group Interventions 21 Relevance of the World Bank Group Early Response������������������������������������������30 3.  Relevance for Countries and Sectors 32 Informed Design 42 Comparative Advantages 44 Data Gaps for a Targeted Response 46 Quality of the World Bank Group Early Response������������������������������������������������50 4.  Influence on Client Strategies and Actions 52 Coordination 63 Monitoring, Safeguards, and Governance 70 Factors Affecting the Quality of the World Bank Group’s Response 74 Conclusions and Recommendations�������������������������������������������������������������������� 78 5.  Bibliography���������������������������������������������������������������������������������������������������������������82 ii Boxes Box 1.1. Sovereign Debt and COVID-19 5 Nigeria: Client Structural Issues as Constraints on World Bank Support Box 3.1.  39 The International Finance Corporation and Multilateral Box 3.2.  Investment Guarantee Agency Early Response 41 Box 3.3. Assessing Country Conditions: The Panama and Africa Cases 44 Box 3.4. Tracking World Bank Funds in Croatia 47 Box 4.1. Effect of the World Bank COVID-19 Early Response 53 Box 4.2. Adaptive Management in Pakistan 55 Box 4.3. The Heavily Indebted Poor Countries Initiative 60 Recent Independent Evaluation Group Evaluations on Box 4.4.  Public Finance and External Debt 62 World Bank–International Monetary Fund Collaboration Box 4.5.  in Ecuador and Pakistan 64 World Bank Group Coordination in Colombia and Europe and Central Asia65 Box 4.6.  World Bank Group’s Approach to COVID-19 Response in Georgia Box 4.7.  69 Box 4.8. Slow Disbursement in the Lao People’s Democratic Republic 72 Figures Change in Gross Domestic Product and Gross Domestic Figure 1.1.  Product Growth from 2019 to 2020 by Region 4 Figure 1.2. Main Transmission Channels of the COVID-19 Economic Crisis 7 Theory of Change for World Bank Early Response to Figure 1.3.  Economic Implications of COVID-19 9 Annual Average Multilateral Development Bank Commitments Figure 2.1.  before and during the COVID-19 Crisis 22 World Bank and IFC Commitments and Disbursements of Figure 2.2.  COVID-19 Early Response Addressing Economic Implications, by Region (April 2020–June 2021) 25 World Bank Group COVID-19 Response—New Approaches Figure 2.3.  and Existing Instruments 27 iii Relationship between Countries’ Income Levels Figure 3.1.  and Need Score Estimates 34 Relevance of World Bank Group Support in Relation Figure 3.2.  to Initial Conditions 35 Within-Country Needs Analysis by Sector versus Figure 3.3.  World Bank Group Support in Case Study Countries, Sample 37 Macrofiscal Support via Development Policy Operations, Figure 3.4.  Select Time Periods 40 Figure 3.5. Countries with Extrabudgetary COVID-19 Response Funds 48 Figure 4.1. Comparison of Country Needs with External Debt 57 Table World Bank Group COVID-19 Pandemic Response Plan Evaluation Table 2.1.  Portfolio (April 2020–April 2021) 23 Appendixes Appendix A. Evaluation Methods 88 Appendix B. Adaptive Management and Crisis Learning Framework 117 Appendix C. Country Needs Analysis 122 Appendix D. World Bank Group Reporting to the Board of Executive Directors 126 iv Abbreviations CAT DDO catastrophe deferred drawdown option CERC contingent emergency response component DPF development policy financing DPO development policy operation DSA debt sustainability analysis EBF extrabudgetary fund EOC Emergency Operations Center ESF Environmental and Social Framework FTCF Fast-Track COVID-19 Facility GDP gross domestic product GP Global Practice IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation IMF International Monetary Fund IPF investment project financing LIC low-income country MIGA Multilateral Investment Guarantee Agency MPA Multiphase Programmatic Approach MSME micro, small, and medium enterprise PGC Philippine Guarantee Corporation Independent Evaluation Group World Bank Group    v SDG Sustainable Development Goal WHO World Health Organization All dollar amounts are US dollars unless otherwise indicated. Acknowledgments This report was prepared by an Independent Evaluation Group team led by Raghavan Narayanan, senior evaluation officer, and Željko Bogetić, lead economist, and was conducted under the guidance and supervision of Marialisa Motta, manager; Carmen Nonay, director; and Alison M. Evans, Director-General, Evaluation. The core evaluation team included Nana Sika Ahiabor (research analyst), Aarre Laakso (developmental editor), Anna Sengphet Lattanavong (research The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Acknowledgments analyst), Stephen Mier (research analyst), Dominik Naeher (econometrician), Romayne Pereira (program assistant), Maria Elena Pinglo (senior evaluator), Melvin Vaz (senior evaluator), and Virginia Ziulu (data scientist). Subject matter expert Onno Ruhl advised the team. The team was supported by the methods advisory group of Estelle Raimondo, senior evaluation officer, and Jozef Vaessen, adviser. Technical leads Jenny Gold, Rasmus Heltberg, Lauren Kelly, and Andrew Stone provided valuable insights. Country case studies, firm case studies, and syntheses were conducted by Željko Bogetić, Wasiq Ismail, Stephen Mier, Fernando Montes Negret, Maria Elena Pinglo, Rama Seth, Isha Sharma, Madeleine Varkay, and Melvin Vaz. Rasmus Heltberg and Onno Ruhl conducted key informant interviews. We thank the Approach Paper’s peer reviewers: Afsaneh Beschloss (chief executive officer, RockCreek Capital); Stijn Claessens (head of financial stability policy, Bank for International Settlements); David Ndii (managing director, Africa Economics); and Liliana Rojaz-Suarez (fellow, Center for Global Development). The report was peer reviewed by David Ndii (man- aging director, Africa Economics); Liliana Rojaz-Suarez (fellow, Center for Global Development); and Mark Sundberg (chief economist, Millennium Challenge Corporation). The Independent Evaluation Group evaluation team and management thank numerous staff of the World Bank Country Management Units, and Opera- tional Strategy, Risk, and Country Operations staff (Praveen Kumar, Audra Sedluk, and Sona Verma); staff of the Multilateral Investment Guarantee Agency (Sandro Diez Amigo, Hiroyuki Hatashima, and Vlajko Senic); staff vi of the International Finance Corporation (Belen Castuera, Teng Jiang, Ellen Schwab, and Nuria Perez Tello); and staff of the International Monetary Fund (Jeremie Cohen-Setton and Prakash Loungani). Independent Evaluation Group World Bank Group    vii Overview The COVID-19 pandemic caused a global economic crisis. Since March 2020, COVID-19 has put all aspects of economic and social life under immense strain. The spreading pandemic led to mandatory business closures, limita- tions on contact, and travel bans that contributed to a massive disruption of global economic activities. Unlike past crises (for example, the global financial crisis of 2008), the COVID-19 economic crisis did not affect just one sector; simultaneous supply and demand shocks affected numerous sectors and countries in quick succession. The economic crisis caused by COVID-19 threatens the achievement of The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview global development priorities and has caused a massive debt buildup. In 2020 alone, the pandemic caused direct economic losses of approximate- ly $7.4 trillion. The COVID-19 crisis has accelerated the pace of borrowing globally, especially in highly indebted countries. At more than 65 percent of gross domestic product, government debt in emerging markets is now 25 percentage points higher than in 2010 and continues to increase. Large budgetary reallocations to finance urgent expenses in health and essen- tial services were inevitable but have threatened growth, reversed gains in poverty reduction, and put the achievement of all Sustainable Development Goals at risk. The economic effects of the COVID-19 pandemic materialized through multiple transmission channels. Governments aimed to contain the pandem- ic with lockdowns, which prevented firms from operating and interrupted trade. Household income and investments fell as workers lost their jobs or had their hours cut. Credit risk increased significantly in March–May 2020 (IMF 2020a), sometimes translating to nonperforming loans and increased liabilities for governments. Drops in government revenues and increases in government spending led to increasing sovereign debt. The World Bank Group applied its full capacity to mitigate the economic impacts of the pandemic. It targeted governments, financial institutions, and firms. The Bank Group supported governments’ and firms’ responses viii to the crisis through several instruments, including analytical and advisory activities, lending, emergency liquidity, and direct support to firms. Bank Group support was aimed at “keeping the lights on” and trade flowing for micro, small, and medium enterprises (MSMEs), preserving jobs for households, enhancing the liquidity and capital positions of financial institutions, and supplementing government budgets for crisis spending. We expect that the final outcomes will be reducing firm and household bankruptcies and preventing banking crises and sovereign defaults, with the ultimate impact of protecting livelihoods during the COVID-19 emergency, returning to economic growth, and resuming declines in poverty. The objective of this evaluation was to conduct an early-stage assessment of the Bank Group’s response to the economic implications of the COVID-19 crisis, which is critical to inform the next steps in the response to the cur- rent crisis and help prepare for future crises. Taking stock of the Bank Group response at this point—including early successes and failures—is important to inform the next set of Bank Group support efforts. The next phase of the Bank Group response will need to support both countries still mired in the effects of the COVID-19 pandemic, including its economic implications, and countries engaged in restructuring and recovery. An early assessment is also essential for extracting lessons from the Bank Group’s early response to the COVID-19 crisis to prevent, prepare for, and better address future crises. The findings, lessons, and recommendations in this report are designed to contribute to learning for the Bank Group overall. They may be useful in Independent Evaluation Group World Bank Group    ix adapting to various phases of the current crisis and, potentially, future crises. However, they are intended to be applied at a strategic and institutional lev- el, not to provide ready-made solutions for the unique challenges of specific operations or projects. A parallel Independent Evaluation Group evaluation, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response, examines early World Bank support for addressing the health and social implications of the pandemic. The evaluation assesses the Bank Group’s early response to the COVID-19 economic crisis. We examined the Bank Group’s early response to the cri- sis, defined as interventions over the 15 months of April 2020 through June 2021. Although the case studies cover the whole evaluation period, the port- folio analysis considers a subset of the evaluation period, from April 2020 to April 2021, based on Bank Group COVID-19 response data availability. To assess learning during the crisis, we considered two evaluation windows: the acute crisis phase (April 1, 2020, to December 31, 2020) and the incipi- ent recovery phase (January 1, 2021, to June 30, 2021). The two phases were identified based on the Oxford Stringency Index and country-level economic indicators, including gross domestic product. The objective of identifying the two windows was to assess whether the Bank Group internalized learning from the first period of the crisis to address the challenges that were materi- alizing in the (incipient) recovery phase. We studied the relevance and quality of the Bank Group’s response to the COVID-19 crisis. We assessed the relevance of the Bank Group’s interven- tions on three dimensions: (i) the extent to which the Bank Group targeted The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview its early response based on clients’ and sectors’ needs, (ii) the extent to which the Bank Group used timely diagnostics and lessons from past crises to inform its early response, and (iii) the extent to which the early response leveraged the Bank Group’s comparative advantages. We also studied the quality of the Bank Group response on three dimensions: (i) the extent to which the Bank Group’s early response influenced client strategies, (ii) the extent to which the Bank Group coordinated its early response among its constituent institutions and with development partners, and (iii) how well the Bank Group’s early response handled monitoring, safeguards, and gov- ernance. Finally, we assessed how well the Bank Group learned during the crisis, including how well it learned from its own work on COVID-19 (intrac- risis learning), developed countries’ approaches to COVID-19, and past crises (intercrisis learning). The evaluation has limitations because of its scope. The evaluation focuses on assessing the Bank Group’s response to the economic implications of the COVID-19 crisis (“the impact of the crisis on livelihoods”). It was developed concurrently with—and is complemented by—an evaluation of the World Bank’s health, human capital, and social response to the COVID-19 crisis (which looks at “the impact of the crisis on lives”). The decision to have two separate reports stems from the recognition that assessing the Bank Group’s work on lives and livelihoods in a single evaluation would have limited the x depth of the analysis. Both evaluations will be further complemented by future, ex post assessments of the impact of the crisis. This thematic evaluation is limited to the Bank Group’s response to the eco- nomic implications of the COVID-19 crisis. It is not a corporate evaluation or a comprehensive evaluation of the COVID-19 crisis responses of all three institutions. It was developed in concurrence with—and it is complement- ed by—a parallel evaluation of the World Bank’s health, human capital, and social response to the COVID-19 crisis. Neither evaluation assesses overall corporate response to the crisis, be it of the World Bank, the International Finance Corporation (IFC), or the Multilateral Investment Guarantee Agency (MIGA). Rather, they look at the Bank Group’s response on their respective topics. For example, activities under the IFC Global Health Platform are out- side the scope of this evaluation because the underlying projects are related to the health sector. The evaluation also has limitations because of its nature as an early-stage learning assessment. Because the COVID-19 response is ongoing and oc- curred in a context of high uncertainty about how to address the global economic crisis quickly and effectively, this early-stage evaluation does not attempt to estimate the probability that early response projects will succeed. The identification of the two windows in the evaluation period—the acute crisis and incipient recovery phases—is based on a preliminary assessment. The evaluation of the incipient recovery phase—which lasts only 6 months, including 4 months of portfolio data—should be regarded as a preliminary Independent Evaluation Group World Bank Group    xi assessment of the Bank Group’s work supporting countries’ and firms’ initial recovery, with the aim of informing future Bank Group actions to support the next phase of the recovery. Activities under the IFC Base of the Pyramid Platform, which the Board of Executive Directors approved in February 2021, are outside the scope of this evaluation because the underlying projects are outside the evaluation period. Finally, the evaluation is limited because of certain methodological con- straints. We treated the IFC and MIGA portfolios only via a sample of firm-level cases. Therefore, we could not assess the relevance or quality of the overall IFC and MIGA response across all sectors and industry groups. We used a novel framework to analyze country needs, and there were data limitations on the analysis in some countries and sectors. The availability of internal data (such as supervision and portfolio documents), external data, and key informants also limited the evaluation. This evaluation assesses only IFC advisory services and World Bank advisory services and analytics to the extent that they were linked to country- and firm-level case studies because extensive coverage of advisory work was not feasible. The World Bank Group Delivered the Largest Crisis Response in Its History Thanks to Strategic and Agile Decision-Making and Learning from Past Crises At the onset of the pandemic, Bank Group senior management demonstrated The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview strategic thinking, agile corporate decision-making, and learning from past crises. Senior management articulated the Bank Group strategy early in March 2020. The World Bank’s strategy included front-loading International Development Association (IDA) spending allocations and seeking an unprecedented IDA Replenishment a year ahead of schedule. It also included activating the International Bank for Reconstruction and Development’s crisis buffer to release additional financing. The World Bank aligned with World Health Organization guidance on technical health issues. With the International Monetary Fund (IMF), it launched a global convening effort on debt suspension via the Debt Suspension Service Initiative to help indebted client countries free up fiscal space for crisis response. The World Bank established the Emergency Operations Center and worked across various groups and functional areas to address different facets of the crisis. Similarly, IFC and MIGA organized emergency response committees working across industry groups and launched fast-track COVID-19 facilities, guarantee program envelopes, and a joint trade finance initiative. The design and agile deployment of the MIGA and IFC fast-track COVID-19 facilities and the increased use of the World Bank Multiphase Programmatic Approach demonstrated learning from previous crises. As a result of agile corporate actions, the Bank Group delivered the largest crisis response in its history and the largest COVID-19 response across de- velopment partners. The Bank Group committed $157 billion to address the xii COVID-19 crisis, of which it specifically committed $65 billion to address the economic implications of COVID-19 (International Bank for Reconstruction and Development and IDA: $49 billion; IFC: $10 billion; MIGA: $6 billion). World Bank commitments went mainly to Africa and Latin America and the Caribbean, Regions with many IDA clients that had greater economic needs to respond to the crisis. The Equitable Growth, Finance, and Institutions Practice Group and the Social Protection and Jobs Global Practice provided the main share of the World Bank response. The Financial Institutions Group provided the main share of the IFC investment services response. World Bank disbursements to IDA clients in Africa and the Middle East and North Africa Regions were relatively low. IFC disbursements to clients were high in all regions. The Bank Group’s response drew on a mix of existing instruments and new approaches. The World Bank expanded its use of the Multiphase Programmatic Approach—in addition to individual investment projects and development policy operations—to address the crisis. The World Bank response also leveraged contingent emergency response components and catastrophe deferred drawdown options as part of its lending operations based on countries’ conditions. IFC developed new envelopes under its fast-track COVID-19 facility and programs driven by the $2 billion Real Sector Crisis Response Envelope and the $6 billion Financial Institutions Group Response Envelope. MIGA launched two guarantee programs totaling $5 billion focused on the financial sector: credit enhancement and capital Independent Evaluation Group World Bank Group    xiii optimization. MIGA also supported IFC’s trade finance activities by issuing risk coverage support to commercial banks. The Bank Group’s support contributed to saving jobs and protecting house- holds’ incomes. Its response to the economic implications of COVID-19 sought to directly meet the needs of governments, financial institutions, and firms, addressing the needs of households indirectly. The Bank Group’s swift and broad interventions supported macrofiscal and financial stability, public sector institutional improvements, and capital enhancement and liquidity for financial institutions and MSMEs in sectors affected by the pandemic and the lockdowns, resulting in jobs and incomes saved. The World Bank Group Response Was Mostly Relevant Overall and Highly Relevant in Low- Income Countries, through Targeting, Informed Design, and Comparative Advantages The Bank Group’s early response was highly relevant to low-income coun- tries, which had greater needs. Bank Group support focused on low-income countries, which tended to be more vulnerable and needed more support than richer countries during the COVID-19 pandemic in 2020. Yet, some countries with high needs (such as Angola, Gabon, and South Africa) re- ceived limited aggregate support because of various constraints, including, for example, having small, ongoing programs with the World Bank or being nonaccrual status. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview There was considerable variation in how well Bank Group support aligned with the sector-specific needs within each country. In some countries (such as Cabo Verde and Pakistan), the Bank Group supported all sectors in need (economic, public sector, and social protection). In other countries (such as the Philippines and Senegal), it supported select sectors (economic and social protection) and did not support other sectors in need, such as agricul- ture, manufacturing, and services. Macrofiscal support via development policy financing was highly relevant to client governments. The volume of commitments for macrofiscal support was higher than during the global financial crisis of 2008 and higher than prepandemic support, suggesting the relevance of the response in address- ing the economic shock in client countries. The Bank Group directed budget support primarily to social transfers, institutional strengthening, the finan- cial sector, and MSMEs. The IFC and MIGA early response was mainly relevant to local banks with insufficient risk appetite to provide crisis financing. Financial sector support to repeat clients dominated the IFC and MIGA early response. Without IFC and MIGA, existing clients would have either defaulted on loans or cut back on their onlending programs, leading to severe supply chain disruptions and job losses. xiv Existing diagnostics informed the World Bank’s early response. The World Bank’s early response aligned well with the World Development Report 2022: Finance for an Equitable Recovery, country program frameworks, and analyti- cal instruments such as Financial Sector Assessment Programs and Country Private Sector Diagnostics. The Bank Group used lessons from past crises well to inform its early re- sponse. Consistent with lessons learned from past crises, country analytical work contributed to the World Bank’s early response, which aimed to address the pandemic through a three-pronged approach: budget, public health, and MSME support (for example, in Cabo Verde, Georgia, and the Philippines). Similarly, IFC adapted its response to the local context well based on les- sons from the global financial crisis (for example, IFC’s project with Liquid Telecom). MIGA supported trade finance activities in partnership with IFC, backstopping losses for client banks. The IFC-MIGA Global Trade Finance Initiative, an innovative new product based on lessons from past crises, intended to support MSME value chains in Africa that had insufficient access to credit, training, and certification. Two MIGA projects that did not fully consider country conditions in their designs were less relevant. MIGA overestimated the liquidity needs of banks in Panama and in some African countries (Botswana and Eswatini). Although highly complex at the time of the COVID-19 crisis, assessing country condi- tions carefully is important to channel resources to vulnerable clients. Country office staff needed greater clarity on crisis protocols, mechanisms, Independent Evaluation Group World Bank Group    xv and guidelines for developing targeted approaches to address the econom- ic implications of a crisis. Many country teams welcomed diagnostic tools, real-time sector analysis papers, and the World Development Report 2022 (World Bank 2022d). However, they seek greater clarity and preparedness for day-to-day matters, including real-time dashboards to inform sector tar- geting, guidance on strategic portfolio choices, and surge-capacity plans to promote staff welfare during crises. The Bank Group’s capacity to build its crisis response on its comparative advantages—global footprint and global knowledge that translates into fi- nancing and advisory support—was mixed. The Bank Group used existing di- agnostics and the World Development Report 2022 well to inform its response to the crisis (World Bank 2022e). It also compiled new data—for example, high-frequency phone surveys of households and surveys of businesses and multinational corporations—to inform its analytical work. The Bank Group also leveraged its global footprint well. Country office staff demonstrated leadership at various levels despite facing new crisis-related responsibili- ties: personal and family welfare and the welfare of existing and new clients. However, the Bank Group did not sufficiently build on new global knowledge work to inform portfolio management and strategy or to repurpose policy and institutional strengthening work. The Bank Group also did not system- atically identify and apply lessons from developed countries (such as New Zealand and Switzerland) to its operational work on support to firms and furloughed workers (WEF 2020). The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview Data gaps on recipients’ use of funds may impede targeted approaches in future crises. Limited evidence is available on how the Bank Group lever- aged global data and its access to country budget systems to ensure that its lending reached the most vulnerable households. Croatia (a noncase coun- try) exhibited best practice in this aspect by establishing extensive tracking of funds to recipients via national development banks, the state audit office, and the state financial agency. More than 40 countries (mostly in Africa) established extrabudgetary funds (EBFs) to support firms and citizens. No evidence is available on how the Bank Group used its global reach to align with EBFs to address specific vulnerabilities within countries. Similarly, no evidence is available on the interaction among Bank Group funds and EBFs in supporting vulnerable households. Thus, we could not assess clients’ motivations in setting up EBFs to complement multilateral support during COVID-19 or the risks that such EBFs might entail. Quality through Influence, Coordination, and Safeguards The Bank Group influenced clients’ strategies and actions in the first phase of the evaluation period (the acute crisis phase) but was less on target in the second phase of the evaluation period (the incipient recovery phase). The World Bank’s early response influenced governments’ strategies well, espe- cially their macrofiscal and social safety aspects (for example, in Cabo Verde xvi and Senegal). We did not find evidence, however, that the Bank Group con- sistently demonstrated intracrisis learning. In particular, Bank Group inter- ventions in the first six months of the incipient recovery phase (January 2021 through June 2021) did not reflect lessons from the acute crisis phase (April 2020 through December 2020). During the acute crisis phase, it became clear that firms in contact-intensive industries (such as transport, retail, hotels, tourism, and construction) needed increased support. IFC and MIGA did not target these firms for increased support in the incipient recovery phase (early 2021). We also did not find evidence of the Bank Group switching from emergency interventions to supporting the recovery of countries and private sector clients that were ready to move out of the acute crisis phase (for example, Senegal and Vietnam). One reason was the long disbursement cycles embedded in projects funded outside the fast-track COVID-19 facility (IFC) or via traditional investment project financing (World Bank). Although these findings should be considered preliminary, given the limited evalua- tion period (6 months overall with 4 months of portfolio data), they provide early insights for the Bank Group to take informed actions during the rest of the recovery. The Bank Group has also not yet influenced governments’ strategies and actions to fully reflect the unintended consequences of the massive buildup of sovereign debt and budgetary reallocations by clients in their early responses. Even before the COVID-19 crisis, many poor countries were heavily indebted. The pandemic resulted in massive unavoidable Independent Evaluation Group World Bank Group    xvii expenditures to protect lives and livelihoods. Many countries that needed additional funds to address the COVID-19 shock already had high sovereign debts. Some of them—including Angola, Eritrea, Guinea-Bissau, Lebanon, Mongolia, Mozambique, Nicaragua, Sierra Leone, Sudan, the Syrian Arab Republic, and Zambia—might be in too much distress to recover. In this evaluation, stakeholders of emerging markets and developing economies (9 country case studies and 10 firm case studies) expressed concerns about sovereign debt and borrowing costs. Existing fair debt resolution mechanisms (such as the Common Framework for Debt Treatment and the Debt Service Suspension Initiative) helped during the evaluation period. However, stakeholders perceive them as insufficient to mitigate the looming debt crisis and the potential economic implications in some parts of the world (for example, Sri Lanka’s sovereign default in June 2022). These doubts arise partly because multilateral debt (including IMF and World Bank debt) is not eligible for restructuring. Because the World Bank cannot solely manage sovereign debt crises, it engaged with the IMF in the sovereign debt dialogue at both the global and country levels. The evaluation found, however, limited evidence to date on the extent to which these initiatives have influenced governments to fully reflect in their plans the unintended consequences of the massive buildup of sovereign debt caused by the crisis. This is partly because the World Bank’s loans and other multilateral debt are not included in the Debt Service Suspension Initiative, which also potentially limits countries’ ability to engage with non–Paris Club members and private creditors. Collaboration between the World Bank and the IMF to respond to the pan- demic was effective both at the corporate level and the intervention level. At The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview the corporate level, both institutions organized high-level events (for exam- ple, the Africa high-level panel in April 2020) to crowd in thought leadership and mobilize partners to support the COVID-19 response. The World Bank’s proactive engagement with the IMF promoted effective collaboration on in- terventions, which was critical to addressing economic implications in client countries (for example, Ecuador and Pakistan). The World Bank, IFC, and MIGA coordinated well in their early response to support the financial sector and MSMEs, but collaboration to provide sup- port via structured finance was limited. Latin America and the Caribbean and Europe and Central Asia offer good examples of Bank Group collaboration, with World Bank Partial Credit Guarantee facilities complementing IFC and MIGA support to microfinance institutions and small and medium enter- prise banks. However, especially in the incipient recovery phase (the second part of the evaluation period), we did not find evidence of fast Bank Group responses to clients’ requests for structured finance products to support recovery. Client requests included Partial Credit Guarantee facilities, which coordinated World Bank–IFC interventions could provide. Georgia was an exception and a best practice case. Coordination with regional development banks and other international financial institutions worked well in several cases. Coordination with development partners varied based on country office staffing levels, xviii prepandemic perceptions, and prior arrangements. The World Bank, the Asian Development Bank, and the Asian Infrastructure Investment Bank joined the United Kingdom’s Department for International Development, the United States Agency for International Development, and the Japan International Cooperation Agency to coordinate with the Resilient Institutions for Sustainable Economy project in support of Pakistan’s efforts to strengthen its macrofiscal framework. The World Bank and other international financial institutions also coordinated well to respond with a large aid package to Ecuador: the World Bank committed $1.4 billion through three development policy operations (intended to be disbursed programmatically over three years), the IMF provided $644 million in rapid financing to bridge its extended fund facility support, the Inter-American Development Bank provided a $640 million package, the Development Bank of Latin America lent $500 million, and the Latin American Reserve Fund lent $418 million. IFC’s early response catalyzed other development finance institutions to commit to Banco Davivienda and Ara Tiendas. Neither the Bank Group nor the wider development community has a clear plan for responding to an economic or public health crisis—there is no play- book for either type of crisis. Authorities in client countries and World Bank country teams indicated the need for a dedicated central team in the Bank Group that would analyze the international response to the crisis, includ- ing in developed countries, and feed the learning from this analysis back to country teams in the form of advice. Independent Evaluation Group World Bank Group    xix The Bank Group’s engagements with the Board on the early response were comprehensive. Bank Group management has had frequent and substantive engagements with the Board on the early response since the onset of the global pandemic and throughout the evaluation period. The Bank Group did not set or update specific volume targets during the early response. The Bank Group institutions did not define a corporate re- sults framework to measure the results of Bank Group support in addressing the economic implications of COVID-19. Results frameworks of COVID-19 interventions analyzed in the case studies were not different from the re- sults frameworks of non-COVID-19 operations. We could not, therefore, assess whether the Bank Group is likely to meet its targets and monitor them during the recovery phase of the pandemic. Although the Bank Group streamlined environmental, social, corporate, and fiduciary requirements at the time of the crisis, assessing other stakeholders’ concerns—particularly for the real sector—may shed light on further ways to increase flexibility. The World Bank supported teams and clients with im- plementing the Environmental and Social Framework approved on October 2018 in the context of COVID-19 operations, including by developing tem- plates to facilitate their application at the onset of the crisis. Environmental and social Global Practices at the World Bank created a streamlined, central- ized clearance system to expedite clearance for the initial slate of COVID-19 response projects. IFC made adjustments to its environmental and social appraisals, including by introducing streamlined documentations and virtual appraisals for lower-risk projects. World Bank management also shortened The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview corporate and fiduciary clearance deadlines, delegated some approvals, and briefly paused gender tagging. Stakeholders, however, still perceive envi- ronmental and social safeguards, fiduciary, and corporate requirements as cumbersome in the context of operations addressing the economic impli- cations of the COVID-19 crisis. Some real sector clients indicated that they were overwhelmed by multiple Environmental and Social Framework outputs required by the World Bank. The Bank Group could further assess stakehold- ers’ concerns and consider the possibility of further adapting environmental, social, fiduciary, and corporate requirements to better respond to clients’ needs at times of crisis, especially in high-contact sectors. Streamlining safe- guards for the real sector during a crisis requires the Board’s approval. Six factors affected the quality of the Bank Group’s early response. They were (i) engaging with global partners outside of the development community (such as the International Labour Organization, World Health Organization, and United Nations Conference on Trade and Development); (ii) mobilizing and engaging with local partners and stakeholders (Cabo Verde, Pakistan); (iii) surge-resourcing plans (Cabo Verde); (iv) prioritizing staff welfare during the crisis (for example, Country Management Units in Ecuador and Senegal); (v) underuse of certain Bank Group financing instruments (such as the contingent emergency response component); and (vi) capacity and strengthening of government implementation partners (for example, the Philippine Guarantee Corporation). xx Recommendations We offer two near-term recommendations to strengthen the role of the Bank Group as a crisis responder, which is more critical than ever. Global macro- economic imbalances have reached unprecedented proportions. Inflationary pressures on governments, firms, and households in emerging markets and developing economies have increased along with higher borrowing costs, threatening any nascent recovery from the COVID-19 economic crisis (FES and CBI 2021). The situation is compounded further by other ongoing and new crises emerging in client countries at the national level (for example, inflation), the regional level (for example, the Russian invasion of Ukraine), and the global level (for example, the food crisis). With this context of grow- ing uncertainty at various levels, the role of the Bank Group as a systemic “first responder” to crises has become even more critical. According to the Bank Group’s paper “Navigating Multiple Crises, Staying the Course on Long-Term Development: The World Bank Group’s Response to the Crises Affecting Developing Countries” (World Bank 2022c), the pandemic is one of several related and compounding crises. Others include the war in Ukraine, broader global macroeconomic imbalances, extreme climate-related events, and the combined effects of these circumstances on global food and energy security. Given the broad range of economic support and policy measures implemented across countries in response to these ongoing crises, the Bank Group cannot address COVID-19 recovery in isolation. The Independent Independent Evaluation Group World Bank Group    xxi Evaluation Group offers two recommendations for the Bank Group’s consid- eration to strengthen its role in addressing not only the economic implica- tions of COVID-19 but also other ongoing and future crises. Recommendation 1: To effectively address future crises, codify a global cri- sis response playbook, ideally developed jointly with the IMF. A global crisis response playbook would include considerations on the effectiveness of vari- ous lending instruments at the time of crisis, examples of which instruments worked best under which country conditions and for which sectors and industries, and lessons learned on how to improve their use. The playbook would benefit from describing ways to identify and target countries in high need of support (for example, by developing need scores as conducted in this evaluation) and engage deliberately with highly indebted countries that may need debt resolution mechanisms during recovery from a global crisis. The playbook would include a list of partners, including from the private sector, with whom the Bank Group has successfully collaborated during COVID-19 and other crises. It would also include guidelines on applying safeguards in times of crisis, with a view to selecting only the essential ones, streamlining their application, and incorporating greater flexibility (for example, risk- based tiers and early results-based options) for staff and clients. Given that numerous crises are compounding around the world, it would be prudent for the Bank Group to prioritize staff welfare in planning for its future crisis responses. Staff surge planning has proved to be a driving factor of success during the COVID-19 crisis. Finally, the crisis playbook could include “cri- sis games” to be conducted regularly by Bank Group staff and clients based The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Overview on risk assessments to improve preparedness and strengthen capacity for crisis response. Crisis games would be similar to the recent cybersecurity games conducted jointly by the Bank Group and the IMF with the support of the Boards of the two organizations. The playbook—or parts of it—could be developed jointly with the IMF. The effectiveness of joint Bank Group–IMF action in times of crisis could be further strengthened by an explicit joint statement of principles of collaboration between the two institutions to pre- vent, prepare for, and address crises. Recommendation 2: To respond effectively during the recovery phase of the crisis, explore increasing use of structured finance solutions (such as partial credit guarantees, subordinated debt, and quasi-equity instruments) with a view to supporting small- and medium-size firms. The COVID-19 crisis has left many firms with potential debt overhang. Globally, a large number of small and medium firms were affected to the point of insolvency. The Bank Group’s structured finance solutions offer a relevant response during the incipient recovery phase of this crisis and potentially future crises. It is possible that such solutions would be complex to arrange and require the Bank Group to engage with several partners. However, the benefits are likely higher than the costs, and the trade-offs can be explored further. During the recovery phase of the crisis, it would be useful to explore the feasibility of in- creasing the use of solutions such as subordinated debt, quasi-equity instru- ments, partial risk, and partial credit guarantees to support recapitalizing xxii firms. Some firms coming out of the recovery phase are likely to have limited growth potential because their capital structures were destroyed during the COVID-19 crisis. The Bank Group has the capacity to explore greater use of such options and develop related corporate risk analysis in real time during the crisis. Independent Evaluation Group World Bank Group    xxiii Management Response Management of the World Bank Group thanks the Independent Evaluation Group (IEG) for the opportunity to provide comments on The World Bank Group’s Early Support to Addressing COVID-19: Economic Response, April 2020–June 2021, An Early-Stage Evaluation. Management appreciates IEG’s efforts toward supporting the Bank Group’s COVID-19 response by offering just-in-time lessons and evaluative evidence to inform management choices. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response World Bank Management Response Overall Management welcomes the report’s recognition of the positive prospects of the World Bank’s initial economic response to addressing COVID-19. The report notes that “the World Bank Group’s swift and broad interventions supported macro-fiscal and financial stability, public sector institutional improvements, and capital enhancement and liquidity for financial institutions and MSMEs [micro, small, and medium enterprise] in sectors affected by the pandemic and the lockdowns, resulting in jobs and incomes saved” (xiv). The report also highlights the relevance of the response, especially in low-income countries, and points out that the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA) coordinated well in supporting the financial sector and MSMEs. The report further concludes that the World Bank’s engagement with the International Monetary Fund (IMF) was effective both at the corporate level (both institutions organized high-level events; for example, the Africa high-level panel in April 2020 to crowd-in thought leadership and mobilize partners to support the COVID-19 response) and at the country level (which was critical in addressing economic implications in client countries). Management will reflect on the lessons presented by the report and seize opportunities for further improvement, particularly as it prepares the World Bank Group Evolution Roadmap. xxiv Processes and Policies Management welcomes the conclusion that “the World Bank Group delivered the largest crisis response in its history thanks to strategic and agile deci- sion-making and learning from past crises” (xii). Management confirms that, beyond corporate policies and strategies, each sector gathered its own expe- riences and lessons. For example, with the disruption of agriculture supply chains caused by lock-down measures and the upward pressure on food pric- es, World Bank senior management engaged with the Group of 20, in collab- oration with Food and Agriculture Organization, the International Fund for Agriculture Development, and the World Food Programme to help avoid the adverse trade policies that negatively impacted the food price crisis response in 2007–08. The World Bank also made continuous efforts to generate new information and provide policy guidance to governments using real-time, in-country diagnostics, and a range of advisory services and analytics.1, 2 Frequent surveys were conducted to collect microdata from households and firms at different stages of the crisis for conducting distributional analysis that informed and fine-tuned response strategies. The World Bank’s knowl- edge work and policy engagements at the country level were also adjusted to clients’ evolving priorities during the period.3 Management notes the recognition of the agile way in which safeguards and procurement were treated during the response. Principles-based fiduciary Independent Evaluation Group World Bank Group    xxv policies (with built-in flexibilities) allowed the World Bank to move quickly by permitting higher amounts of advances, retroactive financing, and simpli- fied reporting. This evaluation is focused on economic response to the pan- demic, which was primarily addressed through development policy financing and investment project financing, with very limited procurement. Several of the investment project financing projects provided mainly cash trans- fers and, in some cases, loans to private sector borrowers through financial intermediaries. The procurement policy does not apply to the latter, and the cash transfer programs do not typically involve response projects to expedite clearances. This model was then followed by expedited review decentralized to each region. This initial system led to more-agile quality review for the first round of COVID-19 projects. Debt Management and Sustainability Management underscores that, in addition to engaging on the debt dialogue with country clients at the onset of the COVID-19 pandemic, the World Bank also facilitated jointly with the IMF the Group of 20 bilateral Debt Service Suspension Initiative, which was an example of the World Bank’s convening impact. The World Bank produced several knowledge products and was engaged with the IMF in debt dialogue at both the global and country levels. At the start of the pandemic, the World Bank and the IMF jointly urged the Group of 20 to set up the Debt Service Suspension Initiative.4 The World The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response Bank successfully rolled out the new Sustainable Development Finance Policy, in the challenging context of COVID-19, to help International Development Association countries address core debt vulnerabilities, and it formalized the corporate review process for debt sustainability analyses (DSAs) for countries using the Low-Income Country DSA. At the country level, the World Bank was committed to addressing debt issues in operational contexts, including through development policy financing operations, which in International Development Association contexts sometimes complemented performance and policy actions under the Sustainable Development Finance Policy. The COVID-19 Debt Management Crisis Response Program provided real-time advisory support for government debt managers.5 Approaches were also used to support sustained country efforts to advance public financial management, public investment management, and debt management for overall debt sustainability.6 Along with the IMF, the World Bank supported countries seeking debt restructuring under the Common Framework, providing an input to the process through the joint DSA. While there is a growing number of sovereign debt crisis situations, the World Bank has lead on highlighting the challenges in debt transparency, the Common Framework, and other ad hoc debt relief processes. Recommendations Management agrees with the spirit of the first recommendation and will explore ways to further enhance the Bank Group’s operating model to address key global challenges while maintaining the country-based development model as a foundation. The country model has proved xxvi relatively effective at tackling the response to key global challenges such as the COVID-19 pandemic. Yet, management recognizes that priorities at the national level may not always add up to sufficient progress at the regional or global level, as evidenced by countries’ insufficient preparedness for pandemics. In this regard, one of the key work streams of the World Bank Group Evolution Roadmap proposes strengthening the operating model to further address cross-border issues. Management will use the lessons of this evaluation report to reflect on instruments and financial innovations that could expand the menu of options for all client countries to better prepare and respond to crises. While the report recognizes the flexibilities embedded in World Bank policies, operational innovations are constantly being updated to accompany the broader framework of crises management. Moving this process forward will be done based on several existing workstreams, all informed by the recent IEG synthesis Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations. The most up-to- date statement of management thinking on crises response is the 2022 paper Navigating Multiple Crises, Staying the Course on Long-Term Development; the World Bank Group’s Response to the Crises Affecting Developing Countries. This details the Bank Group’s crisis response and proposes a framework tailored to the current situation affected by multiple, overlapping crisis. An important element of the crisis response is to advocate for and support evidence-based policies and well-tailored financing operations. The paper Independent Evaluation Group World Bank Group    xxvii includes several elements recommended by the IEG report, such as the (i) effectiveness of various instruments, (ii) partnerships, (iii) strategy for countries affected by fragility, conflict, and violence, and highly indebted countries, (iv) operational flexibilities, and (v) others. In International Development Association countries, the increasing application of crisis preparedness assessments, such as the Crisis Preparedness Gap Analysis and other relevant diagnostic tools,7 provides opportunities to adapt global lessons to country contexts and reflect them in country engagements. Management remains committed to ensuring that the right lessons are learned from the COVID-19 and other crises as the Bank Group evolves to respond to a polycrisis environment. Management agrees with the recommendation to explore the increasing use of structured finance solutions to support small and medium enterprises (SMEs). Recent reports, such as the 2021 report on supporting firms in restructuring and recovery and the 2022 World Development Report on finance for an equitable recovery, identified a menu of possible responses to the COVID-19 impact on MSMEs and the private sector more broadly, including structured finance. Structured financing is about finding the needed mix of revenue sources to attract commercial lenders combined with a package of credit enhancements to mitigate risks and, in the end, minimize the cost of borrowing. Depending on country circumstances, the usefulness of these interventions varies greatly. Management will ensure that wherever preconditions exist, structured finance is systematically The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response assessed as the preferred option during project design. Preconditions include reliable and sustainable cash flows, political stability, bankable projects, legislation permitting ring fencing and revenue intercepts, and the use of risk mitigation measures to further reduce the risk of lending. Early Stage Management welcomes this rich report and its findings yet notes that the approach of isolating one aspect of the response may have limited the depth of certain conclusions. As management noted in the comments to the Approach Paper for this report, the separation of economic, social, and health aspects into two evaluations may have missed a salient aspect of the Bank Group’s response to COVID-19, which is its comprehensive approach to a multifaceted policy challenge: “… management believes that the most salient aspect of the World Bank Group’s response to the COVID-19 pandemic is its comprehensiveness—organized around interrelated pillars and implemented through a range of instrument” (World Bank 2022d). Management suggests that the subsequent ex post evaluation on the Bank Group’s COVID-19 response provide a more holistic, less compartmentalized assessment of the unprecedented Bank Group response. International Finance Corporation Management Response IFC management welcomes IEG’s evaluation, the early findings, and recogni- tion of the relevance and agility of IFC’s support. xxviii The evaluation comes at a timely juncture, when multiple crises are envelop- ing the global economy, and notes an increased need for impactful and agile response strategies. IFC thanks IEG for the work it has done to formulate recommendations to strengthen IFC’s crisis response. While the report itself acknowledges the limitations of scope and depth that such early-stage assessments entail, IFC management would like to point to the potential shortcomings such very early evaluations with limited sample size may have for formulating actionable recommendations that can help inform learning. Given the short period under evaluation, and the limited portfolio sample reviewed, it would be unreasonable to expect an overwhelming amount of adaptive learning to be mainstreamed within the stated evaluation period. A case in point are the expedited project processing procedures developed under the umbrella of the Fast-Track COVID-19 Facility (FTCF) in March 2020. The FTCF enabled IFC to make innovative real-time changes to its operating model, leading to quantifiable efficiency gains.8 Examples include virtual site visits for due diligence and supervision, shorter documentation, and streamlined decision-making for lower-risk transactions, none of which were noted in the evaluation. While IFC was keenly aware of the potential of these models for being mainstreamed into operations outside of the facility, it took time to assess their relevance for processing efficiency and adequacy for risk mitigation. These processing enhancements have ultimately been Independent Evaluation Group World Bank Group    xxix mainstreamed under the Expedited Processing for Existing Clients. However, it was not operational until April 2022. In IFC’s view, the time frame for the evaluation was too short to demonstrate meaningful adaptive learning. Compounding the temporal issue were the conflicting periods used for dif- ferent portions of the evaluation, as well as the fragmented focus on IFC’s crisis response activities and industries where it was delivered. This resulted in the exclusion of initiatives that demonstrate the vast amount of adaptive learning that happened as IFC mounted its pandemic response. A case in point is the Global Health Platform, which was excluded because it “focused on health,” but which was also omitted in the parallel evaluation of the COVID-19 social and health response.9 As a result, a significant part of IFC’s crisis toolbox has not been covered in any COVID-19 evaluation. Given that the platform was launched to help firms diversify the production of health care products, thus saving businesses and livelihoods in emerging markets, management argues that it is not only a critical part of IFC’s “economic response” but is a good example of adaptive learning. The Global Health Platform is a comprehensive platform encompassing more immediate as well as long-term interventions for strengthening the resilience of local health care systems beyond just the current pandemic. Finally, while the bulk of IFC’s immediate crisis response was channeled through financial institutions because of the need for quick delivery early The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response on—a strategic choice given the market uncertainties and limitations around the operating environment at the time—IFC would like to highlight that IEG’s findings regarding its response in the real sector do not capture the full picture. IFC delivered a robust response in contact-intensive industries, including tourism, manufacturing, construction, education, and retail. The striking omission of health, specifically health services, pharmaceuticals, and medical technology manufacturing and distribution in the definition of “contact-intensive” industries in this evaluation further dilutes the comprehensiveness of IFC’s crisis response.10 During the evaluation period (April 1, 2020–June 30, 2021), IFC own-account COVID-19–related commitments under Manufacturing, Agriculture, and Services totaled $2.25 billion. The real-sector response was further strengthened through advisory and upstream engagements, none of which have been adequately captured in the evaluation. With these gaps clearly illustrated, IFC would like to be cautious on con- clusions around the relevance and comprehensiveness of its COVID-19 response based on a truncated sample and siloed approach, an issue raised explicitly at the Approach Paper stage. Hence, IFC management would like the Committee on Development Effectiveness to consider the method- ological limitations of these early-stage evaluations and omissions in this particular evaluation for formulating learning insights. As the Committee on Development Effectiveness is aware, methodology concerns regarding early-stage evaluations were raised in the 2022 External Review, which noted their limited learning value for the Bank Group. xxx Recommendations Recommendation 1 While management broadly agrees with recommendation 1 (to effectively address future crises, codify a global crisis response playbook, ideally devel- oped jointly with the IMF), we find it somewhat belated in light of what IFC has already been doing. More importantly, anticipating what shape a future crisis may take—while potentially useful—is unlikely to capture the actual form of the next crisis, with the global pandemic being a case in point. Since the outbreak of the COVID-19 pandemic, IFC has been diligently assessing the processes and results of its crisis response and adjusting its operational model accordingly, as stated previously. IFC continues to collaborate, identi- fy opportunities and synergies across the Bank Group, and place the learning from its interventions in a broader context for maximum impact in address- ing future crises. We believe that the Bank Group has already laid out a clear and compre- hensive strategy, included in the Bank Group roadmap (April 2022) and the follow-up paper, Navigating Multiple Crises, Staying the Course on Long-Term Development” (July 2022), which builds on the 2020 COVID-19 paper Relief, Restructuring, and Resilient Recovery and the Green, Resilient and Inclusive Development approach. Independent Evaluation Group World Bank Group    xxxi It is important to note that IFC’s ongoing crisis response is aligned with the Bank Group and goes beyond COVID-19 to address the multiple, ongoing crises. The follow-up paper proposes a robust framework of four pillars that each support the three stages of relief, restructuring, and resilient recovery. For example, IFC’s Global Food Security Platform will help support private sector activities close the financing gap for agribusinesses across the food supply chain to support pillar 1. In addition, IFC’s Africa Trade and Supply Chain Recovery Initiative, which is a trade-value-chain initiative for food and nutrition security, is designed to complement the Global Food Security Platform with a focus on African trade. Both these initiatives leverage many of the learnings from the FTCF. Through the Global Health Platform, IFC will continue efforts to support the supply of health care products and support private sector resilience under pillar 3. Recommendation 2 IFC management agrees with recommendation 2 (to respond effectively during the recovery phase of the crisis, explore the increasing use of struc- tured finance solutions [such as partial credit guarantees, subordinated debt, and quasi-equity instruments] to support SMEs) in theory. However, direct lending to SMEs entails significant cost and resources, which would limit the number of such interventions that IFC can undertake. More impor- tantly, during crisis, client demand is typically driven by short-term needs and instruments. IFC views structured finance solutions as one of the many The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response options available in its toolbox to support clients during their growth phase. IFC’s structured finance solutions (which include risk-sharing facilities, par- tial credit guarantees, and quasi-equity instruments but typically not sub- ordinated debt) have been refined and developed over the years, taking into account lessons learned and client demand. IFC has a documented track re- cord of offering structured risk-sharing products to financial institutions for building their SME loan portfolios in target markets and expanding lending to their clients in times of crisis. In fact, it is one of IFC’s flagship structured finance offerings for financial institutions to help channel their lending toward SMEs. What has been particularly challenging in times of crisis is the limited information and the compromised ability to assess the financial via- bility of SMEs in IFC’s countries of operation. IFC management would have welcomed more insights from the IEG evaluation, particularly good practice case studies in the context of the COVID-19 crisis and how structured fi- nance products can be a good fit for SMEs facing capital shortages and debt overhang. Management would like to note that there appeared to be limited demand for “financial structuring” from six of the seven firms covered by the case studies in IEG’s evaluation, as reported by these clients in the recently completed IFC fiscal year 2022 Integrated Client Survey. This periodic survey of IFC clients assesses client satisfaction, the relevance of IFC’s products and services, and clients’ changing needs. Furthermore, four of the six firms rated IFC’s support during times of crisis (for example, COVID-19) as “excellent.” One rated IFC’s support as “good” and another as “average” in the survey that covers the period of the pandemic response captured in the evaluation. xxxii Going Forward Management looks forward to working closely with IEG on the upcoming ex post evaluation of the COVID-19 response. Management is confident that IEG will put in place a process to ensure evaluation independence for the ex post report from the two (Health and Social Response and Economic Re- sponse) early-stage evaluations of COVID-19. This is in line with the system of governance and good practice standards outlined in the World Bank Group Evaluation Principles. Multilateral Investment Guarantee Agency Management Response MIGA welcomes the evaluation report, which aims to provide an early-stage assessment of the relevance and quality of the Bank Group’s early response to the COVID-19 crisis, which is critical for shaping the ongoing response and preparing for future crises. As a part of the Bank Group response, MIGA issued $7.64 billion of guaran- tees under the Agency’s COVID-19 Response Program between fiscal years 2020 and 2022. MIGA appreciates IEG’s recognition of the Bank Group’s agile decision-making and learning from past crises, which contributed to deliv- ering the largest rescue package in the Bank Group’s history and the largest Independent Evaluation Group World Bank Group    xxxiii crisis response among development partners. IEG has also noted the effec- tive Bank Group collaboration with the IMF and other development institu- tions in supporting highly relevant operations in low-income countries. The evaluation report’s findings on the Bank Group’s assistance to different economic sectors are useful for understanding the kinds of projects support- ed in relation to the varied nature of the COVID-19 economic crisis across countries. This evaluation report aimed to complement another IEG eval- uation, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response, An Early-Stage Evaluation. It is important for readers of this evaluation to be aware of the limitations of these evaluations. IEG did not intend to assess the relevance or quality of the overall Bank Group response across all sectors and industry groups. The report analyzes IFC and MIGA portfolios through only a sample of firm-level case studies as acknowledged in the report itself. In particular, MIGA’s support to host governments in their efforts to procure medical supplies and services during the COVID-19 pandemic (pillar one of MIGA’s COVID-19 response package), accounting for $875 million in guarantees by the end of fiscal year 2022, were not covered in the evaluation report nor the previous IEG evaluation on the health and social response to COVID-19. The evaluation report is also designed to give early feedback; therefore, by design, the evaluation period is relatively short (April 2020–June 2021). Within this short time period, the evaluation report formulated a two-win- The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response dow analytical scheme for assessing the Bank Group’s response: “the acute crisis phase” (April 1, 2020–December 31, 2020) and “the incipient recovery phase” (January 1, 2021–June 30, 2021). Based on the two-window analytical scheme, the report concludes that the Bank Group’s influence on govern- ments’—and especially firms’—actions during the incipient recovery phase were “less on target” compared with during the acute crisis phase. The report shows that the nature and extent of the COVID-19 economic crisis has varied widely across countries. In addition, the March 2022 Bank Group COVID-19 Crisis Response Operational Update has emphasized that the impacts of COVID-19 and the compounding climate change and conflict crises cannot be separated. Intracrisis and Intercrisis Learning As noted in the overview, the report assessed how well the Bank Group learned during the crisis. The analysis included how well the Bank Group learned from two types of activities: (i) learning during the crisis, from its work on COVID-19 (termed intracrisis learning) and (ii) learning from past crises and from developed countries’ approaches to COVID-19 (termed intercrisis learning). The report’s assessment of intracrisis and intercrisis learning is closely linked to the two-window analytical scheme, concluding that intracrisis learning had been limited, as the Bank Group did not reflect lessons from the acute crisis phase, corresponding to the first part of the evaluation period. The report also states that the Bank Group provided a low level of support to contact-intensive industries like transport, retail, hotels, tourism, and construction, and countries and firms during the incip- xxxiv ient recovery phase. Further, the report states that learning from developed countries was limited, many of which kept workers (partially) paid during the COVID-19 crisis (different from what was done during the 2008 global financial crisis), in addition to providing support to enterprises to prevent them from going bankrupt. MIGA questions the validity of the IEG finding that the Bank Group’s in- tracrisis learning was limited since it is solely based on the Bank Group’s low support to contact-intensive industries, anchored in the two-window analytical scheme. The Bank Group also applied learning from past crises to inform its early response to the COVID-19 economic crisis. Moreover, the limited evaluation period for the Bank Group learning assessment also restricts IEG’s analysis because it failed to consider projects that had yet to become operational. Therefore, MIGA remains cautious regarding the evaluation findings and questions the learning value for operational teams who sought to respond quickly to unprecedented events in the face of high uncertainty. Independent Evaluation Group Assessment of the Relevance of Multilateral Investment Guarantee Support MIGA notes with emphasis that the Agency’s COVID-19 response was for- mulated in a context characterized by deep uncertainty and fluidity. The report has recognized MIGA’s countercyclical role in deteriorating country Independent Evaluation Group World Bank Group    xxxv risk situations in stating, “The rising risk points to MIGA’s relevant role in risk mitigation and capital mobilization under guaranteeing project risk in countries where commercial financing was needed” (42). The evaluation reviewed three MIGA operations at the early stage of the pandemic, concluding, “Two MIGA projects that did not fully consider coun- try conditions in their designs were less relevant. MIGA overestimated the liquidity needs of banks in Panama and some African countries (Botswana and Eswatini)” (xv). MIGA disagrees with the insufficient country conditions analysis conclusion, noting MIGA’s regular and consistent assessment of country conditions through the quarterly country risk updates and extensive due diligence for underwriting MIGA guarantee projects. Therefore, to derive substantive learning from experience, MIGA would appreciate having concrete evidence supporting the evaluation’s conclu- sion that MIGA’s decision to support Banco Nacional de Panamá was made without sufficient analysis of country conditions. The report states and recognizes that the IMF endorsed the MIGA guarantee, and the IMF contrib- uted financing to Banco Nacional de Panamá’s liquidity. Given this, MIGA wonders how IEG can be so certain in stating the evaluation report’s finding of, “its low relevance could have been expected” (44) because of the fully dollarized economy (box 3.3). The fully dollarized economy is the under- lying vulnerability that the MIGA supported guarantee was addressing as The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response the economy is partly dependent on dollar inflows into the economy for its liquidity, which could come under pressure amid the uncertainties of the pandemic. The IMF’s May 2020 report on “Panama: Request for Purchase Under the Rapid Financing Instrument” states, “Building liquidity buffers remain critical to maintaining financial and external stability. Staff urged the authorities to consider the introduction of an emergency liquidity facility, which the BNP [Banco Nacional de Panamá] could operate” (IMF 2020b, 7). These were the difficult conditions that the MIGA guaranteed funding was responding to. As of August 2022, although the banking sector is assessed as sound and the liquidity facility funded by IMF’s Rapid Financing Instru- ment was not used, the IMF still maintains liquidity support to Panama as a precautionary measure (IMF 2022). The IMF considers the liquidity facility as a dual lender of last resort and credit facility—in the absence of a central bank and an institution acting as a lender of last resort function—given the overall context of elevated risks and uncertainties, viewing the efforts of the Panamanian authorities to enhance overall liquidity as appropriate. Similarly, the statement in the evaluation that “capital relief was granted to tier 1 bank subsidiaries with relatively large market share (Botswana and Eswatini) and not to tier 2 bank subsidiaries with smaller market shares in countries in need (for example, Ghana, Mozambique, and Nigeria)” (44; box 3.3) However, IEG’s analysis is unclear and potentially based on a misunderstanding of the product’s design in this case. The Capital Optimization product generates additional lending capacity at the consolidated group level (MIGA guarantee holder), with the expectation that xxxvi such additional lending capacity is allocated to each group’s subsidiary under separate contracts of guarantees. Such capacity does not automatically entail its use, as banks can have the capacity to grow (as endorsed and approved through the group’s budget process) but not grow because of a variety of reasons (demand, economy, regulation, and so on). The team at underwriting assesses such risks and benefits, specifically focusing on likelihood of MIGA- enabled capacity being used during the tenor of the guarantees. On such a backdrop, pointing to the “granting” of the relief to one subsidiary compared with another, appears misleading and incorrect. The ex post evaluation should be underpinned by comparing projected loans with the benefit of the MIGA guarantee, relative to actual loans disbursed, for each country/subsidiary and, in the event of deviations, try to articulate as to why there was a deviation. Moreover, the transaction was signed in late quarter 2 of 2020, at the earliest stage of the pandemic. The analysis presented in the evaluation report did not reference the project context at inception, which was at the earliest stage of the pandemic, when uncertainties were extremely high. Recommendation 1 MIGA agrees with recommendation 1 (to effectively address future crises, codify a global crisis response playbook, ideally developed jointly with the IMF) but would emphasize that action for implementing this recommendation is already in place through a Bank Group crisis response strategy. MIGA also considers more value would be gained and captured Independent Evaluation Group World Bank Group    xxxvii by memorializing the processes that the Bank Group institutions have implemented during the crisis, how the Bank Group and the constituent institutions coordinated with other development partners, including the IMF, and how management sought to communicate with the Board and other stakeholders, and what worked well and what worked less well and why. This could be updated as additional evidence and results become available. Such an exercise would help ensure that Bank Group management and staff, and future Bank Group management and staff, would have this information readily available and could use it or modify its recommendations and conclusions as needed, given the specifics of the crisis at hand. This would be a “light” exercise, since staff and management are already dealing with “future crises”—the war in Ukraine, increased fragility, the debt crisis, the food and fuel challenges, and increasing climate disasters. The idea of staging “crisis games” seems redundant, given that we are currently addressing multiple crises simultaneously. Furthermore, this approach would help ensure that the recommendation is sufficiently targeted to be relevant and appropriate to MIGA and IFC. However, the recommendation’s inference regarding “greater flexibility” in applying safeguards in times of crisis needs extra scrutiny. In many sections, the report states that compliance requirements constrained Bank Group deployment of COVID-19 support. For example, “The Bank Group could have assessed the need to adjust compliance (including safeguard) requirements to xxxviii The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response facilitate supporting the real sector during the crisis,” (51) and “some Board members were hesitant to relax compliance and risk controls” (73). The state- ment needs more context and elaboration to be useful. For instance, what evidence is there to corroborate the conclusion for MIGA, and which safe- guards, in particular, should the Bank Group have been more flexible on, and what would the implications have been? The flexibility measures proposed by the evaluation report (for example, risk-based tiers and early results-based options) are essentially what MIGA has been doing as standard practice in its guarantee operations, including during the COVID-19 pandemic. Finally, any discussion regarding flexibility around environmental and social standards would also need to include a discussion on how MIGA would look at those projects from the potential harm to project-affected people as a result of relaxed standards from an accountability perspective, including po- tential Compliance Advisor/Ombudsman complaints and Compliance Advi- sor/Ombudsman cases that could result, and future IEG evaluations (project and thematic). Moreover, MIGA’s Board of Directors and civil society have been clear in the past that relaxing environmental and social and fiduciary safeguards is not an appropriate response to crises. However, if this principle has changed, it would be useful for management to discuss it with the Board, IEG, Compliance Advisor/Ombudsman, and other stakeholders (including civil society). Recommendation 2 MIGA broadly agrees with recommendation 2 (to respond effectively during the recovery phase of the crisis, explore the increasing use of structured finance solutions [such as partial credit guarantees, subordinated debt, and quasi-equity instruments] to support SMEs) that structured finance solu- tions can be useful in addressing crisis situations. The report makes repeat- ed references to “structured finance solutions being limited.” However, the report does not explain which challenges could have exclusively or been better addressed through structured finance solutions. MIGA notes that structured finance solutions typically are complex, involve multiple parties, and, most importantly, for this evaluation, take time to structure. Given the need for timely client support during the COVID-19 crisis, the report is not clear on why structured finance solutions should be preferred. Regardless of the potential role of structured finance, it is important to remember that the objective is not to deliver certain structures per se, but rather to deliv- er value to our clients and promote development. The recommendation to specify a specific form of financial packaging rather than adopting a menu approach that delivers the appropriate solution in the context of the coun- try, market and challenge to be addressed would seem most suitable for a crisis response package. As the two IEG early-stage evaluations (“health, human capital, and social response” and “economic implications”) will be complemented in the future by the IEG ex post assessment of the impact of the COVID-19 crisis response, MIGA management is looking forward to working with IEG to make the forthcoming evaluation as comprehensive as possible, covering the entire Independent Evaluation Group World Bank Group    xxxix spectrum of the Bank Group activities and instruments that were delivered to address the unprecedented challenges. MIGA management is also looking forward to having a fresh discussion with IEG on the approach and meth- odology for the comprehensive evaluation, which will be independent from these two early-stage evaluations. 1  COVID-19 Research, Analysis & Policy Responses internal website, accessible at http:// covid19research. 2  Significant efforts were made to use data, existing diagnostics, real-time indicators, and les- sons from past crises to support the World Bank’s response to COVID-19. Findings from these informed operation designs. For example, the following were tasked in 2020: (i) the COVID-19 Mobility Analytics Task Force, (ii) the COVID-19 Open Data website, and (iii) data tools and research, among others. At the country level, the Business Pulse Surveys were conducted to measure the impact of the pandemic on firms, companies, and public policy responses; phone surveys were conducted to collect sector related microdata from households for distributional The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Management Response analysis; real-time indicators were used to track the economic impact. https://www.world- bank.org/en/about/annual-report/advancing-knowledge 3  Examples include the following: (i) Global Investment Competitiveness Report: Rebuilding Investor Confidence in Times of Uncertainty (May 2020); (ii) A Phased Approach of Investment Climate Policy Responses to COVID-19 (June 2020); (iii) COVID-19 and FDI: How Should Govern- ments Respond? (October 2022); (iv) Re-Thinking the Approach to Informal Businesses: Typol- ogies, Evidence and Future Exploration (November 2020); (v) Supporting Firms in Restructuring and Recovery (March 2021); and (vi) FCI Policy Compendium, which provided an overview of the “real-time” financial sector policy that was deployed during the crisis; (vi) infrastructure accessible at INF Covid19 Hub. 4  Established in May 2020, the Debt Service Suspension Initiative helped countries concen- trate their resources on fighting the pandemic and safeguarding the lives and livelihoods of millions of the most vulnerable people. Forty-eight of 73 eligible countries participated in the initiative before it expired at the end of December 2021. From May 2020 to December 2021, the initiative suspended $12.9 billion in debt-service payments owed by participat- ing countries to their creditors, according to the latest estimates. The World Bank and the IMF supported the implementation of the Debt Service Suspension Initiative by monitoring spending, enhancing public debt transparency, and ensuring prudent borrowing. https://www. worldbank.org/en/topic/debt/brief/covid-19-debt-service-suspension-initiative 5  The support has been provided mostly through training sessions (including with the Mac- roeconomic and Financial Management Institute of Eastern and Southern Africa), as well as through bilateral calls or videoconferences on topics such as the revision of the issuance program given increased financing needs, implementation of liability management operations (bond buybacks, switches), and so on. xl 6  This is consistent with past Independent Evaluation Group evaluations, which called for closer integration of public financial management and debt management. https://ieg.world- bankgroup.org/evaluations/public-financial-and-debt-management 7  The World Bank’s Crisis preparedness diagnostic instruments include Ready2Respond, Emergency Preparedness & Response Programs, Social Protection Stress Testing Methodolo- gy, Disaster Risk Finance diagnostics, Pandemic preparedness diagnostics, and the DRR-PFM Toolkit. 8  In fiscal year 2021, median mandate to first disbursement stood at 139 days for Fast-Track COVID-19 Facility projects, against 313 days for other projects. 9  It is worth noting the biased treatment of health in this evaluation, which included health in the needs analysis (appendix A) but chose to exclude it in the response analysis. 10  World Bank Group pharmaceutical experts confirm that the Bank Group’s work related to pharmaceuticals is downstream and should be classified under the health sector and not tech- nology, as defined by the IEG team. Independent Evaluation Group World Bank Group    xli 1 | Background and Context Highlights The economic problems intertwined with the COVID-19 pandemic are markedly different from past economic crises in their origins, scope, and timing. They threaten all Sustainable Development Goals, especially Sustainable Development Goal 1 (end poverty) and Sustainable Development Goal 8 (promote economic growth and shared prosperity). They are also causing a rapid buildup of sovereign debt. The crisis resulting from the pandemic, which caused direct eco- nomic losses of approximately $7.4 trillion in 2020 alone, has had deep economic impacts through numerous transmission channels affecting firms (trade and production disruptions), households (job and income losses), financial institutions (nonperforming loans), and governments (public spending, income losses, and sovereign debt). The World Bank Group has drawn on its full capacity to re- spond to urgent issues with support to public budgets, bank liquid- ity, firm solvency, and trade during both the acute crisis phase in 2020 and the incipient recovery phase in 2021. The objective of the evaluation is to foster learning and adaptive management to strengthen the Bank Group’s ongoing response to the economic dimensions of the pandemic and help it prepare strategically and institutionally for early responses to future crises. This early-stage evaluation builds on and adds to previous and on- going Bank Group efforts to learn from crises, including The World Bank’s Early Support to Addressing COVID-19: Health and Social Response, a parallel evaluation of the World Bank’s early response to the health, human capital, and social protection emergencies related to COVID-19. This evaluation will be complemented by a future ex post evaluation. 1  The main evaluation question is, “How well did the Bank Group respond to global, country, and firm needs in its early response to the COVID-19 economic crisis?” We assessed both the relevance of the Bank Group response (how well it was tailored to countries’ and sectors’ needs and the Bank Group’s comparative advantages) and its quality (the extent to which it influenced governments and firms, was well coordinated, and maintained adequate monitoring, safe- guards, and governance). We used a mixed methods approach, including a structured litera- ture review, case-based analysis, key informant interviews, portfo- lio review and analysis, and econometric and comparative analysis. The evaluation applies a “learning for adaptive management at the time of a crisis” lens to its quality assessment, aimed at under- standing whether the Bank Group applied lessons learned from its own work during the crisis (intracrisis learning). The evaluation has several limitations because of its scope and its nature as a real-time learning evaluation. 2   The Economic Impact of the COVID-19 Pandemic The COVID-19 economic crisis is markedly different from past crises. There have been previous public health crises (such as the Ebola crisis, which was more regional but had a large impact), but the economic dimensions of the COVID-19 crisis are unprecedented. First, unlike many past economic cri- ses, the COVID-19 economic crisis did not arise because of macroeconomic or financial imbalances but because of a worldwide health crisis that halted economic activities. Second, COVID-19 affected both supply and demand, whereas financial crises are demand shocks (Benguria and Taylor 2020). The two processes were mutually reinforcing, creating a downward spiral. Finally, the COVID-19 economic crisis hit most countries and sectors of the economy simultaneously. The economic problems intertwined with the COVID-19 pandemic threaten the realization of World Bank strategic goals and the Sustainable Development Goals (SDGs). The World Bank (2020d) estimates that between 88 million and 115 million people were pushed into extreme poverty in 2020 because of the pandemic. The economic burden of the pandemic and its impact on government spending threaten all SDGs because they threaten the means of implementation. Poverty reduction (SDG 1, “end poverty in all its forms everywhere”); economic growth and shared prosperity (SDG 8, “promote sustained, inclusive, and sustainable economic growth, full and productive employment, and decent work for all”); and progress on debt relief and debt resolution (SDG 17, “revitalize the Global Partnership for Independent Evaluation Group World Bank Group    3 Sustainable Development”) are especially at risk.1 The economic impact of the pandemic has been enormous. In 2020 alone, world gross domestic product (GDP) shrank by 3.3 percent (World Bank 2022b). However, before the pandemic, the economy was forecast to grow by 2.5 percent in 2020 (World Bank 2020c). Thus, the pandemic slashed GDP by approximately 5.8 percent just in 2020 (figure 1.1). The impact is equivalent to approximately $7.4 trillion, or four times the GDP of Sub-Saharan Africa (2020). For longer periods, the costs are higher: the International Monetary Fund (IMF) forecast that the COVID-19 pandemic will cost the global econ- omy $12.5 trillion through 2024. Moreover, these estimates include only direct economic losses. They omit the economic effects of premature deaths, declining health (including mental health), reduced quality of life, and dete- rioration in human capital caused by job losses and school closures. Each of these factors increased the cost of the pandemic by trillions of dollars (Cutler and Summers 2020). Finally, the massive debt that governments and firms took on to weather the pandemic may dampen future economic growth and have far-reaching social costs (box 1.1).  hange in Gross Domestic Product and Gross Domestic Figure 1.1. C Product Growth from 2019 to 2020 by Region 0 0 (constant 2017 international $, trillions) -1 -2 Change in GDP growth� Change in GDP, PPP (annual %, pp) The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 -2 -4 -3 -6 -4 -8 -5 -10 EAP ECA LAC MENA SAR SSA WLD Region Change in GDP Change in GDP growth Source: World Development Indicators. Note: EAP = East Asia and Pacific; ECA = Europe and Central Asia; GDP = gross domestic product; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; pp = percentage points; PPP = purchasing power parity; SAR = South Asia; SSA = Sub-Saharan Africa; WLD = World. In parallel, the world is witnessing a rapid buildup of sovereign and corpo- rate debt, especially in highly indebted countries. Sovereign debt in many emerging and developing economies has been trending upward since the early 2010s. The COVID-19 crisis has accelerated the pace of borrowing globally because sovereign nations have undertaken massive debt to sup- port firms and citizens during the crisis (box 1.1). At more than 65 percent of GDP, emerging markets government debt is now 25 percentage points higher than in 2010 and continues to increase, with many countries running per- sistently large budget deficits (Gudmundsson et al. 2022). The rapid buildup in emerging market sovereign debt levels has greatly increased the likeli- hood of further debt strains. Sri Lanka defaulted in May 2022, and sovereign defaults are looming on the horizon for highly indebted countries such as 4 South Sudan and Zambia (IIF 2022). The massive debt that governments and firms took on to weather the pandemic may dampen future economic growth and have far-reaching social costs, even in countries that do not default. Box 1.1. Sovereign Debt and COVID-19 Sovereign debt was high even before the COVID-19 pandemic. Many emerging econ- omies entered the pandemic with record levels of sovereign debt. By 2019, approxi- mately half of the countries eligible for International Development Association assis- tance were in or at high risk of debt distress. Global debt surged as a result of the COVID-19 pandemic. Countries took on additional debt to pay for programs to limit the pandemic’s economic and human costs. Average total debt burdens among low- and middle-income countries increased by rough- ly 9 percent of gross domestic product during 2020, compared with an average of 1.9 percent per year over the previous decade (figure B1.1.1).  eneral Government Gross Debt, by Country Income Figure B1.1.1. G Group, 2010–20 80 70 Share of GDP (%) 60 Independent Evaluation Group World Bank Group    5 50 40 30 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 Year High income Upper middle income Lower middle income Low income Source: World Bank 2022e. Note: The figure shows the general public debt stock as a share of the gross domestic product by World Bank income classification. GDP = gross domestic product. (continued) Box 1.1. Sovereign Debt and COVID-19 (cont.) Because of these vulnerabilities, the global community will need to stand ready to provide debt relief equitably and efficiently. Managing and resolving elevated levels of sovereign debt are essential to ensuring an equitable recovery from the COVID-19 crisis. Sovereign debt crises are costly for sustained growth—every year a country re- mains in default reduces its gross domestic product growth by an estimated 1–1.5 per- centage points. High sovereign debt also has far-reaching social costs. High sovereign debt reduces a government’s ability to spend on social safety nets and public goods such as education and public health. These consequences are typically borne dispro- portionately by vulnerable populations, low-income households, and small businesses and tend to worsen preexisting poverty and inequality. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 Sources: Borensztein and Panizza 2009; Economist 2022; World Bank 2022b, 2022e. The economic effects of the COVID-19 pandemic materialized through mul- tiple channels (figure 1.2). Both the pandemic and the lockdowns prevented firms from operating and interrupted trade flows, creating a sudden and deep supply shock. Household income and investments fell quickly as workers lost their jobs or had their hours cut, which in turn constrained demand for goods and services. The massive, combined supply and demand shocks created a downward spiral, eventually affecting banks’ balance sheets through the rise of nonperforming loans, some of which become liabilities for governments. Government budgets were affected primarily through the simultaneous drop in revenues and pressures to increase expenditures on social protection, unemployment benefits, and subsidy programs aimed at supporting recovery for the most affected micro, small, and medium enterprises (MSMEs). Some contact-intensive sectors of the economy were particularly affected by the crisis, including tourism, manufacturing, construction, education, and retail services. Many small countries and island nations relying heavily on tourism lost precious export earnings and government revenues when their econo- mies most needed fiscal support. Commodity exporters were affected simi- larly by the collapse of commodity prices. 6 Figure 1.2. Main Transmission Channels of the COVID-19 Economic Crisis Pandemic Lockdowns Households Firms Job losses Trade disruptions Production disruptions Household income losses Firm income losses Social protection programs Firm support programs Financial institutions Nonperforming loans Bank support programs Government Public spending Government income losses Independent Evaluation Group World Bank Group    7 Public budget Macrofiscal programs Sovereign debt Source: Independent Evaluation Group. The World Bank Group Response The World Bank Group applied its full capacity to mitigating the economic impacts of the pandemic (see the theory of change in figure 1.3). Bank Group capacity included enablers such as the Bank Group’s strategic positioning and convening, its emergency preparedness, and its risk management prac- tices. It also included resources such as data, analytics, and knowledge as captured in dashboards, among other means; policies and procedures; and human and financial resource management. Finally, Bank Group capacity included coordination and partnerships with the IMF and other development partners, government and local stakeholders, and new clients and partner- ships to address the pandemic. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 Bank Group interventions targeted governments, financial institutions, and firms. The World Bank provided development policy financing (DPF) to help governments cope with the crisis and improve the transparency of governments’ macro and fiscal measures during the COVID-19 crisis and beyond. The International Finance Corporation (IFC) provided emergency liquidity and credit to firms through financial intermediaries (such as credit lines and guarantees). IFC and the Multilateral Investment Guarantee Agency (MIGA) developed lending envelopes and guarantee programs to provide fast disbursement or coverage options to financial institutions and firms affected by the pandemic and the lockdowns. Together, these Bank Group operations addressed the immediate needs of MSMEs—especially those in contact-intensive sectors—to “keep the lights on” and thereby save jobs, enhanced the liquidity and capital position of financial institutions, and supplemented government budgets for crisis spending. On the basis of these intermediate outcomes, we expect that the final outcomes would be reducing firm and household bankruptcies and preventing banking crises and sovereign defaults, with the ultimate impact of protecting livelihoods during the COVID-19 emergency, returning to economic growth, and resuming declines in poverty. However, final outcomes and impact are out of scope for this early-stage evaluation. 8  heory of Change for World Bank Early Response to Economic Implications of COVID-19 Figure 1.3. T Transmission channels Economic impacts (Figure 1.2) • Firm income losses • Household income losses • Nonperforming loans • Government spending Outputs • Lending governments World Bank Group capacity (MPA, DPF, IPF) • Emergency liquidity and Enablers credit to firms through • Strategic positioning and financial intermediaries convening (credit lines, guarantees) • Emergency preparedness • Structural issues • Risk management Resources • Dashboards, analytics, data knowledge Intermediate outcomes • Staff guidance on policies • “Lights kept on” and trade and procedures flowing for firms, especially • Human and financial MSMEs in contact-intensive resource issues Final outcomes Impact sectors Coordination • Jobs preserved for • Firms surviving • Livelihoods and partnerships households pandemic protected • Liquidity and capital • Households avoiding • Resumption of • IMF and other deployment position enhanced for bankruptcy economic growth partners financial institutions • Banking crises • Return to • Government and local • Government budgets prevented declines in poverty stakeholders supplemented for crisis • Sovereign defaults • New clients and partner- spending prevented ships specific to pandemic • Communications and advocacy Out of scope Source: Independent Evaluation Group. Note: DPF = development policy financing; IMF = International Monetary Fund; IPF = investment project financing; MPA = Multiphase Programmatic Approach; MSME = micro, small, and medium enterprise. Independent Evaluation Group World Bank Group    9 With support from the Bank Group and other sources, most countries pro- gressed from the acute crisis phase to the incipient recovery phase during the evaluation period. The acute crisis phase was marked by widespread lockdowns and steep dives in economic growth. The incipient recovery phase was marked by the easing of lockdowns and the resumption of economic growth. The resumption of economic growth largely occurred in the first half of 2021. For instance, growth returned to positive territory by first quarter of 2021 in Nigeria and Serbia (case study countries for this evaluation) and in Mozambique and Zambia (heavily indebted countries), and it rebounded by the second quarter of 2021 in Georgia and the Philippines (also case study countries). By the end of the evaluation period (June 2021), nearly every client country had also eased restrictions from their peak stringency, as The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 measured by the Oxford Stringency Index.2 In Serbia and Zambia, lockdowns were eased as early as June 2020, in Georgia by August 2020, in Nigeria by November 2020, and in Mozambique and the Philippines by January 2021. Some countries reimposed more stringent restrictions later in the pandemic, but few returned to the level of stringency of second quarter 2020. Nearly every country had both resumed economic growth and loosened lockdowns by June 2021. Objective and Scope of the Evaluation This evaluation assesses the relevance and quality of the Bank Group’s early response in addressing the economic implications of COVID-19. Its purpose is to foster learning and adaptive management to strengthen the Bank Group’s response to the economic dimensions of the COVID-19 crisis— protecting livelihoods—and help the Bank Group prepare for future crises. This evaluation covers the Bank Group’s early response to specifically ad- dress the economic implications of COVID-19. It examines Bank Group interventions over the 15-month period from April 1, 2020, to June 30, 2021. Although the case studies cover the whole evaluation period, the portfolio analysis considers a subset of the evaluation period, from April 2020 to April 2021.3 For assessment of intracrisis learning, the first part of the evaluation period (the acute crisis phase) is defined as April 1 to December 31, 2020. 10 The second part of the evaluation period (the incipient recovery phase) is defined as January 1 to June 30, 2021. As described in the previous section, the two phases were identified based on the Oxford Stringency Index and country-level economic indicators (including GDP). Although the distinction is preliminary and has limitations (described in the next section), it allows making an early assessment of whether the Bank Group internalized learning from the acute phase of the crisis to address the challenges that were materi- alizing in the (incipient) recovery phase. An assessment of the Bank Group’s early response to the COVID-19 crisis is important for informing the recovery phase of the response to this crisis and early responses to future crises. The world is still developing an opti- mal policy response to COVID-19 economic implications, where the aim is to preserve economic value to the extent possible and keep physical capital and essential productive assets intact via macroeconomic, financial, and fiscal support. On this basis, a stocktaking of the Bank Group response at this early point—including successes and failures and overall consistency among Bank Group interventions and its comparative advantages—is warranted to support the next set of support efforts to address the COVID-19 crisis and to prepare for future crises. We offer findings, lessons, and recommendations to promote learning within the Bank Group overall. They are intended to be applied at a strategic and institutional level, not to provide ready-made solu- tions for the unique challenges of specific operations or projects. Independent Evaluation Group World Bank Group    11 Evaluation Questions, Methods, Limitations, and Links with Other Evaluations The overarching evaluation question is, “How well did the Bank Group respond to global, country, and firm needs in its early response to the COVID-19 economic crisis?” The report seeks to address this overarching question by assessing the relevance and quality of the Bank Group response as articulated in the following two evaluation questions: 1. What has been the relevance of the Bank Group COVID-19 response in addressing the economic needs of clients, and what lessons can be drawn? » Tailored approach. In what ways and to what extent did the Bank Group tailor its response to country conditions? » Design. To what extent was the Bank Group support to help clients address economic vulnerabilities informed by timely diagnostics, lessons from past crises, and understanding of trade-offs? » Comparative advantage. To what extent did the Bank Group use its com- parative advantages (for example, by developing and sharing knowledge; by leveraging internal synergies among the World Bank, IFC, and MIGA; and by convening partners) to tailor its client response? 2. What has been the quality of the early Bank Group COVID-19 response to address the economic needs of clients, and what lessons can be drawn? » Influence. To what extent have Bank Group interventions influenced gov- ernment policies and the actions of governments and firms? The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 » Coordination. How well has coordination within the Bank Group, between the Bank Group and the IMF, and between the Bank Group and other part- ners supported the design and delivery of Bank Group interventions? » Monitoring and governance. To what extent do Bank Group support efforts have adequate results frameworks, safeguards, and governance? We used a mixed methods approach. The methods included a structured literature review, case-based analysis, key informant interviews, portfolio review and analysis, and econometric and comparative analysis. The econo- metric and comparative analysis consisted of combining the coded portfolio data with data from public sources on COVID-19 (on countries’ situations and needs, including vulnerabilities, preparedness, capacities, socioeconom- ic impact, response, and spread of the virus) and using the combined data to select cases and address the relevance of the Bank Group’s response to the crisis, given countries’ needs. We studied the relevance and quality of the Bank Group response to the COVID-19 crisis at three levels: global, countries, and firms. At the global level, the units of analysis are Bank Group global support efforts, including establishment of partnerships and development of real-time knowledge on the economic implications of the COVID-19 crisis. We assessed these global support efforts and their contributions to the relevance and quality of the response across the three institutions of the Bank Group. 12 We studied the relevance of the Bank Group response on three dimensions. The relevance criterion assesses whether interventions are doing the right things (OECD DAC Network on Development Evaluation 2019). For the pur- pose of early-stage assessment, we assessed relevance in three ways: (i) evi- dence of a targeted approach based on the needs of the clients; (ii) the extent to which the targeted support was informed by timely diagnostics, lessons from past crises, and understanding of trade-offs; and (iii) the extent to which the Bank Group used its comparative advantages to tailor the response. We used three dimensions to study the quality of the Bank Group response. Quality is not strictly an evaluation criterion and is typically assessed as part of effectiveness. We did not comprehensively assess the effectiveness of the Bank Group responses to countries and firms because it is too early to do so. We do, however, provide evidence of the quality of Bank Group actions when available. Given the early-stage nature of this evaluation, we studied the quality of the Bank Group response on three dimensions: (i) the extent to which the Bank Group response influenced client strategies in response to the pandemic; (ii) the extent of coordination within the Bank Group, with the IMF, and with other development partners; and (iii) the extent of moni- toring, safeguards, and governance frameworks in place. In evaluating the quality of the Bank Group’s response, we assessed its ability to learn and apply lessons in real time, including from its own work (intracrisis learning). To evaluate the quality of the Bank Group’s response to COVID-19, we assessed whether the Bank Group was learning lessons Independent Evaluation Group World Bank Group    13 derived from its COVID-19 work as the crisis unfolded (intracrisis learning). The intracrisis learning assessment complemented a more traditional assessment of whether the Bank Group applied to its COVID-19 work lessons learned from previous crisis and real-time lessons from developed countries’ approaches to COVID-19. To assess intracrisis learning, we used the principles of adaptive management and crisis learning. Adaptive management is collecting, sharing, and studying data from ongoing efforts to identify and understand the drivers of results, then adapting processes to continually improve an organization’s ability to achieve the desired results (Moynihan 2009; appendix B). Adaptation should occur both during execution (adjusting course based on evidence) and during planning (setting the course for new efforts based on evidence). Crisis learning is adaptive management applied to the management of crises. Effective crisis response maintains what works well, expands resources as necessary, pivots to address new challenges, innovates, and increases risk appetite judiciously. We embedded these principles in our assessment of the quality of the Bank Group response to the crisis. The evaluation was developed in close coordination with a parallel Independent Evaluation Group (IEG) evaluation focused on protecting lives, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response, and with an IMF evaluation of the IMF’s early response to the pandemic. IEG has conducted an evaluation that focuses on assessing the World Bank’s early response to support governments’ responses to the The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 health, human capital, and social protection emergencies related to the COVID-19 crisis. The two IEG teams have collaborated closely, especially on country case studies, by jointly engaging with clients and stakeholders. We have also coordinated findings and messages, to the extent possible, with the IMF evaluation team that has conducted a review of the IMF’s early response to the COVID-19 pandemic at the country level by jointly engaging with clients and stakeholders regarding macrofiscal response. The evaluation adds to previous Bank Group efforts to learn from crisis sup- port. These efforts include the synthesis report Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations and the evaluations The World Bank Group’s Response to the Global Economic Crisis: Phase I and Phase II, and The World Bank Group and the Global Food Crisis: An Evaluation of the World Bank Group Response. The evaluation also leverages evidence gath- ered through just-in-time notes and validations on trade finance, support to small and medium enterprises in times of crisis, and distressed assets man- agement. See appendix A for more details. The evaluation has limitations because of its scope. The evaluation focuses on assessing the Bank Group’s response to the economic implications of the COVID-19 crisis (“the impact of the crisis on livelihoods”). As mentioned earlier, it was developed in concurrence with—and it is complemented by— an evaluation of the World Bank’s health, human capital, and social response to the COVID-19 crisis (which focuses on “the impact of the crisis on lives”). 14 Neither evaluation assesses the World Bank’s, IFC’s, or MIGA’s overall corpo- rate response to the crisis. Rather, they look at the Bank Group’s response on selected topics. For example, activities under the IFC Global Health Platform are outside the scope of this evaluation because the underlying projects are related to the health sector. Moreover, none of the projects financed by the Global Health Platform would have met the selection criteria for case studies. The decision to have two separate reports stemmed from the recognition that assessing the Bank Group’s work on both lives and livelihoods in a single eval- uation would have limited the depth of the analysis. Both evaluations will be complemented by future, ex post assessments of the impact of the crisis. The evaluation also has limitations because of its real-time learning nature. Because of the limited time available to deliver an early-stage evaluation with lessons useful for shaping the ongoing response, the evaluation Approach Paper was approved with the agreement that the evaluation would have limited scope and depth. As such, this evaluation has several limitations. First, the assessment overlaps with the ongoing Bank Group response to the crisis. The Bank Group was adapting its response during the evaluation period to improve its actions in light of the evolving and uncertain global-, country-, and firm-level contexts of COVID-19. Activities under the IFC Base of the Pyramid Platform, which the Board of Executive Directors approved in February 2021, are outside the scope of this evaluation because the underlying projects are outside the evaluation period. Given Independent Evaluation Group World Bank Group    15 the early-stage nature of the evaluation, we do not attempt to estimate the probability that early response projects will succeed. Second, the identification of the two windows in the evaluation period (the acute crisis and incipient recovery phases) is based on a preliminary assessment. The evaluation of the incipient recovery phase of the evaluation—which lasts only 6 months, including 4 months of portfolio data—should be regarded as a first assessment of the Bank Group’s work supporting countries’ and firms’ first recovery, with the aim of informing future Bank Group actions to support the next phase of the recovery. Third, because we treated the IFC and MIGA portfolio only via a sample of firm-level cases, we could not assess the relevance or quality of the overall IFC and MIGA response across all COVID-19–tagged projects. Fourth, IFC advisory services and World Bank advisory services and analytics were assessed only to the extent they were linked to country- and firm-level case studies because extensive coverage of analytical and advisory work was not feasible. IFC upstream activities were outside the scope of this evaluation. Fifth, the analysis was limited by the availability and quality of data tagged as relevant to the COVID-19 economic response, supervision data, interviewees, and strong indicators for successful economic policy responses. Gaps were mitigated by relying on new macroeconomic policy response trackers from external sources (such as Oxford University and the IMF). Data availability was mostly consistent on country case studies but uneven on firm case studies because of confidentiality, ongoing restructuring of projects, and data paucity regarding onlending to real sector firms. We used a novel framework to analyze country needs (the need score described in chapter 3), and there were data The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 1 limitations to the analysis in some countries and sectors. We operated on the premise that “rough approximations delivered at the right time are better than precise results delivered too late for decision” (Bellavita, Wholey, and Abramson 1986). 16 1  See the United Nations, Department of Economic and Social Affairs, “Sustainable Develop- ment,” https://sdgs.un.org/goals#goals. 2  The Oxford Stringency Index aggregates countries’ policy responses to the pandemic over time on a scale from 0 (no restrictions on everyday activities) to 100 (complete lockdown of the country; Hale et al. 2020). 3  Based on World Bank Group COVID-19 response data availability. Independent Evaluation Group World Bank Group    17 2 | Agile Corporate Response to Address the Economic Implications of the COVID-19 Crisis Highlights The World Bank Group COVID-19 crisis support—the largest among the development partners at $157 billion between April 2020 and June 2021 ($65 billion alone for the economic response)—was underpinned by a strong and agile corporate response. The World Bank demonstrated a strong and agile corporate response, includ- ing agile decision-making, by creating the Emergency Operations Center. The agile response of the International Finance Corporation (IFC) was evidenced in the creation of a fast-track facility based on lessons from past crises. World Bank commitments went mainly to Africa and Latin America and the Caribbean, Regions with many International Development Association clients that had greater economic needs to respond to the crisis. The Equitable Growth, Finance, and Institutions Practice Group and the Social Protection and Jobs Global Practice provided the main share of the World Bank response, whereas the Financial Institutions Group provided the main share of IFC investment ser- vices’ response. World Bank disbursements to International Devel- opment Association clients in Africa and the Middle East and North Africa were relatively low. IFC disbursements to clients were high in all Regions. The Bank Group acted quickly and strategically to help client countries weather the economic implications of the COVID-19 crisis. It articulated its approach to the crisis early, front-loaded 18   its efforts to support low-income countries, and organized new decision-making committees to facilitate an agile response. It reflected lessons from past crises (including the 2008 global financial crisis) well at the onset of the COVID-19 pandemic, including to facilitate fast disbursements to clients. The Bank Group relied on a variety of existing and new approach- es and instruments to respond to the economic implications of the crisis, including the World Bank’s Multiphase Programmatic Approach, four new IFC fast-track COVID-19 envelopes under one facility, and two Multilateral Investment Guarantee Agency guaran- tee programs. 19  The World Bank Group’s Corporate Actions to Respond to the COVID-19 Crisis In a context of high uncertainty about how to address the global economic crisis, Bank Group senior management acted quickly and strategically to address the COVID-19 crisis. Bank Group senior management articulated its strategy early. In March 2020, Bank Group senior management announced that the Bank Group would mount a large-scale effort to fight the pandemic. The World Bank’s strategy included front-loading International Develop- ment Association (IDA) spending allocations and seeking an unprecedented IDA Replenishment a year ahead of schedule, activating the International Bank for Reconstruction and Development’s crisis buffer to release addition- The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 2 al financing, aligning with the World Health Organization (WHO) guidance on technical health issues, and launching with the IMF a global convening effort on debt suspension to help indebted client countries free up fiscal space for crisis response. The parallel IEG evaluation focused on protecting lives also found that the World Bank response exhibited “unprecedented scale and speed” (World Bank 2022d, x). Similarly, IFC and MIGA announced fast-track COVID-19 facilities and guarantee program envelopes to respond to the pandemic. In its strategy Approach Paper, the Bank Group framed the technical contents of its response in relation to four pillars and three phases, including addressing staff safety concerns in headquarters and country offic- es (World Bank 2020e). The Bank Group early response demonstrated agile corporate decision- making. The World Bank quickly established the Emergency Operations Center (EOC), which proved to be a useful innovation for operational coordination. As the parallel IEG evaluation focused on protecting lives also notes (World Bank 2022d), the EOC had an agile governance structure with a director-level World Bank–wide steering committee. It worked across Global Practices (GPs), bringing technical, operational policy, legal, and fiduciary functions together and relying on subcommittees to work on technical issues. The three institutions created new corporate arrangements (for example, new committee structures for management approvals and adjustments to policies and procedures) to respond to the crisis. The Bank 20 Group calibrated the new arrangements based on feedback from clients (such as rotating the chair of the EOC) and the findings of diagnostics conducted by the practice and industry groups. Speed of decision-making improved over time, for example, by delegating authority from the Board to the EOC to approve commitments. The agile decision-making efforts of the World Bank were organized in relation to COVID-19 rapid-response committees reporting to the EOC, with representation from multiple vice presidential units. The decisions of the rapid response committees on strategic areas to prioritize (for example, health systems or the financial sector) cascaded to the Regions and countries at the onset of the pandemic. Country- and sector- specific prioritization efforts by the Regions and Country Management Units further strengthened the Bank Group’s corporate response across the three institutions, for example, by contributing to the focus on low-income countries (LICs), strengthening public institutions, and supporting the financial sector. Similarly, IFC and MIGA fast-tracked decision-making on investment services and new guarantee issuances via emergency operations committees led by senior industry directors and MIGA operations directors. The Bank Group reflected lessons from past crises in the design of COVID-19–related corporate arrangements and sought delegated authority from the Bank Group shareholders for faster disbursements. The corporate initiatives of the Bank Group as part of its early response demonstrated deliberate actions to learn from past crises, for example, the global financial crisis of 2008. One of the key weaknesses in the Bank Group’s crisis response Independent Evaluation Group World Bank Group    21 in 2008 was slow decision-making and slow disbursements. At the onset of COVID-19, the design and agile deployment of an IFC fast-track COVID-19 facility and the expansion of the World Bank Multiphase Programmatic Approach (MPA) led to faster disbursements to clients. For example, half of the $8 billion approved under the IFC fast-track COVID-19 facility was com- mitted within six months, including the full use of the Global Trade Finance Program envelope. This rapid response suggests that management embraced the key lessons from prior crises. Portfolio of World Bank Group Interventions As a result of the corporate actions, the Bank Group delivered the largest overall response to COVID-19 among development partners. The Bank Group committed $157 billion to alleviate the pandemic’s health, economic, and social impacts during the first 15 months of the crisis (April 1, 2020, to June 30, 2021). This is the largest crisis response of any such period in the Bank Group’s history. The World Bank alone increased its overall commitments by more than 60 percent over the 15-month period before the pandemic, from $42 billion on average for the 2016–19 period to $67 billion on average for 2021–22, significantly above any other development partner (figure 2.1).  nnual Average Multilateral Development Bank Figure 2.1. A Commitments before and during the COVID-19 Crisis World Bank +60% (IBRD and IDA) The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 2 ADB +23% IDB +31% AIIB +200% NDB +121% IsDB -10% AfDB -50% 15 30 45 60 75 Commitments (US$, millions) CY20—21 (avg) CY16—19 (avg) % %change Source: World Bank. Note: ADB = Asian Development Bank; AfDB = African Development Bank; AIIB = Asian Infrastructure Investment Bank; CY = calendar year; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IDB = Inter-American Development Bank; IsDB = Islamic Development Bank; NDB = New Development Bank. The Bank Group response to specifically address the economic implica- tions of COVID-19 accounted for $65 billion and covered an array of themes and sectors. Over the same 15 months, the Bank Group financed 1,130 COVID-19–related interventions with a volume of $65.08 billion to address the economic implications of the pandemic (table 2.1). International Bank for Reconstruction and Development and IDA lending and advisory projects accounted for 74 percent and 75 percent in project count and volume, re- spectively, with investment project financing (IPF) representing the largest share of the portfolio, followed by DPF. IFC projects accounted for 21 percent 22 in project count and 16 percent in project volume, with new and restructured investment services representing more than 80 percent of the portfolio. MIGA, given its business model, had a smaller share of projects related to the economic implications of COVID-19, reflected by 5 percent and 9 percent in project count and project volume, respectively.  orld Bank Group COVID-19 Pandemic Response Plan Table 2.1. W Evaluation Portfolio (April 2020–April 2021) Projects Amount Institution or Instrument (no.) (US$, billions) Total 1,130 65.08 IBRD and IDA 841 48.50 Advisory services and analytics 450 0.04 Lending 391 48.46 Development policy lending 81 19.82 Investment project financing 292 24.07 Program-for-Results financing 17 4.55 Special fund 1 0.01 International Finance Corporation 234 10.42 Global Trade Finance Program 18 0.20 Special operations transfer 24 1.96 New investment services 82 4.45 Restructured investment ser- 61 3.72 vicesa Independent Evaluation Group World Bank Group    23 Advisory services 49 0.10 Multilateral Investment Guaran- 55 6.16 tee Agency Non-honoring guarantees 10 3.54 Political risk insurance 45 2.62 Source: Independent Evaluation Group. Note: IBRD = International Bank for Reconstruction and Development; IDA = International Development Association. a. Projects restructured because of COVID-19, which are treated differently from new investment commitments by the International Finance Corporation portfolio team. Social Protection and Jobs Global Practice projects were covered in this evaluation to the extent that they included relevant components to household support. The total number of Social Protection and Jobs projects included in the study was 154 for total commitment volume of $15.38 billion. Social Protection and Jobs commitment data specific to economic implications were not available. The stated Global Trade Finance Program volume covers only increases in trade finance limits during this evaluation period and not the total envelope used for this program. The total presented in this table should not be treated as the sum of all the International Finance Corporation envelopes’ total size but the increases during this evaluation period. Activities of the International Finance Corporation Global Health Platform and Base of the Pyramid Platform are outside the scope of this evaluation. Africa received the largest share of the Bank Group’s commitments to re- spond to the economic implications of COVID-19. Most Bank Group com- mitments went to Africa, Latin America and the Caribbean, and Europe and Central Asia. These three Regions received about 65 percent of the commit- ments. South Asia and the Middle East and North Africa received relatively fewer projects (figure 2.2). Africa and Latin America and the Caribbean had more IDA countries with greater needs to respond to the economic shocks of COVID-19. The Equitable Growth, Finance, and Institutions Practice Group and the Social Protection and Jobs GP provided the main share of the World Bank early response to COVID-19 economic implications. Together, they made up 77 percent of all lending commitments ($37 billion out of $48.5 billion) and The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 2 52 percent by number of interventions (204 out of 391) in the early response to address economic implications. The IFC Financial Institutions Group provided the main share of IFC invest- ment services’ early response to COVID-19 economic implications. Financial Institutions Group support made up 76 percent of IFC investment services’ early response to address COVID-19 economic implications ($3.5 billion out of $4.6 billion, excluding restructured projects and special operations trans- ferred projects). Financial Institutions Group support is expected to prevail in the acute phase of the crisis, while real sector support should increase in the recovery phase. World Bank disbursements in Latin America and the Caribbean and East Asia and Pacific were high, but disbursements in Africa and the Middle East and North Africa were relatively low. World Bank disbursements in Latin America and the Caribbean and East Asia and Pacific were higher than precrisis levels at 82 percent and 75 percent, respectively. World Bank disbursements in Africa and the Middle East and North Africa were lower than precrisis levels at 57 percent and 65 percent, respectively. Both Africa and the Middle East and North Africa include several IDA countries. Disbursements were partly affected by the high number of IPFs and the necessary safeguard compliance requirements before disbursements. 24  orld Bank and IFC Commitments and Disbursements of Figure 2.2. W COVID-19 Early Response Addressing Economic Implications, by Region (April 2020–June 2021) 20 100 Disbursement/commitment ratio (%) 18 90 16 80 Amount (US$, billions) 14 70 12 60 10 50 8 40 6 30 4 20 2 10 0 0 AFR EAP ECA LAC MENA SAR AFR EAP ECA LAC MENA OTH SAR WLD IBRD and IDA IFC Commitment Disbursement Disbursement/Commitment ratio Source: Independent Evaluation Group. Note: The numerator in the disbursement to commitment ratio percentage is calculated based on ag- gregate disbursement flows during the evaluation period (vintage October 2021) for projects in this eval- uation’s portfolio and is intended to be a high-level view of total disbursements to the Region. This ratio is not based on the World Bank’s traditional definition of disbursement ratio as annual disbursements to projects within a fiscal year. OTH refers to a special program with the Ministry of Finance and Health in the Republic of Yemen. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = International Finance Corporation; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; OTH = other; SAR = South Asia; WLD = World. IFC disbursements to clients were high, except for those in the Middle East Independent Evaluation Group World Bank Group    25 and North Africa region, and broadly in line with IFC’s expected countercy- clical role. Across all regions, IFC’s disbursement ratio from early response projects was above 90 percent.1 This was higher than its long-term average disbursements (85 percent during 2015–18) and higher than the disburse- ments observed in the early response during the global financial crisis (60 percent during 2008–10), suggesting IFC’s agile response. IFC disburse- ment to clients from the early response in Middle East and North Africa is 44 percent and the lowest among the six regions. The Bank Group response drew on a mix of existing instruments and new approaches. The Bank Group responded to the COVID-19 crisis by develop- ing new approaches in addition to existing instruments and providing addi- tional financing to existing loans (figure 2.3). The World Bank deployed the MPA—in addition to individual investment projects and development policy operations (DPOs)2—to address the crisis. The World Bank’s response lever- aged contingent emergency response components (CERCs) and catastrophe deferred drawdown options (CAT DDOs) as part of its lending operations based on countries’ conditions. All seven CAT DDO operations approved to address the economic implications of COVID-19 were linked to DPF; CERCs were linked to DPF or IPF. The World Bank’s country engagement process was a critical feature of its approach to supporting client countries. Country engage- ment products (Country Partnership Frameworks and Performance and Learn- ing Reviews) helped adapt the Bank Group’s country programs to respond to this crisis in its acute phase. The World Bank made efforts to generate new information and provide policy guidance to governments using in-country diagnostics and a range of advisory services and analytics.3 For example, (i) The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 2 high-frequency phone surveys were conducted at the household level, often in partnership with national statistics offices, in more than 100 countries (with repeated rounds in a sizable subset);4 (ii) new data on private sector perfor- mance and impact were collected, primarily through business surveys such as the Business Pulse Surveys and Multinational Corporation Pulse Surveys; and (iii) various trackers were established to monitor governments’ policy responses by collecting new data on public policy responses.5 IFC developed a new facility and programs to respond to the crisis, including (i) the Financial Institutions Group Response Envelope, (ii) the Real Sector Crisis Response Envelope, (iii) an allocation for COVID-19 work to the Global Trade Finance Program, (iv) the Working Capital Solutions program, and (v) an allocation for COVID-19 work for the Global Trade Liquidity Program and the Critical Com- modities Financing Program. IFC provided portfolio-level standstill arrange- ments to many existing clients to prevent loan defaults. Between June 2020 and June 2021, IFC launched two platforms to augment its early response: (i) the Global Health Platform and (ii) the Base of the Pyramid Platform; both are outside the scope of this evaluation because the Global Health Platform focuses on health, and the recent launch of the Base of the Pyramid Platform did not allow the evaluation team to assess its relevance and quality. MIGA launched two guarantee programs—credit enhancement and capital optimiza- tion—focused on the financial sector. MIGA also supported IFC’s trade finance activities by issuing risk coverage support to commercial banks. 26  orld Bank Group COVID-19 Response—New Approaches Figure 2.3. W and Existing Instruments COVID-19 response (new Existing instruments, portfolio, approaches, fast-track and risk management facilities, and programs) ASA DPF IPF The World Bank Multiphase PforR Programmatic Approach CAT DDO Additional finance CERC Repurpose existing exposure Portfolio suspension arrangements Advisory services IFC RSE envelope Debt financing Global Trade Finance Equity financing Working Capital Investment services Solutions Subordinated and Trade Liquidity and structured finance Critical Commodities Finance Independent Evaluation Group World Bank Group    27 BoP program FIG envelope FTCF facility Credit Enhancement Non-Honoring Program Guarantees MIGA Support to IFC Trade Finance Political Risk Capital Optimization Insurance Program Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; BoP = Base of the Pyramid; CAT DDO = catastrophe deferred drawdown option; CERC = contingent emergency response component; DPF = development policy financing; FIG = Financial Institutions Group; FTCF = fast-track COVID-19 financing; IFC = International Finance Corporation; IPF = investment project financing; MIGA = Multilateral Investment Guarantee Agency; PforR = Program-for-Results; RSE = Real Sector Crisis Response Envelope. Broadly, the Bank Group’s response to the economic implications of COVID-19 sought to directly meet the needs of governments and firms, in turn addressing the needs of households. It provided support for macrofiscal and financial stability, public sector institutional improvements, and capital enhancement and liquidity for financial institutions and MSMEs in sectors affected by the pandemic and the lockdowns. The Bank Group support pro- vided through the existing instruments and new approaches described in the previous paragraph aimed at (i) supporting fiscal and financial stability by enhancing macroeconomic policy reforms in debt management and reorien- tation of government expenditures, and through enhancement of financial sector supervision and strengthening of deposit insurance; (ii) improving public sector institutional capacity to respond to the pandemic; (iii) provid- ing emergency liquidity and credit to allow firms (including financial institu- The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 2 tions) to continue to operate; and (iv) supporting workers by financing and refining pension plans, and digitalizing transfers related to the social safety net and cash transfers. 28 1 Disbursement ratio is calculated as the ratio of total disbursements to date to project com- mitments at approval.  2  We distinguish between development policy operations and how they are financed—develop- ment policy financing. 3  COVID-19 Research, Analysis, and Policy Responses internal website, accessible at http:// covid19research. 4  The surveys filled a major gap in data created by the speed of the crisis and provided key information on the types of households and sectors most affected by economic losses, the extent of job losses, coping strategies, and food insecurity. These data were valuable to inform and fine-tune the World Bank’s and the governments’ responses and take heterogeneity with- in countries into account. 5  For example, the Foreign Direct Investment Entry Tracker identified a rise in foreign direct investment barriers, especially among Organisation for Economic Co-operation and Development countries. An assessment of responses and programs by investment promotion agencies provided key insights on investment retention strategies. Independent Evaluation Group World Bank Group    29 3 | Relevance of the World Bank Group Early Response Highlights The relevance assessment focused on whether the World Bank Group COVID-19 economic interventions were (i) tailored to country and sector needs; (ii) designed based on diagnostics, lessons from past crises, and understanding of trade-offs; and (iii) based on the Bank Group’s comparative advantages. The Bank Group’s early response generally was highly relevant to low-income countries, which were the ones most in need. Within countries, there was considerable variation in the extent to which Bank Group support went to sectors with greater needs. Macrofiscal support (including social safety nets) via development policy financing was relevant to client governments. International Finance Corporation and Multilateral Investment Guarantee Agency support to client firms in the financial sector; micro, small, and medium enterprises; and onlending to facilitate trade flows was mostly relevant. The Bank Group used lessons from past crises well to inform its early response designs and approaches, including through country analytical work. Similarly, the International Finance Corporation adapted its response to the local context well based on lessons from the 2008 global financial crisis and disbursed effectively to regional clients. The Multilateral Investment Guarantee Agency’s support to trade finance activities in partnership with the International Finance Corporation also built on past lessons. The Bank Group’s assistance built on its comparative advantages. 30   The World Development Report 2022 provided a strong knowledge base to inform country programs, but global knowledge work on the uniqueness of the crisis was missing (World Bank 2022e). The Bank Group also did not reflect in its operations lessons from de- veloped countries on addressing the economic implications of the COVID-19 crisis. Although the early response benefited from the Bank Group’s global footprint, country office staff needed greater clarity on crisis protocols to be more relevant. Finally, data gaps ex- ist that may impede targeted responses in future crises. 31  We assessed the relevance of the Bank Group’s response on three di- mensions. Relevance means “doing the right things in the right places,” that is, deploying interventions that address countries’ and sectors’ needs to cope with the economic implications of COVID-19. To assess relevance, we evaluated three dimensions: (i) the extent to which Bank Group tailored its response to country and sector needs; (ii) the extent to which Bank Group’s response was informed by diagnostics, lessons from past crises, and under- standing of trade-offs; and (iii) the extent to which the Bank Group used its comparative advantages (global knowledge, financing, and global footprint) to tailor its response. We also assessed whether the Bank Group prepared for a future targeted response by collecting the right data. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 Relevance for Countries and Sectors We assessed the relevance of Bank Group support to countries and sectors by comparing their needs with the support that the Bank Group provided to them. First, we constructed need scores related to the economic implica- tions of COVID-19 at the country and sector levels. Second, we measured the strength of Bank Group support received as the ratio between the country’s total amount of received support and its GDP (that is, the support received per dollar of GDP). We calculated sector need scores using harmonized indicators from the World Bank Business Pulse Surveys and the World Bank Enterprise Survey follow-up on COVID-19.1 Third, we compared the need scores to the strength of Bank Group support received to assess the relevance of the Bank Group’s interventions. (For a full description of the methodology, see appendix A and Naeher, Narayanan, and Ziulu 2022.) To assess country and sector relevance, we developed country and sector need scores as proxy measures of the overall degree to which a country or a sector was adversely affected by the COVID-19 pandemic. The country need score aggregates micro (firm-level) and macro (country-level) data across nine aspects of the economy into a single country-level composite index. The sector need score aggregates micro (firm-level) and macro (country-level) data for each aspect of the economy within a country. The nine aspects of the economy used to measure needs are (i) education, (ii) health, (iii) social protection, (iv) public finance, (v) financial sector, (vi) economic fitness, 32 (vii) agriculture, (viii) manufacturing, and (ix) services.2 As an example, the analysis assumes that countries with fewer hospital beds and more reported deaths tend on average to be more vulnerable to the shocks triggered by COVID-19. We selected the nine sectors and the main indicators and subindicators associated with them based on data availability and similar existing approaches in the economic literature (appendix C). Overall, Bank Group support to countries was relevant because it focused on LICs, which tended to have greater needs for support to address the economic implications of COVID-19. Figure 3.1 shows that need score estimates were higher in LICs. Figure 3.1, panel a depicts the relationship between the need score and per capita GDP. Figure 3.1, panel b shows the ranges of the need score estimates for each income group. Both graphs show a clear negative relation between countries’ need score and income levels, indicating that LICs tended to be more vulnerable and in need of support than richer countries during the COVID-19 pandemic in 2020. It is important to note that the need score estimates for poor countries vary widely, with estimates ranging from 0.26 to 0.62 in the group of LICs. This indicates that not all poor countries suffered equally under COVID-19. In fact, there are several examples of LICs with need scores well below the average scores of higher-income groups. Figure 3.2 shows the relationships between Bank Group support (expressed as the total amount of support per dollar of GDP) and countries’ per capita GDP (panel a) and between Bank Group support Independent Evaluation Group World Bank Group    33 and the need score (panel b). The negative relation between Bank Group support and per capita GDP indicates that the Bank Group’s early support in response to COVID-19 tended to be larger (relative to the size of the benefiting countries’ economies) in poorer countries. Given the greater vulnerability of poorer countries, as shown in figure 3.1, the Bank Group’s early response to COVID-19 provided relatively more support to countries with greater needs for support.  elationship between Countries’ Income Levels and Need Figure 3.1. R Score Estimates a. Need score versus GDP per capita 0.8 0.6 Need score 0.4 0.2 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 0 0 25,000 50,000 75,000 100,000 GDP per capta b. Need score by income level 0.8 0.6 Need score 0.4 0.2 0 LIC LMIC UMIC HIC Country income classification Source: Independent Evaluation Group. Note: The line in panel a represents the fitted values of a bivariate linear regression of need score on GDP per capita. In panel b, the boxes represent the range of values between the 25th and 75th percen- tile (including the median), the ends of the whiskers represent the lower and upper adjacent values, and the dots represent outliers. Countries with World Bank Group support (amount/GDP) greater than 0.1 are excluded. The need score is a normalized composite measure constructed using the methodology described in appendix C. Data are for 2020. GDP = gross domestic product; HIC = high-income country; LIC = low-income country; LMIC = lower-middle-income country; UMIC = upper-middle-income country. 34  elevance of World Bank Group Support in Relation to Initial Figure 3.2. R Conditions a. GDP per capita b. Client needs Group support (amount/GDP) World Bank Group support (amount/GDP) 0.08 0.08 World Bank Group support (amount/GDP) World Bank Group support (amount/GDP) 0.06 0.06 0.04 0.04 0.02 0.02 World Bank 0 0 0 10,000 20,000 30,000 0 0.2 0.4 0.6 Need score GDP per capita Need score Source: Independent Evaluation Group. Note: The line in each graph represents the fitted values of a bivariate linear regression of the variable on the y-axis on the variable on the x-axis. High-income (nonclient) countries and countries with World Bank Group support (amount/GDP) greater than 0.1 are excluded. Data are for 2020. Robustness checks are described in appendix C. GDP = gross domestic product. However, some countries with high needs received limited Bank Group support on the aggregate nine aspects of the economy used to measure the country need scores. Angola, Bangladesh, Gabon, India, Lebanon, Mongolia, Nigeria, the Philippines, South Africa, and Zimbabwe had country need scores above the median, taking into account all nine aspects of the Independent Evaluation Group World Bank Group    35 economy (education, health, social protection, public finance, financial sector, economic fitness, agriculture, manufacturing, and services). Nevertheless, they received limited support from the Bank Group in aggregate on these nine areas. It is important to note that these countries may have not requested assistance from the Bank Group because they relied on other partners or on their own internal resources to address sectors’ needs, may have limited ongoing programs with the Bank Group, or may be in nonaccrual status. Within countries, the Bank Group’s support for sectors with greater needs varied. Figure 3.3 shows the sector-specific need score estimates and nor- malized distribution of Bank Group support for select country case studies (note that for some countries, data are missing for some sectors). Each graph also reports the relevance score, which is calculated based on comparing the distribution of Bank Group support (indicated by the blue bars in figure 3.3) with the need score (orange bars). These results give rise to several inter- esting insights. There is considerable variation in the degree to which Bank Group support aligned with the sector-specific need scores in each country. For example, in Cabo Verde and Pakistan, there was a strong correlation between the need scores and the relative magnitude of Bank Group support to each sector, corresponding to relevance scores of 78 percent and 69 per- cent, respectively. By contrast, the match between sector-specific needs and Bank Group support was weaker in the Philippines and Senegal,3 where the relevance scores were only 31 percent and 34 percent, respectively. Figure 3.3, panels c and d show, for example, that the Bank Group’s support in these countries focused mainly on social protection and economic fitness, as highlighted in the parallel IEG evaluation on preserving lives (World Bank The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 2022d). However, the Bank Group did not provide support for agriculture, manufacturing, or services in either country, although these sectors are esti- mated to have had relatively large needs (or vulnerabilities) compared with other sectors. The World Bank held back support to a few countries, both in terms of commitments and, in some cases, disbursements, but these cases do not affect the overall relevance of the response because they were limited by government capacity. Because of governance gaps (for example, anticor- ruption efforts) and regressing policy reforms (for example, structuring fast disbursements to citizens), a few clients (including Nigeria and Zambia) did not get the necessary commitments during the pandemic (according to IEG interviews). In some cases, DPF may have been an appropriate instrument, but the World Bank held back its use based on country conditions and risk to development outcomes (for example, in South Africa). In a few cases (for example, Ecuador), a sound overlapping engagement with the IMF helped clients emerge out of the crisis faster than peers within the Region. In other cases (for example, Nigeria), clients’ structural issues limited the uptake of World Bank support within a reasonable time frame (12–18 months since commitment) during a pandemic (box 3.1). 36  ithin-Country Needs Analysis by Sector versus World Bank Figure 3.3. W Group Support in Case Study Countries, Sample a. Cabo Verde (relevance: 78 percent) 1.0 0.8 0.6 0.4 0.2 0.0 lth n re m n es g t ss tio en tio rin tu te ea ic ne ec ca nm ul ys tu rv H fit ic Se ot ac u ls er r Ed ic pr Ag ia uf ov om nc an al G ci na on M So Fi Ec World Bank Group support Need score b. Pakistan (relevance: 69 percent) 1.0 0.8 Independent Evaluation Group World Bank Group    37 0.6 0.4 0.2 0.0 lth n es re n m g s t en tio io s in tu te ic ea ne at ur nm c ul rv ys H uc te fit t ric Se ac ls ro er Ed ic Ag uf ia p ov om nc an al G on ci na M So Fi Ec World Bank Group support Need score c. Philippines (relevance: 31 percent) 1.0 0.8 u t s ro er i S a Ed ic r al Ag uf p ov om i nc an al G on ci na M So Fi Ec World Bank Group support Need score c. Philippines (relevance: 31 percent) 1.0 0.8 0.6 0.4 0.2 0.0 m n n re es g lth t ss en tio tio rin te tu ic ne ea The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 nm ec ca ys tu ul rv fit H ic ls ac Se u ot er Ed ic r pr ia Ag uf ov om nc an al G na ci on M So Fi Ec World Bank Group support Need score d. Senegal (relevance: 34 percent) 1.0 0.8 0.6 0.4 0.2 0.0 re g n lth n es ss m t en io rin tio tu te ne ic ea at nm tu ec ul rv ys uc fit H ic ac Se ot ls er ic Ed r uf Ag pr ia om ov an nc al G on M ci na So Ec Fi World Bank Group support Need score Source: Independent Evaluation Group. Note: Depicted values of World Bank Group support, need score, and relevance score are normalized values constructed using the need score analysis methodology described in appendix C. The relevance score (ranging from 0 to 1) is converted to a percentage here for illustrative purposes for case study countries. If data on the indicators used in the construction of the need score for a sector are missing, then no (orange) bar is shown for that sector. Data are for 2020. 38  igeria: Client Structural Issues as Constraints on World Bank Box 3.1. N Support Complexity in Nigeria starts with the intergovernmental fiscal relations among the federal government, the 36 states of the federation, and 774 local governments. To date, the World Bank Group has also not done sufficient advisory services and analytics (World Bank) or advisory services work (International Finance Corporation) to clarify the responsibilities and accountability of the various levels of government, including at the state level. In this context, the World Bank supported a COVID-19 response financed by (i) a US$100 million International Development Association credit allocated to Nigeria through the fast-track COVID-19 facility under the crisis response window and Nigeria’s allocation under the 19th Replenishment of the International Development Association and (ii) a US$14 million grant through the Pandemic Emergency Financing Insurance Window, which provides financial support to International Development Association–eligible countries if major multicountry disease outbreaks occur. Despite major efforts by World Bank staff, at the time of this evaluation, the Pandemic Emergency Financing Project had not been disbursed because of the complexity of the federal structure and the well-documented dysfunction of the state and national governments. An upcoming Program-for-Results (2022) offers some hope for addressing the incentives for states to align with the federal government. Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    39 Macrofiscal support via DPF was highly relevant to clients. Given its flex- ibility and relatively agile disbursing ability, the World Bank used the full suite of DPF instrument variations to address the economic implications of COVID-19. Macrofiscal support was higher in volume commitments than during the global financial crisis (2008) and higher than the prepandemic level of support, suggesting the relevance of the response in addressing the economic shock felt in client countries (figure 3.4). Besides the stand-alone and series DPOs, the World Bank approved seven DPF CAT DDOs and one DPF policy-based guarantee. Under a DPF CAT DDO, budget support is dis- bursed on the activation of a catastrophe trigger, typically the declaration of a national emergency or disaster. Generally, the DPOs with additional com- ponents disbursed faster than the investment projects and played a relevant role in budget support in pillar 4 (institution strengthening), followed by pillar 3 (sector support). This focus broadly aligned with the Bank Group’s June 2020 COVID-19 response Approach Paper (World Bank 2020e).  acrofiscal Support via Development Policy Operations, Figure 3.4. M Select Time Periods 200 150 DPO (no.) 100 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 50 0 FY07–08 FY18–19 FY20–21 Time period Source: Independent Evaluation Group calculations and Operations Policy and Country Services Devel- opment Policy Actions Database (April 2022). Note: DPO = development policy operation; FY = fiscal year. The IFC and MIGA early response was essential to supporting local banks in providing crisis financing. Without IFC and MIGA’s response, existing clients in the financial sector would have either defaulted on loans or cut back on their onlending programs, leading to severe disruption for firms (especially MSMEs) and supply chains. Without IFC and MIGA support, clients would have also launched employee reduction programs, leading to income reduc- tions for workers and households (per key informant interviews and case studies). See box 3.2 for examples. 40  he International Finance Corporation and Multilateral Box 3.2. T Investment Guarantee Agency Early Response In Indonesia, local banks did not have the risk appetite to provide large financing to large real sector firms, mainly because of uncertainty about the direction of the pandemic. One example was Trans Corp, an existing International Finance Corporation (IFC) client that operated in the consumer segment. Trans Corp’s businesses largely counted on local consumers, and the pandemic severely affected local demand. Trans Corp faced short-term liquidity problems in 2020 as revenues fell because of tempo- rary closures of its facilities in response to the government’s containment measures. Without IFC’s liquidity support to Trans Corp during the pandemic, Trans Corp would have laid off most of its employees. Moreover, the layoffs would have affected the company’s relationship with its suppliers, most of which were small and medium enterprises (SMEs) in the retail segment of its business. Absent IFC intervention, the company would have had to find resources to reduce its expenses or sell some of its real estate assets. The vast majority of VP Bank’s portfolio in Vietnam focused on the retail and SME seg- ment. Without IFC’s support, VP Bank would have raised liquidity support from other sources and collected only approximately half the amount it received from IFC. This lower liquidity would have prevented VP Bank from restructuring half of its US$1 billion loans affected by the pandemic, which would have harmed its SME clients. VP Bank’s full use of the Global Trade Finance Program limits during the pandemic indicates Independent Evaluation Group World Bank Group    41 liquidity shortages in the market and limited support from international banks in Vietnam, even though Vietnam was one of the better performing countries during the pandemic (in terms of low-cost response to managing COVID-19, for example, rapidly rolling out moratoria on debt repayments, suspending social insurance payments, and increasing the firm-level interest deductibility cap). The relevant support of IFC and the Multilateral Investment Guarantee Agency en- abled ProCredit’s subsidiaries (Western Balkan countries) to onlend to SMEs, including for climate financing, and manage liquidity risks in a timely manner. The Multilateral Investment Guarantee Agency and IFC collaborated at the time of the due diligence of ProCredit to assess the environment and social aspects of the project and agreed with ProCredit on specific targets for climate financing to SMEs. Source: Independent Evaluation Group. MIGA’s early response supported firms in the substantial- to high-risk cat- egories, predominantly in Africa and in Europe and Central Asia, and was relevant. Based on MIGA corporate portfolio risk analysis,4 most political risk insurance exposure gained via the MIGA early response was at the high tier of the country risk spectrum (44 percent at entry, 47 percent at the end of the evaluation period).5 Analysis of the non-honoring guarantee portfolio indicates that the MIGA non-honoring guarantee projects were in countries with risk assessed as substantial (84 percent at entry, 72 percent at the end of the evaluation period).6 The Africa and Europe and Central Asia portfolios experienced rising country risk throughout MIGA support. The rising risk points to MIGA’s relevant role in risk mitigation and capital mobilization under guaranteeing project risk in countries where commercial financing The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 was needed. Informed Design The Bank Group used lessons from past crises well to inform its early re- sponse designs and approaches. Consistent with lessons learned from past crises, country analytical work contributed to the World Bank’s ear- ly response and aimed to address the pandemic through a three-pronged approach: budget, public health, and MSME support. In Serbia, the World Bank’s analytical work via Country Economic Memorandum and the “West- ern Balkans Regular Economic Report” informed the early response pro- grams (World Bank 2020a, 2020f). Similarly, IFC adapted its response to the local context well based on lessons from the 2008 global financial crisis. IFC launched two new platforms—the Global Health Platform and Base of the Pyramid Platform—to better respond to clients’ needs at the time of the crisis. In Georgia, prior high-quality and tailored analytical work had identi- 42 fied key vulnerabilities (World Bank 2018); this work helped the World Bank quickly put together a customized program focused on those vulnerabilities. IFC, for example, provided Liquid Telecom with a flexible and efficient sup- port package for the Sub-Saharan economy to digitalize firm activities. This project was critical at a time when many offline duties were shifted online because of COVID-19, allowing more people to perform their jobs from home and permitting Liquid Telecom to dispose of more capital to expand the company’s digital infrastructure network in fragile markets (such as Chad, Sudan, and Zambia). In South Africa and the Balkans, MIGA had expected severe pressures in some of its guarantee holder’s subsidiaries during the pandemic and facilitated capital relief solutions. MIGA’s support to trade finance activities in partnership with IFC is an innovative product based on lessons from past crises that backstops losses for client banks and supports MSMEs that have insufficient access to credit, training, and certification to become part of value chains. MIGA’s early response was less relevant when it did not fully consider coun- try conditions. Although highly complex at the time of crisis, including the COVID-19 crisis, assessing which countries are most in need of support is important to ensure that resources are targeted to vulnerable clients (box 3.3). Country office staff continue to need greater clarity on specific crisis proto- Independent Evaluation Group World Bank Group    43 cols, mechanisms, and guidelines for developing targeted approaches ad- dressing economic implications of the COVID-19 crisis. Many country teams welcomed diagnostic tools, real-time sector analysis papers, and the World Development Report 2022. However, for day-to-day matters, they seek greater clarity and preparedness for the next crisis in (i) a country-level emergency procedures playbook that outlines the flexibility of instruments, policies, and procedures and informs crisis-time diagnostics at the country level; (ii) real-time dashboards and lighter versions of the World Development Report to inform integrated sector approaches; (iii) real-time guidance on strategic portfolio choices informed by continuing analysis of global response strat- egies; and (iv) surge-capacity plans (when and how to increase fixed- and variable-cost resources in country offices) to support staff welfare. Box 3.3. Assessing Country Conditions: The Panama and Africa Cases In Panama, the Multilateral Investment Guarantee Agency (MIGA) issued a Non- Honoring of Financial Obligations of a state-owned enterprise guarantee to help Banco Nacional de Panama raise financing for a US$1 billion trust fund established by the Ministry of Economy and Finance to mitigate the economic effects of the pandemic. The trust fund was expected to provide loans to onshore commercial banks for emergency liquidity and lending to small and medium enterprises and companies in priority sectors. The MIGA project provided non-honoring guarantees to Goldman Sachs and Société Générale for providing senior unsecured loans totaling US$500 million to Banco Nacional de Panama. The Ministry of Economy and Finance contributed an additional US$500 million to the trust fund through the Rapid Financing Instrument from the The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 International Monetary Fund. As of mid-May 2021, liquidity under the trust fund has not been drawn by any bank. Although assessing country conditions is complex, especially at the time of a crisis, and although the MIGA non-honoring guarantee was endorsed by the International Monetary Fund, its low relevance could have been expected in view of the strong standing of Panama’s financial sector in both capital markets and private credit markets and the fact that Panama is a fully dollarized economy. The design of the trust fund was also such that small and medium enterprises were not the primary project beneficiary, despite the original project’s objectives. MIGA issued guarantees to the headquarters of FirstRand (in South Africa) as part of its COVID-19 response under the capital optimization program. MIGA expected that the capital relief generated from these guarantees would be deployed to seven FirstRand Group subsidiaries (across Sub-Saharan Africa) to absorb deleveraging pressures during the pandemic. However, capital relief was granted to tier 1 bank subsidiaries with relatively large market share (Botswana and Eswatini) and not to tier 2 bank subsidiaries with smaller market shares in countries in need (for example, Ghana, Mozambique, and Nigeria). Source: Independent Evaluation Group. Comparative Advantages The Bank Group’s comparative advantages during a crisis predominantly lie 44 in its ability to advance global knowledge and translate it into financial and advisory support to governments and firms, and in its global footprint. IEG corporate evaluations (2010–22) have provided extensive evidence that these are areas where the Bank Group has comparative advantages over other multilateral and regional development partners in responding to an economic crisis. The World Bank’s early response was well informed by existing diagnos- tic tools. The World Bank’s early response was broadly aligned with—and structured based on—diagnostics, with additional financing for emergency government budget response to the social crisis and measures to maintain liquidity in the financial sector. The Bank Group relied on several existing analytical instruments to tailor its emergency COVID-19 response mecha- nisms. These include the IMF’s Article IV consultations, the debt sustain- ability analysis, the Financial Sector Assessment Program, household and business enterprise surveys, Country Economic Memorandums, and the three-year Country Program Frameworks, which the World Bank augmented with targeted additional financing to support country-specific fiscal, mone- tary, and financial policy. Yet the World Bank did not conduct novel global knowledge work to deepen understanding of the unique characteristics of the evolving crisis or reflect on relevant lessons from developed countries. Despite the novelty of the pandemic and its implications globally, the World Bank did not conduct new Independent Evaluation Group World Bank Group    45 global diagnostics to inform existing portfolio management and strategy or repurpose policy and institutional strengthening work. Also, the Bank Group did not seem to identify, analyze, and apply lessons from developed countries on how to address the economic implications of COVID-19. This conclusion regarding the gap in novel global knowledge work was based on two sets of reviews: (i) IEG’s review of the dashboards of the Multilateral Leaders Task Force on COVID-19 and the World Bank COVID-19 response was conducted to assess the availability of global knowledge work at the lev- el of the World Development Report,7 and (ii) IEG’s review of global economic or meta-economic policy trackers was conducted to assess the Bank Group’s contributions to global knowledge work.8 Developed countries’ COVID-19 response programs (for example, in New Zealand and Switzerland) focused on, for example, payments to affected workers and households, including in high-contact sectors, which the Bank Group could have considered for some client countries (WEF 2020). Although some lessons from advanced countries may not be relevant for or applicable to the Bank Group’s clients, stakehold- ers we interviewed indicated that the Bank Group did not seem to have a way to systematically tap into the knowledge and experience of advanced coun- tries to inform the dialogue with clients and to discuss whether lessons from these countries were relevant and applicable (or not). The Bank Group’s early response leveraged its global footprint well. The Bank Group’s response benefited from its global footprint. Moreover, as the parallel IEG evaluation on preserving lives notes, “World Bank staff respond- ed to enormous demands to deliver extraordinary support in unprecedented circumstances” (World Bank 2022d, 92). Bank Group staff adjusted well to The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 remote and hybrid work overall, enabling them to react to client needs in an agile fashion. Bank Group staff managed the early response despite ongoing disruptions in engagement with country authorities and continued mobility restrictions that affected supervision efforts and clients’ implementation capacity. Staff in country offices demonstrated leadership at various lev- els despite facing a “triple whammy” of new crisis-related responsibilities: personal and family welfare, the welfare of existing clients, and the welfare needs of potential new clients (according to key informant interviews and case studies). Data Gaps for a Targeted Response Limited evidence is available on how the World Bank leveraged global data and its access to country budget systems to ensure that its support for the COVID-19 crisis reached the most vulnerable households. Tracking the flow of funds through the case study country budget departments to the provin- cial and community levels will require substantial analytics on community employment programs; local digital communities can help track the impact on household and enterprise finances of credits to bank cash cards and e-wallets. Furthermore, in the social sector–linked DPFs, the most visible pandemic response of the World Bank has been the conditional cash transfer programs. In response to advocacy group requests, the validation of these 46 programs—particularly to public and commercial stakeholders—needs to be articulated better via a consultative group of real economy representatives. The support to Croatia, a noncase country, may be a best practice so far (box 3.4). Box 3.4. Tracking World Bank Funds in Croatia The World Bank’s budget support to Croatia (Crisis Response and Recovery Development Policy Operation, US$300 million, 2020) and investment project support (Helping Enterprises Access Liquidity in Croatia investment project financing, US$240 million) both used the capacity of the national development bank (the Croatian Bank for Reconstruction and Development), state audit office, and state financial agency for tracking trade, financial, and export data. Disbursement information and progress across key results indicators (for example, the number of exporters receiving subloans, the number of firms served in lagging regions, and the increased use of the European Union’s parallel financing by the Croatian Bank for Reconstruction and Development) were set up to be monitored at various levels of government for both projects, seamlessly leveraging data systems from the three organizations. Source: Independent Evaluation Group. No evidence is available on how the Bank Group used its global reach to align with extrabudgetary funds (EBFs) that many client countries set up in paral- Independent Evaluation Group World Bank Group    47 lel with the crisis but conflated with the specific vulnerabilities of the pan- demic’s economic implications. More than 40 client countries set up EBFs to support firms and citizens, especially in Africa (figure 3.5). Such COVID-19 response funds from governments draw mainly on budgetary resources, but they usually target pooling private donations, public resources, and external sources of finance. Although most funds are not appropriated in the budget (that is, they are off-budget), some have used on-budget arrangements (for example, appropriations through specially created programs or subprograms of the budget). Most funds operate through separate banking, financial management, and reporting arrangements outside regular public finance management channels. No evidence was available to show the interactions between the World Bank support and the EBFs. Figure 3.5. Countries with Extrabudgetary COVID-19 Response Funds The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 3 Sources: Independent Evaluation Group; International Monetary Fund. Note: Dots indicate the countries that set up extrabudgetary funds to address the economic implica- tions of COVID-19. We could not assess the motivations and risks related to EBFs set up by cli- ents to complement multilateral support during COVID-19. There are several motivations behind creating EBFs in the current crisis; a central one has been to accelerate government spending. Other reasons include the need for a high level of government discretion, to pool public and private resources, to coordinate interventions across different sectors and levels of govern- ment, or to ring-fence COVID-19 spending. EBFs are often regarded as sub- optimal (Rahim et al. 2020). In the absence of strong safeguards, funds with independent spending authority—bypassing normal budgetary and expendi- ture controls—can dilute accountability and weaken fiscal control, creating significant fiscal risks and corruption vulnerabilities. The COVID-19 crisis heightens many of these risks. The rush to set up funds has led in some cases to a legal vacuum in which their purpose, management, and oversight are in- sufficiently defined. The pressure on governments to respond swiftly to the emergency has often led to the relaxation of ex ante financial controls and procurement processes and the weakening of oversight mechanisms. WHO conducted a global survey on EBFs in 2020, the findings of which revealed a wide array of approaches by client countries, but a central question remained 48 on the level of transparency of the EBFs. 1  The World Bank Business Pulse Surveys and World Bank Enterprise Survey data set capture 120,000 businesses’ performance across 60 countries in the real sector, including manufac- turing, agriculture, and services. The data set captures sector impacts based on sales, labor, wages, liquidity, solvency, and operational performance. 2  Many of these aspects of the economy (for example, health and social protection) fall under the subject of the parallel Independent Evaluation Group evaluation on preserving lives, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response. We used them here simply because needs in any of these aspects of the economy could trigger economic dis- tress. Additionally, in this case, we are trying to model a country’s total need to compare with gross domestic product. Finally, using this broad set of aspects of the economy allows for a within-country analysis that permits comparing the World Bank Group response to one aspect of the economy with its responses to other aspects. 3  International Finance Corporation support in the Philippines included support to the rural microfinance sector via the Working Capital Solutions envelope. 4  The portfolio analysis was conducted via a participatory approach with Multilateral Investment Guarantee Agency (MIGA) economics and risk teams. 5  For this analysis, MIGA staff provided a streamlined version of their country risk categories aggregated on a four-point scale characterized in the following descending order: high, substantial, moderate, and low. These four categories consolidate MIGA’s internal political risk insurance ratings from a 10-point scale and its Non-Honoring of Sovereign Financial Obligations and Non-Honoring of Financial Obligations ratings from a 21-point scale. Non-Honoring of Sovereign Financial Obligations coverage protects against losses resulting Independent Evaluation Group World Bank Group    49 6  from a government’s failure to make a payment when due under an unconditional and irre- vocable financial payment obligation or guarantee given in favor of a project that otherwise meets all of MIGA’s normal requirements. 7  See the “Multilateral Leaders Task Force on COVID-19 Vaccines, Therapeutics, and Diagnos- tics” (http://covid19taskforce.com) and “The World Bank Group’s Response to the COVID-19 (coronavirus) Pandemic” (https://www.worldbank.org/en/who-we-are/news/coronavi- rus-covid19). 8  See “Oxford Supertracker” (https://supertracker.spi.ox.ac.uk/). 4 | Quality of the World Bank Group Early Response Highlights The quality assessment focused on whether the World Bank Group COVID-19 economic interventions (i) influenced government policies and the actions of governments and firms; (ii) were coordinated internally, between the Bank Group and the International Monetary Fund, and between the Bank Group and other partners; and (iii) had adequate results and monitoring frameworks, safeguards, and governance. The assessment also looked at whether the quality of the Bank Group’s response was informed by intracrisis learning. The Bank Group early response positively influenced client country strategies, especially regarding macrofiscal and social safety nets. The Bank Group’s influence on governments’ actions and espe- cially on firms’ actions in the incipient recovery phase (the second part of the evaluation period) was less on target because the Bank Group’s capacity to learn from developed countries and from its own experience (intracrisis learning) was limited. Support to firms operating in contact-intensive industries, which were highly affect- ed by the crisis, was low. The World Bank produced several knowl- edge products and engaged with the International Monetary Fund in debt dialogue at both the global and country levels. However, there is limited evidence on the extent to which these initiatives influenced governments to collectively develop solutions to the massive buildup of sovereign debt caused by the crisis. The World Bank’s proactive engagement with the International Monetary Fund improved the quality of the response to the crisis, but coordination with regional development banks and other international financial institutions was mixed. The World Bank, the 50   International Finance Corporation, and the Multilateral Investment Guarantee Agency coordinated well to support the financial sector and micro, small, and medium enterprises, but Bank Group– structured finance solutions were limited. The Bank Group did not set or update specific volume targets during the early response, and thus we could not assess whether it is likely to monitor or meet its targets during the recovery phase of the pandemic. The Bank Group could have assessed the need to adjust compliance (including safeguard) requirements to facilitate supporting the real sector during the crisis. Such flexibility requires the Board of Executive Directors’ involvement and approvals. Six factors affected the quality of the Bank Group’s early response: (i) engaging with partners outside of the development community, (ii) mobilizing local partners and stakeholders, (iii) adopting clear country office staff surge plans, (iv) prioritizing staff welfare, (v) using instruments that can disburse quickly and at scale, and (vi) selecting and strengthening Project Implementation Units. 51  This chapter is organized in three sections. First, we review the quality of the Bank Group early response. Second, we identify factors affecting the quality of the response. Finally, we describe opportunities to improve the quality of the Bank Group response in the recovery phase of the COVID-19 pandemic and future pandemics. The quality of the Bank Group’s response is assessed based on three dimen- sions and through a real-time adaptive management lens. The dimensions of the assessment are (i) the extent of the Bank Group’s influence on client strategies and actions; (ii) the extent of coordination within the Bank Group (World Bank, IFC, and MIGA) and with partners; and (iii) the adequacy of the Bank Group’s monitoring, safeguarding, and governance activities. We applied a real-time adaptive management lens to the quality assessment, The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 looking at whether interventions in the incipient recovery phase (the second part of the evaluation period) reflected learning from the acute crisis phase (the first part of the evaluation period; intracrisis learning), and whether the Bank Group applied lessons learned from previous crises and from developed countries’ experience with COVID-19. Influence on Client Strategies and Actions From the inception of the COVID-19 crisis, the Bank Group quickly developed new sector knowledge and repurposed its commitments to influence client strategies and actions. The Bank Group responded quickly to help clients address the economic implications of the COVID-19 crisis by (i) developing new sector knowledge and frameworks (including by establishing a partnership on COVID-19 with WHO and the International Labour Organization); developing the COVID-19 Approach Paper in April 2020 (World Bank 2020e), which identified the broad needs of the client governments at the time of the crisis; and focusing the World Development Report 2022 on the needs of the financial sector during a pandemic shock (World Bank 2022e); (ii) conducting timely assessments of country needs related to COVID-19, including through the use of big data and data science; (iii) increasing its overall commitments from $67 billion to $121 billion, of which $65 billion was specifically dedicated to respond to the economic implications of the COVID-19 crisis (April 2020–June 2021); and (iv) 52 repurposing existing projects to help governments address the economic consequences of the crisis (World Bank) and support firms’ liquidity and avoid loan defaults (IFC and MIGA). The World Bank’s early response had a positive effect on client governments’ macrofiscal strategies and social safety net approaches, and on households and MSMEs. The Senegal case study exemplifies these findings (box 4.1). Box 4.1. Effect of the World Bank COVID-19 Early Response As part of the early response, the World Bank developed an investment project to support the government of Senegal in addressing youth employment, social safety nets, and resilient recovery. The World Bank leveraged its comparative advantages— knowledge of the Senegalese labor markets and private sector and the possibility to quickly translate knowledge into financial support—to provide local informal sector workers and small and medium enterprises with economic opportunities amid the pandemic. For instance, the project implementation agency outsourced the produc- tion and delivery of meals to public schools to approximately 200 small suppliers (mostly women’s associations) who produce food packs in frontier provinces. Through these local community partnerships, the project allowed microbusinesses whose eco- nomic resources were cut away by the pandemic to continue to earn an income. The early response supported by the project provided the implementation agency with the opportunity to better connect to frontier provinces, strengthen service delivery mech- Independent Evaluation Group World Bank Group    53 anisms, and build new relationships with local communities. Source: Independent Evaluation Group. The Bank Group’s influence on governments’ and firms’ actions in the incip- ient recovery phase was less on target, as demonstrated by limited support to contact-intensive industries and to countries and firms that entered the incipient recovery phase. We did not find evidence of innovative approach- es to target segments of the economy that were particularly affected by the crisis or to adjustments in Bank Group support as the needs of clients changed over time. For example, during the acute crisis phase in the first nine months of the pandemic (April 2020–December 2020), it became clear that the COVID-19 crisis was having a disproportionately negative impact on sectors that depended on in-person contacts (such as tourism, manufactur- ing, construction, education, and retail). Analysis of IFC and MIGA portfolios in the incipient recovery phase suggests agile disbursements to the finan- cial sector but limited support for contact-intensive industries. The average share of support going to contact-intensive industries during the incipient recovery phase was less than 5 percent; the remaining 95.4 percent went to noncontact-intensive industries. In 60 of the 81 countries receiving IFC or MIGA support across both the acute crisis phase and the incipient recov- ery phase, no support went to contact-intensive industries. Firms in these industries needed liquidity support at a greater scale and size than did firms in the financial and telecommunications sectors. Yet, across countries, a small share of Bank Group support went to contact-intensive industries (less The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 than 10 percent of total commitments) and particularly to firms operating in contact-intensive industries. Even in countries with stronger IFC and MIGA support directly to firms in the real sector, a small share (less than 5 percent of total commitments) went to these industries. Similarly, in the incipient recovery phase (the second part of the evaluation period), the Bank Group— particularly IFC and MIGA—did not switch from supporting emergency interventions to supporting the recovery of countries (such as Senegal) and private clients that were ready to move out of the acute crisis phase. The World Bank also did not increase much needed support for social assistance in the incipient recovery phase (the second part of the evaluation period). The Pakistan case study, however, provides anecdotal evidence of an effective shift between the acute crisis phase (the first part of the evaluation period) and the incipient recovery phase (the second part of the evaluation period; box 4.2). The limited evidence on changes to the Bank Group’s support pro- vided during the incipient recovery phase to better respond to clients’ needs is influenced by the fact that the incipient recovery period covered in the evaluation is short (only 6 months for the case studies and 4 months for the portfolio). Follow-up ex post evaluations will allow us to assess the quality of the Bank Group’s response during the recovery phase—including a possible shift of support from the financial to the real sector—more comprehensively. 54 Box 4.2. Adaptive Management in Pakistan In Pakistan, early response to the crisis started with emergency repurposing of the portfolio and pipeline enabled by an early consultative approach of engaging local stakeholders, engaging the federal and provincial government simultaneously (the ap- proach used in prior crises), implementing fast-track initiatives, and soliciting propos- als from all Global Practices without restructuring. Together, these efforts resulted in US$40 million being repurposed from existing projects and positioned the World Bank as one of the first responders. Another US$40 million was canceled and added to the Pandemic Response Effectiveness in Pakistan Project. Debt management and enhanced transparency were covered under pillar 1 of the Resilient Institutions for Sustainable Economy Project. For a country like Pakistan, the macrofiscal implications of debt management are significant. To free up fiscal space for the pandemic response, the World Bank and the International Monetary Fund urged the Group of Twenty to set up a Debt Service Suspension Initiative. Pakistan also requested that the Group of Twenty activate the Debt Service Suspension Initiative on May 1, 2020, which was expected to free up approximately US$1.8 billion that would be used to increase social, health, and economic recovery spending in response to the COVID-19 crisis. Resilient Institutions for Sustainable Economy provided support to the government of Pakistan to ensure that the government raises financing from all sources with the least cost and risk as it deals with the COVID-19 pandemic. The deferred debt payment translates to about 0.5 percent of Pakistan’s gross domestic product.a Independent Evaluation Group World Bank Group    55 Direct and targeted payments to low-income groups in Pakistan drove financial inclu- sion for the country, especially during the pandemic. The flagship safety net program of the government of Pakistan during COVID-19—Ehsaas—had initially envisioned fi- nancial inclusion for 7 million beneficiaries (90 percent of them women). However, with the pandemic’s socioeconomic implications, the program ended up reaching nearly 15 million beneficiaries through one-time direct payments into their newly opened bank accounts. It demonstrated the government of Pakistan’s ability to execute well and at scale. Source: Independent Evaluation Group. Note: a. https://www.worldbank.org/en/topic/debt/brief/COVID-19-debt-service-suspension-initia- tive. The Bank Group’s capacity to learn from its own experience (intracrisis learning) and from developed countries was limited. The World Bank’s usual learning frameworks—based on South-South learning and recognized glob- al best practices—were of limited relevance during the COVID-19 crisis. Low support to contact-intensive industries and to countries and firms that entered the incipient recovery phase pointed to limited intracrisis learning. Learning from developed countries was also limited. In addition to keeping enterprises from going under, several developed countries aimed to keep workers (partially) paid during the COVID-19 crisis (unlike what they did during the 2008 crisis). Although paying workers was costly, it allowed gov- ernments to keep the human capital in the economy in the right place during the “pause” forced by COVID-19 so the economy would be ready to rebound more quickly once the “play” button was pressed again. The Bank Group’s The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 client countries could have benefited from similar policies. COVID-19 led to a further increase in sovereign debt, one of the largest buildups since the creation of the Bretton Woods institutions. Budget sup- port from the World Bank and development partners will not be sufficient for countries in debt distress. Commercial debt and multilateral debt incurred by client countries surged during the evaluation period, leading to one of the largest buildups of debt in clients since World War II (World Bank 2022e; see also figure B1.1.1). Fifty-eight percent of the world’s poorest countries are in debt distress or at high risk of it.1 The World Bank and its development partners’ early response via strong budget support was necessary for client countries but will likely be insufficient for subsequent phases of the response (staff and client interviews; IMF 2021b). A further comparison of client country needs and sovereign debt increases during COVID-19 suggests that some countries are even more vulnerable on the economic front. Even before the COVID-19 crisis, there was high debt distress in a significant number of poor countries. Although this was not caused by the pandemic and instead is the result of a range of prior issues, it led to a number of the case study countries and countries not part of the case study group being in a vulnerable position, even at the start of the unprec- edented shock caused by COVID-19. Many of these countries also had high need scores, which meant that they needed significant additional money to 56 address the COVID-19 shock in its multiple facets. Figure 4.1 shows the posi- tion in 2020 for a number of these countries. Figure 4.1. Comparison of Country Needs with External Debt a. Countries with high external debt in the COVID-19 early response portfolio 0.8 0.7 0.6 Need score SLE SOM AGO ZMB 0.5 GABZWE MNG MOZ LBN ECU SDN SLV LKA COL TUN CPV DJI NIC PAN MNE 0.4 SEN MRT ARG LAO GEO 0.3 0.2 0.1 0 100 200 300 External debt stocks (% of GNI), 2020 Countries in COVID-19 portfolio with high debt distress Other countries Linear fit (full sample) b. Case study countries with high debt distress and growing debt 0.8 0.7 0.6 Need score 0.5 NGA PHL ECU CPV GEO 0.4 PAK SEN LAO Independent Evaluation Group World Bank Group    57 0.3 SRB 0.2 0.1 0 100 200 300 External debt stocks (% of GNI), 2020 Case study countries with high debt distress Other countries Linear fit (full sample) c. Low-income countries not in the COVID-19 early response portfolio with high debt distress 0.8 0.7 0.6 Need score 0.5 0.4 GNB 0.3 0.2 0.1 0 100 200 300 External debt stocks (% of GNI), 2020 External debt stocks (% of GNI), 2020 Case study countries with high debt distress Other countries Linear fit (full sample) c. Low-income countries not in the COVID-19 early response portfolio with high debt distress 0.8 0.7 0.6 Need score 0.5 0.4 GNB 0.3 0.2 0.1 0 100 200 300 External debt stocks (% of GNI), 2020 LIC not in COVID-19 portfolio with high debt distress Other countries Linear fit (full sample) The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 d. Heavily indebted poor countries as defined by the World Bank and the IMF 0.8 0.7 0.6 Need score SLE SOM ZMB 0.5 MOZ SDN 0.4 SEN MRT NIC 0.3 0.2 0.1 0 100 200 300 External debt stocks (% of GNI), 2020 Highly indebted countries Other countries Linear fit (full sample) Source: Independent Evaluation Group. Note: Dashed lines indicate median values (full sample). AGO = Angola; CPV = Cabo Verde; ECU = Ecua- dor; GAB = Gabon; GEO = Georgia; GNB = Guinea-Bissau; GNI = gross national income; IMF = International Monetary Fund; LAO = Lao People’s Democratic Republic; LBN = Lebanon; LIC = low-income country; MNE = Montenegro; MNG = Mongolia; MOZ = Mozambique; MRT = Mauritania; NIC = Nicaragua; PAK = Pakistan; PAN = Panama; PHL = Philippines; SDN = Sudan; SEN = Senegal; SLE = Sierra Leone; SOM = Somalia; SRB = Serbia; TUN = Tunisia; ZMB = Zambia; ZWE = Zimbabwe. 58 Multilateral debt in the form of COVID-19 response was necessary but adds to medium-term risks for clients. Because of the urgent need to help client countries address the COVID-19 shock and the fact that this was only pos- sible at the speed required by using existing financing modalities (that is, additional debt), the Bank Group and other financiers not only had no choice but also had a compelling justification to provide these debt-distressed countries with additional lending. Quite clearly, however, this necessary short-term measure had medium-term risks: even higher debt burdens at a time when these countries need to start recovering from the COVID-19 crisis and building resilience against future shocks. Although a detailed analysis of the debt situation of individual countries goes beyond the scope of this eval- uation, the starting position indicates that a significant number of countries might be in too much distress to recover successfully. These include Angola, Eritrea, Guinea-Bissau, Lebanon, Mongolia, Mozambique, Nicaragua, Sierra Leone, Sudan, the Syrian Arab Republic, and Zambia. The Bank Group has also not yet influenced governments’ strategies and actions to fully reflect the unintended consequences of the massive build- up of sovereign debt and budgetary reallocations by clients in their early response. Emerging markets and developing economies’ stakeholders (9 country case studies and 10 firm case studies) expressed concerns about sovereign debt and borrowing costs. Although existing fair debt resolution mechanisms such as the Common Framework for Debt Treatment and the Independent Evaluation Group World Bank Group    59 Debt Service Suspension Initiative helped during the evaluation period, stakeholders perceive them as insufficient to mitigate the looming debt crisis and the potential economic implications in some parts of the world (for example, in Sri Lanka during June 2022). At the time of this writing, several highly indebted countries (pre-COVID-19 debt buildup) faced food and energy crises, inflationary pressures, and exposure to foreign currency debt from non–Paris Club lenders. These issues create differences in the debt profile between the various creditors. Clients expect nuanced and globally coordinated approaches from the World Bank and its development partners to address the likely new wave of crisis looming in highly indebted countries. The World Bank produced several knowledge products and en- gaged with the IMF in debt dialogue at both the global and country levels; however, there is limited evidence on the extent to which these initiatives have influenced governments to collectively develop solutions to the mas- sive buildup of sovereign debt caused by the crisis. Past crises and debt resolution frameworks offer lessons for the Bank Group’s influence on and support to highly indebted countries. Lessons from oper- ating debt resolution facilities such as the Heavily Indebted Poor Countries Initiative in 1996 and 1999 were yet to be linked with the global solidarity packages or embraced fully by the development community during the eval- uation period (box 4.3). Box 4.3. The Heavily Indebted Poor Countries Initiative The World Bank and the International Monetary Fund launched the Heavily Indebted The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 Poor Countries (HIPC) Initiative in 1996. The subsequent enhanced HIPC Initiative in 1999 aimed to accelerate debt relief, and the Multilateral Debt Relief Initiative brought on board the African Development Bank in 2005 and the Inter-American Development Bank in 2007. The objective was to ensure that low-income countries could continue poverty reduction–related expenditures. The enhanced HIPC Initiative required countries to spend fiscal savings from debt relief on poverty-reducing programs, such as in health and education. The HIPC Initiative was highly standardized. Creditors granted debt relief to debtor countries on common principles that helped address the information asymmetries and coordination problems among multiple creditors that can hinder restructuring agreements. The HIPC Initiative offered relief on debt held by the International Monetary Fund and World Bank. Participation of multilateral institutions is important because evidence suggests that creditors and debtors should aim for comprehensive debt relief when debt is unsustainable. Debt relief has been substantial. These initiatives provided 38 countries with debt relief totaling more than US$100 billion. External debt in low-income countries fell dramat- ically from its peak in 1994 (figure B4.3.1). Before the HIPC Initiative, eligible countries spent on average more on debt service than on health and education combined. Their 60 (continued) Box 4.3. The Heavily Indebted Poor Countries Initiative (cont.) health, education, and other social services expenditures are now approximately five times their debt service payments. Figure B4.3.1. External Debt in Low-Income Countries 120 Heavily Indebted Poor Countries 100 initiative begins 80 Share of GDP (%) 60 40 20 0 1970 1976 1982 1988 1994 2000 2006 2012 2018 Year Source: World Bank 2022b. Note: GDP = gross domestic product. Sources: IMF 2021a; World Bank 2022b. Independent Evaluation Group World Bank Group    61 Similarly, recent IEG evaluations offered important lessons for the Bank Group’s influence on client countries’ public finance management and ex- ternal debt assessments during the pandemic. Two parallel IEG evaluations concluded during the evaluation period offered lessons on the Bank Group’s strategies for influencing clients, with a focus on public finance and external debt (World Bank 2021a, 2021b; box 4.4). For example, the challenge of tack- ling non–Paris Club members’ and private creditors’ debt needs to be collec- tively addressed by the multilaterals, client governments, and the creditors.  ecent Independent Evaluation Group Evaluations on Public Box 4.4. R Finance and External Debt The Independent Evaluation Group has conducted two recent evaluations on pub- lic finance and external debt. The first is The International Development Association’s Sustainable Development Finance Policy: An Early-Stage Evaluation (World Bank 2021a), which aimed to provide early insights into the rollout of the policy and how it may be improved to minimize the risk of debt distress. As the World Bank considers its actions in the post-COVID-19 environment characterized by a further significant deterioration of public debt distress in member countries and much reduced fiscal space, three principles recommended by this evaluation are worth reiterating: (i) expand perfor- mance and policy actions, (ii) set them explicitly within a longer-term reform agenda, The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 and (iii) use them to target the main country-specific drivers of debt stress and risk. The second Independent Evaluation Group evaluation, World Bank Support for Public Financial and Debt Management in IDA [International Development Association]-Eligible Countries (World Bank 2021b), identified important gaps in complementarities be- tween World Bank support for public finance management and for debt management. Specifically, the World Bank has provided significant and well-coordinated support to improve public debt management in many countries eligible for IDA assistance and facing rising debt vulnerabilities. However, this support has not been systemat- ically accompanied by efforts to improve public financial management (and public investment management in particular), despite widely recognized synergies among borrowing, fiscal transparency, and the quality of public investment. The principles suggested in this evaluation are (i) regularly monitoring the quality of the key pillars of public financial and debt management for each IDA-eligible country, and (ii) prioritiz- ing and sequencing World Bank support for public financial and debt management capacity building and reform in IDA-eligible countries. Such a framework could inform the design of budget support operations, investment projects, and country-specific performance and policy actions under the newly adopted sustainable development finance policy. Source: Independent Evaluation Group. 62 Coordination Collaboration between the World Bank and the IMF to respond to the pandem- ic was effective both at the corporate level and at the intervention level. Both institutions increased their commitments significantly after COVID-19 hit: the World Bank by about 60 percent (from $43 billion to $71 billion; calendar years 2019–20) and the IMF by three times during the first year of the pandemic. At the corporate level, both institutions organized high-level knowledge events and mobilized partners to support the COVID-19 response (for example, the Africa high-level panel in April 2020). Although the two institutions took slightly different approaches on the level of conditionality to extend support (the World Bank is more stringent than the IMF),2 key informants expressed positive views on the coordination mechanisms at the corporate level. Inter- views with staff from both the World Bank and the IMF suggest that an ex ante joint crisis response plan would further improve the effectiveness of the collaboration between the two institutions in reacting to future crises, given that the World Bank cannot solely manage sovereign debt issues. The World Bank coordinated well with the IMF and development finance institutions on macrofiscal issues and debt service suspension, including through joint policy dialogue with governments. At the start of the pandem- ic, the World Bank and the IMF urged the Group of Twenty to set up the Debt Service Suspension Initiative. Along with the IMF, the World Bank actively Independent Evaluation Group World Bank Group    63 supported countries seeking debt restructuring under the Common Frame- work, providing a critical input to the process through the joint debt sustain- ability analysis. In the cases analyzed, the Bank Group demonstrated good coordination with the IMF and more frequent World Bank–IMF joint mac- rofiscal policy dialogue with client governments, compared with prepandem- ic times. In some cases (for example, Pakistan and Serbia), the World Bank and the IMF established working groups to discuss and agree on reforms to help client governments weather the crisis. World Bank–IMF collaboration in Ecuador and Pakistan exemplifies this finding (box 4.5).  orld Bank–International Monetary Fund Collaboration in Box 4.5. W Ecuador and Pakistan In Ecuador, the International Monetary Fund (IMF) and the World Bank coordinated on the design of the Extended Fund Facility program and the development policy financing series to ensure consistency and complementarity of the policy measures being supported.a The World Bank’s early response (April 2020) on the budget reform and on Ecuador’s social safety net and social assistance needs was perceived by the government of Ecuador as high quality. The World Bank advised and led the design of the conditionality in the IMF program in these areas. The World Bank’s work revealed incorrect reporting of the budget figures by the country authorities, which led to a delay in the initial extended fund facility support by the IMF and the reassessment of the IMF program with an interim emergency The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 bridging loan and the preparation of a new, larger extended fund facility operation. The close coordination between the two institutions continues in jointly monitoring the remain- ing and large hidden fiscal contingencies in three areas: state-owned banks, state-owned enterprises, and the pension system. More needs to be done to quantify and address them. In Pakistan, the World Bank early response via budget and liquidity support showed strong coordination between the World Bank and the IMF, ranging from frequent policy dialogue on macrofiscal constraints to project sequencing to address the various dimensions of the crisis (such as balance of payments, institutions to respond to the crisis, and social safety nets). The IMF initiated the first response in Pakistan via the US$1.4 billion Rapid Financing Instrument and support in April 2020 to meet Pakistan’s balance of payment needs stem- ming from the COVID-19 outbreak a month earlier. The World Bank coordinated with the IMF on its early response, the US$500 million Resilient Institutions for Sustainable Economy Project in June 2020. This project was followed in May 2021 by a US$500 million World Bank development policy financing series for Pakistan, Securing Human Investments to Foster Transformation, which supplemented the existing social safety net program in Paki- stan (Ehsaas). Besides the IMF, the World Bank Group also interacted with other internation- al financial institutions to develop joint projects, maintaining periodic meetings to coordinate their support. The World Bank also helped start a development partners working group, which has met regularly since the start of the pandemic to coordinate the response on the procurement, administration, and logistics of vaccinations. Source: Independent Evaluation Group. Note: a. The IMF established the extended fund facility to provide assistance to countries expe- riencing serious payment imbalances because of structural impediments or slow growth and an 64 inherently weak balance-of-payments position. An extended fund facility supports comprehensive programs, including the policies needed to correct structural imbalances over an extended period. The early response to support the financial sector and MSMEs was well coor- dinated within the Bank Group, especially between IFC and MIGA. The World Bank’s Partial Credit Guarantee facilities complemented IFC’s and MIGA’s focus on maintaining liquidity in commercial microfinance institutions and small and medium enterprise banks in Africa and in Latin America and the Caribbean. Colombia and Europe and Central Asia exemplify this finding (box 4.6).  orld Bank Group Coordination in Colombia and Europe and Box 4.6. W Central Asia In Colombia, the initial International Finance Corporation (IFC) firm-level support in Latin America and the Caribbean (for example, a US$600 million senior loan to Davivienda at the onset of the pandemic) had a catalytic effect during the second year of the pan- demic by crowding in new investor financing. The initial support package to Davivienda consisted of a US$100 million senior loan for onlending to women-owned small and medium enterprises, and a US$500 million Basel III–Tier 2 loan. The Basel III–Tier 2 loan catalyzed new investors (such as the Organization of the Petroleum Exporting Countries Fund for International Development [OFID]) to the region; increased the client firm’s market share in the micro, small, and medium enterprise space from 14 percent in 2019 to 20 percent in 2022; and increased support to low-income housing and green build- ings. A year after IFC’s financing package invested in Davivienda, the Inter-American Independent Evaluation Group World Bank Group    65 Development Bank, the International Development Finance Corporation, and FinDev Canada jointly provided subordinated loans for a long tenor of 10 years (US$390 mil- lion). In parallel, the Multilateral Investment Guarantee Agency (MIGA) and IFC coordi- nated well to support this systemically important local bank. MIGA guarantee support sat adjacent to the first US$600 million IFC financing package. Furthermore, MIGA supported a loan package from OFID through a guarantee, which allowed OFID to par- ticipate in the IFC-led financing as a parallel lender. The MIGA guarantee protected the lender (OFID) against the risks of currency inconvertibility and transfer restrictions. In Europe and Central Asia, MIGA and IFC coordinated activities to support a sys- temically important commercial bank client, ProCredit Holding. Under its COVID-19 response program pillar (Countering Adverse Economic Impacts during the COVID-19 Crisis), MIGA issued capital optimization guarantees of €218.5 million to ProCredit (continued)  orld Bank Group Coordination in Colombia and Europe and Box 4.6. W Central Asia (cont.) Holding to cover €230 million of its equity investments in eight ProCredit subsidiar- ies (Albania, Bosnia and Herzegovina, Georgia, Kosovo, Moldova, North Macedonia, Serbia, and Ukraine) in central, eastern, and southeastern Europe against the risk of expropriation of funds with respect to the mandatory cash reserves of the subsidiar- ies held at their respective central banks for a period of five years. MIGA’s support to ProCredit was mainly to guarantee client lending for climate financing and increased efficiency of onlending to micro, small, and medium enterprises. MIGA and IFC col- laborated mostly on environmental and social aspects (such as data sharing and joint supervision plans) and during the initial stages of due diligence of ProCredit Holding The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 (for example, reviewing client firms’ existing environmental and social systems at the holding company level). IFC has information on ProCredit at the holding company lev- el; MIGA has this information at the subsidiary level. Without continued MIGA support, ProCredit Group would have faced added regulatory costs at a time of unprecedented pressure because of the health and economic crisis,a affecting its ability to maintain its loan book in the medium to long term. Source: Independent Evaluation Group. Note: a. The ProCredit Group subsidiaries are required by their local regulatory authorities to maintain a minimum amount of mandatory cash reserves with their respective central banks. Under current regulations, a 100 percent risk-weighted assets rule is applied to these mandatory reserves, result- ing in capital consumption at the consolidated group level. The Multilateral Investment Guarantee Agency’s guarantees would help reduce the risk weighting of the mandatory reserves from 100 to 0 percent, providing risk-weighted assets consumption relief of up to €302.3 million. This creates ad- ditional capital headroom at the consolidated group level, which ProCredit Holding would redeploy to the eight subsidiaries to expand their lending, primarily to small and medium enterprises and in support of climate finance activities. Bank Group coordination with donors, regional development partners, and other international financial institutions worked well in several cases. Development partner coordination and a clear division of labor toward the COVID-19 early response were strong in Ecuador, Georgia, and Serbia (case study countries). When formal donor coordination frameworks were absent, the World Bank faced more difficulties in field coordination needed to maximize the synergies of donor funding and nonfunding efforts. Such difficulties were registered in cases of Cabo Verde, Nigeria, Pakistan, the Philippines, and Senegal. To be sure, in some of these countries, the World Bank made 66 progress in this area even in the absence of these frameworks, which bodes well for future potential synergies and results. For example, the World Bank, using its convening power, established a development partners’ working group to coordinate response efforts in Pakistan. The World Bank, the Asian Development Bank, and the Asian Infrastructure Investment Bank joined the United Kingdom’s Department for International Development, the United States Agency for International Development, and the Japan International Cooperation Agency to coordinate their responses with the Resilient Institutions for Sustainable Economy Project in support of Pakistan’s efforts to strengthen its macrofiscal framework. In Senegal, the World Bank initiated a joint policy reform matrix with defined roles and efforts for joint donor action to avoid duplication in response to the crisis. Such an arrangement was also observed in Cabo Verde and Ecuador. In Ecuador, the government was in the middle of a shift in its overall development approach (moving from public sector–led growth to private sector–led growth) when the COVID-19 crisis hit. The World Bank and other international financial institutions coordinated well to respond with a large aid package to Ecuador: the World Bank committed $1.4 billion through three DPOs (intended to be disbursed programmatically over three years), the IMF provided $644 million in rapid financing to bridge its extended fund facility support, the Inter-American Development Bank provided a $640 million package, the Development Bank of Latin America lent $500 million, and the Latin American Reserve Fund lent $418 million. IFC coordinated with 14 donor governments’ development finance institutions Independent Evaluation Group World Bank Group    67 to crowd in blended finance support toward small and medium enterprise financing to augment its fast-track COVID-19 facility. World Bank GPs responsible for the economic response to COVID-19 shared knowledge and collaborated well; they also contributed to the health and social response. IEG’s knowledge flow and collaboration evaluation, conducted before the 2019 realignment of operational staff, documented how challenging it was for the World Bank to work across GPs on many multisector issues (World Bank 2019). Collaboration on lending and global knowledge tasks for the COVID-19 economic response was easier. Interviewees reported flexibility, ease of accessing advice and cross-support, collaborative attitudes, good information sharing, speedy clearances, and close relationships between headquarters and country offices. Operations Policy and Country Services guided the lending response, streamlined some review and approval processes (such as fast-tracking DPF disbursements), and strove to offer updated and accessible guidance and templates. Many of the World Bank’s knowledge networks also mobilized to create and share data and knowledge on issues that cut across the economic response and the health and social response. For example, both the Macroeconomics, Trade, and Investment and the Social Protection and Jobs GPs created central groups, supplemented by regional focal points, to support teams preparing operations. The World Bank (2022d) finds that the Finance, Competitiveness, and Innovation GP, which was one of the GPs at the core of the economic response to the COVID-19 crisis, was a key contributor to early health and social response projects. It also finds that DPFs led by the Macroeconomics, Trade, and Investment GP were critical for the health and social response. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 The Bank Group institutions did not sufficiently collaborate to support clients via structured finance in the incipient recovery phase (the second part of the evaluation period). Based on key informant feedback and data on the loosening of restrictions and resumption of GDP growth, several clients were prepared for COVID-19 recovery in 2021. They were keen on exploring structured finance products such as Partial Credit Guarantee facilities, sub- ordinated debt for MSMEs, and innovative products needed to recapitalize firms with lower growth potential in the near term, which could be provided by coordinated World Bank–IFC interventions supported by the World Bank Partial Credit Guarantee team and IFC teams providing credit lines and trade finance. The Bank Group’s COVID-19 response package in Georgia serves as a best practice case (box 4.7). The development community does not have a playbook for future crises. Authorities in client countries and the World Bank’s country teams indicated the importance of establishing a dedicated central team in the Bank Group that would analyze the international response to the crisis, including in developed countries, and share the learning from this analysis with country teams (IEG interviews). Operations Policy and Country Services management indicated that this support and analysis was done through sector boards, but country teams did not corroborate this impression. As the IEG evaluation (World Bank 2022d) notes, the Organisation for Economic Co-operation and 68 Development emphasizes that pandemic preparedness requires detailed and up-to-date operational plans and processes describing the different roles of staff, procedures, and uses of instruments in responding to crises. It is also important to note that the need is not just to make sectoral advice available but also for the Bank Group to provide an overall strategic response to crises that can be reflected in country strategies and the portfolio.  orld Bank Group’s Approach to COVID-19 Response in Box 4.7. W Georgia The World Bank Group provided a high-quality, coordinated COVID-19 response to Georgia. The total financing provided by the World Bank was approximately 2 percent of Georgia’s gross domestic product, which was a substantial part of the 5–7 percent fiscal financing gap that opened up because of the COVID-19 crisis. This helped the Georgian government mount a significant fiscal stimulus of approximately 6.5 percent of gross domestic product to finance social assistance and unemployment programs and support for firms. The World Bank’s Practice Groups responded well to client needs, coordinated by the Country Management Unit, which has maintained excellent relationships with the government of Georgia. Given the International Finance Corporation’s strong presence in Georgia and its experiences in conducting due diligence of financial intermediaries, its advisory inputs to the World Bank package— especially on environmental, social, and governance and corporate governance—were crucial for the Partial Credit Guarantee plan subcomponent of the World Bank’s micro, Independent Evaluation Group World Bank Group    69 small, and medium enterprise project. Engagement among the World Bank, the International Monetary Fund (IMF), and other development partners was well coordinated, resulting in a timely multidonor response to Georgia. The World Bank used the macro framework that the IMF had developed in April 2020 in its development policy financing supplemental loan approved in late 2020. A World Bank US$50 million development policy operation and the subsequent equal supplemental financing was closely coordinated with a parallel US$50 million support from the Asian Infrastructure Investment Bank and another US$180 million from KfW. Evidence from document reviews and interviews suggests that without the World Bank development policy financing support, the Asian Infrastructure Investment Bank and KfW likely would have provided less support that would have arrived later. (continued)  orld Bank Group’s Approach to COVID-19 Response in Box 4.7. W Georgia (cont.) The response package was underpinned by an existing trusted relationship between the World Bank and the government of Georgia, the client’s track record of sound macroeconomic management, and the World Bank’s attention to results frameworks. This relationship has supported intensive, real-time knowledge transfer and ongoing policy dialogue, especially during the acute crisis phase (the first part of the evaluation period), when uncertainty was extreme. In that environment, the World Bank proved to be a reliable partner of the government. The real-time, high-quality knowledge transfer was possible because it was based on previous analytical work of the World Bank that identified relevant vulnerabilities (World Bank 2017b, 2018, 2020b), for example, in public The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 health; competitiveness; skills; and micro, small, and medium enterprises, which came into full force during the pandemic. Development policy financing and investment proj- ect financing projects, which supported economic recovery (including for micro, small, and medium enterprise), incorporated sound results frameworks and result chains. Early indications of effectiveness of the response are positive. The World Bank’s mac- rofiscal support during the time of extreme budgetary need in the acute crisis phase helped the government finance critical budgetary services, including social assistance and unemployment benefits in 2020. The reforms have continued, as documented by the most recent IMF report, which notes the rapid, V-shaped recovery during 2021 and the government’s commitment to the structural reform program supported by the IMF and the World Bank (IMF 2021b). Source: Independent Evaluation Group. Monitoring, Safeguards, and Governance From a reporting and governance perspective (narrowly defined), Bank Group early response–related engagements with the Board were comprehensive. Bank Group management has held frequent and substantive engagements with the Board on the early response since the onset of the global pandemic. Interactions with the Board have covered everything from weekly fact up- dates to requests for approval of emergency response operations (see full list 70 in appendix D). Because the Bank Group did not set or update specific volume targets during the early response, we could not assess whether it is likely to monitor or meet its targets during the recovery phase of the pandemic. None of the Bank Group institutions defined a corporate results framework to measure the results of Bank Group support addressing the economic implications of COVID-19. The results frameworks of COVID-19 interventions analyzed in the case studies were not different from the results frameworks of non- COVID-19 operations. Similarly, Bank Group early response interventions did not identify and measure outcomes specifically related to the economic implications of COVID-19. No significant differences were observed between the outcomes of COVID-19 response interventions and non-COVID-19 interventions during the evaluation period. The Bank Group streamlined its environmental and social clearances and created templates to facilitate their application in crisis operations. The World Bank supported teams and clients with implementing the Environ- mental and Social Framework (ESF) approved on October 2018 in the context of COVID-19 operations, including by developing templates and examples to facilitate their application at the onset of the crisis. Environmental and social GPs at the World Bank did indeed create a streamlined, centralized clearance system to expedite clearance for the initial slate of COVID-19 response projects under the purview of the respective directors and specially Independent Evaluation Group World Bank Group    71 designated practice managers. This model was then followed by expedited review decentralized to each Region at the level of environmental and social practice managers and regional environmental and social standards advis- ers. This initial system led to more agile quality review for the first round of COVID-19 projects. IFC introduced a suite of process adjustments for envi- ronmental and social appraisals, including streamlined documentation and virtual appraisals for lower-risk projects and reliance on local consulting expertise, when available. The Bank Group could assess the feasibility of further adapting environmental and social safeguards to respond to clients’ needs during the COVID-19 crisis, especially in contact-intensive sectors. Stakeholders interviewed for the evaluation still perceived environmental and social safeguards as cumbersome in the context of operations addressing the economic implications of the COVID-19 crisis. They mentioned that ESF policies and procedures tend to be heavy on up-front paperwork and light on substantive risk controls during implementation. Waivers sometimes allowed for speedy project approval, but delays ensued during the implementation phase. Some staff perceived Operations Policy and Country Services as overly restrictive in its interpretation of policies, procedures, and Board guidance. Some clients lacked familiarity with ESF requirements and were overwhelmed by multiple ESF outputs required by the World Bank: Environmental and Social Review Summary, Environmental and Social Commitment Plan, Stakeholder Engagement Plan, and detailed documents for high-risk topics, such as medical waste, social inclusion, and nondiscrimination. Recurring changes to project document templates and delays on decisions regarding waivers complicated teams’ and clients’ The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 project preparation. Overlaps in responsibilities between the chief standards adviser and the central and regional environmental and social teams (now addressed via organizational change in 2022) complicated efforts to resolve the identified ESF implementation issues. There may also have been insufficient staff capacity in the country offices to effectively apply the safeguards. The Lao People’s Democratic Republic is an example of a client country where disproportionate safeguards for a time of crisis slowed disbursement (box 4.8). The evaluation team also did not find sufficient evidence that greater flexibility to adjust to the crisis in 2020 was substantively discussed in the EOC and with the Board. Box 4.8. Slow Disbursement in the Lao People’s Democratic Republic In the Lao People’s Democratic Republic, the World Bank correctly identified the needs of micro, small, and medium enterprises to access finance during the pandem- ic. It combined lines of credit, capacity-building support to the Department of Small and Medium Enterprise Promotion via technical assistance, and targeted emergency support to the hospitality and transport sector in a single intervention (October 2020; US$40 million commitment). One challenging but relevant component was setting up a Partial Credit Guarantee Facility to backstop bank lending to micro, small, and medium enterprises. This project involved coordination with five commercial banks: (continued) 72  low Disbursement in the Lao People’s Democratic Republic Box 4.8. S (cont.) Lao China Bank, Lao Viet, Maruhan Japan Bank, Sacombank, and VietinBank. Howev- er, at the time of evaluation (October 2021), only 1.5 percent of the commitment had been disbursed, and the government’s project steering committee had not identified the international consultants on technical assistance support. The slow disbursement was rooted in the need to ensure environmental management safeguard systems that were mandated consistently by all five commercial banks in coordination with the Department of Small and Medium Enterprise Promotion. The client expected financial institution projects to have differentiated treatment of safeguards from real sector or infrastructure projects, especially during a crisis. Source: Independent Evaluation Group. The Bank Group maintained yet simplified corporate and fiduciary require- ments at the project level at the time of the COVID-19 crisis. Management shortened clearance deadlines, delegated some approvals, and briefly paused gender tagging but maintained most standard corporate and fiduciary pri- orities and requirements. The corporate priorities of gender, citizen engage- ment, grievance redress mechanisms, climate co-benefits, and their related processes remained in effect. The fiduciary, procurement, and safeguards risk controls also remained in effect. Management created new written and Independent Evaluation Group World Bank Group    73 unwritten subprocesses that tightened controls on DPF, Program-for-Re- sults, vaccine operations, and approvals of level 2 project restructurings. The World Bank had used additional finance at the time of other crises to pro- vide flexible and fast-disbursing support but was limited in its ability to do so for COVID-19 operations because projects approved under the previous safeguards policies were no longer eligible for additional finance with the transition to the new ESF (October 2018). The corporate and fiduciary prior- ities and requirements exist for valid reasons, including policy commitments the World Bank has made to its shareholders, and some Board members were hesitant to relax compliance and risk controls. However, stakeholders report- ed that maintaining the procedures and priorities while mounting the emer- gency response resulted in a delayed, suboptimal response. Factors Affecting the Quality of the World Bank Group’s Response Six factors affected the quality of the Bank Group’ early response. They were (i) engaging with global partners outside of the development community, (ii) mobilizing and engaging with local partners and stakeholders, (iii) surge-resourcing plans, (iv) prioritizing staff welfare, (v) underuse of certain Bank Group financing instruments, and (vi) influence on the project implementation agencies. Engagements with partners outside of the development community helped improve the quality of the response. Given the widespread, compounded risks created by the COVID-19 pandemic, addressing economic implica- The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 tions required embracing and advocating for engagements with new part- ners outside of the development community. Several clients sought crisis diagnostics and support from the International Labour Organization, the United Nations Conference on Trade and Development, and the Organisa- tion for Economic Co-operation and Development, in addition to official development assistance at the global and country levels. The Bank Group crowded in new global partners to support its policy dialogue with clients (for example, WHO in Senegal). Mobilizing and engaging local partners and stakeholders helped improve the quality of the early response in some cases. Many country offices could not gather local intelligence because Bank Group staff worked remotely or in hybrid fashion during the evaluation period. Much of the intervention design was informed by headquarters diagnostics, which were relevant but not sufficiently enriched with local data. Cabo Verde was a good reference in this respect because local staff engaged local partners to adjust the project designs in real time. The Bank Group used media reports to react to budget support requirements (for example, in Kenya, a noncase coun- try) and repurpose existing projects (toward safety nets) in some cases (for example, in Nigeria). Adopting a clear country office surge-resourcing plan during the crisis improved the quality of the response. Beyond intervention designs 74 and surge-financing plans, country offices needed corporate-level commitments to increase and reallocate human resources to respond to the crisis and manage client expectations. Smaller country office teams (such as in Cabo Verde) reacted well to the crisis, repurposing their staff and consultants fairly quickly. Staff and consultants of large country office teams were dedicated to highly complex existing programs, and it was not easy to unwind and repurpose them (as in Nigeria, for example). Prioritizing staff welfare during crises to build trust and respect improved the quality of the response. Staff in country offices operated in a highly stressful environment, as revealed through key informant interviews and case study–related interviews of country staff. Many informants had family members who had passed away or been hospitalized. Sometimes, staff man- aged Bank Group work commitments while managing their own COVID-19 symptoms. The Country Management Units of Ecuador and Senegal exhibit- ed best practices in balancing clients’ and staff needs through well-thought- out crisis response plans and a positive work environment. Although the crisis unfolded, and the Country Management Units in countries with rela- tively weak capacity to handle a pandemic faced unprecedented challenges, prioritizing staff welfare while balancing the expectations of clients and partners was a successful approach. Interviews suggest that such actions increased the level of trust between senior management and staff in country offices and between country office staff and headquarters. The World Bank’s choice of financing modalities and the use of fast- Independent Evaluation Group World Bank Group    75 disbursing, scalable instruments at the onset of the crisis influenced the quality of its response. Key attributes for crisis response instruments are timely design, timely approval processing, timely disbursement, and scale. Instruments with timely processing include CAT DDOs and CERCs. CERCs generated mixed responses from clients and staff who attempted to activate them at times of need. Interviews with key informants indicated that in Serbia, for example, it was difficult to activate CERCs. An existing CAT DDO was disbursed effectively, and (based on early discussions with the implementing agency of the earlier CAT DDO) a follow-up DDO might have been more appropriate than the CERC (which was originally prepared as part of an IPF but could not be activated or repurposed during the acute crisis phase or the incipient recovery phase). DPF was well suited to provide substantial financing, but IPF was slow to disburse, especially when clients with weak capacity ran into procurement and other delays. Repurposing and additional financing of existing projects (investment projects, DPOs, and Program-for-Results) proved to address the clients’ crisis needs quickly and well. As the IEG evaluation (World Bank 2022d) notes, the MPA also proved to be an effective and timely instrument. MPAs allowed the implications of the crisis that a country faced to be addressed in a coordinated, strategic way. MPA document templates were also standardized yet customizable, facilitating project design, including by providing distilled technical knowledge on pandemic response that simplified teams’ work with clients. Giving teams more flexibility to choose financing and implementation modalities (for example, using MPA, CERCs, or DDOs) and target clients (that is, regional or other subnational actors) would have further improved the quality of this instrument to address the economic implications of the The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 4 COVID-19 crisis. Government implementation partners that can effectively implement World Bank projects are a condition for success. The capacity of government imple- mentation partners was uneven among the case study countries. One of the implementation partners of the DPO to the Philippine government provides an example. As part of the DPO, the World Bank planned to provide tech- nical assistance to the Ministry of Finance and to the Philippine Guarantee Corporation (PGC), a subsidiary of the Ministry of Finance. The PGC was responsible for the implementation of the project’s pillar aimed at expand- ing credit guarantees to MSMEs to support business continuity (pillar 2.6). Yet the local commercial banks and the microfinance institutions did not use the PGC support and viewed the PGC as overly bureaucratic and risk averse. Local financial institutions questioned the capacity of the PGC to manage the volume of risk sharing, specifically of commercial banks’ small and me- dium enterprise portfolios and the larger microfinance institution portfolio. The PGC did not have sufficient staff resources or risk control procedures to respond effectively to significant market demand during the pandemic. 76 1  Data from the International Monetary Fund Global Debt Database, International Monetary Fund, Washington, DC, https://www.imf.org/external/datamapper/datasets/GDD. 2  Several countries received International Monetary Fund support in 2020 without condition- ality on structural adjustments, but the World Bank held back disbursements during 2020 on government policy concerns (for example, in Nigeria). Independent Evaluation Group World Bank Group    77 5 | Conclusions and Recommendations The Bank Group developed an agile, strategic corporate response to address the economic implications of the COVID-19 crisis, which led to the institution’s largest-ever commitment to address a crisis. The crisis had a direct economic impact of approximately $7.4 trillion in 2020 alone. At $157 billion between April 2020 and June 2021 ($65 billion alone for the eco- nomic response), the Bank Group response to the COVID-19 crisis was the largest among the development partners and the largest in its crisis response history. The Bank Group’s response was made possible by senior manage- ment’s swift action at the inception of the crisis, the establishment of ad hoc decision-making committees to facilitate an agile response, the expansion of existing instruments (such as MPAs), and the deployment of new approach- es, including a new IFC fast-track COVID-19 facility and new MIGA guaran- tee programs. The Bank Group’s response focused on the macrofiscal needs of client governments, the liquidity needs of banks and MSMEs, and support to trade finance, helping many client countries progress from the acute crisis phase to the incipient recovery phase (the second part of the evaluation pe- riod). One important concern, however, is that the pandemic has contributed to a rapid buildup of sovereign and corporate debt that has left highly in- debted countries more economically vulnerable in the medium term. Recent IEG evaluations on public finance management and external debt assess- ments (World Bank 2021a, 2021b) offer important lessons in this regard. The Bank Group’s early response was relevant to client governments and mostly relevant to client firms. It was relevant because it was geared toward countries that are highly vulnerable to economic shocks, especially LICs. However, support to sectors within countries varied, with some countries not getting sufficient support to sectors in need. The World Bank’s concentra- tion on DPF and MPAs—primarily through the Equitable Growth, Finance, and Institutions Practice Group and the Social Protection and Jobs GP—was relevant to client governments, especially in the Africa and Latin America 78   and the Caribbean Regions, although disbursements to IDA clients in the Africa and Middle East and North Africa Regions were relatively low. Prior actions supported both short-term reforms for economic and social recov- ery and longer-term regulatory reforms, including the green, resilient, and inclusive development agenda. The IFC and MIGA early response—especially through the IFC Financial Institutions Group—was mostly relevant to firms and in line with IFC’s expected countercyclical role. The relevance of the Bank Group’s support was limited by the lack of global knowledge work on the unique characteristics of the COVID-19 crisis. Country office staff need- ed greater clarity on crisis protocols to be better prepared. The Bank Group’s early response was generally of good quality in design, was influential with clients, and was coordinated well with the IMF, although it had room for improvement in some areas. The Bank Group’s early response positively influenced client country strategies, especially regarding macrofis- cal and social safety nets. The Bank Group’s influence on governments’ and firms’ actions in the incipient recovery phase (the second part of the evalu- ation period) was less on target based on the six-month period of incipient recovery evaluated. Recent IEG evaluations on sustainable development finance policy and public financial and debt management offer important lessons relevant to the recovery phase of the pandemic (World Bank 2021a, 2021b). We did not find evidence that the Bank Group learned consistently from developed countries or from its own experience (intracrisis learning) during the evaluation period. As a result, firms in high-contact industries Independent Evaluation Group World Bank Group    79 that had been severely affected by the crisis and governments and firms that needed support for the incipient recovery phase were underserviced. The World Bank’s proactive engagement with the IMF improved the quality of the response to the crisis, but coordination with regional development banks and other international financial institutions was mixed. The Bank Group’s support to the financial sector and MSMEs was well coordinated, but Bank Group–structured finance solutions were limited. Because the Bank Group did not set targets related to the economic implications of COVID-19, we could not assess whether it is likely to monitor or meet them during the recovery phase. The Bank Group could have assessed whether existing com- pliance (including safeguards) requirements facilitated providing support during the crisis. Six factors affected the quality of the Bank Group’s early response. They were (i) engaging with partners outside of the development community, (ii) mobilizing local partners and stakeholders, (iii) adopting clear staff surge plans for country offices, (iv) prioritizing staff welfare, (v) using instruments that can disburse quickly and at scale, and (vi) selecting and strengthening Project Implementation Units. We offer two near-term recommendations to strengthen the role of the Bank Group as a crisis responder, which is more critical than ever. Global macro- economic imbalances have reached unprecedented proportions. The situa- tion is compounded by early warning signs of new crises emerging in client countries at the national level (for example, inflation), the regional level (for example, the Russian invasion of Ukraine), and the global level (for example, The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Chapter 5 the food crisis). With this context of growing uncertainty at several levels, the role of the Bank Group as a systemic first responder to crises has become even more critical. We offer two recommendations for the Bank Group’s con- sideration to strengthen its role in addressing the economic implications of COVID-19 and future crises. Recommendation 1: To effectively address future crises, codify a global cri- sis response playbook, ideally developed jointly with the IMF. A global crisis response playbook would include considerations on the effectiveness of vari- ous lending instruments at the time of crisis, examples of which instruments worked best under which country conditions and for which sectors and industries, and lessons learned on how to improve their use. The playbook would benefit from describing ways to identify and target countries in high need of support (for example, by developing need scores as conducted in this evaluation) and engage deliberately with highly indebted countries that may need debt resolution mechanisms during recovery from a global crisis. The playbook would include a list of partners, including from the private sector, with whom the Bank Group has successfully collaborated during COVID-19 and other crises. It would also include guidelines on applying safeguards in times of crisis, with a view to selecting only the essential ones, streamlining their application, and incorporating greater flexibility (for example, risk- based tiers and early results-based options) for staff and clients. Given that numerous crises are compounding around the world, it would be prudent 80 for the Bank Group to prioritize staff welfare in planning for its future crisis responses. Staff surge planning has proved to be a driving factor of success during the COVID-19 crisis. Finally, the crisis playbook could include “cri- sis games” to be conducted regularly by Bank Group staff and clients based on risk assessments to improve preparedness and strengthen capacity for crisis response. Crisis games would be similar to the recent cybersecurity games conducted jointly by the Bank Group and the IMF with the support of the Boards of the two organizations. The playbook—or parts of it—could be developed jointly with the IMF. The effectiveness of joint Bank Group–IMF action in times of crisis could be further strengthened by an explicit joint statement of principles of collaboration between the two institutions to pre- vent, prepare for, and address crises. Recommendation 2: To respond effectively during the recovery phase of the crisis, explore increasing use of structured finance solutions (such as partial credit guarantees, subordinated debt, and quasi-equity instruments) with a view to supporting small- and medium-size firms. The COVID-19 crisis has left many firms with potential debt overhang. Globally, a large number of small and medium firms were affected to the point of insolvency. The Bank Group’s structured finance solutions offer a relevant response during the incipient recovery phase of this crisis and potentially future crises. It is possible that such solutions would be complex to arrange and require the Bank Group to engage with several partners. However, the benefits are likely Independent Evaluation Group World Bank Group    81 higher than the costs, and the trade-offs can be explored further. During the recovery phase of the crisis, it would be useful to explore the feasibility of in- creasing the use of solutions such as subordinated debt, quasi-equity instru- ments, partial risk, and partial credit guarantees to support recapitalizing firms. Some firms coming out of the recovery phase are likely to have limited growth potential because their capital structures were destroyed during the COVID-19 crisis. The Bank Group has the capacity to explore greater use of such options and develop related corporate risk analysis in real time during the crisis. Bibliography Bellavita, Christopher, Joseph S. Wholey, and Mark A. Abramson. 1986. “Perfor- mance-Oriented Evaluation: Prospects for the Future.” In Performance and Credibility: Developing Excellence in Public and Nonprofit Organizations, edited by Joseph S. Wholey, Mark A. Abramson, and Christopher Bellavita. Lexington, MA: Lexington Books. Benguria, Felipe, and Alan M. Taylor. 2020. “After the Panic: Are Financial Crises De- mand or Supply Shocks? Evidence from International Trade.” American Economic Review: Insights 2 (4): 509–26. https://doi.org/10.1257/aeri.20190533. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Bibliography Borensztein, Eduardo, and Ugo Panizza. 2009. “The Costs of Sovereign Default.” IMF Staff Papers 56 (4): 683–741. Cutler, David M., and Lawrence H. Summers. 2020. “The COVID-19 Pandemic and the $16 Trillion Virus.” JAMA 324 (15): 1495–96. https://doi.org/10.1001/ jama.2020.19759. Economist. 2022. “The 53 Fragile Emerging Economies.” July 15, 2022. https://www. economist.com/finance-and-economics/2022/07/20/the-53-fragile-emerg- ing-economies. FES (Friedrich-Ebert-Stiftung) and CBI (Consensus Building Institute). 2021. “Re- sponding to Risks of COVID Debt Distress.” Roundtable Report. FES and CBI, New York. Gudmundsson, Tryggvi, Vladimir Klyuev, Leandro Medina, Boaz Nandwa, Dmitry Plotnikov, Francisco Schiffrer, and Di Yang. 2022. “Emerging Markets: Pros- pects and Challenges.” IMF Working Paper 22/35, International Monetary Fund, Washington, DC. Hale, Thomas, Sam Webster, Anna Petherick, Toby Phillips, and Beatriz Kira. 2020. Ox- ford COVID-19 Government Response Tracker. Blavatnik School of Government, University of Oxford, Oxford. IIF (Institute of International Finance). 2022. “IIF Weekly Insight: Prevention and Resolution of Sovereign Debt Crises—The Importance of Market-Based 82 Approaches.” IIF Weekly Insight (April 28), IIF, Washington, DC. IMF (International Monetary Fund). 2020a. “Financial Soundness Indicators.” Data- set, IMF, Washington, DC. https://data.imf.org/FSI. IMF (International Monetary Fund). 2020b. “Panama: Request for Purchase under the Rapid Financing Instrument.” IMF Country Report 20/147, IMF, Washington, DC. IMF (International Monetary Fund). 2021a. “Debt Relief under the Heavily Indebted Poor Countries (HIPC) Initiative.” Factsheet (March 23), IMF, Washington, DC. https://www.imf.org/en/About/Factsheets/Sheets/2016/08/01/16/11/Debt-Re- lief-Under-the-Heavily-Indebted-Poor-Countries-Initiative. IMF (International Monetary Fund). 2021b. “Issues in Restructuring of Sovereign Domestic Debt.” Policy Paper 2021/071, IMF, Washington, DC. IMF (International Monetary Fund). 2022. “Panama: Second Review Under the Arrangement Under the Precautionary and Liquidity Line.” IMF Country Report 22/276, IMF, Washington, DC. Lee, Nancy, and Rakan Aboneaaj. 2021. “MDB COVID-19 Crisis Response: Where Did the Money Go?” CGD Note (November 30), Center for Global Development, Washington, DC. Moynihan, Donald P. 2009. “From Intercrisis to Intracrisis Learning.” Journal of Con- tingencies and Crisis Management 17 (3): 189–98. https://doi.org/10.1111/j.1468- 5973.2009.00579.x. Independent Evaluation Group World Bank Group    83 Naeher, Dominik, Raghavan Narayanan, and Virginia Ziulu. 2022. “Relevance of the World Bank Group’s Early Response to COVID-19: A Cross-Country Sector Anal- ysis.” Policy Research Working Paper 9935, World Bank, Washington, DC. OECD (Organisation for Economic Co-operation and Development) DAC (Develop- ment Assistance Committee) Network on Development Evaluation. 2019. Better Criteria for Better Evaluation: Revised Evaluation Criteria Definitions and Princi- ples for Use. Paris: OECD. https://www.oecd.org/dac/evaluation/revised-evalua- tion-criteria-dec-2019.pdf. Rahim, Fazeer, Richard Allen, Hélène Barroy, Laura Gores, and Joseph Kutzin. 2020. “COVID-19 Funds in Response to the Pandemic.” Special Series on COVID-19 (August 26), International Monetary Fund, Washington, DC. Reinhart, Carmen M., and Kenneth S. Rogoff. 2011. “From Financial Crash to Debt Crisis.” American Economic Review 101 (5): 1676–1706. WEF (World Economic Forum). 2020. The Global Competitiveness Report Special Edi­ tion 2020: How Countries Are Performing on the Road to Recovery. Geneva: WEF. WHO (World Health Organization). 2020. Global Spending on Health 2020: Weathering the Storm. Geneva: WHO. World Bank. 2011. The World Bank Group’s Response to the Global Economic Crisis: Phase I. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2012. The World Bank Group’s Response to the Global Economic Crisis: Phase II. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2013. The World Bank Group and the Global Food Crisis: An Evaluation of the World Bank Group Response. Independent Evaluation Group. Washington, The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Bibliography DC: World Bank. World Bank. 2017a. Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2017b. Georgia Public Expenditure Review: Building a Sustainable Future. Washington, DC: World Bank. World Bank. 2018. Georgia—From Reformer to Performer. Systematic Country Diag- nostic. Washington, DC: World Bank. World Bank. 2019. Knowledge Flow and Collaboration under the World Bank’s New Operating Model: An Independent Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2020a. “The Economic and Social Impact of COVID-19: Setting the Stage.” Western Balkans Regular Economic Report 17, World Bank, Washington, DC. World Bank. 2020b. Georgia: Towards Green and Resilient Growth. Washington, DC: World Bank. http://hdl.handle.net/10986/34862. World Bank. 2020c. January 2020 Global Economic Prospects: Slow Growth, Policy Chal- lenges. Washington, DC: World Bank. World Bank. 2020d. Poverty and Shared Prosperity 2020: Reversals of Fortune. Wash- ington, DC: World Bank. https://openknowledge.worldbank.org/bitstream/han- dle/10986/34496/9781464816024.pdf. 84 World Bank. 2020e. Saving Lives, Scaling-up Impact and Getting Back on Track: World Bank Group COVID-19 Crisis Response Approach Paper. Washington, DC: World Bank. http://documents1.worldbank.org/curated/en/136631594937150795/pdf/ World-Bank-Group-COVID-19-Crisis-Response-Approach-Paper-Saving-Lives- Scaling-up-Impact-and-Getting-Back-on-Track.pdf. World Bank. 2020f. Serbia’s New Growth Agenda: Job and Training-Ready. Country Economic Memorandum. Washington, DC: World Bank. World Bank. 2021a. The International Development Association’s Sustainable Develop- ment Finance Policy: An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021b. World Bank Support for Public Financial and Debt Management in IDA-Eligible Countries. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2022a. “Debt Service Suspension Initiative.” World Bank Brief (March 10), World Bank, Washington, DC. https://www.worldbank.org/en/topic/debt/ brief/COVID-19-debt-service-suspension-initiative. World Bank. 2022b. January 2022 Global Economic Prospects. Washington, DC: World Bank. World Bank. 2022c. “Navigating Multiple Crises, Staying the Course on Long-Term Development: The World Bank Group’s Response to the Crises Affecting Devel- oping Countries.” Working Paper 174677, World Bank, Washington, DC. https:// Independent Evaluation Group World Bank Group    85 openknowledge.worldbank.org/handle/10986/37826. World Bank. 2022d. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response. Independent Evaluation Group. Washington, DC: World Bank. World Bank Group. 2019. World Bank Group Evaluation Principles. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/reports/ WorldBankEvaluationPrinciples.pdf. World Bank. 2022e. World Development Report 2022: Finance for an Equitable Recov- ery. Washington, DC: World Bank. https://openknowledge.worldbank.org/bit- stream/handle/10986/36883/9781464817304.pdf. APPENDIXES Independent Evaluation Group The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A. Evaluation Methods Scope The analysis covers International Bank for Reconstruction and Development (IBRD), International Development Association (IDA), International Finance Corporation (IFC), and Multilateral Investment Guarantee Agency (MIGA) projects that relate to portions of pillars 3 and 4 in the relief and restruc- turing stages of the World Bank Group COVID-19 response. Specifically, it covers saving livelihoods, preserving jobs, and ensuring more sustainable business growth and job creation (pillar 3) and strengthening policies, The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A institutions, and investments (pillar 4; see table A.1.) The evaluation does not cover recovery-stage projects because it is too early to assess them. We collaborated closely with the Independent Evaluation Group team conduct- ing the evaluation on protecting lives, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response, which focuses on pillar 1 and components of pillar 2 and pillar 4 (World Bank 2022).  orld Bank Group COVID-19 Response Plan and Focus of the Table A.1. W Independent Evaluation Group Evaluation Pillar Relief Stage Restructuring Stage Recovery Stage 1 Public health Restructuring health Pandemic-ready health systems systems 2 Social emergency Restoring human capital Building equity and inclusion 3 Economic emergency Firm restructuring and Green business, growth, debt resolution and job creation 4 Line of sight to goals Policy and institutional Investment to rebuild reforms better Source: Independent Evaluation Group, based on World Bank 2020. Note: This evaluation’s scope is outlined in red. 88 Methodology Overview We used a mixed methods approach. The methods include a structured litera- ture review, econometrics analysis, case-based analysis, informant interviews, and portfolio analysis. Case-based analysis is particularly important, given that the portfolio is recent (see figure A.1). Table A.2 shows in more detail which methods were used to answer each evaluation question for the three units of analysis: country, firm, and global. The paragraphs that follow the ta- ble briefly describe how the methods helped answer the evaluation questions. Figure A.1. Evaluation Methods Overview Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    89 Table A.2. Methods by Evaluation Question and Unit of Analysis Unit of Analysis Country (including Global sublevel (partnerships Evaluation of national and real-time Question government) Firm knowledge) 1. What has been » Literature review » Literature review » Literature review the relevance of the (government-led (firm-led actions) (relevance of World Bank Group actions) » Case-based anal- global support COVID-19 response » Case-based ysis (firm cases) efforts) to address the analysis (country » Key informant » Key informant The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A economic needs of cases) interviews (rele- interviews (rele- clients, and what vance of global » Key informant vance to firms) lessons can be support efforts) interviews » Firm-level project drawn? (relevance to portfolio analysis countries) (relevance to » Country-level firms) project port- folio analysis (relevance to countries) 2. What has been » Case-based » Case-based anal- » Key informant in- the quality of the analysis (country ysis (firm cases) terviews (quality early Bank Group cases) of global support COVID-19 response efforts) to address the » Global portfolio economic needs of analysis (quality clients, and what of global support lessons can be efforts) drawn? Source: Independent Evaluation Group. The two evaluation questions contain several subquestions. The relevance question includes three subquestions regarding the Bank Group’s tailored approach for client governments and firms, informed design of the inter- ventions, and the extent to which the Bank Group leveraged its comparative advantages. The quality question includes three subquestions regarding the level of Bank Group influence on clients to take necessary actions, the extent of coordination within the Bank Group, coordination with the International 90 Monetary Fund and other development partners, and the extent to which the Bank Group support has adequate results frameworks, monitoring, and governance. Ensuring the Validity of Findings We applied a consistent approach using existing evaluation frameworks and triangulating with internal sources for internal validity. We applied multiple levels of triangulation during the evaluation. We used common templates when conducting portfolio reviews, case studies, and interviews. We also routinely cross-checked the portfolio review results to reach consensus on the alignment of review templates and evaluation questions. A participatory approach with Bank Group staff ensured consistency on data sources and internal validity. Furthermore, we applied external validation mechanisms. We consulted and cross-validated with external data management teams in the World Bank, IFC, and MIGA to compare our scoped portfolio to ensure the accuracy of project identification and records of mobilized capital. In addition, several rounds of peer reviews provided feedback during the evaluation process to guarantee the evaluation’s relevance and effectiveness. Limitations Independent Evaluation Group World Bank Group    91 The Approach Paper was approved with the agreement that this would be an early-stage evaluation with limited scope and depth. Because of the limited time available to deliver an early-stage evaluation with lessons useful for shaping the ongoing response, we were authorized at the Approach Paper review meeting to proceed with a focused early-stage evaluation (rather than a full thematic evaluation) of limited scope and depth. As such, this evalua- tion has several limitations. Given that the COVID-19 response is ongoing, this is an early-stage evaluation that does not judge whether early response projects will succeed. Because we treat the IFC and MIGA portfolio only via a sample of firm-level cases, we could not assess the relevance or quality of the overall IFC and MIGA response across the entire universe (for example, the health sector response). We used a novel framework to analyze country needs, and there were data limitations to the analysis in some countries and sectors. We operated on the premise that “rough approximations delivered at the right time are better than precise results delivered too late for deci- sion” (Bellavita, Wholey, and Abramson 1986). This evaluation defined two phases to understand the evolution of the Bank Group interventions: the acute phase (April 2020–December 2020) and the incipient recovery phase (January 2021–June 2021) and based this on data regarding emerging market and developing economies’ gross domestic product fluctuations, evolution of lockdowns, and the Oxford Stringency Index, a highly respected and quanti- tative measure suggesting early recovery. The portfolio for this evaluation includes data from IBRD and IDA, IFC, and The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A MIGA, which all report project information using different protocols. There- fore, we used several assumptions to harmonize the data across the different institutions. This approach had the advantage that it allowed for the data from all the institutions to be analyzed jointly, but some of the data granu- larity might have been lost. Given the ongoing nature of the COVID-19 pandemic at the time the evalua- tion was conducted, external data on the economic effects of COVID-19 were limited. To mitigate this, we used some novel data sources such as the World Bank Business Pulse Surveys, the World Bank Enterprise Survey follow-up on COVID-19, and the World Bank COVID-19 Vulnerability Index. Qualitative Methods Document and Literature Review The purpose of the document and literature review was to create a synthesis of “what works” to limit the economic impact of epidemics or other crises, providing a standard by which to judge the relevance of the Bank Group’s support efforts. To create this synthesis, we reviewed and analyzed the in- ternal and external literature on recent evidence regarding the effectiveness of Bank Group global support efforts and government and firm actions to address crises. This review included caveats on the extent to which one can extrapolate from previous crises to the current one. 92 The text analysis was conducted on project documents from specific coun- tries to be selected based on their vulnerability to COVID-19 and the magni- tude of Bank Group support they are receiving. The objective of the analysis was to identify the common and specific themes related to the COVID-19 response emerging from the text in project documents in the selected coun- tries based on a vocabulary of key words and phrases. Sampled Interventions We conducted case-based analyses (case studies) at the country and firm levels to study the relevance of the Bank Group response. The country-level analysis included, when relevant, the sublevel of national government. The case-based analyses began by purposively sampling country-level and firm- level cases to study. In contrast to random sampling, purposeful sampling is a technique widely used in qualitative research for the identification and selection of information-rich cases for the most effective use of limited resources (Patton 2005). Country-level case studies. To help in assessing the relevance of Bank Group interventions, we identified case study countries from which les- sons could be extracted. We selected the case study countries based on regional diversity, economic vulnerability in the face of COVID-19, World Bank commitment level, and country income level (table A.3; see the case Independent Evaluation Group World Bank Group    93 identification protocol section in this appendix for further explanation of the vulnerability index and case identification protocol). We prioritized lower-middle-income and low-income countries because they had greater challenges in responding to COVID-19. Table A.3. Country Case Studies World Bank Vulnerability Commitment Region Country Level Level Income Level AFR Cabo Verde High Medium LMIC AFR Nigeria High High LIC AFR Senegal High Medium LMIC EAP Lao PDR High High LMIC EAP Philippines High High LMIC ECA Georgia Medium Medium LMIC ECA Serbia Medium Medium UMIC LAC Ecuador High High UMIC The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A SAR Pakistan High High LIC Source: Independent Evaluation Group. Note: Cabo Verde, Ecuador, Lao PDR, and Senegal are not commonly studied country case studies for Independent Evaluation Group thematic evaluations. AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; Lao PDR = Lao People’s Democratic Republic; LIC = low-income country; LMIC = lower-middle-income country; SAR = South Asia; UMIC = upper-middle-income country. Firm-level case studies. The evaluation incorporates the findings of 10 deep dive case studies at the firm level. The case study firms were selected based on client vulnerability to COVID-19, existing IFC and MIGA portfolio expo- sure (through loans outstanding, equity exposure, guarantees issued), new commitment levels, repeat exposure at the client and client group levels, and diversity of response by business lines such as credit to small and medium enterprises, working capital or liquidity support, or trade credit (table A.4). Additionally, we held consultations with IFC to refine the selection of case studies and included Liquid Bond as a firm-level case study. 94 Table A.4. Firm Case Studies Portfolio New Firm Country Region Exposure Commitment Ara Tiendas Colombia LAC High High Banco Davivienda Colombia LAC High High BNP Panama LAC High High CI Bank Egypt, Arab MENA High High Rep. FirstRand Multiple AFR High High Hikma Group Multiple MENA High High Liquid Bond Multiple AFR Medium High ProCredit Multiple ECA High High Trans Corpora Indonesia EAP High High VP Bank Vietnam EAP High Medium Source: Independent Evaluation Group. Note: Analysis of firms with projects across multiple countries may cover multiple business lines such as small and medium enterprise finance, micro finance, and trade finance. The qualification of high, low, and medium was based on IFC exposure to the client group or the client firm at the time of portfolio identification and the Approach Paper preparation (vintage February 2021). This portfolio was updated again during this report’s preparation (vintage February 2022, to cover IFC activities from April 2020 to April 2021). The IFC exposure to the firms fluctuated during the COVID-19 pandemic, and the qualifiers (high, low, and medium) may be different at the time of this evaluation report’s preparation. The case study identification and selection was discussed with IFC management and staff during the preparation of the Approach Paper. AFR = Africa; BNP = Banco Nacional de Panama; EAP = East Asia and Pacific; ECA = Europe and Central Asia; IFC = International Finance Corporation; LAC = Latin America and the Carib- bean; MENA = Middle East and North Africa. We relied on available documentation from approval documents and port- Independent Evaluation Group World Bank Group    95 folio documents for country and firm cases. We used Board of Executive Directors approval documents (such as Project Appraisal Documents, the President’s Report, and Condition of Guarantees) and portfolio supervision documents (Client Quarterly Supervision Report, Implementation Supervi- sion Reports) for data collection and analysis. Key Informant Interviews We conducted key informant interviews to help in assessing both the quality of Bank Group global support efforts and the relevance of the Bank Group interventions in case study countries and case study firms. We engaged key informants, such as Bank Group staff, client country representatives, client firm representatives, and potential beneficiaries through interviews. Key informants provided more detail about matters that are simply not record- ed in project documentation (because they are of the wrong type, might be politically sensitive, were too detailed, and so on). To help in evaluating the relevance of Bank Group interventions in case study countries and firms, the interviews aimed to understand whether the intervention involved such things as diagnosing economic needs, targeting, and tailoring to needs and priorities, and choosing the right instruments in ways not recorded in project documents. Similarly, to help in assessing the quality of Bank Group global support efforts, the interviews aimed to understand whether the interven- tions in a project involved, for example, leveraging technology, systematic monitoring, and using evidence to make adaptive decisions in ways not recorded in project documentation. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A The availability of key informants targeted for this evaluation in client countries and firms was limited. We interviewed more than 250 key informants from vari- ous groups (table A.5). For all case studies, we interviewed the task team lead, the country manager, the country economist and the government counterpart, Bank Group Board members, and donors to the extent feasible. In firm cases, the client counterpart was interviewed based on availability; interviewing all counterparts was not feasible, and we relied on staff interviews and internal documents. In some country cases, we interviewed external actors (such as academics, think tanks, and civil society organizations) and regional partners (such as develop- ment banks and international financial institutions). Table A.5 shows the number of interviews conducted in this early-stage evaluation. Table A.5. Key Informant Interviews (number) Type of Informant Country Cases Firm Cases Headquarters staff 46 30 Country office staff 58 60 Government officials 14 0 Private sector clients 0 22 Development partners and cofinanciers 7 0 Think tanks and civil society organizations 2 0 Academics 5 0 Board members, donors, and other multilateral actors 12 3 Total 144 115 96 Source: Independent Evaluation Group. Quantitative Methods COVID-19 Evaluation Database We built a customized database to store all key data needed from the mul- tiple quantitative analyses conducted for this evaluation. The database was built following relational database best practices to ensure both ease of use and data integrity and quality control. A key consideration when building the databases was the fact that the evaluation portfolio includes projects from IBRD and IDA, IFC, and MIGA, which all report project data using different formats. Consequently, we consolidated and harmonized the data from the different institutions under a common taxonomy. The database includes the following components: » Bank Group portfolio data. These data have been collected from the data provided by the institutions, the team’s own research, and the use of semiau- tomatic portfolio techniques (for IBRD and IDA data). » Country-level indicators, including more than 100 microeconomic, macroeco- nomic, and financial indicators we collected from multiple sources (such as the World Bank, the International Monetary Fund, the World Health Organi- zation, the World Trade Organization, and Standard & Poor’s). The two components of the data set can be joined through a common key (coun- Independent Evaluation Group World Bank Group    97 try code), which allows mapping any variables across the tables (figure A.2). Figure A.2. Structure of the Database The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Source: Independent Evaluation Group. Portfolio Review and Analysis Project portfolio analyses (country and firm levels). To support the gener- alizability of findings and identify unique actions worthy of replication, we used country- and firm-level project portfolio analyses to help assess the relevance of the World Bank interventions supporting countries and IFC and MIGA interventions supporting firms. We classified the COVID-19 portfolio by transmission channels. These included the following: » The broad types of Bank Group interventions used, for example, facilitating policy changes; » The specific instruments and approaches employed, for instance, contingent emergency response component or Multiphase Programmatic Approach; » The portfolio management and operational processes followed, such as fast- track approvals, restructuring, or due diligence; 98 » The outputs expected at the meso level; and » Results indicators and drivers of early success and failure (where available). We used key informant interviews to assess the Bank Group’s response, in- cluding whether it paid sufficient attention to monitoring and supervision. Project portfolio analysis (global level). Beyond the specific interventions that the Bank Group makes to support countries and firms, it also makes a variety of efforts to support all clients via global initiatives on knowledge sharing and other topics. An analysis of the portfolio of these broad global support efforts helped us assess their quality. Econometric Analysis The purpose of this econometric analysis was to determine the relevance of the Bank Group early response to COVID-19 in each country and across sectors. For the purpose of this study, relevance was defined as the match between the types and scopes of provided support and the types and scopes of support that were most needed in each country during the first year of the COVID-19 pandemic. The analysis was conducted along the following three steps. First, a set of sector-specific scores capturing vulnerability and need for support in each country were constructed based on empirical indicators (the need score). Independent Evaluation Group World Bank Group    99 Second, Bank Group support in response to COVID-19 was mapped to the same sector-specific themes, yielding a normalized measure of the relative intensities of Bank Group support to each sector in a given country. Finally, the two measures were compared for each country using a distance function, which yielded an empirical measure of the degree of relevance of Bank Group support within and across countries. In operationalizing this approach, the analysis aggregated micro (firm-level) and macro (country-level) data across several sectors of the economy into a single composite index. For this purpose, we relied on a newly released har- monized data set of variables from the World Bank Business Pulse Surveys and the World Bank Enterprise Survey follow-up on COVID-19. This novel data set captures more than 120,000 businesses across 60 countries and measures the impact of the pandemic across several important dimensions such as sales, labor adjustments, liquidity and insolvency, operations of the business, firm responses, and accessibility to public policy support. Case Identification Protocol We sampled country-level cases to study based on vulnerability—includ- ing economic vulnerability—to COVID-19, among other factors. To do so, we primarily relied on the COVID-19 Vulnerability Index prepared by the World Bank, which was designed to summarize COVID-19 vulnerabilities by country. The index was derived from nearly 50 indicators on six dimensions: (i) COVID-19 exposure and response (composite indicator summarizing COVID-19 cases and crisis response indexes); (ii) health score (composite The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A indicator summarizing health system and preparedness indexes); (iii) com- modity exposure (composite indicator summarizing commodity exposure indexes); (iv) fitness and trade exposure (composite index summarizing economic fitness, trade, and global value chain exposure indexes); (v) inter- nal vulnerability (gauges the overall macroeconomic health and potential internal imbalances in the economy, such as gross domestic product growth, inflation, and government debt); and (vi) external vulnerability (gauges the overall macroeconomic health and potential external imbalances in the economy, such as current account balances, debt service, and reserves). In addition, the score assigned to each of these six dimensions was combined (by an unweighted average) to derive a composite index reflecting the level of vulnerability of each country to COVID-19. We used this index in conjunc- tion with portfolio data to conduct a multidimensional analysis to under- stand the level of Bank Group support (in terms of number of projects and of commitments) across different levels of vulnerability. Criterion 1 The first criterion considered countries with high vulnerability to COVID-19 and high Bank Group engagement. Vulnerability to COVID-19 was measured as the average of country rank on all six dimensions: (i) COVID-19 exposure and response, (ii) health score, (iii) commodity exposure, (iv) fitness and trade exposure, (v) internal vulnerability, and (vi) external vulnerability. 100 Figure A.3 shows countries highlighted by this criterion, and table A.6 lists them. Figure A.3. Countries Highlighted by Criterion 1 Source: Independent Evaluation Group. Note: Countries are colored in a gradation of four shades, representing the overall level of vulnerability to COVID-19 (0–25, 26–50, 51–75, and 76–100). Countries in darker shades have higher vulnerability. The circles overlaid on the map represent the level of World Bank Group engagement in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. Independent Evaluation Group World Bank Group    101 102 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.6. Countries Highlighted by Criterion 1 World Bank Projects by Institution Projects by Type Group Projects (no.) (no.) IBRD or IDA IFC MIGA World World IFC Income US$, proj- proj- proj- Bank Bank invest- IFC ad- MIGA MIGA Country Level No. billions ects ects ects ASA lending ment visory NH PRI Brazil UM 22 1.78 14 8 0 8 6 8 0 0 0 Colombia UM 15 1.83 8 6 1 5 3 4 2 1 0 Ecuador UM 13 1.41 9 4 0 4 5 4 0 0 0 Ethiopia L 13 0.95 9 4 0 4 5 1 3 0 0 Indonesia LM 15 2.61 11 3 1 6 5 2 1 1 0 Kenya LM 22 0.88 15 7 0 9 6 3 4 0 0 Mexico UM 15 2.84 10 4 1 8 2 4 0 1 0 Morocco LM 15 0.79 12 3 0 7 5 1 2 0 0 Nigeria LM 23 3.46 7 15 1 3 4 8 7 0 1 Pakistan LM 24 1.54 21 3 0 7 14 1 2 0 0 South UM 13 0.95 6 7 0 5 1 4 3 0 0 Africa Türkiye UM 19 1.62 13 6 0 7 6 6 0 0 0 Total 209 20.68 135 70 4 73 62 46 24 3 1 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = Interna- tional Finance Corporation; L = lower; LM = low-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Criterion 2 The second criterion considered countries with weak COVID-19 response and high Bank Group engagement. Weak COVID-19 response was measured using the COVID-19 exposure and response dimension (that is, a composite indicator summarizing COVID-19 cases and crisis response indexes). Figure A.4 shows countries highlighted by this criterion, and table A.7 lists them. Figure A.4. Countries Highlighted by Criterion 2 Source: Independent Evaluation Group. Note: Countries are colored in a gradation of four shades, representing the quality of the countries’ COVID-19 response (0–25, 26–50, 51–75, and 76–100). Countries in darker shades have weaker respons- Independent Evaluation Group World Bank Group    103 es. The circles overlaid on the map represent the level of World Bank Group engagement in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. 104 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.7. Countries Highlighted by Criterion 2 World Bank Group Projects by Institution Projects by Type Projects (no.) (no.) World IBRD or IFC MIGA World Bank Income US$, bil- IDA proj- proj- proj- Bank lend- IFC in- IFC ad- MIGA MIGA Country Level No. lions ects ects ects ASA ing vestment visory NH PRI Brazil UM 22 1.78 14 8 0 8 6 8 0 0 0 Colombia UM 15 1.83 8 6 1 5 3 4 2 1 0 Ecuador UM 13 1.41 9 4 0 4 5 4 0 0 0 India LM 22 3.56 9 13 0 4 5 12 1 0 0 Indonesia LM 15 2.61 11 3 1 6 5 2 1 1 0 Mexico UM 15 2.84 10 4 1 8 2 4 0 1 0 Nigeria LM 23 3.46 7 15 1 3 4 8 7 0 1 South UM 13 0.95 6 7 0 5 1 4 3 0 0 Africa Total 138 18.45 74 60 4 43 31 46 14 3 1 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = Inter- national Finance Corporation; LM = lower-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Criterion 3 The third criterion considered countries with inadequate health systems and preparedness and high Bank Group engagement. Inadequate health systems and preparedness was measured using the health score dimension (that is, a compos- ite indicator summarizing health system and preparedness indexes). Figure A.5 shows countries highlighted by this criterion, and table A.8 lists them. Figure A.5. Countries Highlighted by Criterion 3 Source: Independent Evaluation Group. Note: Countries are colored in a gradation of four shades, representing the quality of the countries’ health systems and COVID-19 preparedness (0–25, 26–50, 51–75, and 76–100). Countries in darker Independent Evaluation Group World Bank Group    105 shades have worse health systems and were less prepared. The circles overlaid on the map represent the level of World Bank Group engagement in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. 106 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.8. Countries Highlighted by Criterion 3 World Bank Projects by Institution Projects by Type Group Projects (no.) (no.) IBRD or IDA IFC MIGA World World IFC Income US$, proj- proj- proj- Bank Bank invest- IFC ad- MIGA MIGA Country Level No. billions ects ects ects ASA lending ment visory NH PRI Bangladesh LM 22 2.18 12 10 0 4 8 7 3 0 0 China UM 15 0.94 6 9 0 5 1 9 0 0 0 Colombia UM 15 1.83 8 6 1 5 3 4 2 1 0 Ecuador UM 13 1.41 9 4 0 4 5 4 0 0 0 Ethiopia L 13 0.95 9 4 0 4 5 1 3 0 0 India LM 22 3.56 9 13 0 4 5 12 1 0 0 Indonesia LM 15 2.61 11 3 1 6 5 2 1 1 0 Kenya LM 22 0.88 15 7 0 9 6 3 4 0 0 Mexico UM 15 2.84 10 4 1 8 2 4 0 1 0 Morocco LM 15 0.79 12 3 0 7 5 1 2 0 0 Nigeria LM 23 3.46 7 15 1 3 4 8 7 0 1 Pakistan LM 24 1.54 21 3 0 7 14 1 2 0 0 Philippines LM 21 1.92 16 5 0 11 5 3 2 0 0 Türkiye UM 19 1.62 13 6 0 7 6 6 0 0 0 Total 254 26.55 158 92 4 84 74 65 27 3 1 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = Interna- tional Finance Corporation; L = low; LM = lower-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Criterion 4 The fourth criterion considered countries with high commodities exposure and high Bank Group engagement. The commodities exposure dimension was measured using a composite indicator summarizing commodity expo- sure indexes. Figure A.6 shows countries highlighted by this criterion, and table A.9 lists them. Figure A.6. Countries Highlighted by Criterion 4 Source: Independent Evaluation Group. Note: Countries are colored in a gradation of four shades, representing the countries’ level of commod- Independent Evaluation Group World Bank Group    107 ities exposure (0–25, 26–50, 51–75, and 76–100). Countries in darker shades have higher exposure. The circles overlaid on the map represent the level of World Bank Group engagement in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. 108 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.9. Countries Highlighted by Criterion 4 World Bank Projects by Institution Projects by Type Group Projects (no.) (no.) IBRD or IDA IFC MIGA World World IFC Income US$, proj- proj- proj- Bank Bank invest- IFC ad- MIGA MIGA Country Level No. billions ects ects ects ASA lending ment visory NH PRI Brazil UM 22 1.78 14 8 0 8 6 8 0 0 0 Colombia UM 15 1.81 8 6 1 5 3 4 2 1 0 Ecuador UM 13 1.41 9 4 0 4 5 4 0 0 0 Ethiopia L 13 0.95 9 4 0 4 5 1 3 0 0 Indonesia LM 15 2.61 11 3 1 6 5 2 1 1 0 Kenya LM 22 0.88 15 7 0 9 6 3 4 0 0 Malaysia UM 17 0 17 0 0 17 0 0 0 0 0 Mexico UM 15 2.84 10 4 1 8 2 4 0 1 0 Nigeria LM 23 3.46 7 15 1 3 4 8 7 0 1 South UM 13 0.95 6 7 0 5 1 4 3 0 0 Africa Total 168 16.72 106 58 4 69 37 38 20 3 1 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = Interna- tional Finance Corporation; L = low; LM = lower-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Criterion 5 The fifth criterion considered countries with low rankings in terms of eco- nomic fitness, trade and global value chain exposure, and high Bank Group engagement. This criterion was measured using the fitness and trade expo- sure dimension (that is, a composite index summarizing economic fitness, trade, and global value chain exposure indexes). Figure A.7 shows countries highlighted by this criterion, and table A.10 lists them. Figure A.7. Countries Highlighted by Criterion 5 Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    109 Note: Countries are colored in a gradation of four shades, representing a composite of economic fitness, trade, and global value chain exposure (0–25, 26–50, 51–75, and 76–100). Countries in darker shades have lower fitness, more trade exposure, and more global value chain exposure. The circles overlaid on the map represent the level of World Bank Group engagement in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. 110 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.10. Countries Highlighted by Criterion 5 World Bank Projects by Institution Projects by Type Group Projects (no.) (no.) IBRD or IDA IFC MIGA World World IFC Income US$, proj- proj- proj- Bank Bank invest- IFC ad- MIGA MIGA Country Level No. billions ects ects ects ASA lending ment visory NH PRI Malaysia UM 17 0 17 0 0 17 0 0 0 0 0 Morocco LM 15 0.79 12 3 0 7 5 1 2 0 0 Vietnam LM 14 0.23 9 5 0 8 1 4 1 0 0 Total 46 1.02 38 8 0 32 6 5 3 0 0 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = Inter- national Finance Corporation; LM = lower-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Criterion 6 The sixth criterion considered countries with internal macroeconomic vulnerabilities and high Bank Group engagement. The internal macroeco- nomic vulnerability dimension gauges the overall macroeconomic health and potential internal imbalances in the economy, such as gross domestic product growth, inflation, and government debt. Figure A.8 shows countries highlighted by this criterion, and table A.11 lists them. Figure A.8. Countries Highlighted by Criterion 6 Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    111 Note: Countries are colored in a gradation of four shades, representing the countries’ internal macroeco- nomic vulnerabilities (0–25, 26–50, 51–75, and 76–100). Countries in darker shades have higher vulner- ability. The circles overlaid on the map represent the level of World Bank Group engagement in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. 112 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.11. Countries Highlighted by Criterion 6 World Bank Projects by Institution Projects by Type Group Projects (no.) (no.) IBRD or IDA IFC MIGA World World IFC Income US$, proj- proj- proj- Bank Bank invest- IFC ad- MIGA MIGA Country Level No. billions ects ects ects ASA lending ment visory NH PRI Brazil UM 22 1.78 14 8 0 8 6 8 0 0 0 India LM 22 3.56 9 13 0 4 5 12 1 0 0 Kenya LM 22 0.88 15 7 0 9 6 3 4 0 0 Mexico UM 15 2.84 10 4 1 8 2 4 0 1 0 Nigeria LM 23 3.46 7 15 1 3 4 8 7 0 1 Pakistan LM 24 1.54 21 3 0 7 14 1 2 0 0 South UM 13 0.95 6 7 0 5 1 4 3 0 0 Africa Türkiye UM 19 1.62 13 6 0 7 6 6 0 0 0 Total 160 16.65 95 63 2 51 44 46 17 1 1 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = International Finance Corporation; L = lower; LM = lower-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Criterion 7 The seventh criterion considered countries with external macroeconomic vulnerabilities and high Bank Group engagement. The external macroeco- nomic vulnerability dimension gauges the overall macroeconomic health and potential external imbalances in the economy, such as current account balance, debt service, and reserves. Figure A.9 shows countries highlighted by this criterion, and table A.12 lists them. Figure A.9. Countries Highlighted by Criterion 7 Independent Evaluation Group World Bank Group    113 Source: Independent Evaluation Group. Note: Countries are colored in a gradation of four shades, representing the countries’ external macro- economic vulnerability to COVID-19 (0–25, 26–50, 51–75, and 76–100). Countries in darker shades have higher vulnerability. The circles overlaid on the map represent the level of World Bank Group engage- ment in terms of number of projects. The size of each circle is proportional to the number of projects, including active, closed, and pipeline projects. 114 The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A Table A.12. Countries Highlighted by Criterion 7 World Bank Projects by Institution Projects by Type Group Projects (no.) (no.) IBRD or IDA IFC MIGA World World IFC Income US$, proj- proj- proj- Bank Bank invest- IFC ad- MIGA MIGA Country level No. billions ects ects ects ASA lending ment visory NH PRI Colombia UM 15 1.83 8 6 1 5 3 4 2 1 0 Ecuador UM 13 1.41 9 4 0 4 5 4 0 0 0 Pakistan LM 24 1.54 21 3 0 7 14 1 2 0 0 Türkiye UM 19 1.62 13 6 0 7 6 6 0 0 0 Total 71 6.40 51 19 1 23 28 15 4 1 0 Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = Inter- national Finance Corporation; LM = lower-middle; MIGA = Multilateral Investment Guarantee Agency; NH = non-honoring guarantees; PRI = political risk insurance; UM = upper-middle. Summary The countries selected as potential cases of interest for the evaluation had high vulnerability and high Bank Group engagement. Specifically, they met these two criteria: (i) high vulnerability (a value of 50 or higher in the chosen dimension) and (ii) high Bank Group engagement (top two quartiles). Fig- ure A.10 lists the countries selected as potential countries of interest when applying each criterion. The numbers on the top of the matrix (1 to 7) refer to each of the mapping criteria described in the previous pages. The numbers inside the matrix (0 or 1) indicate whether each country was picked (1) or not (0) by each mapping criterion. Figure A.10. Case Selection Matrix Independent Evaluation Group World Bank Group    115 Source: Independent Evaluation Group. References Bellavita, Christopher, Joseph S. Wholey, and Mark A. Abramson. 1986. “Perfor- mance-Oriented Evaluation: Prospects for the Future.” In Performance and Credibility: Developing Excellence in Public and Nonprofit Organizations, edited by Joseph S. Wholey, Mark A. Abramson, and Christopher Bellavita. Lexington: Lexington Books. Patton, Michael Quinn. 2005. “Qualitative Research.” In Encyclopedia of Statistics in Behavioral Science, edited by Brian S. Everitt and David C. Howell. Chichester, UK: John Wiley & Sons. https://doi.org/10.1002/0470013192.bsa514. World Bank. 2020. Saving Lives, Scaling-up Impact and Getting Back on Track: World Bank Group COVID-19 Crisis Response Approach Paper. Washington, DC: World Bank. http://documents1.worldbank.org/curated/en/136631594937150795/pdf/ World-Bank-Group-COVID-19-Crisis-Response-Approach-Paper-Saving-Lives- Scaling-up-Impact-and-Getting-Back-on-Track.pdf. World Bank. 2022. The World Bank’s Early Support to Addressing COVID-19: Health The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix A and Social Response. Independent Evaluation Group. Washington, DC: World Bank. 116 Appendix B. Adaptive Management and Crisis Learning Framework We used the adaptive management and crisis learning analytical frame- works to assess the quality of the World Bank Group’s early response to the COVID-19 crisis. Adaptive management is collecting, sharing, and studying data from ongoing efforts to identify and understand the drivers of results, then adapting processes to continually improve an organization’s ability to achieve the desired results (Moynihan 2009). Adaptation should occur both during execution (adjusting course based on evidence) and during planning (setting the course for new efforts based on evidence). Crisis learn- ing is adaptive management applied to the management of crises. Crisis learning introduces the concept of ad hoc adaptation in the early stage of a crisis, when first responders adapt to the unexpected situation in real time absent prior institutional learning because of the unprecedented nature of the crisis. The crisis response literature calls this stage I ad hoc adaptation, and it is a subject of this evaluation, which assesses how well the Bank Group managed its response in the earliest stages of the crisis—the acute crisis phase (the first part of the evaluation period)—when the scope of the crisis and its comparability to previous crises were unknown (figure B.1). Stage II, intracrisis learning, refers to the learning that occurs during a crisis, includ- Independent Evaluation Group World Bank Group    117 ing assessments of current performance and efforts to improve responses to the current incident—a form of course correction in adaptive management. In contrast to stage I, it draws on and contributes to organizational knowl- edge about crisis response. It is also a subject of this evaluation, which as- sesses how well the Bank Group managed its response to the evolving crisis as its scope and its similarities to and differences from previous crises were becoming clear during the incipient recovery phase (the second part of the evaluation period). Stage III, intercrisis learning, involves learning from past crisis experiences and drawing lessons to be prepared for future incidents. It is a form of course setting in adaptive management. Because of its long-term nature, it is not a subject of this evaluation. Figure B.1. Adaptive Management and Crisis Learning The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Appendix B Source: Independent Evaluation Group. Effective adaptive management and crisis learning has five characteristics: 1. First, an institution needs to maintain what works well—continue best practices and lessons learned from prior crises. An organization that maintains what works well will show evidence of a countercyclical re- sponse and the use of real-time diagnostics and analytics. Although the Bank Group response and the objectives of its projects vary by country and phase of the crisis within the country, the Bank Group’s overall support efforts should be underpinned by its comparative advantages,1 including its knowledge of the affected sectors, the experience it acquired in dealing with previous crises, and its ability to convene and work with partners. 2. Second, an institution needs to expand resources as necessary to address the crisis—address resource constraints on the response. An organization that expands its response might show adaptation in procure- ment, human resources, and new instruments to facilitate quick response. It would show evidence of process change that facilitated faster actions, such as disbursements or behavior change and monitoring. 3. Third, an institution needs to pivot to address major new challenges brought on by the crisis. The economic effects of the COVID-19 crisis hit 118 contact-intensive industries harder than others. Because of the high trans- missibility of the disease, lockdowns focused on industries that involved close personal interaction. Contact-intensive industries—such as retail and hardware, hotels and restaurants, education, hospitals, transportation, construction, and leisure and recreation—were most affected. In these in- dustries, lockdowns and job losses were widespread. Noncontact-intensive industries such as banking, manufacturing, and telecommunications were also less affected by lockdowns and their economic consequences. As a consequence, we would expect that the Bank Group response would begin to disproportionately favor contact-intensive industries as these distinc- tions became clear. 4. Fourth, an institution needs to innovate—respond to create new, large- scale effects. Innovations relevant to the Bank Group response might include, for example, digitalized and targeted cash transfers or new in- struments to address novelty of the crisis. An organization that innovates would show evidence of new or changed processes or project designs. 5. Fifth, in a crisis, it is usually necessary to increase risk appetite. A crisis, by definition, exceeds the capacity of the normal mechanisms for maintaining equilibrium. Responding effectively usually requires some risk-taking. The COVID-19 pandemic affected high-risk countries dispro- portionately because of their generally low preparedness for systemic eco- nomic shocks. We would therefore expect that the Bank Group response to Independent Evaluation Group World Bank Group    119 COVID-19 would favor higher-risk countries more than the prepandemic portfolio. Furthermore, the Bank Group aspired to provide support in its early response that was exceptional in speed, scale, and selectivity (World Bank 2020). This implies an increase in Bank Group risk tolerance, which would affect project design and implementation, requiring innovation, fast-track delivery, frequent monitoring of progress, and project restruc- turing to ensure course correction based on early learning. The evaluation assesses the extent to which the Bank Group response to the COVID-19 economic crisis exhibited these five characteristics. References Moynihan, Donald P. 2009. “From Intercrisis to Intracrisis Learning.” Journal of Con- tingencies and Crisis Management 17 (3): 189–98. https://doi.org/10.1111/j.1468- 5973.2009.00579.x. World Bank. 2020. Saving Lives, Scaling-up Impact and Getting Back on Track: World Bank Group COVID-19 Crisis Response Approach Paper. Washington, DC: World Bank. http://documents1.worldbank.org/curated/en/136631594937150795/pdf/ World-Bank-Group-COVID-19-Crisis-Response-Approach-Paper-Saving-Lives- Scaling-up-Impact-and-Getting-Back-on-Track.pdf. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response  Appendix B 120 1  There is no simple agreed-on terminology for referring to “what the World Bank Group does” as distinct from “what firms and governments do” in development. The Organisation for Economic Co-operation and Development’s Development Assistance Committee Network on Development Evaluation uses the term intervention to cover both cases, but this may obscure important differences between the two. In this evaluation report, we use three terms. (i) Ac- tions refer to the things that firms and governments do, such as designing and implementing projects. (ii) Interventions refer to the things the Bank Group does to directly support govern- ments and firms, such as providing loans and technical assistance. (iii) Support efforts refer more broadly to any kind of effort the Bank Group makes, including not only interventions but also, for example, knowledge sharing and partnering. Independent Evaluation Group World Bank Group    121 Appendix C. Country Needs Analysis This appendix presents the list of nine sectors, indicators, and data sources used to calculate the need scores at the country level (table C.1). Robustness checks on the needs analysis are presented further in table C.2. Table C.1. Need Score Indicators and Data Sources Sector Indicator Description Source Education Teachers Pupil-teacher ratio, secondary World Bank Expenditure Government expenditure per student, World Bank The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix C secondary (percent of GDP per capita) Enrollment School enrollment, secondary (percent World Bank net) Health Hospital Hospital beds (no. per 1,000 people) World Bank beds Medical staff Physicians and nurses (no. per 1,000 World Bank people) Deaths Average 14-day notification rate of re- European ported deaths per million population Centre for Disease Pre- vention and Control Social protection Poverty Poverty head count ratio at $5.50 a day, World Bank, 2011 PPP (percent of population) WDI Inequality Gini index (World Bank estimate) World Bank, WDI Unemploy- Unemployment, total (percent of total World Bank ment labor force) Public finance Medium- Composite index based on various fac- World Bank, term tors, such as government gross financ- Vulnerability vulnerability ing needs (percent GDP), debt, interest Index expense, and debt service ratio Financial system Regulatory Regulatory capital to risk-weighted as- IMF, FSI capital sets (percent) Loans Nonperforming loans net of provisions to IMF, FSI 122 capital (percent) (continued) Sector Indicator Description Source Liquidity Liquid assets to total assets (percent) IMF, FSI Foreign Net open position in foreign exchange to IMF, FSI exchange capital (percent) Economic fitness Recession Change in the GDP growth rate in 2020 World Bank compared with the average growth rate in 2017–19 Agriculture, manufacturing, services Sales Percentage of establishments with World Bank, decreased monthly sales compared with BPS and ES year before Arrears Share of establishments in arrears or World Bank, expected to fail in the next six months BPS and ES Workers Share of establishments that fired work- World Bank, ers in the past 30 days BPS and ES Wages Share of establishments that reduced World Bank, wages in the past 30 days BPS and ES Agriculture Agriculture, forestry, and fishing, value World Bank, share added (percent of GDP) WDI Manufactur- Manufacturing, value added (percent of World Bank, ing share GDP) WDI Services Services, value added (percent of GDP) World Bank, share WDI Source: Independent Evaluation Group. Note: BPS = COVID-19 Business Pulse Survey; ES = Enterprise Survey; FSI = Financial Soundness Indica- tors; GDP = gross domestic product; IMF = International Monetary Fund; PPP = purchasing power parity; Independent Evaluation Group World Bank Group    123 WDI = World Development Indicators. The findings on needs analysis are robust to moderate changes in the con- struction of the need scores. Specifically, table C.2 reports the results of estimating the regressions constructed based on a subset of the considered dimensions. In columns 1 and 2 of table C.2, the need score is construct- ed without the education dimension; in columns 3 and 4, it is constructed without the health dimension; and so on (as indicated in the table head- ing). Across all alternative specifications, the results are very similar to the baseline estimates (with all dimensions). In particular, the results in the even-numbered columns in table C.2 indicate that the finding that Bank Group support was not strongly targeted at countries with greater needs is not driven by a single dimension of the constructed need scores. This result on the relevance of the Bank Group’s response to COVID-19 applies only to the macro level (that is, the allocation of support across countries). It is still possible that at the micro level, the Bank Group’s early response to COVID-19 successfully targeted the sectors with the greatest needs within each country. The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix C 124  obustness of Regression Results—Dependent Variable: World Bank Group Support Table C.2. R (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Need score 0.035*** 0.010 0.025** −0.001 0.032** 0.012 0.037*** 0.019 0.035*** 0.011 0.029*** −0.001 (0.009) (0.522) (0.039) (0.921) (0.015) (0.419) (0.005) (0.254) (0.004) (0.482) (0.008) (0.968) GDP per capita −0.00** −0.00** −0.00** −0.00** −0.00** −0.00** (0.039) (0.022) (0.047) (0.057) (0.058) (0.026) Region FE no yes no yes no yes no yes no yes no yes Observations 107 105 107 105 107 105 107 105 107 105 107 105 R-squared 0.062 0.199 0.040 0.195 0.055 0.201 0.071 0.206 0.076 0.199 0.065 0.195 Source: Independent Evaluation Group. Note: p values are in parentheses. Estimated via ordinary least squares (some specifications include region fixed effects). High-income (nonclient) countries and countries with World Bank Group support (amount/GDP) greater than 0.1 are excluded. The dependent variable is the amount of World Bank Group support received per dollar of GDP. The need score is a normalized composite measure constructed using the methodology described in appendix A. FE = fixed effects; GDP = gross domestic product. *p < 0.10 **p < 0.05 ***p < 0.01. Independent Evaluation Group World Bank Group    125 Appendix D. World Bank Group Reporting to the Board of Executive Directors  tylized View of the World Bank Group’s Reporting to the Box D.1. S Board of Executive Directors, February–September 2020 February » February 4: Board: Oral Briefing on Coronavirus and Human Development presen- The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix D tation of “The Novel Coronavirus Outbreak” (Chair: MDCFO) March » March 3: Board: Update on World Bank Group Response to COVID-19. Paper on “Briefing Note on Launch of World Bank Group Fast-Track COVID-19 Facility” (Chair: World Bank Group President) » March 10: COW: The Implications of COVID-19 Pandemic on Public Debt and the Need for Sound Borrowing and Lending Practices for Sustainable Development (Chair: World Bank Group President) » March 17: Board: “Proposal for a World Bank Fast-Track COVID-19 Facility” (Chair: MDOPS, MDCOO) » March 25: Board: Additional Response to COVID-19 Addressing the Economic and Social Implications. Paper on “World Bank Group’s Proposed Operational Re- sponse to the Economic Impact of COVID-19” (Chair: World Bank Group President) April » April 2: Board: COVID-19 SPRP utilizing the MPA (25 ops) (Chair: MDOPS) » April 2: (AOB) West Bank and Gaza COVID-19 Emergency Response under the SPRP MPA » April 7: COVID-19 SPRP Weekly Fact Sheet #1 126 (continued)  tylized View of the World Bank Group’s Reporting to the Box D.1. S Board of Executive Directors, February–September 2020 (cont.) » April 9: Board: COVID-19 Implementation (Chair: MDOPS) » April 15: COVID-19 SPRP Weekly Fact Sheet #2 » April 17: Development Committee: Update on World Bank Group Response to the COVID-19 Emergency » April 22: COVID-19 SPRP Weekly Fact Sheet #3 » April 23: Board: Second Oral Briefing on COVID-19 by MDOPS » April 24: Board: Proposed Investment in GTFP and Real Sector Envelope COVID-19 Crisis Response (Chair: MDOPS, MDCOO) » April 28: COVID-19 SPRP Weekly Fact Sheet #4 » April 30: (AOB) Update on IDA Contribution to COVID-19 Pandemic Response May » May 5: Technical Briefing: IBRD Financial Sustainability Framework Enhancing IBRD’s Ability to Respond to COVID-19 (Chair: CRO, OPCS, DFI) » May 5: COVID-19 SPRP Weekly Fact Sheet #5 Independent Evaluation Group World Bank Group    127 » May 12: COVID-19 SPRP Weekly Fact Sheet #6 » May 20: COVID-19 SPRP Weekly Fact Sheet #7 » May 22: (AOB) Indonesia COVID-19 Emergency Response under the SPRP MPA » May 26: (AOB) Islamic Republic of Iran COVID-19 Emergency Response under the SPRP MPA » May 27: COVID-19 SPRP Weekly Fact Sheet #8 » May 28: Board: FY20 Q3 Operations Update to the Board and COVID-19 Oral Up- date (Chair: MDOPS) (continued)  tylized View of the World Bank Group’s Reporting to the Box D.1. S Board of Executive Directors, February–September 2020 (cont.) June » June 2: COVID-19 SPRP Weekly Fact Sheet #9 » June 10: COVID-19 SPRP Weekly Fact Sheet #10 » June 17: COVID-19 SPRP Weekly Fact Sheet #11 » June 22: (AOB) Sri Lanka COVID-19 Emergency Response AF under the SPRP MPA » June 23: Board: Paper on World Bank Group COVID-19 Crisis Response Approach Paper: Saving Lives, Scaling up Impact and Getting Back on Track (Chair: World The World Bank Group’s Early Support to Addressing the COVID-19 Economic Response Appendix D Bank Group President) » June 25: COVID-19 SPRP Weekly Fact Sheet #12 » June 30: COVID-19 SPRP Weekly Fact Sheet #13 July » July 9: COVID-19 SPRP Weekly Fact Sheet #14 » July 14: COVID-19 SPRP Weekly Fact Sheet #15 » July 22: COVID-19 SPRP Weekly Fact Sheet #16 » July 30: COVID-19 SPRP Weekly Fact Sheet #17 August » August 4: Board: FY20 Q4 Operations Update to the Board and FY20 Retrospec- tive (Chair: MDOPS) » August 6: Technical Briefing: “COVID-19 Resource Allocation” (Chair: CRO, OPCS, DFI) » August 6: (AOB) Nigeria COVID-19 Emergency Response under the SPRP MPA » August 31: COVID-19 SPRP Weekly Fact Sheet #18 (continued) 128  tylized View of the World Bank Group’s Reporting to the Box D.1. S Board of Executive Directors, February–September 2020 (cont.) September » September 9: COVID-19 SPRP Weekly Fact Sheet #19 » September 15: COVID-19 SPRP Weekly Fact Sheet #20 » September 17: COW: Paper on “Leaning Forward to Save Lives, Scale up Impact and Get Back on Track: World Bank Group COVID-19 Crisis Response Update” (Chair: World Bank Group President) » September 21: COVID-19 SPRP Weekly Fact Sheet #21 » September 28: ED’s Seminar: 2020 World Bank Group Operations and Direction: The Pandemic’s Deepening Impact on Developing Countries (Chair: MDOPS, MDDPP) Source: Fiscal year 2022 second quarter World Bank Operations Update. Note: AF = additional financing; AOB = absence-of-objection; COW = committee of the whole; CRO = chief risk officer; DFI = development finance institution; ED = economic development; FY= fiscal year; GTFP = Global Trade Finance Program; IBRD = International Bank for Reconstruction and Develop- ment; IDA = International Development Association; MDCFO = managing director and chief financial officer; MDCOO = managing director and chief operating officer; MDDPP = managing director, devel- opment policy and partnership; MDOPS = managing director, operations; MPA = Multiphase Pro- grammatic Approach; OPCS = Operations Policy and Country Services; Q = quarter; SPRP = Strategic Preparedness and Response Program. Independent Evaluation Group World Bank Group    129 The World Bank 1818 H Street NW Washington, DC 20433