The World Bank’s Early Support to Addressing COVID-19 Health and Social Response An Early-Stage Evaluation © 2022 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org ATTRIBUTION Please cite the report as: World Bank. 2022. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response. An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. COVER PHOTO Shutterstock / Kelly Ermis. EDITING AND PRODUCTION Amanda O’Brien GRAPHIC DESIGN Luísa Ulhoa Rafaela Sarinho This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The bound- aries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissem- ination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response An Early-Stage Evaluation November 15, 2022 Contents Abbreviations vii Acknowledgments viii Overview x Management Response xxii Report to the Board from the Committee on Development Effectiveness xxxiv 1. Introduction��������������������������������������������������������������������������������������������������������������1 Response 2 Evaluation Purpose and Scope 3 Evaluation Design 10 Methodology 13 Quality of Response: Relevance����������������������������������������������������������������������������20 2.  Addressing Health and Social Needs 21 Building on Human Capital Capacities 29 Reorientation of the World Bank Portfolio to Respond to Needs 32 Integration of Support for Institutional Strengthening and Recovery 38 Quality of Response: Early Successes, Challenges, Learning, and Adjustment�������44 3.  Implementation Status and Facilitating Factors 46 Building on Lessons and Evidence from Past Crises 56 Innovation and Learning 60 Coordination, Dialogue, and Adjustment in Countries 63 Regional Knowledge Sharing and Cooperation 69 4. Quality of the Response: Operational Policies and Partnerships������������������������� 74 Internal Coordination 76 Instruments Supporting COVID-19 Crisis Response, Streamlined Processes, Corporate Requirements, and Procurement 82 Monitoring and Reporting 97 Partnerships to Facilitate Response 103 Vaccine Support 110 ii Conclusions and Way Forward�����������������������������������������������������������������������������118 5.  Bibliography������������������������������������������������������������������������������������������������������������� 123 Boxes Three Questions That Guide the Evaluation Box 1.1.  8 The Logic of the Conceptual Framework Box 1.2.  12 Evaluation Components at Each Level Box 1.3.  16 Examples of Health and Social Support for the Early COVID-19 Response Box 2.1.  23 Key Areas to Strengthen Preparedness to Address Needs Box 2.2.  in Crisis Response 27 Alignment of COVID-19 Support with Health Response Box 2.3.  31 Country Situations Regarding Prioritization of Box 2.4.  COVID-19 Response Actions 35 Examples of Institutional Strengthening Support in COVID-19 Response 39 Box 2.5.  Box 3.1. Examples of Early Results from Case Study Countries 50 Examples of Successes and Challenges of Early COVID-19 Support  Box 3.2.  54 Intervention Areas with Positive Evidence and Areas for Learning Box 3.3.  59 Box 3.4. Examples of Innovations Supporting the COVID-19 Response 62 Lessons on Multisector Coordination for an Integrated Response Box 3.5.  64 Examples of Real-Time Data Systems and Tools for Decision-Making Box 3.6.  67 Box 3.7. Examples of Early Results Contributed by Regional Projects 70 Box 4.1. Cross-Sectoral and Unit Teams Supported Internal Innovations 77 Box 4.2. Examples of Instrument Use 84 Box 4.3. Examples of Indicators Monitoring the Response 100 Examples of the Global Mobilization of Data and Knowledge Resources Box 4.4.  103 Examples of Global Partnerships Supporting COVID-19 Response Box 4.5.  104 Box 4.6. Vaccination in Mozambique 114 iii Figures Figure 1.1. Global Milestones and Timeline of World Bank Group Response 4 Conceptual Framework for COVID-19 Health and Social Response Figure 1.2.  11 Theory of Action to Assess the Quality of the Early COVID-19 Response Figure 1.3.  15 Figure 2.1. Dimensions Assessed for Quality of Support to Need 21 Figure 2.2. Areas of Health and Social Response Support in Countries 22 Figure 2.3. Alignment of Project Portfolio with Identified Country Needs 26 Figure 2.4. Extent of Gender Equality Support in Country Portfolios 29 Figure 2.5. Extent of Portfolio Reorientation in Countries 33 Dimensions Assessed for Quality of Implementation and Learning Figure 3.1.  46 Figure 3.2. Implementation Progress Ratings of Projects in Countries 47 Factors Important to Satisfactory Implementation of Country Support 48 Figure 3.3.  Coverage of Country Support to Areas Important Figure 3.4.  to Facilitating Satisfactory Implementation 49 Intended Beneficiaries of World Bank Country Support Figure 3.5.  by Global Practice 53 Areas of Implementation Successes and Challenges Figure 3.6.  in COVID-19 Response 58 Dimensions Assessed for Quality of World Bank Operational Figure 4.1.  Processes and Partnerships to Support Country Responses 76 Global Practices Contributing to Projects for Early Health Figure 4.2.  and Social Response 81 Mix of Instruments Used by Global Practices to Support Figure 4.3.  the Response 83 Figure 4.4. Crisis Instruments in Country Portfolios 87 Disbursement of COVID-19 Resources by Instrument and Time Figure 4.5.  88 Operational Process–Related Success and Figure 4.6.  Challenges Reported by Projects 92 Figure 4.7. Cumulative Procurements for Goods by Category, by Date 96 Country-Level Alignment of Indicators to Measure Figure 4.8.  the COVID-19 Response 98 Figure 4.9. Extent of Monitoring in Countries by Response Area 99 Countries by Extent of Level of Progress on COVID-19 Figure 4.10.  Response Indicators 102 iv Tables Examples of Advisory Services and Analytics Supporting the Response37 Table 2.1.  Application of Operational Lessons from Past Crises in COVID-19 Table 3.1.  57 Constraints and Opportunities of Instruments in COVID-19 Table 4.1.  Response by Timing of Financing 89 Summary Timeline of World Bank Vaccine Response Table 4.2.  112 v Appendixes Appendix A. Methodology 134 Appendix B. COVID-19 Portfolio Analysis 152 Appendix C. Case Study Findings 186 Review of Country Situations: Analysis of Country Support Types, Appendix D.  Needs, and Implementation Status 231  apid Review of Evidence on What Works in a Crisis and Alignment Appendix E. R with the COVID-19 Response 257 Review of World Bank Successes and Challenges from Past Crises to Appendix F.  Inform the COVID-19 Response 283 Appendix G. Review of COVID-19 Support by Regional Projects 294 Appendix H. Analysis of Multiphase Programmatic Approach of Health 315 Abbreviations Africa CDC Africa Centres for Disease Control and Prevention ASA advisory services and analytics AVATT African Vaccine Acquisition Task Team CAT DDO catastrophe deferred drawdown option CERC Contingency Emergency Response Component COVAX COVID-19 Vaccines Global Access Facility DPF development policy financing FCS fragile and conflict-affected situation GP Global Practice GPE Global Partnership for Education HNP Health, Nutrition, Population IDA International Development Association IEG Independent Evaluation Group IPF investment project financing MPA Multiphase Programmatic Approach PEF Pandemic Emergency Financing Facility PPR prevention, preparedness, and response SMS short messaging service SPJ Social Protection and Jobs Independent Evaluation Group World Bank Group    vii UNICEF United Nations Children’s Fund WHO World Health Organization All dollar amounts are US dollars unless otherwise indicated. Acknowledgments This evaluation was prepared by an Independent Evaluation Group team led by Jenny Gold, senior evaluation officer, and Stephen Porter, senior mon- itoring and evaluation officer, under the overall direction of Alison Evans, Director-General, Evaluation, and with the guidance and supervision of Galina Sotirova, manager, Corporate and Human Development, and Oscar Calvo-Gonzalez, director, Human Development and Economic Management. Santiago Ramirez Rodriguez was responsible for the portfolio analysis, with The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Acknowledgments support from Brian Allen, Mohammad Anas, and Harsh Anuj. Harsh Anuj, Yingjia Liu, and Stephen Porter led text analytics of past project lessons and procurements. Case studies were conducted by Jenny Gold, Elissar Harati, Aliza Inbal, Victor Malca, Eduardo Fernandez Maldonado, Lourdes Pagaran, Stephen Porter, and Denise Vaillancourt, with support from Emilia Alduvin, Mamka Anyona, Pooja Churamani, Ventura Mufume, Louise Pierre, Amadou Hassane Sylla, Robert Waswaga, and Maribelle Zonaga. Mamka Anyona prepared the rapid review of evidence. Dawn Roberts pre- pared the innovation stocktaking, review of monitoring, and analyses of advisory services and analytics. Dawn Roberts also provided advice on the evaluation methods. Ryoko Sato supported the heat map analysis, and Luis Da Silvia supported the clustering and decision tree analysis. Goele Scheers and Richard Smith led the regional project analysis, with support from Elissar Harati. Rasmus Heltberg led the review of operational processes and partnerships, with support from Mamka Anyona and Neeta Sirur. Estelle Raimondo provided overall methods support. Gaby Loibl provided administrative support to the team. William Hurlbut and Janet MacMillan edited the report, and Sharon Fisher provided design and editorial support. The peer reviewers for this evaluation were Tamar Manuelyan Atinc (senior fellow at Brookings Institution and former vice president for human development at the World Bank), David Zakus (adjunct professor at the University of Toronto, former director of global health at the Canadian Public Health Association, and founding director of the Centre for Global viii Health at the University of Toronto), and Marine Buissonnière (independent researcher in epidemic preparedness, and response and former director of public health at the Open Society Foundations). The advisory panel for this evaluation was composed of Richard Seifman (former senior health adviser at the World Bank), Bruno Marchal (professor and evaluation methods expert at the Institute of Tropical Medicine, Antwerp), and Shanta Devarajan (professor at Georgetown University and former acting chief economist at the World Bank). The team is grateful to all the staff who generously shared documents, insights, and experiences and engaged with us throughout the evaluation. The team acknowledges the country offices of Djibouti, Honduras, India, Mozambique, the Philippines, Senegal, Tajikistan, and Uganda for their support during the case studies, and the program leaders and staff who shared innovations. Independent Evaluation Group World Bank Group    ix Overview This evaluation assesses the quality of the World Bank’s early response to the COVID-19 crisis and the initial steps toward recovery, focusing on the health and social response. It concentrates on the relief stage and support to restruc- ture systems in the first 15 months of the pandemic (February 1, 2020, to April 30, 2021) in 106 countries. A parallel Independent Evaluation Group evalua- tion looks at the World Bank Group support to address the economic implica- tions of the pandemic. To assess the quality of the response, the evaluation is guided by a theory of action that synthesizes evidence in three dimensions: relevance of support to the needs of countries; implementation, learning, and adjustment; and operational policy and partnerships to support smooth The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Overview responses in countries. As the response is ongoing, the evaluation does not assess effectiveness but considers early results and pathways that are expected to lead to outcomes. Main Findings In a context of high uncertainty, the World Bank delivered a response of unprecedented scale and speed. The immediate support was particularly swift in the most vulnerable countries. In the first 15 months, the World Bank pro- vided financing of an estimated $30 billion for the health and social response in 106 countries with high or medium vulnerability to human capital and development losses. Support to small states, less-prepared countries, and fragile and conflict-affected situations was emphasized. About 20 percent of financing was disbursed in the first months of 2020, and 40 percent was dis- bursed by April 2021. Staff and clients worked long hours to deliver new and repurposed operations, all while learning to use remote connectivity tools and adapting to home-based work and personal stresses. Relevance of Support to Country Needs The evaluation looks at how well the World Bank responded to the immedi- ate health threat of COVID-19, how well it focused on protecting vulnerable groups against human capital losses, and how well it integrated institutional x strengthening in the relief stage to help sustain preparedness and resilience postcrisis. The World Bank support was relevant to the needs of countries and well aligned with most emergency areas in their COVID-19 responses. Emergency support expanded critical health services to prevent and control the spread of disease, including infection prevention and control, case management, surveillance, and provision of laboratories. The support prioritized social protection for poor and vulnerable people. For example, Djibouti, Hondu- ras, India, Senegal, and Tajikistan expanded emergency health and social protection actions through World Bank operations. World Bank support in countries aligned well with national COVID-19 plans of governments, which coordinated emergency support of development partners to the response. World Bank support addressed country needs most comprehensively where earlier work on human capital had built preparedness and where cross- sectoral coordination among Global Practices (GPs) and sectors in countries was stronger. Knowledge and relationships developed before COVID-19 helped reorient country portfolios in human development and other sectors to accommodate newly emerged needs. For example, in Uganda, the response built on existing relationships in health, education, water, agriculture, and nutrition. In the Philippines, new relationships needed to be developed in health, initially slowing the early response, while work before COVID-19 in social protection and community development enabled the rapid expansion Independent Evaluation Group World Bank Group    xi of cash transfer programs and support in communities. Coordination across sectors was weak in most countries. However, where coordination was stronger (for example, in India and Senegal), it helped quickly mobilize a range of GPs and sectors in the country to address needs related to testing, surveillance, laboratories, social protection, child learning, and nutrition, and involved women’s groups and the informal sector. The early months of the World Bank response had a strong emergency focus, followed in about half of countries with efforts to protect human capital. The World Bank’s knowledge work on gender, epidemic preparedness, supply chains, social protection, and behavior change communication helped prioritize actions in some countries (Djibouti, Honduras, India, and Uganda). Strong government leadership helped some countries rapidly adapt World Bank support to both emergency and human capital needs. In the remaining half of countries, less attention was given in the first 15 months to continu- ing maternal and child health and education services, protecting women and girls from the shock of COVID-19, and engaging communities. The challenge of responding to urgent needs while protecting human capital was especially acute in countries with weaker systems for rapid health response and extensive human capital vulnerabilities (such as Chad and Niger). Integrating institutional strengthening in the early COVID-19 response helped focus on sustaining public health preparedness and building resil- ience in health, education, and social protection systems. In more than 90 percent of countries, institutional strengthening was part of World Bank support. For example, in countries such as India, the Philippines, and Tajikistan, the World Bank helped strengthen and rapidly expand social The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Overview protection systems, often to a national scale. In Djibouti and Uganda, exten- sive support in education helped develop and strengthen remote learning networks. Honduras emphasized early health support to strengthen labora- tories. However, most countries still need strategies to sustain preparedness and ensure systems resilience after the crisis. Regional disease-focused projects, such as in Senegal, Zambia, and the countries of the Organisation of East Caribbean States, often helped countries to put better strategies in place for sustaining public health preparedness and to strengthen capacities in areas such as laboratories, testing, and case management. Early Successes, Challenges, Learning, and Adjustment The evaluation examines how well the World Bank supported implemen- tation and adjustment to ensure a strong response. It looks at how well the World Bank supported countries to achieve early results, built on past lessons and evidence, and introduced innovation. It also examines how the World Bank used dialogue and coordination, knowledge sharing through regional projects, and data to inform decisions and adjust the response. Although too early to observe outcomes, case studies provide promising evidence of early outputs that are key to satisfactory implementation and a good indicator that positive outcomes can be expected. Examples include the rapid expansion of critical health services, such as COVID-19 testing, xii social protection benefits, and remote learning for children. These interven- tions likely helped reduce the health threat of COVID-19 and protect human capital. About 40 percent of countries had projects that included monitoring data, and a mix of interventions that provide critical health services for disease prevention and control and limited coverage of interventions to protect human capital and engage communities. Broadly, the World Bank used its experience from past crises and existing knowledge about effective interventions. Most health projects built on past lessons and incorporated effective disease prevention and control interven- tions. For example, countries received widespread support for laboratories and infection prevention and control for COVID-19. At the same time, sup- port to local government and service providers, community-based interven- tions, and support to address gender equality (such as psychosocial care and sexual and reproductive health interventions) were limited, despite consis- tent evidence of effectiveness and lessons from past crises and risk commu- nication. The focus on broad national response was strong, and attention to local-level implementation challenges in reaching vulnerable groups was less prominent. The burden of the pandemic on frontline health workers was heavy, yet innovations in service delivery during the crisis were rare. In education, while there was often a focus on local learning, case studies suggest that countries faced challenges in supporting teachers and vulnera- ble children to continue learning during the crisis (India, Mozambique, Independent Evaluation Group World Bank Group    xiii and Uganda). In its effort to respond quickly and effectively, the World Bank innovated— its response included some form of innovation in more than 80 percent of countries. This evaluation found more than 200 examples of innovations supported by the World Bank in its COVID-19 response, such as for health communication and vaccine monitoring (Tajikistan), surveillance (Colom- bia), expanding cash transfers (the Democratic Republic of Congo), remote coaching of teachers (Lebanon), and multisector coordination (Haiti). In Senegal, community-based disease surveillance and multistakeholder en- gagement supported community health workers and volunteers to detect COVID-19 and report cases to health facilities and local government. In Mali, a new national call center provided advice for implementing COVID-19 pro- tocols. Global partnerships and knowledge sharing by regional projects were useful to successfully promote innovation. For example, the World Bank’s Education Technology team helped countries to expand remote learning; re- gional disease-focused projects helped expand country innovations in infec- tion prevention and control, point of entry control, testing, and surveillance. The World Bank engaged in frequent dialogue with governments and part- ners to coordinate and adjust implementation. Supporting government coordination to implement responses at the national and subnational levels worked best where there were country-led structures that predated COVID-19. Coordination structures facilitated dialogue on emerging needs, strengthened responses, and involved frontline services and communities for oversight, learning, communication, and problem-solving. One Health struc- tures, which coordinate multisectoral disease response actions, in Senegal and Zambia helped coordinate actions in health with other sectors. Subna- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Overview tional nutrition structures were key for COVID-19 messaging and for disease surveillance in Honduras, Senegal, and Uganda. Parent-teacher networks were important for supporting learning. New structures for coordination took time to set up during COVID-19, for example, in Haiti and the Philippines. Regional projects facilitated knowledge sharing and were particularly helpful for countries with limited capacity to respond independently to COVID-19. Regional projects supported technical cooperation (such as for planning and reporting on the response) between ministries and public health insti- tutes, encouraged leadership, developed human capacity, and coordinat- ed technical sharing and financing for COVID-19 responses in countries. Longer-running regional projects had more established networks, which had successfully built some preparedness before the pandemic to support COVID-19 responses, although even newer regional projects added value, mainly through convening and technical and learning support. The Econom- ic Community of West African States was wellprepared to support coun- tries during COVID-19, largely thanks to earlier support under the Regional Disease Surveillance Systems Enhancement Project. Despite being a newer organization, the Africa Centres for Disease Control and Prevention, sup- ported by a World Bank regional project, quickly developed convening struc- tures in Africa, such as for collaboration for disease surveillance, testing, and xiv vaccines. The Organisation of East Caribbean States Regional Health Project also quickly coordinated support for testing and case management. Few countries possessed real-time data systems and adequate data to inform decisions and adapt the response. Where they existed, diagnostics (Djibou- ti), geo-enabled monitoring (Tajikistan), iterative beneficiary monitoring, short messaging systems (Lesotho), online surveys (Tunisia), and dashboards (Colombia) supplied timely data to inform decisions, monitor behavioral change, and adjust actions. Where available, frequent data on the quality of health and education services in communities were critical for course cor- rections. Honduras and Uganda used remote supervision systems to monitor and improve local nutrition services during COVID-19. Tajikistan and Zambia used short messaging systems to track vaccine services and commu- nicate with teachers. Djibouti, India, the Philippines, Senegal, and Tajikistan used real-time survey data to adjust social protection responses. Operational Policies and Partnerships The assessment of the operational policies and partnership looks at how well the World Bank’s internal coordination, instruments for financing the COVID-19 response, and internal systems for reporting and monitoring supported the response. It looks also at the World Bank’s financing and technical partnership, including support of the Pandemic Emergency Financing Facility and support to vaccine financing. At the onset of the pandemic, Bank Group senior management demonstrat- Independent Evaluation Group World Bank Group    xv ed strategic and agile decision-making. Bank Group senior management articulated its approach early in March 2020 and delivered an Approach Paper to its Executive Directors in June 2020. This included front-loading International Development Association spending allocations and seeking an unprecedented International Development Association replenishment a year ahead of schedule, activating the International Bank for Reconstruction and Development’s crisis buffer to release additional financing, and aligning with the World Health Organization technical guidance on health issues. Within the World Bank, the Emergency Operations Center facilitated good internal coordination across GPs and operational support units, which was critical for action alignment and technical problem-solving. Policy guidance and knowledge sharing in GPs helped guide World Bank teams’ design projects in the early months of the response. Country portfolio reviews led by World Bank country management facilitated coordination of support across GPs and project teams in countries. To quickly process projects, managers in the health sector mobilized surge capacity involving retirees, exchanging staff, and increasing the responsibilities of country office staff. Wider engage- ment of GPs outside Human Development could have drawn on more staff resources and financing to help countries and coordinate efforts to process project support in the early months of the response. At the country level, having a pre–COVID-19 World Bank program with a good mix of instruments, including crisis instruments that could support timely financing in the first weeks of the crisis, facilitated a swift response. Crisis instruments, repurposed projects, regional projects, trust funds, and The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Overview grants, where available in country program portfolios, helped rapid financ- ing and just-in-time assistance in the early weeks and months of the crisis. Other instruments built on this support but often took longer to process: the Multiphase Programmatic Approach (MPA) was key to expanding new lending in more vulnerable countries for the health emergency response; development policy financing provided important funding for early sys- tems strengthening in areas where it could achieve quick wins by building on previous policy dialogue on human capital; and Pandemic Emergency Financing Facility grants supported COVID-19 plans and coordination with United Nations partners, although the small amounts of funding took time to process. The early responses in Senegal and Uganda relied on crisis in- struments, repurposed projects, and trust funds, which were complemented by development policy financing, Pandemic Emergency Financing Facility, and MPA support once available. Tajikistan used repurposed projects for its early COVID-19 response and then used the MPA financing in health when it became available. The World Bank introduced operational flexibility, which facilitated rapid processing of new financing for the MPA. This included shortened clearance times and delegation of approvals. The first MPA projects disbursed in about two months compared with about five months in previous crises. This quick timing was important because there was less reliance on additional financ- xvi ing compared with previous crises. Other new investment project financing projects took about five months to disburse, but in some countries, projects disbursed in less than one month. The procurement of medical goods early in the response also happened rapidly; from the first month of the response, personal protective equipment, test kits, and medical equipment were pro- cured for emergency use in countries by using World Bank–facilitated pro- cesses and hands-on assistance or enabling governments to use emergency procedures in projects. Despite the extensive support of safeguard teams, the new Environmental and Social Framework was challenging for new projects in the first months of the crisis, given that ministries were overwhelmed, and it required new learning. Requirements of citizen engagement and gender could have benefited from more hands-on assistance to help teams. It was challenging to collect timely data to report on the progress of support and track and coordinate procurement. Integrated reporting of data on various parts of the World Bank’s country-level COVID-19 response was important for discussions with governments, World Bank teams, partners, and headquarters. In India and other countries, the World Bank country office often lacked timely data to track implementation of projects to inform coordination. A key challenge was the difficulty in coordinating government procurement requests with other development partners in countries so the same items were not purchased. The tracking of goods—from ordering to shipping to arrival in health facilities—was also crucial though rare. Tracking was challenging, given the limited emergency preparedness of procurement Independent Evaluation Group World Bank Group    xvii systems in countries and lack of remote monitoring mechanisms. Having well-established partnerships with development organizations in place before the COVID-19 pandemic facilitated rapid action. For example, the Global Partnership for Education, where available, helped quickly ex- pand education support for children (such as in Uganda). In Mozambique, Senegal, and Uganda, partnership with the Global Financing Facility helped expand maternal and child health services and risk communication, though actions could have been quicker. In Tajikistan, the Global Partnership for Social Accountability supported efforts to involve civil society to monitor the COVID-19 response, and partnership with Gavi, the Vaccine Alliance sup- ported early planning to access vaccines. Existing country-level development partnerships enabled coordinated financing and actions for the response (for example, in India and the Philippines). Collaboration with nongovernmental organizations and the private sector in World Bank projects (such as in Belize, India, Peru, and Togo) helped expand community-based implementation, innovation, and use of technology and digital payments in social protection. In the uncertain early months, the World Bank made good efforts in engag- ing with partners to help prepare countries to deliver vaccines and expedite access, but the World Bank lacked an instrument to rapidly facilitate advance market commitments. In the first months of the pandemic, the Health, Nutrition, and Population GP convened global partners to explore ways to help low-income countries access vaccines. This was followed by intense internal dialogue about how the World Bank could best support vaccine readiness and access, focusing on supporting country-level efforts for vac- cines, given the lack of a global instrument to help finance advance market The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Overview commitments. Partnerships worked well at the country level, and the MPA fi- nancing for vaccines was timely. But implementation of vaccine support was initially slow because countries had limited health systems capacities to sup- port delivery, and they often could not access vaccines early in the response. The key was having access to financing for advance resource commitments, pooling resources with other partners in countries to support procurement, and aligning efforts in countries for vaccine safety and delivery. Earlier engagement with partners—namely, the COVID-19 Vaccines Global Access initiative and the African Vaccine Acquisition Trust—could have helped ensure advanced vaccine supplies for countries but also facilitated earlier preparedness and communication about vaccines. Overall, the quality of the early World Bank response was good. Looking ahead, a number of areas need attention by the World Bank and its clients: better preparedness of countries to deliver emergency services that reach local levels; more resilient systems in countries to protect health, education, and gender equality; improved support for cross-sectoral coordination; data for managing quality implementation; regional learning and cooperation; and stronger internal preparedness to respond quickly in a crisis, including coordination with partners. xviii Recommendations The findings from the evaluation inform four recommendations for ensuring stronger future preparedness. Recommendation 1. Use the World Bank’s crisis recovery efforts to strengthen the resilience of essential health and education services to en- sure that human capital is protected in a crisis. Proposed Actions » In health, build on innovations developed during COVID-19 to help countries strengthen telehealth and other platforms for continuing essential health services in an emergency. Help countries improve the quality of frontline services, including the availability of data to inform decisions for quality improvements. Services could be improved to better manage supplies, deliver vaccines, support health workers to deliver effective care, and ensure infection prevention and control measures. The availability and use of feedback from beneficiaries and coverage of vulnerable groups are also important. The World Bank could also help develop new capacities to deliver services, such as in psychosocial care. » In education, draw on evidence and innovations of the COVID-19 response to strengthen platforms for continuous learning in a crisis. Strengthen com- Independent Evaluation Group World Bank Group    xix munity networks that have been established to support learning. To avoid learning losses, facilitate knowledge building to uptake effective approaches to help children in and out of school catch up. Help countries increase the reach to vulnerable groups that may have been missed by remote learning. Strengthen monitoring of beneficiary feedback on the quality of learning. Recommendation 2. Apply a gender equality lens to health and social crisis response actions across sectors. Proposed Actions » Develop actions across sectors (in health, education, urban, and social pro- tection) for protecting women and girls from shocks, which can be drawn on in a crisis response. This is especially important in countries with high needs for addressing gender equality. Examples of areas to support gender equality include psychosocial support, sexual and reproductive health, income and asset accumulation, and community engagement. Recommendation 3. Help countries strengthen regional cooperation and crisis response capacities for public health preparedness. Proposed Actions » Support regional organizations to facilitate cooperation, political leader- ship, and technical learning, especially in Africa. Such support could help strengthen preparedness in countries and regional mechanisms for crisis response, facilitate financing and technical partnerships, encourage innova- tion, and expand evidence to scale up effective approaches. Regional support The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Overview could also facilitate evidence-based and data tools to help countries monitor crisis response actions. » Help countries strengthen national and subnational platforms to coordinate and deliver crisis interventions, such as One Health platforms, with great- er emphasis on critical health services and demand-side activities, such as citizen engagement. At the national level, invest in platforms that coordinate action and prepare various sectors to take on specific roles in crisis. At the subnational level, invest in platforms that can reach local government and communities for disease surveillance, risk communication, delivery of health and social services, and monitoring support. Recommendation 4. Build on the COVID-19 experience to strengthen the World Bank’s internal crisis preparedness so that it has the tools and proce- dures ready to respond in future emergencies. Proposed Actions » Review and expand operational flexibilities for processing new projects in crises and develop guidance on the effective use of instruments at different stages of crisis response. The World Bank could also explore innovative ways to strengthen the use of crisis instruments in countries, such as through support to communities, and expand guidance on hands-on assistance for xx citizen engagement and gender, learning from the provision of such support in procurement. » Expand and strengthen the World Bank’s partnerships and instruments to enable coordinated financing, advance market commitments, and technical support that will help countries strengthen crisis preparedness. The partner- ships could be at the global, regional, and country levels. They could include technical partnerships to expand knowledge for quality implementation of preparedness activities, partnerships with nongovernment and the private sector to support community-based implementation, feedback on services and use of technology, and global partnership for aligning financing, plans, and guidance to support countries. » Strengthen tools to allow for the integrated management and frequent re- porting of monitoring data on projects in World Bank portfolios. Independent Evaluation Group World Bank Group    xxi Management Response Management of the World Bank thanks the Independent Evaluation Group (IEG) for the opportunity to respond to the IEG report The World Bank’s Early Support to Addressing COVID-19: Health and Social Response—An Early-Stage Evaluation. Management appreciates IEG for supporting the World Bank’s COVID-19 response by offering just-in-time lessons and evaluative evidence to inform management’s choices. Management recognizes the usefulness of this evaluation for informing the World Bank’s efforts to support countries The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Management Response to prevent, prepare for, and respond to present and future crises. Global Leadership, Partnerships, and Learning Management welcomes the report’s recognition of the World Bank’s global leadership in the delivery of an unprecedented crisis response. The report underscores the importance of prior World Bank country engagement and the mix of instruments that enabled a swift response comprising new lend- ing and repurposed projects.1 The World Bank’s COVID-19 response has been, from multiple angles, an extraordinary one. The World Bank responded at speed and at scale in an unprecedented fashion. The World Bank » Completed pathbreaking analytical work to help understand how an un- known pandemic was evolving and impacting social and economic circum- stances, globally and at the country level; » Provided affected countries with a sharp increase in financing tailored to their circumstances, including by accelerating an International Development Association Replenishment by a full year; » Innovated on instruments, especially the health Multiphase Programmatic Ap- proach (MPA), which was the initial Fast Track COVID-19 Facility of $6 billion;2 » Achieved record commitments and disbursements; xxii » Made more financing for vaccines available than any multilateral development bank or international organization—this is especially true once the vaccine donations provided by countries that are members of the Organisation for Economic Co-operation and Development are excluded from the calculation; and » Built on lessons from past crises, including the need to employ a variety of instruments like the Contingency Emergency Response Component; re- purposed projects, including regional projects, trust funds, and grants; and guidelines on a variety of topics for projects in emergency situations. Management acknowledges IEG’s recognition that the flexibility of the World Bank’s processes for project financing improved in COVID-19 from past crises and allowed an agile and swift response that shortened time to disbursement by half compared with previous crises. Procurement plans for vaccine projects also disbursed nearly 10 times faster than other investment project financing health projects. Moreover, the World Bank continued to be a full-service development bank, addressing all key aspects of the pandemic and its impacts while maintaining a focus on its corporate commitments, including a large increase in climate financing and on preserving the basis for a resumption of progress toward long-term development outcomes. Management believes that the World Bank’s early coordination with global and regional partners on interventions in relation to vaccines were instru- Independent Evaluation Group World Bank Group    xxiii mental for the effectiveness of the response. From the beginning, the World Bank’s COVID-19 response recognized the centrality of vaccination for containing the pandemic—once an effective vaccine was available—but prior to vaccines becoming available in 2021, the interim emphasis of the re- sponse was on prevention, testing, treatment, and surveillance. Management notes the strength of its partnership with external partners, including with COVID-19 Vaccines Global Access (COVAX)3 on the following areas: (i) as- sessing countries’ readiness to deliver vaccines (World Bank 2021a) through the Vaccine Introduction Readiness Assessment Tool and Vaccine Readiness Assessment Framework (VRAF); (ii) streamlining the vaccine acquisition process by setting up a cost-sharing mechanism (World Bank 2021b) with COVAX that supported countries willing to procure doses in addition to free doses through COVAX; and (iii) monitoring countries’ capacity to deliver vaccines by monitoring, sharing, and coordinating information, data, and activity regarding vaccine availability and countries’ readiness through the Multilateral Leaders’ Task Force, which included the World Bank, World Health Organization (WHO), the International Monetary Fund, and the World Trade Organization. Management also appreciates the report’s ac- knowledgment in relation to the challenges of having a global instrument to support advanced market commitments for vaccines, but notes that the Gavi Alliance and World Bank do have a long track record of financial innovation at the global level, most notably through the International Finance Facility for Immunization, for which the World Bank serves as Treasury Manager. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Management Response This frontloading tool, which creates vaccine bonds through raising finance on capital markets backed by long-term donor pledges, meant that a glob- al mechanism was in place to raise additional funds through COVAX, and several donor countries made their contributions to the COVAX Advanced Market Commitment through this mechanism. The World Bank, through its cost-sharing mechanism developed with COVAX, allowed COVAX to make advance purchases from vaccine manufacturers based on aggregated de- mand across countries, using financing from the World Bank. Additionally, the World Bank’s efforts to strengthen regional capacity was one of the most notable highlights of the vaccine response, particularly the support offered to the African Union to make use of the African Vaccine Acquisition Task Team (AVATT) initiative. By the time the announcement was made, many of the 36 countries with approved vaccine operations had already formalized plans to procure vaccines through AVATT. Both COVAX and AVATT financing arrangements were part of the World Bank’s effort to ensure countries had flexibility in financing in alignment with country preferences. The World Bank, as a country-based model, demonstrated the ability to complement the other mechanisms, and specifically, to take advantage of the centralized procurement capacity of COVAX and AVATT. To date, 630 million vaccine doses have been purchased with World Bank financing ($6.5 billion) through a variety of procurement channels. Management believes that the efforts made pre-COVID-19 in helping strengthen regional capacity, especially in Africa, yielded results during the COVID-19 pandemic. Regional projects facilitated knowledge sharing and were particularly helpful for countries with limited capacity to respond xxiv independently to COVID-19. They also supported technical cooperation (such as for planning and reporting on the response) among ministries and public health institutes, encouraged leadership, developed human capacity, and coordinated technical sharing and financing for COVID-19 responses in countries. World Bank operations have also helped strengthen institutional capacity of Africa Center for Disease Control, and regional projects such as the World Bank project Regional Disease Surveillance Systems Enhancement have improved prevention, preparedness, and response (PPR) capacity using a One Health approach. The World Bank is building on these partnerships through its global PPR program, including the PPR Financial Intermediary Funds. Adaptive Management and Internal Coordination Management emphasizes its adaptive response in a context characterized by deep uncertainty and fluidity. As the crisis evolved and as new informa- tion became available, the World Bank’s response remained flexible and adapted continuously as country and regional needs evolved throughout the 15-month period of this review. The World Bank remained relevant by analyzing the evolution of the virus, enhancing its understanding of it, and calibrating its response to the changing external environment. As a global institution working across all regions in a context of high uncertainty and shifting landscape of vaccine development and regulatory approvals, man- agement had to recalibrate safeguards carefully and continuously in rela- Independent Evaluation Group World Bank Group    xxv tion to financing for vaccines. The early use of a waiver for the first vaccine project in Lebanon (as mentioned by the report), was critical for upholding both speed and safety. When more data became available, the World Bank aligned with WHO regulatory approvals, and the focus shifted to helping countries to navigate the severe supply constraints at the global level and working with countries to match supply and demand in a context where do- nation timelines were highly uncertain. Adaptive management is an essen- tial element of the World Bank’s outcome orientation. There was sufficient flexibility built into the World Bank’s operational policies and approaches, building on lessons learned from earlier crises, and the speed and agility with which these were triggered (allowing substantial additional commit- ments to be made within months of the WHO declaration that COVID-19 constituted a pandemic). Management highlights the contribution of the Social Protection and Jobs (SPJ) Global Practice as one of the most dynamic aspects of the World Bank’s COVID-19 response. During the period covered by the report, SPJ had the highest lending volume (in fiscal year [FY]21 $8,837 million, almost four times that of the Health, Nutrition, Population [HNP] Global Practice). In addition, significant effort was devoted to adapting existing operations and safety net programs, topping-up benefits to existing beneficiaries, or expanding the beneficiary pool without new lending. These efforts led to securing financial support to households to face health-related restrictions. As businesses closed and movement was restricted to essential services only, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Management Response social protection (through new or repurposed SPJ-led projects) provided essential intermediate income that allowed people to stay home instead of continuing to work and mingle, risking disease transmission. It should be noted that over 40 evaluations have demonstrated the effectiveness of SPJ programs in saving lives and protecting or enhancing well-being across a range of dimensions. A new “lessons learned” paper, recently released by SPJ, includes an overview of those evaluations (Gentilini 2022). Similarly, the upcoming Poverty and Shared Prosperity Report shows evidence from microsimulations for low- and middle-income countries showing that pov- erty would have been significantly higher without safety net responses, and that countries with better social protection systems were able to mitigate the impact of the pandemic more effectively. Management took unprecedented steps for an effective internal coordination to help manage its engagement globally and will reflect on ways to further improve for future crises. Management established coordination arrangements that permitted the delivery of the MPA in just three weeks—the fastest and largest response in the history of the World Bank. The success of the effort was a combination of top-down and bottom- up creativity of many teams across the institution that found innovative ways to quickly resolve challenges. Among the many actions taken, management highlights the adoption of streamlined processes and efforts for cross-fertilization; regular coordination meetings starting in February 2020 within the Human Development practice; and design of the global blueprint for the MPA Program, with close coordination of headquarters and field offices. Management also compiled operational updates from xxvi countries about restructuring ongoing projects and reallocating funds to support initial national responses. Moreover, management established the Emergency Operations Center as the engine of internal coordination. The Emergency Operations Center was established quickly with experienced staff; it shared information, held weekly cross-functional coordination meetings, resolved queries with a daily turnaround, and maintained regular communication between headquarters and country offices for the health sector.4,5 It was instrumental in coordinating operational responses and facilitating problem solving. Management also notes that the World Bank gained considerable experience engaging stakeholders and built on this experience over time, including using electronic platforms, stronger engagement with civil society networks to ensure governments’ accountability to citizens, and strengthening opportunities for citizen engagement through the World Bank’s Global Partnership for Social Accountability. Gender and Targeted Beneficiaries Management points out that the World Bank projects continued to focus on women, as teams were given the flexibility to waive the gender tag require- ments in the early stages of the COVID-19 response. While reviewing the FY20 MPAs as they were approved, based on the gender priorities detailed in Independent Evaluation Group World Bank Group    xxvii the Gender HNP Guidance Note, management observed that a good share of the MPA projects did consider gender issues to the extent possible, even if not all of them were able to specify project interventions due to the limited scope and time frame. Although the early response projects did not adopt entry points of gender that would have been used under “normal” circum- stances (such as psychosocial support or sexual and reproductive health), HNP’s COVID-19 projects acknowledged and responded to gender-based gaps directly related to the pandemic response, such as ensuring women received critical health information, training of female service providers, supporting countries in providing compensation packages to frontline workers (mostly women) who were at high risk, and psychosocial support for frontline workers. By April 2021, HNP, in collaboration with the Gender Group, also produced a second set of guidelines on reducing gender gaps in vaccine delivery for COVID-19, which includes recommendations for contin- uation of essential health services including sexual and reproductive health services, community engagement, and provision of psychosocial support. These recommendations are reflected in subsequent COVID-19 projects. The subsequent additional financing operations covered cross-sectoral issues more systematically, based on the learnings and lessons of the initial MPA operations along with a growing body of global evidence and data showing the impact of the pandemic on health, education, and social protection.6 By phase II of the pandemic response, the gender tag was resumed. From the early stages of the response, management provided multiple trainings to staff on identifying and addressing gender gaps in their pandemic response. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Management Response Key guidelines from this training have also been incorporated into the HNP Flagship Course (aimed at client countries) as part of the gender and health training module. Currently, as part of the gender strategy update, HNP is de- veloping its action plan, which covers pandemic preparedness and ensuring continuity of health services including sexual and reproductive health and psychosocial services along with community engagement. Management clarifies that the World Bank’s COVID-19 response targeted vulnerable populations from the onset. Although in health, vulnerability is a broad concept (it includes women, children, adolescent girls, poor peo- ple, farmers, and so on), the wider World Bank COVID-19 response target- ed vulnerable populations as more narrowly defined based on their risk of COVID-19 mortality and morbidity. This prioritization process was aided by the WHO Strategic Advisory Group of Experts Road map for prioritizing use of COVID-19 vaccines, which was referenced in project documents and which countries then adapted to their specific country needs. Key vulnerable groups were defined as people most at risk of COVID-19 infection, (severe) illness, and death, including health workers, and adults over 60, and people with comorbidities. Children were not prioritized, given their lower risk and lack of approval for COVID-19 vaccines early in the pandemic. Still, the SPJ projects succeeded in reaching 92 percent of the vulnerable population they intended to reach at design, including 95 percent of the intended women and children. These projects also reached the ”last mile” of vulnerable and marginalized beneficiaries from the outset, using existing platforms for behavior change communication (that usually accompanies cash transfers) to deliver COVID-19 xxviii messages. Given the broad impact of the crisis, universal programs have a greater likelihood of reaching most, if not all, vulnerable populations. That said, the decision regarding whether a program should be universal or targeted requires consideration of the trade-offs in coverage, cost, and efficiency. Management underscores that essential service delivery to meet human cap- ital needs was supported by the ongoing Human Development portfolio (not tagged as COVID-19 response) and through COVID-19 response operations. At the global level, the World Bank was one of the first large-scale develop- ment organizations to point to the risk and impact of disruptions in essential health services resulting from COVID-19. The report correctly notes that beyond its immediate impact on health outcomes (mortality and morbidity), the pandemic also had a dramatic impact through disruptions in essential health services, especially for maternal and child health and gender-related services.7 The World Bank’s ongoing portfolio of projects complemented the emergency response: the long-term investment portfolio in health systems amounted to $30 billion in more than 200 countries. The World Bank has been supporting countries to strengthen the resilience of essential health services, expand reach of telemedicine, strengthen data to inform decision-making, and strengthen citizen engagement.8 To mitigate drops in coverage rates of childhood immunization, the World Bank continues to work closely with Gavi Alliance partners to find ways to protect financing of routine vaccines in the highest risk countries, drawing on World Bank financing to comple- ment domestic financing through existing health projects when necessary. Independent Evaluation Group World Bank Group    xxix Furthermore, health systems strengthening, including preparedness planning for delivery of essential health services, was represented in about one-third of all MPA commitments. Some of the COVID-19 response investments un- der the MPA project had positive spillovers for the delivery of essential public health services for dealing with comorbidities that increased the risk of severe COVID-19 disease, hospitalization, and death.9 Examples include the Essential Health Services Recipient Executed Trust Fund grant program and the Global Financing Facility, launched toward the end of calendar year (CY)2020 to help incorporate support for essential health services into COVID-19 operations. Recommendations Management welcomes the report’s recommendations as the World Bank continues to scale-up its engagement in crises preparedness and response in a world affected by compounding crises, not just COVID-19. As stated in the Management Action Record FY22, management has observed that the effects of IEG’s evaluations often start long before the issuance of the formal report, as evaluation processes highlight key issues, spark new ways of thinking, and trigger real-time learning and adaptation. This is particularly true for this evaluation given IEG’s effective collaboration with management in building The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Management Response the knowledge base to confront the crises. Most of the recommendations are therefore being internalized in existing engagements and the report will only help advance their implementation. Management agrees with the first recommendation to use the World Bank’s crisis recovery efforts to strengthen the resilience of essential health and education services to ensure that human capital is protected in a crisis. The World Bank is already working toward strengthening resilience of essential health and education services, including through operational design that bet- ter addresses the intersection of primary health care and pandemic prepared- ness and response, and through the sharing of experiences on mechanisms to strengthen telehealth and other relevant platforms for use in emergencies, and for education through evidence and innovations and an expanding port- folio in addressing learning losses and accelerating long-term learning. Management agrees with the second recommendation to apply a gender equality lens to health and social crisis response actions across sectors. The World Bank’s gender strategy and guidelines will continue to provide support and capacity building to country teams to implement these recommenda- tions, making the report’s findings operational. However, there is scope to do more, and HNP will leverage new opportunities to ensure a gender lens in its analytical and operational work, for example, through capacity building and knowledge exchanges to encourage more gender focused analytics; through support for the collection of gender-disaggregated data; by documenting lessons learned; and by developing a thematic paper to feed into the update of the gender strategy in 2023. The World Bank will also apply a gender lens xxx to its strategic priorities (Global Solutions). Management agrees with the third recommendation to help countries strengthen regional cooperation and crisis response capacities for public health preparedness. The World Bank’s work to strengthen regional cooperation is articulated in both the position paper on pandemic prevention, preparedness, and response (which will be launched soon), and PPR Financial Intermediary Fund communications.10 The 20th Replenishment of International Development Association regional window would continue to support countries in this regard. The World Bank recognizes the need to find new ways to better engage civil society and increase stakeholder engagement, and it is also looking for opportunities to further strengthen platforms for coordination and to be more inclusive of civil society organizations. Management also agrees with the fourth recommendation to build on the COVID-19 experience to strengthen the World Bank’s internal crisis pre- paredness so that it has the tools and procedures ready to respond in future emergencies. Applying the lessons from COVID-19 and other crises, the World Bank will build on the experience of developing guidance notes for World Bank teams to operate more efficiently and effectively as it supports countries to strengthen capacities for pandemic PPR (including through PPR Financial Intermediary Funds), for example, by providing hands-on support for Environmental and Social Framework, promoting cross-country learning, and strengthening the monitoring and use of data on World Bank portfolios. Independent Evaluation Group World Bank Group    xxxi Additionally, the World Bank has a long history of involvement working on global innovative financing mechanisms (for example, Treasury Manager for International Finance Facility for Immunization, and pneumococcal Ad- vance Market Commitment), and is actively involved in dialogue with other partners. World Bank will explore further to shape and redesign global and regional financing instruments to be more “fit-for-purpose” during crises. Reference World Bank. 2021. “COVAX and World Bank to Accelerate Vaccine Access for Developing Countries” Press release no. 2022/006/HNP, July 26, 2021. https:// www.worldbank.org/en/news/press-release/2021/07/26/covax-and-world-bank- to-accelerate-vaccine-access-for-developing-countries. 1  For example, the report demonstrates what had been expected: that countries with stronger government leadership; investments in human capital and health system strengthening; prior pandemic and epidemic experience; and prior World Bank–related investment such as the Regional Disease Surveillance Systems Enhancement program and analytical work or both were able to mount a more effective response. Advisory services and analytics were critical for informing the design of COVID-19 operations and for a broader policy dialogue on immediate and longer-term responses (for example, the flagship paper on health financing challenges in developing countries From Double Shock to Double Recovery—Implications and Options for Health Financing in the Time of COVID-19: Technical Update 2. Old Scars, New Wounds). The Multiphase Programmatic Approach offered an umbrella approach with a menu of The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Management Response 2  components and interventions that participating countries could adapt to their needs in line with the World Bank’s country-based model and strengthen to address subsequent stages of the response. This allowed projects to maintain some uniformity in content, with the added advantage of increased speed of design, processing, and approval, and a menu of indicators for countries to tailor to their individual circumstances. 3  The World Bank is a founding member of the Gavi Alliance, the vaccine alliance, and played an important role as an implementing partner of the Gavi Alliance even before COVID-19. The World Bank was part of COVID-19 Vaccines Global Access (COVAX) from its inception, and participated in decision-making on COVAX through the Gavi Board. 4  The Emergency Operations Center team prepared the following: A model Operational Manu- al in April 2020 that was translated into Spanish, French, Portuguese, and Russian to facil- itate the start of implementation of Multiphase Programmatic Approach operations; “how to” guidance notes, including for processing retroactive funding requests; technical notes on several aspects of the health response and challenges that arose during the early months of the pandemic; regular weekly and bi-weekly global learning seminars that facilitated the cross-fertilization of knowledge among country officials, high level experts, and World Bank Group staff; and a template for Project Papers of AF-V operations (October 2020), later taken over by Operations Policy and Country Services. 5  Social Protection and Jobs Global Practice had a similar central resource hub with regional focal points, weekly (later monthly) meetings to provide advice to teams, extensive guidance material on a SharePoint site and a tracking system to monitor the Social Protection and Jobs response, which was used extensively for Senior Management briefings. xxxii 6  For example, women and girls bearing the burden of caring for the sick or of providing child and elderly care during the pandemic; losing jobs and being ineligible for a social safety net due to the informal nature of employment; and the importance of engaging women’s commu- nity groups to carry out knowledge dissemination and service provision. 7  External factors also played a role in the disruption of essential health services. For exam- ple, even when services were available, people were afraid to use them for fear of catching COVID-19; this was particularly true for services like childhood immunization. Although the report correctly identifies the gap in World Bank’s support for demand-side engagement of communities, this should be further qualified by noting that client governments have limited capacity to design and execute demand-side community engagement interventions in both emergency and nonemergency situations. In addition, governments’ and the World Bank teams’ limited attention to community engagement and continuity of essential health and education services should be understood in the context of an overwhelming pandemic with little under- standing of virus behavior—and in the absence of proven preventive and treatment measures. The focus of the response was on early detection and containment through a test and trace strategy, along with wide-scale lockdowns to prepare health systems to handle the pandemic. 8  On March 16, 2022, the World Bank published its report Walking the Talk: Reimagining Pri- mary Health Care After COVID-19, and has completed the latest flagship report Change Cannot Wait: Building Resilient Health Systems in the Shadow of COVID-19—Investing in Health System Resilience for the Anthropocene, which underscores the importance of pandemic preparedness and strengthening systems. Independent Evaluation Group World Bank Group    xxxiii 9  See Multiphase Programmatic Approach projects for Afghanistan, Papua New Guinea, India, Argentina, Ecuador, Indonesia, Haiti, Iran, Senegal, Somalia, and Ukraine. 10  See https://www.worldbank.org/en/news/press-release/2022/09/09/new-fund-for-pandemic- prevention-preparedness-and-response-formally-established. Report to the Board from the Committee on Development Effectiveness The Committee on Development Effectiveness met to consider the World Bank’s Early Support to Addressing COVID-19: Health and Social Response: An Ear- ly-Stage Evaluation and the draft management response. The committee welcomed IEG’s findings and recommendations and manage- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Report to the board ment’s constructive response and echoed their support for the World Bank’s multifaceted and rapid response to the COVID-19 pandemic both in scale and also in quality. Members underscored their appreciation for the unprec- edented World Bank’s efforts and innovative work in the first 15 months of the pandemic (February 1, 2020, to April 30, 2021—the period covered by this evaluation) aimed at strengthening health systems, supporting coun- try needs, prioritizing social protection for poor and vulnerable people, and facilitating knowledge sharing with client countries. They highlighted the operational processes and the number of financing instruments and modal- ities that enabled the rapid response. While recognizing the unprecedented global context, members however noted that the World Bank could have played a more decisive role at the global level, particularly on vaccines, and encouraged management to consider lessons learned from this experience and what could be done differently for a more robust crisis preparedness of the World Bank and client countries. Members commended management for the innovations adopted in the World Bank’s early response to the pandemic. They asked management to provide initial views on lessons learned including which approaches and tools should be retained and applied more systematically, and how the World Bank can promote continuous innovation in its work. They urged the World Bank to continue collaboration with development organizations and regional part- ners to coordinate interventions and achieve sustainable crisis response. Members also recognized the timeliness of the evaluation, given ongoing xxxiv efforts to setup the Financial Intermediary Fund for Pandemic Prevention, Preparedness and Response, and also its usefulness in informing discussions with country delegations on pandemic preparedness and crisis response at the 2022 Annual Meetings. Independent Evaluation Group World Bank Group    xxxv 1 | Introduction The evaluation answers the following question: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? The focus is on the first 15 months of the pan- demic and the large-scale and rapid actions that took place during these months in a very uncertain global context. Outcomes are not assessed; rather, the focus is on how the quality of the early response, design, processes, and outputs supported pathways to outcomes. The evaluation’s purpose is also to draw lessons from the World Bank’s support in the early COVID-19 response to inform recovery efforts and future support for crisis preparedness. 1 Response The World Bank Group quickly launched a large-scale response to help countries address both the health threat and the social and economic impact of the COVID-19 crisis. The support comprised three stages (relief, restructuring, and recovery) and four pillars (saving lives, protecting poor and vulnerable populations, ensuring sustainable business growth and jobs, and strengthening institutions for recovery). Country support was to be tailored to address needs and priorities across these areas. Institutional strengthening was to be undertaken from early in the relief stage to ensure sustained support to countries and maintain a clear route toward their longer-term development priorities. Attention to gender equality, digitali- zation, monitoring, evaluation, learning, and encouraging innovations was The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 intended to cut across all the pillars (World Bank Group 2020b). The Bank Group support to a country was often part of a national COVID-19 response plan, developed in collaboration with the government, partners, and other country stakeholders. In a highly uncertain context, the response was notably swift and unprece- dented, with support provided to more than 100 countries: in February 2020, the Bank Group committed $160 billion in financing for the COVID-19 re- sponse in fiscal years 2020 and 2021. The response was aligned with global ac- tions of the World Health Organization (WHO) and other partners (figure 1.1). The total financing included about $76 billion of World Bank commitments, of which about half supported the health and social response—the focus of this evaluation—in three of the four pillars (saving lives, protecting poor and vulnerable populations, and strengthening institutions for recovery). Other financing was for the economic response in sectors such as agriculture and governance and for the International Finance Corporation response, which are outside the scope of this evaluation. This all took place in a context with much uncertainty about the virus, vaccines, and how to best respond quickly and at scale, with information on the situation evolving daily. Key elements of the response included support through a Multiphase Programmatic Approach (MPA), new projects prepared through a fast-track facility, repurposing of existing projects, grant support from the Pandemic Emergency Financing Facility (PEF), and activation of existing crisis 2 instruments embedded in the portfolio. Although the pandemic continues to destroy lives with waves of infection occurring at different times and inten- sities across countries, even with deployment of effective vaccines, attention is moving toward coping with COVID-19 endemism, protecting vulnerable populations, and restructuring systems for recovery. This will shift the pri- orities in social and health sectors toward a return to stability and building back better to ensure future preparedness and protection of human capital. Evaluation Purpose and Scope Purpose The evaluation provides an early assessment of the quality of the World Bank’s COVID-19 health and social response to save lives and protect people living in poverty. Its purpose is to draw lessons to inform ongoing and future support for crisis preparedness and response. As the response is ongoing, the evaluation pays attention to processes, outputs, and pathways to outcomes, focusing on the relief stage and early support to restructure systems that can inform learning for recovery and future pandemic and crisis preparedness. Independent Evaluation Group World Bank Group    3 4 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 Figure 1.1. Global Milestones and Timeline of World Bank Group Response Milestones in COVID-19 World Bank Group Response First COVID-19 case reported outside China January Statement of support issued for COVID-19 outbreak in China WHO declares COVID-19 public health emergency 2020 Announces plan to mobilize resources, draft policy First COVID-19 death reported outside China February notes WHO finalizes its strategic preparedness and response 2020 Commits up to $160 billion in financing through June 2021; plan includes $104 billion from IDA and IBRD Convenes meetings of multilateral development banks and works with WHO WHO Global Preparedness Monitoring Board estab- Committees to coordinate response; creation lished March of fast-track facility World Bank calls for $8 billion financing 2020 World Bank immediate financing $12 billion, with IFC $8 WHO declares COVID-19 a pandemic billion First countries (Mongolia, Samoa) receive financing HNP MPA with financing up to $6 billion to a number Launch of United Nations supply chain task and ACT of countries Accelerator Suspended countries’ debt service; convened a joint meeting Meeting of WHO emergency committee advises work April 2020 with IMF to mobilize action; COVID-19 termed an unprece- on, among other areas, One Health and support to dented crisis essential health services PEF funds $195 million; financing reaches more than 60 WHO drafts landscape of vaccine candidates countries WHO Supply Portal launched First reports on COVID-19 governance, education, SPJ May 2020 World Health Assembly resolution to fight COVID-19 responses; emergency support reaches 100 countries More than 10 million Global Vaccine Summit; first $0.5 billion for COVAX AMC June 2020 Commitment to work with COVAX facility cases COVAX secures engagement of more than 165 coun- July 2020 IFC launches $4 billion financing to support health care tries (continued) Review committee on functioning of the International August Health Regulations announced 2020 More than WHO Supply Portal launched First reports on COVID-19 governance, education, SPJ May 2020 World Health Assembly resolution to fight COVID-19 responses; emergency support reaches 100 countries More than 10 million Global Vaccine Summit; first $0.5 billion for COVAX AMC June 2020 Commitment to work with COVAX facility cases COVAX secures engagement of more than 165 coun- July 2020 IFC launches $4 billion financing to support health care Milestones tries in COVID-19 World Bank Group Response Review committee on functioning of the International August Health Regulations announced 2020 More than 30 million UNGA discussions on preparedness and response September $48 billion financing already committed for COVID-19 cases 2020 response UNICEF and WHO lead vaccine readiness assessment Approves $12 billion additional financing for MPA on Survey indicates disruption or halting of mental health October vaccines, tests, and treatment More than services in 93 percent of 130 countries 2020 Annual Meetings discuss responses and addressing 40 million cases “IDA cliff” Vaccine readiness assessment of World Bank combined Interim guidance on national deployment and November with WHO and UNICEF process vaccination plans 2020 Releases fact sheet on citizen engagement and stakeholder consultation during COVID-19 December Report indicates school-related closures risk learning UNGASS Special Session on COVID-19 response 2020 poverty Pfizer-BioNTech first vaccine to receive emergency use January Lebanon: support to first vaccine purchase through validation from WHO 2021 reallocation in existing resilience project COVAX signs advance purchase agreement with Pfizer COVAX’s first interim distribution forecast and first February delivery of COVAX outside India to Ghana 2021 (continued) Johnson & Johnson vaccine receives emergency use March Launch of UNICEF, Johns Hopkins, and World Bank tracker validation from WHO; report on virus origins published 2021 measuring education impact More than 150 million Moderna vaccine receives emergency use validation Launch of early IDA Replenishment for recovery from Independent Evaluation Group World Bank Group    5 cases from WHO; COVAX purchase agreement for 500 million April 2021 COVID-19 More than doses 3 million Approved fund for vaccine rollout reaches $2 billion COVAX delivers 38 million doses to 100 economies Pfizer-BioNTech first vaccine to receive emergency use January Lebanon: support to first vaccine purchase through 6 Bank’sfrom validation The World EarlyWHO Support to Addressing COVID-19: Health and Social 2021 Response   Chapter 1 reallocation in existing resilience project COVAX signs advance purchase agreement with Pfizer COVAX’s first interim distribution forecast and first February delivery of COVAX outside India to Ghana 2021 Milestones in COVID-19 World Bank Group Response Johnson & Johnson vaccine receives emergency use March Launch of UNICEF, Johns Hopkins, and World Bank tracker validation from WHO; report on virus origins published 2021 measuring education impact More than 150 million Moderna vaccine receives emergency use validation Launch of early IDA Replenishment for recovery from cases from WHO; COVAX purchase agreement for 500 million April 2021 COVID-19 More than doses 3 million Approved fund for vaccine rollout reaches $2 billion COVAX delivers 38 million doses to 100 economies deaths High-level independent panel releases recommenda- Migration and Development Brief states that remittance tions to curb pandemic May 2021 flows were resilient in 2020, with smaller decline than Launch of new One Health High-Level Expert Panel projected Initiates multilateral leaders task force WHO adds Sinovac vaccine to its emergency use list Commitment to work with AVAT G7 commits to sharing 870 million vaccine doses High-Level Advisory Group on Sustainable and Inclusive The United States plans to buy 500 million Pfizer vac- June 2021 Recovery and Growth with IMF cine doses to donate to more than 90 lower-income HNP report—Walking the Talk: Reimagining Primary Health countries and African Union Care after COVID-19 Rollout of new financing mechanism on advance market July 2021 commitments with COVAX August IMF approves $650 billion in special drawing rights AVAT vaccine shipments begin with World Bank contribution 2021 September Bank Group role in future crises and GRID papers 2021 Only five African nations are on track to fully inoculate (continued) World Bank and IMF Annual Meetings 2021 Development October 40 percent of the population by the end of the year; the Committee Communiqué for Bank Group, IMF, WHO, and 2021 continent faces a shortfall of 275 million vaccine doses World Trade Organization task force WHO and the United States find that more than 22 million Commits $5.8 billion to vaccine support, enabling delivery November children missed a measles vaccine dose in 2020, the of 20.2 million doses, with 245 million in pipeline; $2.8 billion 2021 largest increase in 20 years vaccine contracts signed IDA Replenishment package of $93 billion announced with Omicron variant identified in South Africa; Africa experi- December new human capital and crisis preparedness cross-cutting encing 83 percent spike in new cases 2021 Rollout of new financing mechanism on advance market July 2021 commitments with COVAX August IMF approves $650 billion in special drawing rights AVAT vaccine shipments begin with World Bank contribution 2021 September Bank Group role in future crises and GRID papers Milestones in COVID-19 2021 World Bank Group Response Only five African nations are on track to fully inoculate World Bank and IMF Annual Meetings 2021 Development October 40 percent of the population by the end of the year; the Committee Communiqué for Bank Group, IMF, WHO, and 2021 continent faces a shortfall of 275 million vaccine doses World Trade Organization task force WHO and the United States find that more than 22 million Commits $5.8 billion to vaccine support, enabling delivery November children missed a measles vaccine dose in 2020, the of 20.2 million doses, with 245 million in pipeline; $2.8 billion 2021 largest increase in 20 years vaccine contracts signed IDA Replenishment package of $93 billion announced with Omicron variant identified in South Africa; Africa experi- December new human capital and crisis preparedness cross-cutting encing 83 percent spike in new cases 2021 issues Sources: Barış et al. 2021; Dixon et al. 2021; Independent Panel for Pandemic Preparedness and Response 2021; KNOMAD 2021; Ravelo and Jerving 2022; Saavedra et al. 2020; WHO 2020a, 2020b, 2021b, 2022; World Bank 2020a, 2020b, 2020c, 2020d, 2020e, 2020f; World Bank Group 2021a, 2021b. Note: ACT = Access to COVID-19 Tools; AMC = advance market commitment; AVAT = African Vaccine Acquisition Trust; COVAX = Country Access to COVID-19 Vaccines; G7 = Group of Seven; GRID = Green, Resilient, and Inclusive Development; HNP = Health, Nutrition, and Population; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = International Finance Corporation; IMF = International Monetary Fund; MPA = Multiphase Programmatic Approach; PEF = Pandemic Emergency Financing Facility; SPJ = Social Protection and Jobs; UNGA = United Nations General Assembly; UNGASS = United Nations Gener- al Assembly Special Session; UNICEF = United Nations Children’s Fund; WHO = World Health Organization. Independent Evaluation Group World Bank Group    7 Scope The evaluation answers the following question: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? The evaluation uses three lines of inquiry to answer this question (box 1.1).  hree Questions That Guide the Evaluation Box 1.1. T 1.  What has been the relevance of the World Bank’s early COVID-19 response to addressing the needs of countries in saving lives and protecting poor and vulnerable people (that is, the diagnosis, design, and tailoring of interventions to country situations)? The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 2.  What has facilitated or hindered implementation of the World Bank’s COVID-19 responses in countries, and how is the World Bank supporting learning and adjustments? 3.  How well are operational processes, instruments, and partnerships supporting the World Bank’s COVID-19 responses in countries? Source: Independent Evaluation Group. The evaluation focus is the health and social response during the first 15 months of the COVID-19 pandemic, from February 1, 2020, to April 30, 2021, and a portfolio of 106 countries. The evaluation looks to learn from the World Bank’s health and social support to countries most vulnerable to reversal of development and human capital gains because of COVID-19 (see appendix B for the portfolio identification). Five World Bank Global Practices (GPs) led the early support to countries: Health, Nutrition, and Population; Social Protection and Jobs; Education; Urban, Disaster Risk Management, Resilience, and Land; and Macroeconomics, Trade, and Investment. A concurrent Independent Evaluation Group (IEG) evaluation assesses the economic response to COVID-19. 8 The health and social support evaluation portfolio consists of an estimated $30 billion of commitments to save lives and protect poor and vulnerable people in vulnerable countries, of which about $11 billion (one-third) was committed by May 2020 (appendix B).1 This includes $27 billion in opera- tional financing ($15 billion International Development Association [IDA] and $14 billion International Bank for Reconstruction and Development), $1.5 billion in trust funds, and $60 million in advisory services and ana- lytics (ASA) commitments. Forty percent of this was disbursed in the first 15 months of the response, and about 20 percent was disbursed in the first three months, by May 2020. It is also estimated that there was $1.54 billion in Contingency Emergency Response Component (CERC) commitments from other GPs allocated to the health and social response not covered by the portfolio. The Human Development GP led these commitments. During the early response, Health, Nutrition, and Population had five times more proj- ects approved and managed about 600 percent more in annual allocations, spread across small projects (about 43 percent smaller on average than other health projects) with short durations of two to three years. Social Protection and Jobs almost doubled the number of projects approved, whereas Edu- cation had about 56 percent more projects. For other GPs, lending activity increases were more limited during the early response to COVID-19, whereas lending in areas such as Agriculture and Food increased in the second year of the response. The evaluation portfolio focuses on countries with moderate to high vulnerability to development losses because of the impact of COVID-19.2 Independent Evaluation Group World Bank Group    9 It emphasizes support to less-prepared countries, small states, countries with fragile and conflict-affected situations (FCS), and countries at risk of human capital losses (appendix B). Most financing commitments were allocated to less-prepared countries with pressing needs. Commitments in these countries included support of new projects and repurposing of existing projects and ASA to support the crisis response. Small states received about $26 million per million population in the COVID-19 response compared with about $6 million for other countries. FCS countries received $9 million per million population. New project support focused on countries with lower levels of human capital in Africa and South Asia that included countries with high vulnerability to development losses as a result of COVID-19. Evaluation Design Conceptual Framework The conceptual framework defines the thematic areas and stages of support within the scope of the early COVID-19 response assessed by the evaluation (figure 1.2 and box 1.2). The framework is based on the Bank Group’s COVID-19 response framework (World Bank Group 2020b). The conceptual framework supports a theory-based approach that models the interlinked elements of the health and social response, with the current evaluation focusing on the early World Bank response in the relief and initial restructuring stages. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 10  onceptual Framework for COVID-19 Health and Social Response Figure 1.2. C Relief: To help countries prevent, detect, and respond to the health threat posed by COVID-19 and to protect poor and vulnerable households and communities from the shocks of the crisis. Objective Restructuring and recovery: To strengthen national systems for public health preparedness, to restore human capital, and to promote equity and inclusion in the recovery. Country situation Crisis response experiense | Policy dialogue and systems | Disease situation | Baseline needs | Social impacts Response areas RELIEF STAGE RESTRUCTURING STAGE Ensure health • Ensure critical health services (IPC, case management, surveillance, laboratories) • Strengthen health systems support • Ensure essential health services (maternal and child health, primary care) • Reduce COVID-19 cases • Communicate health risks • Strengthen essential primary services Vaccination • Improve vaccine readiness • Strengthen vaccine systems Protect the poor • Ensure targeted income and in-kind support • Strengthen social protection systems and vulnerable • Provide wage subsidies for informal workers • Improve income generation and asset accumulation Ensure child welfare • Ensure learning of vulnerable children • Facilitate children’s return to school, with compensatory learning and social services • Provide psychosocial support • Continue nutrition support • Ensure nutrition support Community • Improve citizen engagement • Reduce transmission engagement • Improve social cohesion • Improve community resilience Institutional • Improve coordination and planning • Improve systems, policy, and financing to manage crisis and protect human capital strengthening • Expand public health and basic services functions • Reconfigure supply chains and partnerships to promote recovery of response • Ensure local government support RECOVERY STAGE • Improve national and subnational systems and preparedness • Improve equity and inclusion • Improve long-term outcomes, including resilience to future shocks Cross-cutting issues: gender, digitalization, monitoring, evaluation, and learning Source: Independent Evaluation Group. Adapted from World Bank Group 2020b to focus on the health and social aspects of the COVID-19 response. Note: IPC = infection prevention and control. Independent Evaluation Group World Bank Group    11  he Logic of the Conceptual Framework Box 1.2. T The conceptual framework outlines the elements of the World Bank’s health and social support for COVID-19, which are the focus of the evaluation. These elements are anchored in three pillars of the World Bank Group response (saving lives, protect- ing poor and vulnerable populations, and strengthening institutions for recovery). The framework expresses interlinked elements to respond to the health and social shocks of COVID-19 regarding: » The progression of the response through the three stages (relief, restructuring, and recovery) » The menu of areas that could be operationalized through World Bank support and tailored to needs in the country context The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 » The types of results expected at each stage to respond to the health threat and protect human capital, from early outputs and processes to longer-term out- comes for recovery » The integration of institutional strengthening from the early response to improve capacities to manage crises and build more resilient systems to protect against future shocks, including supporting coordination and planning, expanding public health functions (such as disease surveillance), and strengthening local services » The cross-cutting areas important to effective implementation in a country (gender equality, improvements to digitalize systems, and monitoring, evaluation, and learning) The evaluation assesses the World Bank’s health and social support to the early relief stage and initial restructuring stage of COVID-19. The relief stage concentrates on responding to the immediate health and social shocks of COVID-19. In health, relief stage is intended to ensure the supply of critical health services (infection prevention and control, case management, surveillance, and laboratories), early readiness for vaccines, and health risk communication. Continued access to essential health services for primary care of vulnerable groups was also important. The health response is intended to be coupled with tailored support to address social shocks as a result of COVID-19 and to protect accumulated human capital. (continued) 12  he Logic of the Conceptual Framework (Cont.) Box 1.2. T In the relief stage, the social emergency response aims to ensure social protection for poor and vulnerable groups, continued child welfare (access to learning and nutri- tion), psychosocial services for mental health, community and citizen engagement, and social cohesion support for the demand-side aspects of the response. Vulnerable groups include the elderly, people with underlying conditions or comorbidities, poor and marginalized populations, and women and children. The restructuring stage can overlap with and follow the relief stage. Restructuring stage support seeks to strengthen systems and policies for public health prepared- ness and restoring human capital. The restructuring stage in the health response seeks to strengthen health systems, vaccine delivery, and essential health services. The social response seeks to improve social protection systems, education systems, and community resilience. The recovery stage had yet to be reached in the early COVID-19 response, but relief efforts and some initial restructuring are intended to be put in place as building blocks to transition to this stage. The recovery stage is intended to apply the learning from the COVID-19 response to ensure pandemic-ready health systems; improve equity and inclusion through better access to health, education, and social services; and enhance policies that protect human capital. Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    13 Methodology The evaluation adopts a multilevel analysis and a mixed methods approach that combines quantitative and qualitative evidence. It follows a consultative approach to inform the analyses and a modular approach to share prelimi- nary findings. Throughout the evaluation, there was engagement with World Bank GP management and project teams, operational support units, country management, and technical experts to discuss analyses, share preliminary findings, and receive feedback. These consultations are important because the evaluation focuses on an active and evolving portfolio. Theory of Action The evaluation’s theory of action outlines the dimensions for the assessment of the quality of the World Bank’s early COVID-19 response (figure 1.3). The theory of action aligns with the evaluation questions and complements the conceptual framework. It posits three interrelated areas that define the quality of the response, each with specific dimensions for which the evaluation gathered and triangulated evidence: » Support to needs looks at the relevance of the World Bank’s large-scale early response to help address the immediate health threat of COVID-19 and to protect vulnerable groups against human capital losses. » Dimensions: Responsiveness of World Bank support to health and social The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 needs of countries, addressing gender, building on existing human capital capacities, alignment with COVID-19 plans in countries, reorientation of projects in the portfolio, prioritization of support in the portfolio to focus on key areas within needs and vulnerable groups, use of knowledge work to inform needs, and integrating institutional strengthening and support to build resilient systems for recovery » Implementation and learning looks at factors that facilitated and hindered the World Bank early response in countries, which can provide proxy evi- dence of early results, and how there has been learning and adjustment to strengthen the response. » Dimensions: Implementation status of the response, factors facilitating im- plementation, early results, anchoring of support in lessons and evidence from past crises, and emphasis on innovation, learning, dialogue and coor- dination with the government and other stakeholders in countries, use of data and other inputs to make course corrections to ensure a strong mix of support to countries, and regional knowledge sharing » Operational policies and partnerships looks at how well World Bank inter- nal process, instruments, and partnerships supported a smooth and speedy early response. 14 » Dimensions: World Bank internal coordination, the mix of instruments supporting the response, streamlined operational processes, internal re- porting and monitoring, and development partnerships including PEF and vaccines support.  heory of Action to Assess the Quality of the Early Figure 1.3. T COVID-19 Response Addressing needs 1 (relevance) Operational policies • Based on needs and partnerships • Gender • Internal coordination • Building on capacities • Instruments • Aligned with plans • Streamlined processes • Reorientation • Procurement of portfolio • Monitoring and reporting • Prioritized to context • Partnership 3 • Use of knowledge work • Pandemic Emergency • Integrated institutional Financing Facility strengthening • Vaccines • Implementation status and facilitating factors Support to • Early results • implementation Building on past evidence and learning and lessons • Innovation and learning • Dialogue and coordination • Adjustments 2 • Regional knowledge sharing Source: Independent Evaluation Group. Levels of Analysis Independent Evaluation Group World Bank Group    15 The assessment of quality and lessons arise from triangulating evidence gathered by different evaluation components at the country, portfolio, and corporate levels. The evaluation uses a mix of methodological applications to ensure construct, internal, and external validity and reliability of find- ings through a transparent methodological design, with clear justification of choices made (see appendix A for the evaluation methodology). Box 1.3 describes the evaluation components.  valuation Components at Each Level Box 1.3. E At the country level: » Eight country case studies provided in-depth evidence on the COVID-19 response in specific contexts (Djibouti, Honduras, India, Mozambique, the Philippines, Sene- gal, Tajikistan, and Uganda). The team reviewed World Bank projects and analytic work, and interviewed staff, government officials, and representatives of civil society (appendix C). » A review of country situations was conducted to develop a heat map to under- stand the needs of countries early in the COVID-19 pandemic, analyze the extent of addressing these needs in countries in the portfolio, and identify factors that facilitated satisfactory implementation of World Bank support. The analysis used The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 portfolio data on World Bank projects and advisory services and analytics support- ing COVID-19 and publicly available health and social data on country indicators relevant to areas of the conceptual framework of the evaluation (appendix D). » A rapid review of the literature synthesized evidence on effective crisis interven- tions from systematic reviews and country studies of past epidemic and crises to benchmark World Bank support. The review synthesized existing evidence from the literature on 50 crisis interventions. The findings were used to understand the alignment of interventions supported by the World Bank’s early COVID-19 re- sponse with the available evidence base (appendix E). » A review of past crisis response projects benchmarked successes and challenges from these projects against the early COVID-19 response. The evaluation synthe- sizes lessons from 170 closed projects where the World Bank responded to crises in the past 20 years (appendix F). » A review of regional projects assessed the early results of disease-focused projects for COVID-19. Interviews and document review were used to understand the value added of four disease-focused regional projects to help advance early results of COVID-19 responses in countries (appendix G). » A stocktaking analysis identified innovations to understand how the World Bank undertook new actions to support the COVID-19 context. The innovations were identified through a crowdsourcing survey of country teams, review of the 16 (continued)  valuation Components at Each Level (Cont.) Box 1.3. E project portfolio supporting COVID-19, a review of innovation stories published by Global Practices, and as part of the analysis of the eight case studies conducted for the evaluation. At the portfolio level: » An analysis of operational financing projects and advisory services and analyt- ics supporting the early COVID-19 response was conducted, including analyses of monitoring of the response and procurement. The evaluation undertakes a systematic document and data review focused on internal databases and coded information on a portfolio of COVID-19 projects (appendix B). » An analysis of projects under the Multiphase Programmatic Approach led by Health, Nutrition, and Population was conducted. The analysis uses data from the evaluation portfolio, case studies, regional project analysis, and innovation stocktaking to review Multiphase Programmatic Approach projects in the first year of the response—projects approved by April 30, 2021 (appendix H). At the corporate level: » A review of internal processes and partnerships sought to distill lessons and findings on how the World Bank’s COVID-19 internal coordination and collabora- tion, financing instruments, operational processes, partnerships, knowledge Independent Evaluation Group World Bank Group    17 support, digital tools, and monitoring and reporting guided and supported the early COVID-19 response. The review was based on information from document review and individual or group semistructured interviews. Source: Independent Evaluation Group. The stocktaking of innovations is available in World Bank 2022a. Limitations An important limitation of the evaluation is the dynamic nature of the COVID-19 situation and World Bank response, and the overlap of the evaluation with the ongoing response implementation. The World Bank was adapting the response during implementation to improve its actions, given the evolving and uncertain global and country-level contexts of COVID-19. Moreover, interviews were challenging because of the high number of COVID-19 cases in some countries, the burden of the pandemic on health sector personnel, and illness and personal losses of interviewees as a result of COVID-19. Although the evaluation analyses estimate early support and financing in the countries covered by the portfolio, a rapid update of the portfolio was done at the end of the evaluation to gauge shifts in support as the portfolio continued to evolve throughout the evaluation timeline. A future later-stage evaluation is proposed to look at the effectiveness of the response. Appendix A also outlines limitations of specific methods of the evaluation. Structure The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 1 The report structure is based on the theory of action of the evaluation. Each chapter highlights evidence on early support to countries to effectively respond to the health and social shocks of COVID-19 and to start a process to strengthen systems and policy for better crisis preparedness and protec- tion of human capital. Chapter 2 looks at the extensive scale of the World Bank support and its relevance to needs of countries. Chapter 3 covers the implementation successes and challenges of the World Bank’s early support to countries, which can point to early results, and how the World Bank has adapted and learned during implementation. Chapter 4 covers how well operational policies and partnerships facilitated a smooth and speedy response. Chapter 5 synthesizes key areas of learning from the evaluation, which can inform future preparedness, and presents recommendations for the way forward. 18 1 An in-depth analysis of COVID-19 commitments and financing allocations is outside the scope of the current evaluation. The evaluation provides an estimate from available data on the portfolio for the time period, countries, and Global Practices covered by the analysis.  2 The Inform COVID-19 Risk Index was used to categorize countries based on their vulnera- bility to development achievements being offset by the pandemic. The evaluation adjusted the index to consider the country’s human capital index, given concerns surrounding losses of human capital in countries. The countries were then separated into quartiles based on their vulnerabilities to development and human capital losses (very high vulnerability, high vulnerability, moderate vulnerability, and low vulnerability). Appendix B includes a list of the countries in the portfolio by vulnerability quartile. The Inform COVID-19 Risk Index includes dimensions of social inclusion (such as gender inequality and poverty), economic vulner- ability, governance and institutional capacity, health systems capacity, environment, and population risks (such as access to sanitation and population mobility and density; Poljanšek, Vernaccini, and Marin Ferrer 2020; World Bank 2020f).  Independent Evaluation Group World Bank Group    19 2 | Quality of Response: Relevance World Bank financing quickly expanded emergency support to critical health services and social protection to respond to countries’ needs in a context of uncertainty. Support to essential health services, child welfare, community engagement, and particularly protection of women and girls from the shock of COVID-19 were less prominent in early response actions. The response was quicker and more comprehensive where the World Bank built on existing policy dialogue, analytic work, and support to human capital development, yielding a strong return on earlier investments in human capital. World Bank support aligned with COVID-19 strategies of health ministries. In the few cases where countries planned support involving multiple sectors, integrated emergency health planning helped ensure relevant support for human capital needs. Repurposing existing World Bank operations in country portfolios and adding new support helped mobilize surge capacities across sectors to quickly address needs during the crisis response. Countries with existing health preparedness and health system ca- pacities were well placed to use World Bank support to take rapid actions. Across countries, there were progressive efforts to priori- tize actions for vulnerable groups and to protect human capital. The integration of institutional strengthening brought a longer-term focus on rebuilding health, education, and social protection systems into the early response, but countries have yet to develop strategies to sustain efforts and prioritize preparedness actions. 20   This chapter assesses the quality of the World Bank’s support in terms of its relevance to addressing country needs in saving lives and protecting poor and vulnerable people during the early COVID-19 response. The assessment is based on dimensions of quality from the theory of action in figure 2.1. Figure 2.1. Dimensions Assessed for Quality of Support to Need ADDRESSING NEEDS IMPLEMENTATION OPERATIONAL POLICIES (RELEVANCE) AND LEARNING AND PARTNERSHIPS • Based on needs • Implementation status • Internal coordination • Gender and facilitating factors • Instruments • Building on capacities • Early results • Streamlined processes • Aligned with plans • Building on past evidence • Procurement • Reorientation of portfolio and lessons • Monitoring and reporting • Prioritized to context • Innovation and learning • Partnerships • Use of knowledge work • Dialogue and coordination • Pandemic Emergency • Integrated institutional • Adjustments Financing Facility strengthening • Regional knowledge sharing • Vaccines Source: Independent Evaluation Group portfolio. Addressing Health and Social Needs In a context of uncertainty, the World Bank’s support in the early COVID-19 response helped quickly expand critical health services and social protection across countries. More than 80 percent of countries in the evaluation portfolio received support for critical health services, and 67 percent received support to protect poor and vulnerable persons (social protection and informal economy Independent Evaluation Group World Bank Group    21 support). Support largely focused on the delivery of critical health services, including infection prevention and control, case management, surveillance, and laboratories, and on the expansion of social protection for vulnerable groups, including income support and food support (figure 2.2, panel a). The extensive expansion of social protection during COVID-19 is an improvement from the global financial crisis where the challenges in expanding country social protection systems limited the response (World Bank 2012). Recent studies have shown that the expansion of social protection helped mitigate food insecurity and reduce increases in poverty (Gentilini 2022). The pandemic has had a highly unequal economic impact (World Bank 2022c)—the economic support complemented health and social interventions in about 72 percent of countries (figure 2.2, panel b). As noted in chapter 1, the analysis of the economic response is part of another IEG evaluation focused on COVID-19 and is outside the scope of this evaluation and is covered in a parallel IEG evaluation. Figure 2.2. Areas of Health and Social Response Support in Countries a. Areas of health and social response support in countries Response area Thematic area Ensure health services Infection prevention and control 75 75 Case management 72 Surveillance Laboratories 71 41 Essential health services 31 Vaccination Vaccination Protect the poor Social protection 60 and vulnerable 34 Informal economy Ensure child welfare Child welfare 46 and social services 26 Psychosocial care The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 43 Community engagement, Citizen engagement social cohesion, and 22 resilience Social cohesion 66 Health risk Risk communication communication 0 10 20 30 40 50 60 70 80 Share of countries (percent) b. Balance of health and social support with economic support in countries Lending group IDA (n = 52) IBRD or Blend (n = 54) 0 10 20 30 40 50 60 70 80 90 100 Share of countries (percent) Type of support Economic, health, No economic Only economic and social support  support support Source: Independent Evaluation Group portfolio. Note: Data in both panels are based on 106 eligible countries. Panel a covers the 253 projects in the portfolio. In panel b, support is based on 567 crisis response projects that were active at any point between February 1, 2020, and April 30, 2021. In panel b, economic support is estimated to have been provided for (i) countries where the World Bank provided support to the COVID-19 economic pillar of the World Bank’s response or (ii) countries where the World Bank provided support to the COVID-19 institutional strengthening pillar of the response led by Global Practices outside the Human Develop- ment Practice Group. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association. 22 In the early response, the emphasis was on critical health services, disease prevention and control, and social protection and was aligned with most of the immediate needs to respond to the health and social shocks of COVID-19. The needs analysis shows that about 60 percent of countries’ needs identified at the onset of COVID-19 were addressed by the World Bank’s support (figure 2.3); in about 45 percent of countries, there was a very high alignment with country needs (appendix D). In the early response, there was less attention to child welfare and demand- side engagement of communities. Less emphasis was given to continuing child learning and nutrition when schools and services in communities were closed. Demand-side investments in the community response, in areas of social cohesion and citizen engagement, were also limited (figure 2.2, panel a). Risk communication was planned in two-thirds (66 percent) of countries, but case studies found that the intensity of these activities was limited early in the response. Psychosocial care, essential health services, and vaccines also received less emphasis. Examples of interventions can be found in countries in each of these areas, for example, projects in the Democratic Republic of Congo and Sierra Leone both planned in-depth support to build community trust based on lessons learned from the previous Ebola outbreak. Evaluations conducted by governments and other multilateral organizations identify similar challenges of limitations in support to child welfare, community engagement, and risk communication early in the COVID-19 Independent Evaluation Group World Bank Group    23 response (Johnson and Kennedy-Chouane 2021; OECD 2022; DPME, GTAC, and NRF 2021). Box 2.1 describes the main types of interventions in the early health and social response covered to different extents across countries.  xamples of Health and Social Support for the Early Box 2.1. E COVID-19 Response Health response: » Case management: equip and repurpose health facilities with ventilators, oxygen cylinders, and isolation and quarantine units to care for patients with COVID-19 » Essential health services: finance supply logistics for essential medicines and telehealth to minimize disruptions to health service delivery (continued)  xamples of Health and Social Support for the Early Box 2.1. E COVID-19 Response (Cont.) » Infection prevention and control: equip health workers with medical masks, N95 masks, gloves, eye protection, gowns, hand sanitizer, and other hygiene materials and help facilities develop infection prevention and control protocols » Laboratories: train laboratory staff, update and set up laboratories, and coordinate management of testing data and specimens » Surveillance: strengthen community and event-based surveillance for COVID-19, assess risk, and monitor and evaluate the effectiveness of activities to reduce transmission » Health risk communication: execute communication strategies and campaigns The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 and assess messages for population segments, such as the elderly and vulnera- ble groups » Vaccination: equip countries through procurement and distribution of vaccines and essential equipment, such as syringes, cold chain, and vaccine carriers Social response: » Child welfare: implement safe school reopening plans with sanitation and hygiene protocols, teacher professional development programs, and continuity of child learning » Psychosocial care: establish telepsychiatry systems, toll-free mental health ho- tlines, and psychosocial support for those in isolation » Informal economy: implement public works projects, job training, and informal apprenticeships and improve information systems for informal economic activities » Social protection: provide emergency cash transfers to vulnerable households with an emphasis on women, and pension schemes for the elderly and people with disabilities » Citizen engagement: engage nongovernmental organizations to monitor COVID-19 response, community-based early-warning networks, and SMS com- munication on services 24 (continued)  xamples of Health and Social Support for the Early Box 2.1. E COVID-19 Response (Cont.) » Social cohesion: execute campaigns on gender-based violence, support girls to prevent dropouts, and support community groups and projects to promote behavior change Source: Independent Evaluation Group portfolio analysis. Note: Economic response interventions that complemented the health and social response are cov- ered by another Independent Evaluation Group evaluation. SMS = short messaging service. About half of countries complemented emergency support with early response actions in essential health services for maternal and child health and education to protect human capital. Other countries had limited early emphasis on essential health services and education, key for protecting human capital, especially FCS countries and small states (figure 2.3). A challenge was the lack of preparedness of countries to quickly take actions to address needs to continue essential health and education services in communities in a crisis and support urban risks, especially for vulnerable groups and in countries with weak capacities to deliver services (box 2.2). Case studies and the portfolio review highlight that MPA’s investments in critical health services likely had some spillover effects that supported essential health services. For example, in India and Haiti, the increased infection prevention and control, oxygen, Independent Evaluation Group World Bank Group    25 laboratory, and surveillance capabilities likely helped strengthen health systems and networks to deliver services. » In health, attention to continuing essential health services in the early response was challenging with governments requiring urgent support to expand critical health services for COVID-19 case management—needs in these areas were only met in about 48 percent of countries, which contributes to development losses in maternal and child health, especially for vulnerable groups (GFF 2021; World Bank 2022). Efforts to continue health services were later added to strengthen the COVID-19 response in some countries, building on existing health projects, where available. » Regarding COVID-19, identified needs related to urban risks for the spread of the virus, such as in slums, were addressed in about 16 percent of countries. » In education, where the government requested it, World Bank support helped quickly expand remote learning nationally across countries; however, needs to protect against learning losses were vast, with economic losses estimated in the trillions of dollars (Global Education Evidence Advisory Panel 2022). World Bank education interventions met needs in about 55 percent of countries. Figure 2.3. Alignment of Project Portfolio with Identified Country Needs Critical Health Essential Health Health Risk Social Services Services Communication Child Welfare Protection Global (n=106) FCS (n=32) Small states (n=31) The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 Country-Level Citizen Overall Coordination Engagement Digitalization Urban Average Global (n=106) FCS (n=32) Small states (n=31) Share of countries (percent) Share of countries (percent) 6 20 6 100 40 60 80 100 Source: Independent Evaluation Group portfolio and needs analysis. Note: The figure shows the percentage of countries with needs in areas where the World Bank sup- ported interventions. A need is defined as the underlying needs variable in an area falling in the bottom 50 percent of its distribution across countries. Interventions are based on the analysis of 203 projects coded for the evaluation in 89 countries that had data on needs and World Bank support. Red shading indicates that needs were addressed in less than 50 percent of countries. Gray shading indicates that needs were addressed in 50 percent or more of countries. Small states follow the World Bank definition. Data to assess the need for critical health services, risk communication, and country-level coordina- tion use International Health Regulations data on capacities in the country before COVID-19; needs for essential health services, social protection, community engagement, digitalization, and urban support use data on access and vulnerabilities in these areas from the INFORM COVID-19 Risk Index. Appendix D describes the needs analysis. FCS = fragile and conflict-affected situation. 26  ey Areas to Strengthen Preparedness to Address Needs Box 2.2. K in Crisis Response Health preparedness: Comprehensive support was necessary to ensure both critical health services for preventing the spread of disease and essential health services for protecting against health-related human capital losses of women and children. World Bank support to help countries address these needs in an integrated manner was a lesson from the Ebola crisis and could have helped strengthen response efforts. The case studies and country situation analyses (appendixes C and D) show that the focus on the health emergency diverted attention from essential health services, such as maternal and child health care for vulnerable groups. Health systems were not prepared to continue essential health services during the crisis, given the need for surveillance and case management for COVID-19. Moreover, the intensity of support to frontline health workers and communities for risk communication was limited. Disruptions in the use of essential health services as a result of COVID-19 caused a secondary crisis in some countries, with drops in key maternal and child health indicators. Nutrition was also missed to protect child welfare. Urban preparedness: Few countries with needs at the onset of COVID-19 in terms of urban risks for the spread of disease were prepared with relevant World Bank sanitation and health support for vulnerable populations, such as in slums. Education preparedness: Case studies and the portfolio show good support to expand learning for children in countries receiving such support. The challenge was Independent Evaluation Group World Bank Group    27 the limited coverage of this support across countries in the early response. Moreover, the education sector was underprepared for the situation and lacked a strategy to prevent learning losses among vulnerable groups and girls. Partly, this may be because the sector was often not part of previous multisector crisis response planning. The consequence is a worsening crisis with girls out of school and learning outcomes potentially reduced. Sources: GFF 2021; Global Education Evidence Advisory Panel 2022; Independent Evaluation Group portfolio; World Bank 2021e. Addressing Gender The World Bank’s preparedness to help protect women and girls from the shock of the COVID-19 crisis varied across countries. About half of countries had medium to very high World Bank support for gender equality, with more than 25 percent of projects in the portfolio addressing gender to some extent as part of their crisis response. The greater focus on gender in IDA and FCS countries was promising (figure 2.4). Support to protect women and girls was key to ameliorate impacts on health workers (often women), women caring for children, and adolescents, especially girls. For example, countries are concerned about adverse pregnancy outcomes, school dropouts, early marriage, and pregnancy, which may have adverse long-term consequenc- es (Barış et al. 2021; Nieves, Gaddis, and Muller 2021; World Bank 2022b). The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 However, psychosocial support, sexual and reproductive health, income and asset accumulation, reduction of gender-based violence, continued learning for girls, and community engagement were areas of limited support identi- fied as important in past lessons (Gold and Hutton 2020; World Bank 2021e) and evidence (appendix E). The Social Protection and Jobs GP has shown the strongest address of gender issues as a core element of social protection support (about 95 percent of projects supported gender). Gender-related support of other GPs was limited; however, all GPs focused more on gender in FCS countries compared with other countries, which is promising. Eval- uations from other multilateral and bilateral development organizations highlighted that addressing gender in a crisis requires building on existing approaches and systems already in place (Johnson and Kennedy-Chouane 2021; Vancutsem and Mahieu 2020). Examples of positive outliers are coun- tries that built on their earlier experiences responding to gender equality challenges and received hands-on support: » In Kenya, gender-based violence increased during COVID-19. Joint work between the Social Sustainability and Inclusion and Health, Nutrition, and Population GP teams sought to enhance the quality of gender-based violence services, with a focus on care and treatment by health-care providers, data collection and analysis, health sector systems for response, and the safety of female frontline health workers. 28 » In India, women’s organizations helped ensure the availability of personal protective equipment. Engaging these self-help groups, which have had a long history of World Bank support, ensured the provision of personal protective equipment in communities and directly benefited female-headed households. Figure 2.4. Extent of Gender Equality Support in Country Portfolios IDA (n = 50) Lending group IBRD or Blend (n = 44) FSC (n = 29) 0 10 20 30 40 50 60 70 80 90 100 Share of countries (percent) Gender focus level Low  Medium High Very high Source: Independent Evaluation Group portfolio. Note: Gender focus is defined as the share of projects in a country that were designed to address deter- minants of gender equality. Gender focus levels: Very low = 0 to 24.9 percent of World Bank projects in the country supporting COVID-19, medium= 25 percent to 49.9 percent of projects, high = 50 percent to 74.9 percent of projects, and very high = 75 percent to 100 percent projects. The figure excludes three countries with only regional projects (Grenada, St. Lucia, and St. Vincent and the Grenadines). N = 94 countries. IBRD = International Bank for Reconstruction and Development; IDA = International Develop- ment Association; FCS = fragile and conflict-affected situation. Independent Evaluation Group World Bank Group    29 Building on Human Capital Capacities Previous World Bank support to human capital helped ensure that countries were prepared to respond to needs for the COVID-19 response. Previous- ly developed relationships in human development sectors, ongoing policy dialogue, and earlier investments in human development systems were a good basis on which COVID-19 project support was built. The country needs analysis for the evaluation (appendix D) found that medium to high levels of previous support to human capital development in health, social protec- tion, and education made it almost 1.5 times more likely that the country would address health and social needs during COVID-19 at high or very high levels.1 The strong return on previous investments in human development systems helped build resilience in countries. For example, sustained invest- ments countries made in their social protection systems in digital payments and social registries were used by the World Bank’s COVID-19 support, for example, in India, Jordan, and Morocco. Areas not addressed in the response were often those with limited attention before COVID-19, such as to address urban health risks, psychosocial care, and remote platforms to monitor com- munity services. Case studies show that World Bank country programs with a long history of support and policy dialogue in a sector were well-situated to support the government to quickly draw on existing health and social investments for a fast response to COVID-19. For example: » World Bank projects in Djibouti expanded on education sector networks of teachers and parents’ groups in communities to support remote learning. This reinforced a new platform to help learning in the sector. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 » World Bank programs in India and Tajikistan built on earlier analytic work and projects in social protection to help the government rapidly expand na- tional social protection systems to mitigate COVID-19 shocks. » Senegal drew on its multisectoral One Health platform developed through earlier World Bank and partner investments to assist the COVID-19 response. The World Bank’s support to COVID-19 was able to reinforce this platform quickly and help multisectoral coordination of actions. » World Bank teams supported the government in Uganda in fast-tracking planned reforms in the water sector to create an umbrella organization of service providers to help improve water access in local areas during COVID-19. Alignment with Country Plans Early World Bank support was well aligned with COVID-19 health responses in countries, with some complementary support to responses of other sec- tors. Country COVID-19 response plans often focused on emergency critical health and social protection support. Country responses aligned with WHO guidance (box 2.3). Other responses were fragmented across ministries, with each sector leading its own actions with limited communication across sectors. Where support was identified by the sector as important, World Bank 30 teams often provided support. For example, in Djibouti, Senegal, and Uganda, the World Bank supported education sector strategies to expand remote learning. In Uganda, the World Bank also supported agriculture sec- tor strategies to expand nutrition support and inputs to farmers for planting materials, and areas such as child protection policy, water services, and local government services based on government requests. Where there was integrated cross-sector planning of health and social response actions, World Bank teams could support needs more compre- hensively. Based on experiences in India and Senegal, among others, the integrated planning of health and social response support across sectors— involving health, education, social protection, government, agriculture, wa- ter, and so on—shows potential to improve crisis planning to address needs more comprehensively (appendix D). For example, in Senegal, coordinated planning across sectors (including health, social protection, agriculture, water, and others) allowed sectors to take on strategic roles in the response to cover a wide range of emergency and human capital needs.  lignment of COVID-19 Support with Health Response Box 2.3. A The World Bank’s early support aligned with national COVID-19 plans, which covered countries’ emergency health responses, typically in alignment with World Health Organization (WHO) guidance on strategic preparedness and response areas based Independent Evaluation Group World Bank Group    31 on International Health Regulations (WHO 2021a). About 70 percent of priority areas in national COVID-19 health responses were supported by the World Bank. Figure B2.3.1 shows the alignment of World Bank support with country COVID-19 health priorities. The main area with limited support was essential health services because this was added to WHO’s global guidance later in the response, through discussions with the World Bank and other partners. Aligning with WHO guidance was critical for coordi- nation with partners to support countries, but health responses were often not well integrated with responses of education, water, agriculture, and other sectors to help address broader needs of countries to protect human capital and vulnerable groups. (continued)  lignment of COVID-19 Support with Health Response (Cont.) Box 2.3. A  lignment of World Bank Support to Health Figure B2.3.1. A Priorities in Country COVID-19 Plans Essential Infection Health Health Case Health Prevention Risk System Country-Level Management Services and Control Laboratories Surveillance Communication Capacity Coordination (n=53) (n=41) (n=53) (n=56) (n=55) (n=53) (n=51) (n=64) Share of countries (percent) Share of countries (percent) 37 50 37 75 60 70 75 Sources: Independent Evaluation Group portfolio; country COVID-19 plans from World Health Organization action checklists (WHO 2021d). The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 Note: Early World Bank health support provided limited coverage of vaccination, given the priority of prevention and control in early COVID-19 plans. Vaccination committees were set up in countries in the later months of 2020 and in early 2021. The figure shows the percent of countries that had World Health Organization plans in a response area and received at least one World Bank intervention in that area. The analysis covers 66 countries with com- plete data on COVID-19 plans. Reorientation of the World Bank Portfolio to Respond to Needs Repurposing existing World Bank support in addition to adding new inter- ventions helped quickly address the early needs of the crisis response. About 60 percent of World Bank country programs had a medium to high extent of portfolio reorientation to address changing needs because of COVID-19, with extensive repurposing of projects and ASA in relevant sector areas and add- ing new support (figure 2.5). Repurposing projects already in place allowed the World Bank to rapidly address needs, often within a few days, because it built on existing structures and relationships. It also drew on surge capaci- ties across sectors by mobilizing relevant existing support in the portfolio for the COVID-19 response. GPs were often able to repurpose relevant projects by adjusting components to strengthen the project’s relevance in the evolv- ing context and, in some cases, fast-tracking previously planned support. For example, Djibouti adjusted urban support for slums to support health risk communication and to prevent the spread of infection. India adjusted 32 existing state-level projects to support needs related to education, health, and urban risks, complementing new project support. Uganda adjusted its nutrition support to include health risk communication and ensure contin- ued promotion of nutrition practices throughout COVID-19. Reorientation of the portfolio to address needs was quick in countries with crisis preparedness. Case studies show that World Bank country programs with previous crisis experience had a high extent of portfolio reorientation— reorienting five or more projects and ASA in the portfolio—to engage the support of multiple GPs in the COVID-19 response to address needs. Sixty percent of IDA countries reoriented four or more projects and ASA in their portfolios. About half of countries had a low extent of portfolio reorienta- tion, with limited repurposing of projects to add to new support for the early COVID-19 response. In addition, portfolio reorientation was slightly lower in FCS countries (figure 2.5). Figure 2.5. Extent of Portfolio Reorientation in Countries IDA (n = 50) Lending group IBRD or Blend (n = 45) FSC (n = 29) 0 10 20 30 40 50 60 70 80 90 100 Share of countries (percent) Independent Evaluation Group World Bank Group    33 Reorientation level Low Medium High Source: Independent Evaluation Group portfolio. Note: Reorientation is defined as the number of projects per country identified by the evaluation as responding to COVID-19, including financing projects and advisory services and analytics (ASA) sup- port. Reorientation levels are defined as terciles of its distribution across countries. Low: reorientation ≤ 3 projects or ASA; medium: reorientation = 4 projects or ASA; high: 5 projects or ASA ≤ reorientation ≤ 17 projects or ASA. Figure includes countries with both project and ASA support and excludes three countries with only regional projects (Grenada, St. Lucia, and St. Vincent and the Grenadines). The total number of countries is 95. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; FCS = fragile and conflict-affected situation. Prioritization of Support to Needs in Countries Prioritizing World Bank support to address urgent needs was easier in countries with better health systems capacity and readiness to respond, though case studies show good efforts across most countries to focus re- sponse actions progressively on areas of need and vulnerable groups. The clustering analysis for the evaluation (appendix D) found that the portfo- lio included countries that fall into three main situations in terms of their prioritization of World Bank support to the COVID-19 response to align with needs (box 2.4 describes these three situations). The analysis shows that prioritization was most challenging in countries with weaker health systems, which lead to slower government responsiveness to act on emergency mea- sures, such as gathering restrictions, masks, testing, and contract tracing, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 and extensive health and social needs. These countries often needed support to expand the health response and had multiple needs for protecting human capital losses of vulnerable groups as a result of COVID-19. Case studies show that in countries such as Mozambique and Uganda, where initial prior- itization of the response to address the many urgent needs was challenging, there was a progressive effort to focus COVID-19 support (such as risk com- munication, essential health services, and nutrition support) on vulnerable groups. Focusing interventions on vulnerable groups, local health services, and hot spot geographical areas was an important complement to helping government to quickly expand national disease response capacities. 34  ountry Situations Regarding Prioritization of Box 2.4. C COVID-19 Response Actions Countries where prioritizing World Bank support to address needs was facilitated by strong government responsiveness and preparedness: » About 11 percent of countries in the evaluation portfolio quickly tailored World Bank support to priority needs, including a focus on vulnerable groups. For ex- ample, India had early government responsiveness and some existing epidemic response capacities (relative to other countries in the evaluation) to put health measures in place; India focused the World Bank’s support on the national expan- sion of social protection systems, health services in urban areas, and education for vulnerable groups. The government of Honduras quickly focused World Bank support on laboratories and developing epidemic response capacities. Djibouti had rapid government leadership to focus on needs related to urban slums, ed- ucation networks in communities, and development of disease response capac- ities, including early support to vaccines. In Senegal, early government response and preparedness helped quickly focus the World Bank’s support to reinforce the country’s multisectoral response, which included health, nutrition, social protec- tion, education, and other support to address multiple needs. Countries where prioritizing World Bank support to address needs was facilitated by better capacities to deliver health services before COVID-19: Independent Evaluation Group World Bank Group    35 » About 53 percent of countries in the portfolio had better health systems capaci- ties to deliver services before COVID-19, which helped them focus their response in a few areas to address needs relating to health and social shocks among vul- nerable groups. These countries often faced a high number of cases of COVID-19 in the early response and also had good levels of government responsiveness to put prevention and control measures in place. For example, the World Bank’s response in Tajikistan focused on expanding social protection, laboratories, early vaccination, and citizen engagement. In the Philippines, the World Bank focused its response on community engagement, redeveloping dialogue with the govern- ment to strengthen health systems, and expanding social protection for vulnera- ble groups. (continued)  ountry Situations Regarding Prioritization of Box 2.4. C COVID-19 Response Actions (Cont.) Countries where prioritizing World Bank support to address needs was challenging and progressive throughout the early response, given limited health systems capacities and extensive human capital needs: » About 36 percent of countries in the portfolio had extensive health and social development needs before COVID-19 and low human capital; the key for these countries was protecting against losses of human capital. These countries also often had a lower number of reported cases early in the response and limited surveillance capacities to track cases. Among these countries, health service ca- pacities were often limited, even when there was preparedness before COVID-19. Some countries in this groups (such as Mali and Mauritania) worked with World The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 Bank regional projects during COVID-19, which helped engage government and supported progressive decisions to focus attention on geographical hot spots (such as border areas), laboratory interventions, and case management. Countries such as Niger and Uganda had support across sectors to address needs, but the health response was limited by existing health system capacities; prioritization to focus on the needs of girls and vulnerable youth, for example, was through the progressive strengthening of actions and often through the use of advisory services and analytics to inform actions. Source: Independent Evaluation Group situation analysis. Note: Appendix D includes the clustering analysis. Use of Knowledge Work to Inform Needs Just-in-time ASA to assess emerging needs was key to reorient and prioritize support. ASA was used in about 60 percent of countries. Key in countries was having ASA with some immediate, just-in-time outputs to inform the response. ASA was mainly for diagnostic analysis, technical assistance, studies to monitor the impact of COVID-19, and policy analysis (table 2.1). Conducting just-in-time ASA jointly with the government and partners helped support agreement on response needs and develop actionable 36 strategies. Previous evaluations show that preparatory ASA undertaken to support crisis response helps design effective crisis lending and analytic projects (World Bank 2012, 2017). Moreover, countries need to balance longer-term ASA to inform actions for recovery and just-in-time ASA, which can provide more rapid diagnostics for immediate response needs. The production of global and country knowledge products has continued to increase after the evaluation period to inform the evolution of response actions, for example, the 2022 World Development Report (World Bank 2022c). Examples of just-in-time ASA included the following: » In Djibouti, a gender analysis supported the response to COVID-19 in slums. » In India, the Transport GP conducted a just-in-time diagnostic of supply chain logistics during COVID-19 that helped the government plan for the delivery of oxygen and address the challenge of short supplies. » In Uganda, an assessment of COVID-19 communication helped the government develop a strategy to better engage vulnerable youth and women and girls.  xamples of Advisory Services and Analytics Supporting the Table 2.1. E Response Type of ASA Examples Diagnostic analysis » Inform the government about options to create fiscal space 93 percent (17 percent within existing financial resources to accommodate invest- multicountry) ments for the pandemic and continue routine health-care Independent Evaluation Group World Bank Group    37 delivery. Policy influence » Inform the government on the likely impact of COVID-19 67 percent (9 percent and the implications for policies and programs regarding multicountry) poverty reduction and economic growth Monitoring of COVID-19 » Identify COVID-19 and disaster hot spots using spatial data response analysis and social media data and monitor responses 63 percent (13 percent using digital payment modalities. multicountry) Technical assistance » Help the government strengthen the adaptive social 61 percent (15 percent protection system to increase the resilience of vulnerable multicountry) households to climate-related and other covariant shocks. Knowledge sharing » Exchange knowledge across countries on emerging 42 percent (12 percent guidance and good practices for remote learning to multicountry) reduce the learning losses caused by COVID-19. (continued) Type of ASA Examples Knowledge generation » Document experiences across countries on actions taken 29 percent (11 percent to control COVID-19, including lessons learned and any multicountry) disruption to routine service delivery. New evidence on » Research implementation challenges and successes effectiveness associated with the integration of gender-based violence 28 percent (8 percent prevention programming into social safety net systems. multicountry) Source: Independent Evaluation Group portfolio. Note: ASA are ordered based on the weight of their use in the COVID-19 response. Regional is used to describe multicountry ASA. Percentage in parentheses refers to ASA that addresses more than one country in a region or globally. ASA were coded by their main uses; thus, one ASA could include multi- ple types of analyses. Thirty percent of ASA support was regional. ASA = advisory services and analytics. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 Integration of Support for Institutional Strengthening and Recovery The integration of institutional strengthening in the COVID-19 response framework emphasized the importance of starting to build longer-term pre- paredness capacities from the early emergency response. Early institutional strengthening covered more than 90 percent of countries and focused on ba- sic capacities for the immediate crisis, such as strengthening multisector co- ordination, surveillance, laboratories, remote learning structures, and social registries (box 2.5), with the portfolio analysis showing particular attention to institutional strengthening in FCS countries and countries with region- al project support (appendix B). Support commonly went to early efforts seeking to improve coordination and health systems at the national level. Less attention in the early response was on strengthening local government systems and improving policy and financing (figure B2.5.1). The emphasis on institutional strengthening in the World Bank’s COVID-19 response brought the advantage of a longer-term systems rebuilding focus into the COVID-19 emergency, which has not been seen in past emergencies, such as for avian influenza. Other evaluations of COVID-19 responses note that institutional strengthening has so far been limited and requires further emphasis to sus- tain efforts (Johnson and Kennedy-Chouane 2021). 38 World Bank projects for the COVID-19 response often planned to help address the relief stage and to provide some support for restructuring sys- tems. For example, in health, World Bank project support was for laboratory equipment and training and strengthening laboratory networks. World Bank teams used analytic work and existing projects to help countries plan next steps to strengthen health systems (the Philippines, Senegal, Tajikistan, and Uganda); however, these efforts remain at an early stage. Some countries, such as India and Tajikistan, have also been supported to reconfigure supply chains. In education, projects planned remote learning, which also includ- ed safe reopening of schools (Djibouti, Senegal, and Uganda), but learning losses will need to be addressed. In social protection, projects supported emergency cash transfers and systems strengthening, building on lessons from past crises (World Bank 2012). This support has started to expand pub- lic health functions, although much attention has gone to helping manage continued cycles of emergency with waves of COVID-19 infection.  xamples of Institutional Strengthening Support in COVID-19 Box 2.5. E Response Country-level coordination: support to national and subnational COVID-19 planning, multisectoral coordination, emergency operation units, assessments to enable coordi- nation, operating procedures across sectors and actors, and online tracking of partner contributions Health system capacity: support to health referral systems, human resource planning Independent Evaluation Group World Bank Group    39 and development, use of geographic information systems to track diseases, improve- ments to coordination of surveillance and reporting systems, and laboratory quality Basic service delivery: improvements to education services (such as pedagogy and building safety), and social protection systems, including social registries to cover vulnerable groups such as migrants Policy and finance: policies to protect women and children, disaster and risk mitigation policies, expenditure reviews in human capital sectors, and costing of education sector reform needs (continued)  xamples of Institutional Strengthening Support in COVID-19 Box 2.5. E Response (Cont.) Local government strengthening: information and communications technology platforms for local government, community disease surveillance, expenditure man- agement and budgeting processes to improve service delivery, delivery of essential services (waste management, electricity, and water), and municipal performance grants for civil works. Figure B2.5.1 shows areas of early institutional strengthening support. Figure B2.5.1. Areas of Early Institutional Strengthening Support The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 Country-level coordination (n = 338) Health system capacity (n = 294) Thematic area Basic service delivery (n = 167) Policy and financing (n = 149) Local government strengthening (n = 33) 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 Share of institutional strengthening interventions (percent) Source: Independent Evaluation Group portfolio. Note: Interventions in the chart are based on 253 projects coded for the evaluation. The number of coded interventions for institutional strengthening support is 981; the number of projects is 253. Education and social protection support strongly emphasized strengthen- ing digitalization (more than 80 percent of projects), converting systems and business models to digital technology, often building on work before COVID-19. From the relief stage, a key part of institutional strengthening 40 was support for digitalization (66 percent of projects addressed digitali- zation, and 40 percent of innovations identified included digitalization), especially in FCS countries. In health, 59 percent of projects planned digital support, often for surveillance or case management (appendix B). Digitaliza- tion was expanded quicker where it could build on early foundational work before COVID-19, which was often the case for social protection, where there had been years of sustained investments countries made in creating national identification systems, digital payments systems, and integrated manage- ment information systems and social registries (such as in Brazil, Morocco, Senegal, Türkiye, and many other countries). In some countries, such as Sen- egal, digitalization actions were anchored in national development improve- ments. The evaluation could not assess the effectiveness of digital solutions to support outcomes. Examples of digitalization include the following: » In education, countries supported television, radio, and online pedagogy re- sources for student learning. In Honduras, this included packages for children and parents to follow up on television and radio classes. India developed a digital platform for teacher training. Support of Education Technology the- matic group helped rapidly scale up digital education solutions across coun- tries from early in the COVID-19 response. » In health, countries supported health information systems, contact-tracing applications, and digital surveillance. Mozambique, the Philippines, and Ta- jikistan developed digital tracking systems for vaccine rollout. In Tajikistan, Independent Evaluation Group World Bank Group    41 health sector assistance enabled information hotlines and electronic supply chain management. » In social protection, countries supported expanding digital beneficiary da- tabases and payment systems. India and the Philippines strengthened their national identification systems, with links to digitalized payments for social benefits, social registry data on vulnerable groups, and data on migrant la- borers. Djibouti supported an online platform for tracking food vouchers. Countries do not yet have strategies that will help them develop more resilient systems and sustained capacities for better crisis response preparedness. The needs for capacity building to sustain COVID-19 investments are vast and fall across sectors—health, education, social protection, agriculture, and so on. Case studies and regional project analyses suggest that countries with regional disease-focused projects (such as Senegal and Zambia) often already had approaches for building public health preparedness, which were being developed before COVID-19. Incipient World Bank strategies to help prioritize investments arise from World Bank papers and recent work, for example, in health, social protection, and education (Barış et al. 2021; World Bank 2020e; World Bank Group 2021a, 2021b). Analysis of the cost-effectiveness of interventions fell outside the scope of this evaluation, although it may be useful to optimize the use of resources in the future. Few projects considered the efficiency of scarce resources in the crisis response. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 2 42 1 The human capital data on investment before COVID-19 was coded as part of a separate Independent Evaluation Group analysis. The human capital data cover Health, Nutrition, and Population; Social Protection and Jobs; and Education Global Practice projects between July 3, 2014, and January 15, 2020 (World Bank, forthcoming). Interventions to support human capi- tal in countries before COVID-19 were reviewed in six areas: (i) essential health services (child survival and maternal mortality and improved equitable health access); (ii) critical health services (improved pandemic preparation capacity); (iii) protecting the vulnerable (connect- ing workers to jobs, expanded social program coverage, improved job skill readiness, improved targeting of lowest quintile, increased birth and social registration, and integrated social pro- tection systems); (iv) ensuring child welfare and social services (inclusive education, learning outcomes, quality of teaching, school environment, early childhood development, and stunted growth of children); (v) gender (fertility and adolescent pregnancy, gender-based violence, female higher education and science, technology, engineering, mathematics enrollment, and female labor participation); and (vi) digitalization (information and communication technol- ogy policies, information and communication technology for better targeting and for quality service, and digital skills). The total number of areas supported in a country before COVID-19 was used to identify countries with different levels of human capital support by quartiles: 1 (very low), 2 (low), 3 (high), and 4+ (very high). The analysis includes 80 countries in the evaluation portfolio with available data on human capital support before COVID-19. Independent Evaluation Group World Bank Group    43 3 | Quality of Response: Early Successes, Challenges, Learning, and Adjustment About half of projects had satisfactory implementation status. About 40 percent of countries had support to monitoring, critical health services, essential health services, and community activities—key for satisfactory implementation and a proxy indicator suggesting that countries are on track for results. Case studies point to early successes in countries across health, education, and social protection sectors, with continuous efforts to improve targeting of vulnerable groups and better reach frontline workers. The World Bank made good use of learning from past crises and is implementing intervention types with positive evidence of effec- tiveness from previous responses, although community activities and the intensity of risk communication were limited. Innovations in World Bank support to the early COVID-19 response offer an opportunity for systematic learning about how to implement new approaches for crisis preparedness and systems resilience. Response was swift where World Bank teams and government engaged in useful dialogue and where relationships and national and subnational structures had been developed for coordination and delivery of services before COVID-19. 44   World Bank teams strongly engaged with governments to make iterative adjustments to improve project implementation. Having real-time data on the quality of crisis-related activities in communi- ties facilitated corrective action; however, data systems capacity in countries was limited. Regional projects helped countries act rapidly to implement health interventions, but support to regional approaches was limited, despite the readiness and experience of some regional organiza- tions, particularly in Africa. Regional approaches were key for con- vening, knowledge sharing, and cooperation among government leaders and technical actors implementing responses. 45 This chapter assesses the quality of the World Bank’s implementation of the early COVID-19 response, including successes and challenges (which can point to early results) and learning to adjust and improve actions in countries. The assessment is based on dimensions of quality from the theory of action in figure 3.1.  imensions Assessed for Quality of Implementation Figure 3.1. D and Learning ADDRESSING NEEDS IMPLEMENTATION OPERATIONAL POLICIES (RELEVANCE) AND LEARNING AND PARTNERSHIPS • Based on needs • Implementation status • Internal coordination • Gender and facilitating factors • Instruments • Building on capacities • Early results • Streamlined processes • Aligned with plans • Building on past evidence • Procurement • Reorientation of portfolio and lessons • Monitoring and reporting • Prioritized to context • Innovation and learning • Partnerships • Use of knowledge work • Dialogue and coordination • Pandemic Emergency The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 • Integrated institutional • Adjustments Financing Facility strengthening • Regional knowledge sharing • Vaccines Source: Independent Evaluation Group portfolio. Implementation Status and Facilitating Factors About half of the health and social projects during the early response to COVID-19 had a satisfactory implementation progress rating, with IDA projects reporting better progress than the International Bank for Reconstruction and Development (figure 3.2). At the same time, as could have been expected, the proportion of projects with satisfactory implementation progress is lower than the period before COVID-19, likely as a result of the challenges World Bank teams and countries faced in implementing projects during a pandemic. In countries with more than 40 weeks of community spread of COVID-19 per the WHO classification, more projects have moderately satisfactory or lower implementation progress ratings, suggesting that implementation challenges increase when cases peak. 46 Figure 3.2. Implementation Progress Ratings of Projects in Countries Lending group IBRD or Blend (n = 95) IDA (n = 111) 0 10 20 30 40 50 60 70 80 90 100 Share of projects (percent) Implementation progress rating Unsatisfactory Moderately Moderately unsatisfactory satisfactory Satisfactory Highly satisfactory Source: Independent Evaluation Group portfolio. Note: Projects with no implementation progress rating and projects with financing other than IBRD, IDA, and blend were excluded from the analysis. Implementation Status and Results Report data were from November 5, 2021. The total number of projects is 206. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association. A mix of interventions in a few key areas, such as monitoring, critical health services (especially for laboratories), essential health services, and community activities, suggest that a World Bank country program is on track to facilitate results for the COVID-19 response. Having a mix of interventions appears to be a factor contributing to satisfactory implementation of the World Bank’s response in countries. The decision tree analysis (appendix D) Independent Evaluation Group World Bank Group    47 found that countries with support to monitoring, critical health services (especially laboratories but also in areas of infection prevention and control, case management, and surveillance, where capacities were limited before COVID-19), essential health services such as maternal and child health, and community activities (citizen engagement, gender equality, and urban health, such as communication and sanitation) were more likely to have projects with satisfactory implementation ratings (figure 3.3). About 40 percent of countries had support in most of these areas. Critical health services were well supported, but coverage of citizen engagement, essential health services, gender equality, and urban support was limited (figure 3.4). Interventions in critical health services were especially important in countries less prepared to deliver these services, pointing to the value of supporting countries to prepare for crisis. Case studies and evidence from the literature review (appendixes C and E) also reinforce the finding that essential health services are key for preventing losses of human capital among women and children in a crisis, community activities for health and nutrition messaging, and citizen engagement for trust and communication. A challenge in some case study countries was the limited capacity of the health system to deliver local-level health services, even when there was some crisis preparedness. Addressing gender equality was important in countries where this was a need before COVID-19, pointing to the value of using a gender lens in crisis preparedness and response measures. Providing urban health support within the crisis response was important in countries (such as Haiti, India, Tajikistan, and Uganda) with higher urban risks for the spread of COVID-19 in populations in cities in terms of population density and sanitation, for example.  actors Important to Satisfactory Implementation Figure 3.3. F The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 of Country Support Monitoring 1 Citizen engagement 2 Essential Laboratory 2 support health 2 services Gender 3 equality 3 Urban health risks Critical health services 4 capacities Source: Independent Evaluation Group portfolio and decision tree analysis. Note: The size of the leaf corresponds to its importance for predicting the likelihood that projects in that country will be deemed to have satisfactory implementation progress (satisfactory or highly satisfac- tory): (i) countries with high monitoring and tracking of early evidence of progress (top 50 percent of distribution) were more likely to have projects with satisfactory implementation progress; (ii) countries undertaking even one intervention in these areas were more likely to have projects with satisfactory implementation progress; (iii) in countries with needs in these areas, support to address gender equality and urban health risks was important—having better situations in terms of gender equality (top quartile) and urban health risks (top two quartiles) made it more likely to have projects with satisfactory imple- mentation progress; and (iv) countries with greater preparedness to deliver critical health services (top 50 percent of the distribution) were more likely to have projects with satisfactory implementation status. 48  overage of Country Support to Areas Important Figure 3.4. C to Facilitating Satisfactory Implementation Critical health services (n = 53) Monitoring (n = 106) Laboratories (n = 106) Gender equality (n = 53) Citizen engagement (n = 106) Essential health services (n = 106) Urban support (n = 66) 0 20 40 60 80 100 Share of countries (percent) Source: Independent Evaluation Group portfolio and needs analysis. Note: Percentages reported for critical health services, gender equality, and urban support measure the extent to which World Bank support was aligned with a country’s needs in those areas. The number of countries reported for each of those three areas (53, 53, and 66 countries, respectively) corresponds to the number of countries with needs in those areas in the bottom two quartiles. For all other areas, the number of countries reported is the number of eligible countries for the evaluation (106). Monitoring reflects countries with any indicators on the COVID-19 response monitored, not the level of monitoring. Urban needs consider urban health risks related to sanitation and water access, household size and type, and population density. Urban support includes health and social activities focused on urban communities. Critical health services include infection prevention and control, case management, surveillance, laboratory support, and risk communication. Needs related to critical health services are based on International Health Regulations data on laboratory, surveillance, and human resource capac- Independent Evaluation Group World Bank Group    49 ities in the country before COVID-19 (appendix D). Early Results Although it is too early to observe outcomes, case studies provide some evidence of early outputs, which point to successes of country support (appendix C). Examples of early successes include the expansion of critical health services such as COVID-19 testing, social protection benefits, and remote learning for children (box 3.1). Box 3.1. Examples of Early Results from Case Study Countries Ensuring Health Services In Djibouti, the Multiphase Programmatic Approach (MPA) enabled the development of guidelines and standardized sample collection methods and identified sites for intro- duction of point-of-care diagnostics. The MPA also helped with supplies for health facil- ities, such as polymerase chain reaction machines and COVID-19 test kits. By December 2020, the MPA was helping support the investigation of suspected cases of COVID-19 based on national guidelines, in a context of very limited capacity to deliver critical health services. Health workers were also trained in infection prevention and control per nationally approved protocols, and all acute health-care facilities had triage capacity. Protecting Poor and Vulnerable People The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 In the Philippines, at the onset of COVID-19, the World Bank helped the government to expand existing cash transfer support to provide monetary and food support to vulnerable households, including people with disabilities and indigenous persons and migrant workers. The support targeted 70 million households, of which 85 percent of recipients were women. The government also fast-tracked the use of digital payment and verification systems and links to the national identification system to improve coverage of vulnerable households and build longer-term capacities for managing emergency assistance. The long-running KALAHI Disaster Response Operations Modality Project was adjusted to provide cash transfers to protect populations during COVID-19, including for employees who lost their jobs and returned to their commu- nities and support for community-run projects focused on building local resilience during COVID-19, such as communal gardens and cleaning of facilities. Ensuring Child Welfare In Uganda, the COVID-19 Emergency Education Response Project developed online, paper-based, and radio home-based learning materials for preprimary, primary, and secondary school children and students with social needs, and guided standards and improved sanitation and other conditions in more than 20,000 schools for safe reopening. The project trained more than 10,000 teachers on psychosocial support to counsel learners and school workers on COVID-19 and challenges associated with the lockdown. The World Bank teams also helped the government to develop parenting education and support for early learning continuity, including radio programs. 50 (continued) Box 3.1. Examples of Early Results from Case Study Countries (Cont.) Risk Communication and Community Engagement In Senegal, the pandemic meant that community mobilization activities and house- hold visits had to be scaled down to avoid close contact. As soon as the government confirmed the urgency of prevention measures, the Early Years for Human Develop- ment Project developed and disseminated guidelines on how to conduct community activities in the context of COVID-19. Communication about preventing the spread of COVID-19 used existing networks, including local radio, to get nutrition information to households. The information also accompanied the provision of food and hygiene kits to high-risk groups, reaching more than 90 percent of targeted populations with messages. Source: Independent Evaluation Group case study analysis. Supporting national responses was complemented by adaptive actions during implementation to reach vulnerable groups and support frontline workers. An early success was the focus on vulnerable groups by the Social Protection and Jobs GP and the Macroeconomics, Trade, and Investment GP (more than 90 percent of projects; figure 3.5) and a greater focus on vulner- able groups in FCS countries (appendix B). In Urban, Disaster Risk Manage- ment, Resilience, and Land (58 percent of projects); Health, Nutrition, and Population (50 percent of projects); and Education (14 percent of projects), fewer projects targeted vulnerable groups from the onset, though there were Independent Evaluation Group World Bank Group    51 efforts to continuously improve support to better reach vulnerable groups. Health and education actions had broad population benefits through rap- id expansion of COVID-19 services and online learning. This needed to be complemented with actions to ensure the reach of vulnerable groups, such as women and children, communities with elevated risks of infection, and children in vulnerable households but also frontline workers who were over- whelmed by the crisis. Evidence from the Asian Development Bank, Enabel, the United Nations Children’s Fund (UNICEF), and the International Labour Organization confirms that vulnerable groups needed to be better targeted in the initial design of interventions (Johnson and Kennedy-Chouane 2021; Vancutsem and Mahieu 2020). Case studies also suggest challenges in both health and education in the intensity of support to frontline health work- ers and teachers to continue to provide services in communities during the crisis. However, in health, about 78 percent of countries had some support (such as to train health workers), and there was large-scale procurement of personal protective equipment across countries for health facilities, which likely benefited frontline workers. » Health, Nutrition, and Population support offered broad population benefits by financing national plans, but the success of national support in reaching local services was rarely monitored, and there was a need to adapt actions during implementation to ensure the reach of vulnerable local-level groups. For example, in Senegal and Uganda, supporting frontline health workers and communities and vulnerable women and children required the adjustment of actions during implementation. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 » Education support was often decentralized (63 percent targeted subnational areas and 19 percent communities; see appendix B) to benefit networks of parents, youth, and children in communities and schools and in some cases, children with special needs. Case studies and the portfolio analysis found that continuous attention was needed to focus actions on the most vulner- able groups, such as children in poor households and girls. Some countries started teacher coaching networks to better support teachers at the front line of the response (such as Djibouti and Uganda). » Social Protection and Jobs, by expanding systems to migrant workers and female head of households, stands out for its focus on women and girls and vulnerable groups. » Macroeconomics, Trade, and Investment and Urban, Disaster Risk Manage- ment, Resilience, and Land financing had broad population benefits and often supported policies and actions to benefit vulnerable groups, such as farmers, women, informal sector workers, migrant workers, and people in urban slums. 52 ntended Beneficiaries of World Bank Country Support Figure 3.5. I by Global Practice Broad Population Essential Women Benefits for COVID-19 Frontline Worker and Girls Education 12 16 26 (n = 43) Health, Nutrition, 94 78 22 and Population (n = 111) Macroeconomics, Trade, 77 23 27 and Investment (n = 26) Social Protection and 36 10 56 Jobs (n = 39) Urban, Resilience, Disaster Risk 79 26 18 Management, and Land (n = 34) 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Parents, Adolescents Vulnerable Caregivers, and Children and Youth Groups Education 70 72 14 (n = 43) Health, Nutrition, 14 6 50 and Population (n = 111) Macroeconomics, Trade, 12 23 88 and Investment (n = 26) Social Protection and 38 31 92 Jobs (n = 39) Urban, Resilience, Disaster Risk 15 12 56 Management, and Land (n = 34) 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 Projects (percent) Source: Independent Evaluation Group portfolio. Note: Bars indicate the percentage of projects with at least one World Bank intervention that targeted the beneficiary group. Analysis included 97 countries and 253 coded projects. Case studies point to some early successes where World Bank support Independent Evaluation Group World Bank Group    53 contributed to helping to target vulnerable groups in countries, but challenges remain (box 3.2). The data from available surveys in case study countries suggest some early success with helping countries with risk communication and social protection—key areas to which the World Bank and other partners contributed by, for example, helping to expand social protection responses during COVID-19. Challenges of the early response in terms of reaching vulnerable groups included facilitating access to essential health services (such as for women and children), learning for children, livelihoods of informal workers, and trust and social cohesion.  xamples of Successes and Challenges of Early COVID-19 Box 3.2. E Support Ensuring Health Services Challenges: » Surveys in Djibouti and India reported challenges in access to health care. In India, households reported forgoing health care because of fears associated with COVID-19. In Djibouti, Mozambique, Tajikistan, and Uganda, issues with ongoing access to health care were felt more severely by vulnerable groups. In Djibouti and Mozambique, those issues were more acute for women than men. Protecting Poor and Vulnerable People The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 Successes: » Social protection benefits were identified in surveys from Djibouti, India, the Phil- ippines, Senegal, and Tajikistan. For example, India’s social protection response covered a little more than 87 percent of poor households between May and August 2020. Challenges: » Negative impacts on the livelihoods of informal workers were reported in Djibouti, India, Mozambique, the Philippines, and Uganda. » Disparities were reported in Djibouti’s social assistance, with residents outside urban areas less likely to receive food stamps. Ensuring Child Welfare Challenges: » In Honduras, Mozambique, and the Philippines, children reported being unable to access virtual schooling because of issues related to the internet, equipment, and teachers. (continued) 54  xamples of Successes and Challenges of Early COVID-19 Box 3.2. E Support (Cont.) » In Djibouti, India, and Mozambique, respondents highlighted negative mental health issues among minority groups. In India and Mozambique, female respon- dents reported increased mental health issues linked to COVID-19. » In Uganda, after school reopening, less than half of the children returned to school. Risk Communication and Community Engagement Successes: » In Djibouti, the Philippines, and Tajikistan, high proportions of respondents re- ported adopting COVID-19 preventive measures, such as social distancing and handwashing. Challenges: » In Senegal, disparities in awareness of COVID-19 were reported among women; rural dwellers; and less educated, younger, and poorer populations. » In Uganda, preventive behaviors declined then stabilized by April 2021, except for handwashing, which continued to decline. » In Mozambique, Senegal, and Uganda, distrust of government was a reported challenge. Sources: Afrobarometer 2021a, 2021b, 2021c, 2021d; Bau et al. 2021; Bautista, Balibrea, and Bleza Independent Evaluation Group World Bank Group    55 2020; Bhattacharya and Roy 2021; Grover et al. 2020; Ipsos 2020; Sumalatha, Bhat, and Chitra 2021; Tuppal et al. 2021; WHO 2021b; World Bank 2021c, 2021d, 2021g; World Bank Microdata Library Database (https:/ /microdata.worldbank.org/index.php/catalog/4055; accessed September 2021); UN Women 2021. Building on Lessons and Evidence from Past Crises Compared with past crises, the World Bank’s COVID-19 support to countries was stronger in responsiveness to needs; however, some lessons from past crises were not fully integrated. The evaluation analyzed operational lessons from World Bank projects that supported crises over 20 years and benchmarked these against the early COVID-19 response (appendix F) and evaluations of crisis response (World Bank 2012, 2017, 2019a). Close navigation of the response with government is a key implementation success of the COVID-19 response that stands out against past crisis support, especially given the vast scale of the response compared with past crises (table 3.1). However, persistent challenges related to implementation and The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 learning were carried over from past crises in terms of reaching vulnerable groups, multisector coordination in countries, and engaging government in monitoring and using data to set priorities and inform risk communication messaging and behavior change. Success factors in these areas were limited to a few countries. Routine oversight of implementation was challenging because internet connectivity was limited, and it was difficult to communicate with subnational project implementers. These challenges were evident in case studies and in the analysis of success and challenge factors reported in project Implementation Status and Results Reports (figure 3.6). The World Bank is implementing intervention types with positive evidence of effectiveness from responses to past crises. The evaluation reviewed evidence on effective crisis interventions from systematic reviews and country studies to understand the extent that the current portfolio is positioned to support outcomes in countries (appendix E). The review of evidence identified 70 relevant articles covering 50 interventions relevant to areas of the COVID-19 response framework. Most of the portfolio had interventions with positive evidence of effectiveness from past crises (such as surveillance, case management, infection prevention and control, laboratories, and country-level coordination). 56  pplication of Operational Lessons from Past Crises in Table 3.1. A COVID-19 Past Lessons to Improve Application in Early Challenges in Early Crisis Response COVID-19 Response COVID-19 Response Ongoing responsiveness to » Continuous dialogue » Reach of vulnera- needs during crises was sup- with governments on ble groups required ported by frequently engag- response adapting actions during ing with clients to navigate » Focus on sectors with implementation. in-the-field realities. the highest potential impact—health, social protection, and edu- cation Coordinating roles and » Engaged in existing na- » Coordination capacities response areas with govern- tional platforms to help of government were of- ment and partners through- coordinate implemen- ten not well developed. out implementation was tation important to address emerg- ing priorities. Consistent monitoring of be- » Some country support » The intensity of commu- havior change was important included demand-side nication activities and for effective communication activities for commu- monitoring of behavior approaches. nities. changes was limited. Continuous engagement with » Supported corrective » Data use to inform deci- government helped support actions through weekly sions was limited. corrective actions. exchanges in countries Engaging government in on- » Virtual supervisions of » Some issues were likely going monitoring and review projects conducted missed as a result of helped prioritize support. challenges engaging subnational actors; data Independent Evaluation Group World Bank Group    57 on local responses were limited. Multisector coordination at » Support provided to » Multisector coordination national and subnational lev- existing coordination was limited. els helped ensure an effec- structures in countries tive response. Source: Independent Evaluation Group lessons analysis and case studies. Note: Lessons were extracted from 170 closed past projects that supported crisis response. Those shown in the table relate to the ongoing efforts to address needs during implementation and learning, which are compared with actions and challenges of the early COVID-19 response. Refer to appendix F for lessons analysis.  reas of Implementation Successes and Challenges in Figure 3.6. A COVID-19 Response Areas Continuous Monitoring of Coordinating Ongoing Support for Lesson Behavior Monitoring Roles and Areas Multisector Responsiveness Corrective Direction Change Priorities of Support Coordination to Needs Actions Success (n = 132) Challenge (n = 149) Share of countries (percent) Share of countries (percent) 1 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 10 1 45 20 30 45 Source: Independent Evaluation Group review of project Implementation Status and Results Reports for lessons analysis. Note: The COVID-19 portfolio includes 132 successes and 149 challenges in the selected areas of the figure, extracted from 119 projects coded with those areas. “Ongoing responsiveness to needs” looks at ongoing support to dialogue, diagnostics, drawing on existing capacities, prioritization of support to sectors, and targeting vulnerable groups. “Coordinating roles and areas of support” looks at the alignment with development actors and plans. “Monitoring of behavior change” looks at communica- tion and the monitoring of barriers and behaviors. “Continuous support for corrective actions” looks at adjustments made through project supervision and management. “Monitoring priorities” looks at use of monitoring data with clients to improve the quality of the response in local areas. “Multisector coordina- tion” looks at coordination across sectors nationally and of subnational actors (appendix F). Some interventions with positive evidence of effectiveness from past crises, such as risk communication and demand-side activities in communities, were limited in the response. Box 3.3 summarizes interventions with positive and consistent evidence from the systematic reviews and country studies early in the COVID-19 pandemic. Citizen engagement, risk communication, social cohesion, continuation of essential health services, sexual and reproductive health services, and psychosocial support together account for only about 15 percent of the early response portfolio, despite positive evidence of the effectiveness of these approaches. Psychosocial care in communities is an intervention with positive evidence, which may have helped address increasing distress among children (Loperfido et al. 2020). These areas are important to a prevention-oriented response to protect human capital; they also aligned with lessons for crisis response from Ebola and avian influenza 58 (Gold and Hutton 2020; World Bank 2021f). Interventions that have limited evidence in a crisis context offer opportunities for systematic learning, such as remote learning in schools and use of social media. ntervention Areas with Positive Evidence and Areas Box 3.3. I for Learning Intervention areas with positive evidence: » Building the capacity of community health workers » Provision of masks, respirators, and face coverings and infection prevention and control training for health workers » Combining community prevention measures (masks, hygiene, and physical distancing) » Telehealth for continuation of essential health services » Providing sexual and reproductive health services in emergencies » Surging capacity of human resources and adaptation of health facilities for case management » Active case surveillance and contact tracing, combined with rapid diagnosis and management, and quarantine measures » Strengthening health information and surveillance systems » Community engagement for risk communication, infection prevention and Independent Evaluation Group World Bank Group    59 control, hand hygiene, use of masks, and social distancing » Combining prevention communication with community-based messaging » Engaging existing community leaders and community-based structures » Mental health and psychosocial support programs in community and health structures » Unconditional cash transfers for social protection (continued) ntervention Areas with Positive Evidence and Areas Box 3.3. I for Learning (Cont.) » Financial and social support for protection of vulnerable girls in humanitarian settings » Point-of-care diagnostics, rationing medical supplies » Support for prompt and consistent policies in epidemics and regional coordination Intervention areas where evidence is inconsistent or lacking: » Digital and automated tools for case management and surveillance » Health workers’ use of other personal protective equipment (gloves, gowns, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 and eyewear) » Community-based surveillance » Social media for risk communication and monitoring of response and needs » Remote learning and school reopening measures for vulnerable populations » Models for supporting logistics and medical supply in crisis » Workplace mental health Source: Independent Evaluation Group portfolio. Innovation and Learning Innovations to which the World Bank’s financing and technical support to the early COVID-19 response contributed offer important learning that could be used to build more resilient systems for recovery. Stocktaking analysis identified innovations in more than 80 percent of countries in the portfolio, often reflecting new approaches or practices to strengthen systems (World Bank 2022a). Innovations to support the response were positively associated with the reorientation of World Bank country portfolios, suggesting that reorientation opens opportunities for innovation. Innovations were 60 also encouraged through regional projects, often building on experiences and investments before COVID-19. Assessing the effectiveness of these innovations to understand the benefit that they provided in the country was outside the scope of the evaluation but will be important. Monitoring the quality of health services and expanding delivery (such as through telehealth), expanding social registries using data analytics to identify vulnerable groups, and public-private partnerships to expand digitalization of systems were common areas of innovation. The engagement of local actors, multiple sectors (such as water, technology, and social sectors), and partnerships was observed across innovations (box 3.4). Innovations and ASA often addressed areas where evidence-based learning is important, such as remote learning support for schools. Innovations also often involved civil society and the private sector and point to opportunities to expand engagement of these groups in recovery to strengthen preparedness for crisis response. Further learning could help World Bank teams address areas that are import- ant but received limited attention in the early response. Few innovations (less than 5 percent) were found to support continuation of essential health services, the informal sector, risk communication, psychosocial support, cit- izen engagement, and social cohesion, which are limited in the World Bank’s early COVID-19 response. Moreover, only 10 percent of innovations iden- tified in the COVID-19 response addressed gender disparities (World Bank 2022a). Evidence from the literature and past lessons also emphasize the importance of these areas for crisis response (Gold and Hutton 2020; World Independent Evaluation Group World Bank Group    61 Bank 2021f). Support to manage burnout among local health-care workers, handle misinformation, and garner trust, which case studies and interviews identified as challenges, received very little attention. Learning about the effectiveness of interventions, especially where evidence is inconsistent or lacking, could help ensure the right mix of support to countries. Remote learning support in crises stands out as a widespread intervention undertaken across countries for which the rapid review conducted by the evaluation did not identify evidence, although there may be evidence from noncrisis contexts that is transferable. Other interventions for which evidence of effectiveness is inconsistent or lacking include the use of social media in crises to communicate messages, the use of digital and automated tools for case management, and logistics supply management. Community-level surveillance has also been important in countries where it has been supported. Deepening of evidence could also be helpful in areas with few studies, such as for the expansion of social protection in emergen- cies. For example, evidence was collected on social protection interventions during the early COVID-19 response (Gentilini 2022). Box 3.4. Examples of Innovations Supporting the COVID-19 Response » In Uzbekistan, people receive information about COVID-19 through SMS messag- es, Telegram, WhatsApp, video clips, and infographics. In addition, health services are adapting for telemedicine where possible. » In Mali, a new national 24-hour-a-day, 7-day-a-week call center dedicated to COVID-19 enables free calls and offers advice for implementing coronavirus protocols. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 » In Senegal, community-based disease surveillance and multistakeholder engagement allow community health workers and volunteers to detect COVID-19 and report cases to health facilities and local government agencies. » In Latin America, projects track the presence of COVID-19 in wastewater. Wastewater-based epidemiology supplies real-time information on the extent of virus spread in a community, including asymptomatic cases. Each sample represents a large portion of the community, which is served by a sewerage network; this allows for rapid and cost-effective tracking of disease trends at the population level. » In Cambodia and India, instructional videos, conference calls, and social media supplement coaching services for teachers. Rural teachers receive video lessons on teaching culturally relevant, curricula-aligned content. Source: Independent Evaluation Group innovation stocktaking. Note: SMS = short messaging service. 62 Coordination, Dialogue, and Adjustment in Countries Good pre–COVID-19 relationships were a factor for good dialogue and smoother implementation. Previous evaluations related to financial, social, and environmental shocks also establish the importance of country dialogue in crisis response (World Bank 2012, 2017). Commitment to dialogue with government was strong, often with weekly discussions of emerging challenges and urgent reforms. Good relationships before COVID-19 prepared countries for strong implementation. Examples include the following: » In Honduras, the country’s social response built on long-standing policy dialogue with the government, which deepened through daily exchanges and was informed by ASA. In health, a new policy dialogue developed about emergency measures, which initially slowed the response but later transi- tioned to a dialogue aimed at ensuring better health systems. In education, the dialogue helped with quick project adjustments. » In the Philippines, the MPA supported the first health project after years of no project. Hence, health dialogue with the World Bank deepened in the early COVID-19 response and then expanded to develop early access to vaccines and planning of health systems strengthening. » In Mozambique, without a centrally organized government response, the Independent Evaluation Group World Bank Group    63 World Bank led sector-specific responses in dialogue with relevant ministries and other development partners. Support focused on the health, education, urban, and social protection sectors. The response built on existing sector relationships and accelerated the pace and direction of measures underway before the pandemic. Where available, national platforms helped engage government sectors and development partners to coordinate implementation. Case studies show good engagement in national coordinating structures of government during implementation, even though multisector coordination structures were rare (box 3.5). For example, in India, the Philippines, and Uganda, the World Bank supported coordination platforms during COVID-19. Some World Bank teams helped reinforce government’s coordination capacity, although it was challenging to reinforce amid the crisis. For example, Tajikistan hired a consultant to help with coordination. In Uganda, a dedicated staff member in the World Bank office supported partner coordination with government. Moreover, an important weakness of the World Bank’s support to the re- sponse was the limited engagement with nongovernmental organizations— an issue identified with many COVID-19 responses (OECD 2022).  essons on Multisector Coordination for an Integrated Box 3.5. L Response Where countries had established coordination structures to engage government sectors, partners, and other stakeholders, the World Bank supported them, and the government used them to plan and track integrated COVID-19 actions. About 10 percent The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 of countries had support to multisector coordination groups to implement the response, such as One Health committees. Having these structures set up before COVID-19 to organize the response was key because setting up coordination for the first time is challenging. Having multisector coordination structures was also important for crisis pre- paredness. The World Bank’s COVID-19 recovery efforts are emphasizing the establish- ment of One Health coordination within countries to support multisector responses and strengthen coordination structures. This aligns with the efforts of other agencies—the Food and Agriculture Organization, the World Organisation for Animal Health, the United Nations Environment Programme, and the World Health Organization. When multisector structures were developed, they provided a platform for countries to plan, report on, and take rapid actions with more integration across sectors: » In Haiti, the government created a multisectoral commission to coordinate the COVID-19 response, integrating mechanisms for civil society and the private sector to contribute to emergency preparedness actions, and specific units for crisis response in the health sector. This helped align sector and stakeholder support to the response. » In the Philippines, the national response was coordinated by an intersectoral task force initially led by the health sector, but early in the response, the task force transitioned to central government leadership to ensure multisector support across ministries. 64 (continued)  essons on Multisector Coordination for an Integrated Box 3.5. L Response (Cont.) » In Senegal, the World Bank supported a One Health multisectoral approach to coordinate the COVID-19 response. This approach grew to include ministries responsible for finance, health, social affairs, livestock and animals, agriculture, rural development, environment and sustainable development, and water and sanitation. Since COVID-19, the approach has included education. Knowledge work in COVID-19 reinforced coordination by helping inform how to op- erationalize crisis response actions. Previous evaluations also found that coordinating support with government and partners, combined with knowledge work, enabled the World Bank to develop well-designed financing projects expeditiously (World Bank 2017). For example, the multisectoral response in the Philippines built on long-term knowledge work in social protection and community development. Sources: Independent Evaluation Group portfolio and case studies; FAO et al. 2022. Where available, networks that reached communities were instrumental in risk communication, detecting COVID-19 cases, and providing referrals to health services, but overall, the connection to local government— especially at the community level—was not strong enough. Enabling local government and community groups to support crisis response and ensure Independent Evaluation Group World Bank Group    65 frontline service delivery was a challenge in case study countries, with breaks in communication, coordination, and disruptions in implementation. Evaluations of the responses by Chazaly and Goldman (2021) and the German Institute for Development Evaluation (Schneider et al. 2020) also noted similar issues. Examples of coordination support reaching local levels to a greater degree include the following: » Structures for nutrition (Honduras, Senegal, and Uganda) facilitated COVID-19 messaging to communities and helped engage nongovernmental organizations in the COVID-19 response. The World Bank is undertaking sim- ilar work in developing community networks in other countries (Subandoro, Holschneider, and Ruel-Bergeron 2021). » One Health platforms with networks that reach the community level (Senegal and Zambia) supported contact tracing, case management, communication, and other activities to prevent the spread of disease. » Support to local government (Senegal and Uganda) helped ensure continued delivery of local services, such as health, water, and education. Implementation Adjustment Throughout the early response, World Bank teams strongly engaged with governments to make iterative adjustments to improve implementation. These adjustments were informed by frequent meetings and virtual supervision support, findings from studies, and field supervision by the governments when travel was possible. Discussions often identified The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 implementation bottlenecks and led to corrective actions to adjust projects, such as the following: » In Djibouti, the package of services delivered to vulnerable groups by social protection agents was adapted to include COVID-19 messages and distribu- tion of hygiene kits, based on a challenge identified in country discussions. » In Senegal, the health project added support to help the government recruit contractual staff to surge resources for patient care and provide computer and video equipment for communication between health offices. » In Tajikistan, the country strengthened its support to community engage- ment and risk communication as part of its vaccine response based on learning from the early response. » In Uganda, the country added COVID-19 training support for the private sector based on challenges in support to these facilities. Real-Time Data for Decision-Making Real-time data, where available, were key to informing decision-making about project adjustments. Geo-enabled monitoring, iterative beneficiary monitoring, short messaging service (SMS) texts, online surveys, and dash- boards are tools that supplied timely data to support implementation de- cisions in the early COVID-19 response (box 3.6). Global surveys provided 66 valuable real-time information on the socioeconomic impact of COVID-19, which informed policy dialogue and country economic updates and were critical for discussion with civil society forums in countries. However, these surveys often did not offer real-time information to support immediate course corrections of projects or develop data capacities in countries. In some countries, real-time surveys were adapted to better link to social pro- tection, vaccine promotion, and education projects to inform course correc- tions. Evaluations of COVID-19 responses have also found the application of real-time data collection methods to be useful in adapting projects (Johnson and Kennedy-Chouane 2021; OECD 2022; Vancutsem and Mahieu 2020), and a range of innovative data tools have been used in COVID-19. A challenge is supporting governments to continue real-time data methods that enhance country-level information systems and to coordinate data collection that ensures that the most helpful information for implementation decisions is being collected.  xamples of Real-Time Data Systems and Tools Box 3.6. E for Decision-Making Monitoring of country situations: » In Cambodia, Myanmar, and other countries with high mobile phone coverage, high-frequency phone surveys provided rapid, real-time data and evidence on Independent Evaluation Group World Bank Group    67 the socioeconomic impact of COVID-19 to inform World Bank responses. » In Colombia, a COVID-19 Safe Economic Reactivation Dashboard provides deci- sion makers with real-time information for 1,100 municipalities on key epidemio- logical indicators. To date, the dashboard has more than 15,000 unique users. » In Fiji, the World Bank supported the Ministry of Health and Medical Services to improve communication and data reporting systems with frontline health workers and health facilities, including internet connectivity for case reporting and public health surveillance across health facilities. » In Iraq, Lebanon, Libya, Tunisia, and West Bank and Gaza, a low-cost, just-in- time survey was deployed via a Facebook chatbot to understand attitudes toward vaccines among different social groups. To ensure a degree of national (continued)  xamples of Real-Time Data Systems and Tools Box 3.6. E for Decision-Making (Cont.) representativeness, the survey targeted clusters of individuals according to region, age, and gender. Monitoring of implementation quality: » In Lesotho, a phone-based application sends an SMS message to those who are vaccinated to collect information on the quality of care and track minor side effects that would otherwise not be reported. » In Tunisia, COVID-19 cash transfers are remotely monitored using iterative benefi- ciary monitoring. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 » In Mexico, Nicaragua, and other countries, the Geo-Enabling Monitoring and Supervision (GEMS) initiative has been introduced as a project supervision and monitoring tool for continuous engagement with communities and project super- vision. The governments undertook the capacity-building program on GEMS and internalized the technology tools for monitoring. For example, in Uganda, GEMS is used for real-time project monitoring of support to schools and community groups to promote nutrition. In Mali, GEMS is a platform for third-party monitoring in conflict-affected remote areas. Tajikistan is using GEMS to monitor cash transfers. » In India, the Gujrat Command and Control Centre provides an example of re- al-time performance data for schools, covering online attendance, assessment test results, and a vehicle tracking system. Sources: Independent Evaluation Group portfolio and case studies; World Bank 2022a. Note: SMS = short messaging service. Data on implementation quality in subnational and community responses by local governments remain a key gap in the early COVID-19 response. Few countries had systems established before COVID-19 that could be used to provide beneficiary feedback on services or data on the quality and cover- age of services delivered by health workers, schools, and community actors, 68 especially for vulnerable groups. Another challenge in the response was tracking the success of messaging and communication activities in promot- ing behavior changes in the community related to COVID-19. Strengthening country systems to supply more real-time information on the implementa- tion quality of local government and community-level support could facili- tate preparedness efforts. Regional Knowledge Sharing and Cooperation Regional projects strongly supported implementation of responses. The evalu- ation reviewed the four main regional disease-focused projects that worked on the COVID-19 crisis in countries with weak health systems or limited capaci- ties to respond to crises, highlighting strong early results (box 3.7 and appen- dix G). Coordination facilitated by these regional projects supported political leadership; real-time technical learning to operationalize and review progress of COVID-19 plans; cooperation for efficiencies across countries, such as for procurement of medical goods and equipment; joint training to expand COVID-19 surveillance data and testing; and high-level dialogue to develop public health guidelines. The peer learning of ministries and technical experts across countries that engaged in regional activities helped expand leadership for the response and had a spillover effect on countries not covered by region- al projects that joined the activities. Leaders and technical experts from public health institutions engaged in regional exchanges and then adopted new prac- tices learned from these engagements to expand COVID-19 actions in their Independent Evaluation Group World Bank Group    69 countries. Despite the strong positive early results of regional projects during the COVID-19 response, support to regional projects was limited to only 23 percent of countries in the portfolio, although there has since been some effort to expand regional support in Africa in the second year of the response. A previous IEG evaluation highlighted the untapped potential in fostering re- gional integration initiatives as the World Bank can leverage its global knowl- edge, financing instruments, synergies from acting as one Bank Group, and its ability to catalyze regional actors (World Bank 2019c). Box 3.7. Examples of Early Results Contributed by Regional Projects The evaluation conducted interviews with country actors and teams involved in regional projects. Among the four projects reviewed, Regional Disease Surveillance Systems Enhancement Project in West Africa and East Africa Public Health Laboratory Network had established networks in countries before the COVID-19 pandemic start- ed, whereas the Africa Centres for Disease Control and Prevention and Organisation of Eastern Caribbean States regional health projects were at early stages of implemen- tation. For all projects, the knowledge exchange and real-time dialogue and coordi- nation facilitated by regional engagement was viewed consistently as valuable to help support country actions. Early results were identified for all four regional projects, although there was more ev- idence in countries with a longer duration of regional support before COVID-19 (such The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 as Senegal and Togo, covered by early phases of the Regional Disease Surveillance Systems Enhancement Project). Regional projects were especially well-situated to support critical health services for the COVID-19 response when projects were already established and well-integrated with project support in their respective countries. Early results supported by regional projects include the following: » Regional coordination facilitated rapid country responses to COVID-19, whether of ministerial committees, public health institutes, or project leaders. Coordination mechanisms were used to share real-time information and knowledge on disease detection and best practices, engage with regional and international partners, develop guidance on surveillance, and exchange knowledge with peers to help countries implement COVID-19 support. For example, governments in the Eastern Caribbean used regional knowledge sharing about health waste management and installation of laboratory and health equipment to respond faster. In Africa, public health institutes and health ministries used the Africa Centres for Disease Control and Prevention regional Extension for Community Healthcare Outcomes platforms for interdisciplinary knowledge exchange and training on COVID-19. » Human resources capacities supported by regional projects helped implement the COVID-19 response. For example, countries in West Africa and the Eastern Caribbean deployed field epidemiology graduates in leading strategic and front- line roles for COVID-19. Capacities developed before COVID-19 were crucial to 70 (continued)  xamples of Early Results Contributed by Regional Projects Box 3.7. E (Cont.) rapid responses in East and West Africa. In East Africa, governments deployed lab technicians, assessors, and disease surveillance officers trained through East Africa Public Health Laboratory Network to be at the front line in COVID-19 rapid response teams, conducting testing and contact tracing. Countries used rapid response teams supported by the Regional Disease Surveillance Systems Enhancement Project to implement the COVID-19 response. » Regional coordination and capacity building facilitated the expansion of surveil- lance, testing, border screening, case management, and infection prevention and control for the COVID-19 response. For example, East Africa countries deployed laboratory capacity built by the East Africa Public Health Laboratory Network project for COVID-19 testing. The Africa Centres for Disease Control and Preven- tion contributed to the rollout and expansion of COVID-19 testing in Africa. At the start of COVID-19, only two laboratories in Africa (Senegal and South Africa) could reliably test for the disease. By mid-March 2020, 43 countries had testing capabili- ty, and by August, almost all African Union countries could conduct testing. Source: Independent Evaluation Group regional project review. Regional projects were well situated to support countries in efficiently co- Independent Evaluation Group World Bank Group    71 operating to expand critical health services but were less prepared to help countries plan actions to mitigate COVID-19 impacts. Regional projects helped countries expand testing, surveillance, and case management quick- ly. They were less helpful in sharing knowledge to implement interventions for risk communication, citizen engagement, gender equality, urban public health risks, and essential services. Discussion is ongoing on expanding at- tention to these areas in regional disease-focused projects, based on learning from the early COVID-19 response. Capacities could be developed across countries, such as for citizen engagement, through learning to facilitate actions. Improving essential health services in countries may also be im- portant to address regionally because having this capacity helped countries respond quicker. Strengthening developing capacities of regional organizations for disease response coordination proved to be important. Institutional strengthening before COVID-19 prepared regional organizations to support results during COVID-19, particularly the Economic Community of West African States. Setting up dialogue across countries for the first time during a crisis was more difficult for newer regional projects because actors have limited expe- rience coordinating support and need time to set expectations and develop partnerships and trust. Nevertheless, new platforms (such as the Regional Coordination Center in Zambia for Southern Africa, supported by the Afri- ca Centres for Disease Control and Prevention [Africa CDC] project) were important for learning and cooperation during COVID-19. The Caribbean Public Health Agency, supported by the Organisation of Eastern Caribbean States Regional Health Project, played an important coordination role in The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 3 facilitating country responses during COVID-19. The World Bank did not strengthen support to regional approaches early in COVID-19. In Africa, expanding regional support at the onset of COVID-19 may have facilitated the response, such as through including regional proj- ects in the MPA and by providing financing to regional organizations for knowledge sharing and coordination across countries. The Regional Dis- ease Surveillance Systems Enhancement Project in West Africa and the East Africa Public Health Laboratory Network project already had extensive experience before COVID-19 in demonstrating learning about how to work regionally with countries on disease preparedness and response. Moreover, before COVID-19, there was already extensive preparatory work on the Afri- ca CDC project; although the project was at an early stage of implementation and had limited disbursement, much important work to develop coordina- tion and knowledge sharing structures was already in place and could have been drawn on by the World Bank to facilitate the early actions in countries. Hands-on technical support and facilitating learning, convening, and cooperation across countries were important for regional capacity building. Developing the capacity of regional organizations to facilitate high-level country exchanges among leaders and technical learning and cooperation was critical. Project financing to implement interventions was often from a country-level project, and minimum regional project financing and disburse- 72 ment often were required to influence the leadership and implementation improvements observed through regional engagement. For example, region- al support of the Africa CDC was linked to financing of World Bank projects in Ethiopia and Zambia. Expanded regional support could help develop leadership and coordination in countries, which could then help scale up preparedness capacities and improve the efficient use of resources in World Bank country projects. Learning across countries could be supported through regional projects or ASA focused on regional capacity building and South- South knowledge sharing (World Bank 2019b). Independent Evaluation Group World Bank Group    73 4 | Quality of the Response: Operational Policies and Partnerships Strong leadership at multiple levels in the World Bank enabled innovations and information sharing across sectors, and policy guidance and operational frameworks helped steer the response in the early months. Coordination of country support was done through portfolio review meetings and typically included the health, education, and social protection sectors. In some countries, coordination might have been improved by including sectors beyond human development. Crisis instruments, repurposed projects, regional projects, trust funds, and grants, where available, were important for rapid financing in the early days and weeks of the crisis. The Multiphase Programmatic Approach was innovative and useful to quickly expand new lending for the health response. Operational flexibilities have improved since past crises, but the processing of new projects took several months. Flexibilities in new project processing helped process the Multiphase Programmatic Approach. Countries needed more help with corporate requirements related to gender and citizen engagement. Procurement was successfully expedited; however, tracking procured goods to ensure that they were received by health facilities was challenged by weak country systems. Project monitoring systems provided some timely information during the crisis, but integrated country-level monitoring was not always in place. 74   Existing partnerships involving the World Bank supported preparedness for global and country-level collaboration. Technical partnerships, even though they had limited country coverage, helped expand implementation of interventions. Pandemic Emergency Financing Facility grants supported COVID-19 plans in collaboration with United Nations partners, although funding was small and not rapid. The World Bank engaged in close dialogue with partners in the uncertain early months but lacked a financing instrument to help expedite advance market commitments for countries to access vaccines. Multiphase Programmatic Approach vaccine financing was prompt in countries, but supply constraints slowed initial im- plementation. Stronger regional support in Africa on vaccines earli- er in the response may have helped facilitate access to vaccines. 75 This chapter assesses the quality of the World Bank’s operational poli- cies and partnerships in support of countries during the early response to COVID-19. The assessment is based on dimensions of quality from the theory of action in figure 4.1.  imensions Assessed for Quality of World Bank Operational Figure 4.1. D Processes and Partnerships to Support Country Responses ADDRESSING NEEDS IMPLEMENTATION OPERATIONAL POLICIES (RELEVANCE) AND LEARNING AND PARTNERSHIPS • Based on needs • Implementation status • Internal coordination • Gender and facilitating factors • Instruments • Building on capacities • Early results • Streamlined processes • Aligned with plans • Building on past evidence • Procurement • Reorientation of portfolio and lessons • Monitoring and reporting • Prioritized to context • Innovation and learning • Partnerships • Use of knowledge work • Dialogue and coordination • Pandemic Emergency The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 • Integrated institutional • Adjustments Financing Facility strengthening • Regional knowledge sharing • Vaccines Source: Independent Evaluation Group portfolio. Internal Coordination Strong internal leadership at multiple levels enabled innovations and infor- mation sharing that helped coordinate the response. World Bank leadership led a large-scale response supported by a strategy and undertook internal innovations (box 4.1). IDA spending allocations were front-loaded to ensure resources for COVID-19 through an unprecedented move to start Replenish- ment discussions one year ahead of schedule in April 2021. An emergency operations center led by the Human Development GP and chaired by the Health, Nutrition, and Population GP facilitated technical coordination, adaptive management, and problem-solving for the response. Information sharing among technical staff and across GPs and operational support units increased through, for example, the production of guidance and technical notes, regular learning seminars, and listing answers to frequently asked questions. For example, Social Protection and Jobs; Education; and Health, Nutrition, and Population engaged in the center and developed complemen- tary actions to support the response, building on human capital work done before COVID-19 (World Bank Group 2020a). The Poverty and Equity GP 76 worked with other GPs and the Development Economics Vice Presidency on rapid phone surveys focused on COVID-19. A challenge noted in some cases was aligning actions to draw on technical capabilities of other GPs, such as Water, Transport, Agriculture and Food, and Social Sustainability and Inclu- sion. These sectors had technical knowledge to support risk communication, vaccine delivery, transport of goods, and sanitation—a first line of defense against COVID-19 before vaccines. Box 4.1. Cross-Sectoral and Unit Teams Supported Internal Innovations Internal innovations were stimulated when internal expertise was marshaled rapidly across sectors and units to solve technical issues. The following three examples illustrate the importance of diverse expertise and flexibility for the early response: » Emergency Operations Center (EOC): The EOC was set up in early 2020 to provide internal global technical coordination among Global Practices (GPs) and opera- tional policy and country service units and subcommittees working on specific technical issues. The EOC brought together World Bank experts in health, epide- miology, social protection, agriculture, education, water, operational, legal, and fiduciary functions and staff working with Gavi, the Vaccine Alliance; Pandemic Emergency Financing Facility; Global Financing Facility; Global Partnership for Education; World Health Organization; and countries from all regions. The EOC developed a shared understanding of what the World Bank was doing and coor- Independent Evaluation Group World Bank Group    77 dinated decisions in real time. The EOC had an agile governance structure with a director-level steering committee and members from the GPs and operational support units, and it aimed to work in a nonhierarchical manner and break down operational silos. The EOC successfully mobilized teams across the World Bank through weekly meetings to share information on the COVID-19 response. Inter- viewees praised the EOC’s role in solving problems and providing guidance on technical issues. The GP leadership and ownership inspired knowledge sharing among technical experts across units. Later, the EOC was transferred to the Op- erations Policy and Country Services of the World Bank to focus on vaccines. This decision limited GP leadership in the problem-solving and adaptive management of the response and likely missed an opportunity to build on the achievements of the EOC to strengthen GP collaboration for early COVID-19 vaccine support. (continued)  ross-Sectoral and Unit Teams Supported Internal Innovations Box 4.1. C (Cont.) » Country program flexibility: Good coordination of cross-sector GP engagement at the country level was critical to developing a real-time strategy for individ- ualized responses in dialogue with governments to reorient their portfolios to support country needs. The flexibility given to country teams in the early months of the response was key to rapidly adapting their existing portfolio of projects and advisory services and analytics in different sectors based on local policy dialogue. This adaptation was best done through internal discussion across GP teams where the portfolio before COVID-19 had set a foundation for supporting human capital, and the response could draw on existing partnerships, projects, and relationships. Having relevant cross-sector support in place before COVID-19 facilitated country The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 preparedness. Moreover, ongoing discussions across World Bank GPs in countries were key for continuous adjustments to address bottlenecks and strengthen the response, adding new elements such as communication or urban sanitation. » World Bank–facilitated procurement: Global supply chains for medical goods were severely disrupted early in the pandemic, with countries unable to obtain needed supplies, such as personal protective equipment. In response, the World Bank assembled a team of procurement, human development, legal, and gover- nance experts, which expanded on previous hands-on implementation support. The multidisciplinary team helped countries access critical goods that they could not obtain on the market. For example, World Bank–facilitated procurement helped deliver personal protective equipment for Honduras, oxygen for India, and personal protective equipment and respiratory and diagnostic equipment for Mozambique. Completed World Bank–facilitated procurement was valued at just under US$170 million in those early months, despite challenging turnaround times and complicated logistics and contracting processes. This mechanism accounted for about 4 percent of all procurement of goods delivered to 28 coun- tries. To obtain urgently needed goods quickly, the World Bank team collaborated with partners from United Nations agencies and the private sector. The model could be deployed in future emergencies. Source: Independent Evaluation Group interviews and case studies. 78 Policy guidelines and frameworks helped guide World Bank teams early in the response. Policy notes developed by GPs and other internal units helped guide actions in the first months of the response. A series of internal learn- ing events for staff also helped. In March 2020, the Health, Nutrition, and Population GP approved its Strategic Preparedness and Response Program MPA, the first phase of which committed $6 billion in funding (World Bank 2020b). The MPA, anchored in a cross-sector response framework, helped guide World Bank teams planning the health and social response in coun- tries. Social Protection and Jobs guided technical planning of World Bank teams by providing a continuously updated database and survey of global social protection responses, including a “Living Paper” detailing the design of support to share country experiences and a social response framework. Country directors and World Bank staff and clients highlighted these social protection resources as being useful in their early policy dialogue to guide response actions. The Education team engaged with partners to develop tools that helped clients implement quality responses, such as modules for phone surveys, COVID-19 impact assessment tools, technical notes for school reopening and teacher performance, and examples of content for remote learning and reports on policy responses (World Bank 2020e, 2021e). The Gender Group, with staff from across GPs, provided a range of resourc- es that defined entry points to address gender in the early response, for example, for vaccination projects. Interviewees reported that these frame- works and tools provided timely and useful guidance to help shape country Independent Evaluation Group World Bank Group    79 responses. The early documents and learning developed by GPs and other units also helped shape the Bank Group COVID-19 response framework in June 2020 (World Bank Group 2020b). The portfolio review process, led by country management, coordinated GP support and was important to identify ways to reorient the portfolio quickly and process requests in the early months of the response. Senior staff in countries noted that in the first months of COVID-19, real-time communication between World Bank country management and global technical teams—especially on the health response—was critical, such as on how to involve key sectors to help identify solutions for reorienting assistance in the country portfolio. In Honduras, project implementation and procurement plans were reviewed through weekly exchanges, and collaboration across GP teams helped reorient the portfolio quickly and process requests from the government. In Madagascar, by repurposing project support, the Urban, Disaster Risk Management, Resilience, and Land; Health, Nutrition, and Population; and Social Protection and Jobs GPs collaborated to facilitate social distancing, hygiene services, and handwashing stations for public transport and to provide cash transfers and cash-for-work activities. India and the Philippines conducted frequent portfolio reviews to coordinate support. Case studies and the portfolio review suggest that drawing on a wider range of World Bank sectors helped the response. Forty percent of projects collab- orated with one other GP (figure 4.2). Collaboration varied by instrument— development policy financing (DPF) and catastrophe deferred drawdown option (CAT DDO) had the highest collaboration, as did CERC and regional The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 projects, although to a lesser extent (more than half of these projects had GP collaboration). About 60 percent of World Bank country programs had collaboration involving at least one sector beyond the Human Development GPs, such as Finance, Competitiveness, and Innovation; Agriculture and Food; Social Sustainability and Inclusion; Water; and Governance. More frequent collaboration with a range of GPs would have helped support local government, sanitation, gender, psychosocial support, nutrition, citizen engagement, and other areas. One way that collaboration was encouraged was through joint projects that engaged Health, Nutrition, and Population, but it was also important to align support across GPs, such as to the MPA, to help address interrelated needs. Examples of where Health, Nutrition, and Population collaborated with other GPs to draw on a wider range of support include the following: » In Tajikistan, the MPA led by Health, Nutrition, and Population collaborated with Social Protection and Jobs to implement social protection support that included an aligned ASA led by the Governance GP to assist third-party mon- itoring of the health response. » In the Philippines, the MPA led by Health, Nutrition, and Population collabo- rated with Digital Development to digitalize systems and with Social Sustain- ability and Inclusion to support stakeholder consultations. 80 » In West Africa, Health, Nutrition, and Population and Agriculture collaborated in regional project support to train One Health agents in community-based surveillance. » In Uganda, the DPF led by Macroeconomics, Trade, and Investment provided support across sectors to ensure basic utilities and water services, expand so- cial registration, provide vouchers for farmers, enact a child protection policy, and procure medical supplies, in collaboration with Health, Nutrition, and Population; Agriculture and Food; Social Protection and Jobs; Social Sustain- ability and Inclusion; Water; Energy and Extractives; and Governance.  lobal Practices Contributing to Projects for Early Health Figure 4.2. G and Social Response Social Protection and Jobs Finance, Competitiveness, and Innovation Health, Nutrition, and Population Governance Agriculture and Food Poverty and Equity Contributing Global Practice Energy and Extractives Water Education Urban, Disaster Risk Management, Resilience, and Land Social Sustainability and Inclusion Transport Macroeconomis, Trade, Independent Evaluation Group World Bank Group    81 and Investment Climate Change Environment, Natural Resources, and Blue Economy Digital Development Other 0 5 10 15 20 25 Share of projects (percent) Instruments Regional DPL IPF/PforR Emergency MPA instruments Source: Independent Evaluation Group portfolio. Note: Percentages can add to more than 100 percent across contributing Global Practices or Global Themes because a single project may have multiple contributing Global Practices or Global Themes. The analysis looks at collaboration in parent and additional financing projects. Emergency instruments include COVID-19–activated Contingency Emergency Response Component and catastrophe deferred drawdown option projects. N = 253 parent and 67 additional financing projects. DPL = development policy loan; IPF = investment project financing; MPA = Multiphase Programmatic Approach; PforR = Pro- gram-for-Results. Instruments Supporting COVID-19 Crisis Response, Streamlined Processes, Corporate Requirements, and Procurement The MPA provided an innovative and rapid approach to expand new lend- ing for the health response. The first MPA, approved in April 2020, included project financing in 51 countries, mainly investment project financing (IPF). The additional financing approved in October 2020 expanded support to 70 countries by April 2021. The MPA offered an umbrella approach with a menu of areas that eligible countries could adapt to their needs and build on to address subsequent stages of the response. This allowed projects to maintain some uniformity in content, which increased the speed of design, process- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 ing, and approval while adapting content to country demands. The first MPA projects in countries disbursed in April 2020, then disbursement increased gradually throughout the first 15 months of the response as more projects became active. By the end of May 2020, three months after the crisis was declared, all approved MPA projects had made initial disbursements, with subsequent disbursement by August 2020 (appendix H analyzes the early support of the health MPA). Complementing new project support with advisory services, the use of DPFs and immediate release of financing in the early days and weeks of the re- sponse was important to support quick actions. Ministries of health were overwhelmed and often needed immediate financing and health advisory and technical support to respond to COVID-19. In some countries, such as India, Mozambique, and Senegal, crisis and repurposed resources were used for the immediate response, which provided an opportunity for staff to support the early health dialogue. Drawing on crisis instruments, region- al projects, ASA, and repurposed projects, where available, ensured quick financing by March 2020, which new project support could build on later. The minimum time for countries to process and disburse MPA financing was about two months. New DPFs first disbursed in May 2020 and were useful for rapid expansion of crisis support for vulnerable groups, surveillance systems, or new policies to protect child welfare, but they were limited in coverage. New IPF projects to support social protection and education were processed 82 throughout the first year of the response, with the first disbursement in July 2020. The portfolio analysis shows the mix of instruments used across coun- tries to support the response (figure 4.3). Findings from the portfolio review, case studies, regional project analysis, and operational process review (ap- pendixes B, C, and G, respectively) point to the use of instruments for sup- porting different time frames and aspects of crisis response (box 4.2).  ix of Instruments Used by Global Practices to Support Figure 4.3. M the Response Health, Nutrition, and Population (n = 111) Lead Global Practice Education (n = 43) Social Protection and Jobs (n = 39) Macroeconomics, Trade, and Investment (n = 26) Urban, Disaster Risk Management, Resilience, and Land (n = 34) 0 20 40 60 80 100 Share of projects (percent) Instruments Regional Repurposed Repurposed IPF PforR COVID-19– New IPF New PforR CERC MPA New DPL CAT DDO Independent Evaluation Group World Bank Group    83 Source: Independent Evaluation Group portfolio. Note: The analysis is based on 253 projects coded for the evaluation. COVID-19–activated CERC in- cludes 26 projects in eligible countries and selected lead Global Practices with COVID-19 emergency response tags or keywords “COVID” or “corona” in their titles, project development objectives, indicator, or summary text. This excludes 13 COVID-19–activated CERC projects: 10 projects reported in the Global Facility for Disaster Reduction and Recovery CERC dashboards (May 5, 2021, and June 1, 2021) in eligible countries and lead Global Practices that did not have COVID-19 emergency tags or keywords at the date of data extraction (May 12, 2021), and 3 projects identified separately by the Independent Evalua- tion Group later in the evaluation. CAT DDO = catastrophe deferred drawdown option; CERC = Contin- gency Emergency Response Component; DPL = development policy loan; IPF = investment project financing; MPA = Multiphase Programmatic Approach; PforR = Program-for-Results. Box 4.2. Examples of Instrument Use Regional projects (discussed in chapter 3) could have been used more widely to assist with the health response, especially in Africa (appendix G). Regional projects, where used, provided resources in about one month, by March 2020, to support the response—for example, by establishing links between regional projects in Africa and new Multiphase Programmatic Approach projects in countries for COVID-19, as was done in Mauritania, Senegal, Togo, and Zambia. These projects could have also sup- ported early cooperation on vaccines. Development policy financing (DPF) funded reforms to restructure policies and sys- tems, drawing extensively on dialogue before COVID-19 to support just-in-time policy actions that could be carried out rapidly. New DPFs, where used early in the response, supported financing by May 2020 and then disbursed quickly. Countries with human The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 development–related DPFs increased from about 20 percent before COVID-19 to 27 percent in the evaluation portfolio during COVID-19. The operational processes review noted wider opportunities to use DPFs across more countries to expand policy actions to protect against health and human capital losses. It also noted that some countries could have benefited from supplemental DPFs during COVID-19, which were limited. The challenge was ensuring that DPFs supported relevant policy actions with measurable results, beyond providing rapid financing to national plans. Program-for-Results projects (about 5 percent of countries) and projects with dis- bursement-linked indicators (about 5 percent of new projects) were rare. Although the evaluation did not analyze the benefits of performance-based financing approaches, the operational process review noted a missed opportunity to use these approaches within the health sector to improve early COVID-19 support, given the heavy focus on procuring goods over attention to the quality of services. Tajikistan and Uganda are strengthening performance-based approaches in health systems, for example, for disease management and for infection prevention and control. Repurposed projects were used in about 51 percent of countries and could provide immediate financing (by March 2020) for education, water, communication activities, and health services, though drawing on repurposed projects happened at different time frames of the response. Use of repurposed projects often built on existing policy dialogue and local government or community networks supported by World Bank projects. The projects also procured medical or education goods. These projects 84 (continued) Box 4.2. Examples of Instrument Use (Cont.) performed well in adjusting to the crisis and in addressing needs such as nutrition, although their implementation progress rating may not reflect this because import- ant components for project development objectives were paused. For example, the nutrition project in Uganda successfully added COVID-19 communication, but there were challenges initially to conduct planned nutrition support in schools. The value of repurposing projects has been reinforced in other evaluations of the COVID-19 response. Advisory services and analytics support was available in about 60 percent of countries (discussed in chapter 1). Key was having advisory services and analytics available in the first months of the crisis to inform response actions. Global partnerships and region- al projects also provided knowledge and learning support in some countries. Better linking the Multiphase Programmatic Approach to advisory services and analytics (as in Tajikistan) could have helped address advisory needs of government and supported the implementation of critical health services. Sources: Independent Evaluation Group portfolio, case studies, and regional project analysis; Johnson and Kennedy-Chouane 2021. Crisis instruments helped World Bank country programs to provide resources to governments rapidly and flexibly in the early months of the COVID-19 Independent Evaluation Group World Bank Group    85 response, yet few countries were prepared with good coverage of these instruments in their portfolios. About 65 percent of countries had CERCs in their portfolios in the 15 months leading up to COVID-19. Twenty two percent had a high proportion of CERCs in their portfolios (figure 4.4), and about 6 percent had CAT DDO to support the early health and social response. Case studies found that multiple CERCs in a variety of sector projects provided flexibility to manage resources across the portfolio, but few World Bank country programs had planned CERCs in this way. Moreover, innovating on the design of CERCs—for example, to support local government responses—could improve their use in crises. In the Philippines, an innovative community-based disaster response modality supported by the World Bank provided resources for subnational COVID-19 responses. Challenges in using CERCs were related to their activation timeline, which required countries to declare an emergency to make resources available. Case studies found that this meant it was not possible to access funds for planning activities before the crisis struck—adjusting this requirement could be important to improve the flexibility of CERCs to provide immediate resources for prevention activities to avert a future crisis. There was also concern about crisis instruments diverting financing from projects, without commitments to replenish those resources. However, in countries with experience using crisis instruments, CERCs provided rapid financing, and there was good preparedness with multiple CERCs in the portfolio: » In India and Mozambique, multiple CERCs allowed for flexible management of resources across sectors. Previous crisis experience in Mozambique led to careful planning of CERCs before COVID-19, ensuring availability of resourc- es from these and from repurposed projects for immediate response and cre- ating space to develop an MPA project by June 2021 focused on vaccines and The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 continuity of health services. Where available, countries often used CERCs to procure medical supplies at the onset of the crisis. » In Honduras, the adjusted country portfolio drew on CERCs and a CAT DDO, which supported collaboration across GPs to tackle the impact of hurricanes in addition to COVID-19. The CAT DDO included prior actions to help the national and subnational emergency response plans and to strengthen health surveillance systems. About 70 percent of countries successfully received disbursements from COVID-19 projects in the first three months of the response, starting imme- diately when the crisis was announced in March 2020—a rapid pace for the World Bank. Key was having some early financing from crisis instruments, repurposed projects, regional projects, trust funds, or grants resources within days. The fastest disbursements in the first days and weeks, before other project financing started, were from crisis instruments, repurposed projects, and regional projects, pointing to their role in swift early disbursement in a crisis (figure 4.5, panels a and b). MPA and DPF financing started to disburse several months into the crisis. New IPF and Program-for-Results slowly increased their share of disbursements, indicating their ongoing role in supporting countries for recovery, along with later phases of the MPA. Hav- ing flexible World Bank–executed trust fund resources was key to provide just-in-time assistance to countries, such as to plan activities at the start 86 of the crisis and even before it was officially announced. Case studies also noted the importance of having grant resources for quickly processing new projects in some countries where government was hesitant to use scarce IDA resources for the crisis. In some countries, it took parliament about a year to approve new emergency projects, limiting their value for the emergency response, whereas grant financing could be approved rapidly. Figure 4.4. Crisis Instruments in Country Portfolios Level of emergency High (n = 21) instruments Medium (n = 19) Low (n = 58) 0 10 20 30 40 50 60 70 80 90 100 Share of countries (percent) Country has at least one COVID-19–activated CERC or CAT DDO Yes No Source: Independent Evaluation Group portfolio. Note: The figure is based on a combined 167 parent and additional financing projects from eligible countries tagged with CERC (152 projects) or a CAT DDO (15 projects), irrespective of Global Practice and active between February 1, 2020, and April 30, 2021. The figure includes an estimated COVID-19– activated CERC of 55 projects across Global Practices: 47 projects from the Global Facility for Disaster Reduction and Recovery CERC dashboards updated on May 5, 2021, and June 1, 2021, plus an addition- Independent Evaluation Group World Bank Group    87 al 8 projects identified by the Independent Evaluation Group portfolio analysis. Regional projects with CERC are excluded. The level of emergency instruments is based on the quantity of overall CERC and CAT DDO projects per country, with levels broken down by tercile. Low level of emergency instru- ments: 0 to 1 CERC or CAT DDO project; medium level of emergency instruments: 2 CERC or CAT DDO projects; high level of emergency instruments: 3 to 10 CERC or CAT DDO projects. The total number of countries is 98. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component. Having a mix of instruments in the portfolio that could be used at different times frames in the early weeks and later months of the crisis response was important. Table 4.1 synthesizes findings on various instruments used in the COVID-19 response in terms of their timing, constraints, and opportunities. Countries with regional projects, better embedding of crisis instruments in the portfolio, experience using a range of instruments to support human capital, and trust funds were often better prepared for the crisis response.  isbursement of COVID-19 Resources by Instrument Figure 4.5. D and Time a. Cumulative disbursements by month 12,000 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 Cumulative disbursements (US$, millions) 10,000 8,000 6,000 4,000 2,000 0 0 0 0 20 20 0 20 0 0 21 21 1 21 21 0 02 02 02 02 02 02 02 02 20 20 20 20 20 20 20 .2 .2 .2 .2 .2 .2 2 l. 2 p. ay ay b. ov g ec n. r. n. r. ar ar ct Ap Ap Au Se Fe Ju Ju Ja M M M M N O D Instruments New DPL MPA New IPF New PforR CAT DDO Repurposed IPF COVID-19– Regional Repurposed CERC PforR b. Application of operational lessons from past crises in COVID-19 88 New PforR CAT DDO Repurposed IPF COVID-19– Regional Repurposed CERC PforR b. Application of operational lessons from past crises in COVID-19 Source: Independent Evaluation Group portfolio. Note: Disbursement data are retrieved from the World Bank’s Standard Reports. In panel a, disburse- ments reported are adjusted using the share of COVID-19 response content estimated in the coding of each project by the Independent Evaluation Group and combines parent projects and additional fi- nancing. New projects (approved on or after February 1, 2020) are assumed to have a 100 percent share of COVID-19 response content. Refer to appendix B for portfolio description. Panel b is organized from fastest to slowest time to first disbursement from the start of the crisis. The time to first disbursement is defined as the number of months between February 1, 2020, and the first disbursement date during the evaluation period. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component; DPL = development policy loan; IPF = investment project financing; MPA = Multi- phase Programmatic Approach; PforR = Program-for-Results. N = 246 projects.  onstraints and Opportunities of Instruments in COVID-19 Table 4.1. C Response by Timing of Financing Independent Evaluation Group World Bank Group    89 Instruments Constraints Opportunities Early weeks and first month of COVID-19 response CERC and CAT DDO Emergency activation Innovation of crisis instrument (about 7 percent of May divert funds design financing) Needed in portfolio before Can be used to move resources crisis across the portfolio Prior actions and immediate financing for emergency plans More flexible processing Regional disease Limited across countries Expansion of regional capacity for projects (less than 2 Capacity of regional net- crisis preparedness percent of financing) works before COVID-19 (continued) Instruments Constraints Opportunities Repurposed projects The extent of human capital Built on existing institutional (about 24 percent of portfolio before COVID-19 structures and relationships in countries) sectors to provide rapid support World Bank–execut- Limited availability Immediate diagnostic or technical ed trust funds (less support to carry out emergency than 1 percent of plans financing) Several months into COVID-19 response Multiphase Program- Several months of response The Multiphase Programmatic matic Approach (20 before new projects pro- Approach enabled rapid approval percent of financing) cessed of new projects, drawing on tech- Grant financing processed nical lessons, and can be built on quicker in some countries for later phases of recovery. Pandemic Emergen- Needed to be processed in Financing of United Nations cy Financing Facility a project for use by World partners to support crisis coordi- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 (less than 1 percent of Bank teams nation was often faster. financing) Making available emergency financing for just-in-time advisory services and analytics Grant access was key for coun- tries. Development policy Experience with prior actions Clarity on type and mix of prior financing (23 percent for human development actions to support crisis response of financing) Support to expand services and policies for vulnerable groups New investment New projects take time Strengthening results orientation project financing (24 New systems and institutional percent of financing) strengthening support in a crisis Sources: International Evaluation Group portfolio; World Bank 2012, 2017. Note: The evaluation did not assess the extent of use of trust funds. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component; PforR = Program-for-Results. Streamlined Operational Processes Operational flexibility facilitated rapid processing of new projects for the health response. All projects followed processing guidelines for emergency situations, which bypasses the concept stage—this flexibility was in place before COVID-19. Guidance to process new projects was provided by the fast-track facility. GPs set up technical committees for quick peer review of projects. Additional flexibilities that helped process the MPA quickly were 90 shortened clearance deadlines and delegated approvals to speed up project processing, paused gender tagging for the first MPA, waivers for financing food expenditures under the IPF policy, and encouragement to use project preparation advances without submitting individual requests. These flexibili- ties helped Health, Nutrition, and Population process new projects for the re- sponse, but these flexibilities did not apply to other GPs, and additional steps were introduced for some new projects. For example, new two-page justifica- tions were sometimes required of staff before regional operations committee meetings for projects viewed as risky or large. Given the emergency and its extensive impact, flexibilities applied to the MPA could have been applied to all new financing to help GPs support countries quickly. This was especially important, given the limited use of additional financing compared with past crises. A previous evaluation reported the median time from approval to first disbursement as 4.8 months for new crisis projects, 7.1 months for noncrisis projects, and 1.5 months for DPFs (World Bank 2019a). MPA projects and new DPFs for COVID-19 were very quick to disburse, compared with past crises— the median time from approval to first disbursement for MPA projects was 1.5 months, and the median time was less than 1 month for DPFs. The median time was about 5 months for new IPFs, similar to past crises. However, some projects disbursed in less than 1 month from approval. Compared with past crises, the World Bank’s operational agility improved in COVID-19, but challenges remained. Implementation Status and Results Independent Evaluation Group World Bank Group    91 Reports identified new project processing as a challenge in COVID-19, though less so than in past crisis (figure 4.6; appendix F). There was strong hand-holding support to staff, but templates, technical specifications, and guidance on safeguards and procurement were often developed in real time, resulting in confusion. Templates and guidance were often developed several months into the response and then revised, requiring countries to retrofit project information to the templates. Once guidance stabilized, and there was agreement on how to proceed, project teams often reported easier processes. Teams that processed MPA projects later in the response reported smoother processes because problems had been resolved. Interviewees noted the need to review emergency processes to eliminate confusion on guidance in future crisis responses. A previous evaluation identified the importance of a road map for crisis engagement that defines, for example, broad divisions on roles and responsibilities and the rationale, modalities, and instruments in responses (World Bank 2012). Furthermore, the Organisation for Economic Co-operation and Development emphasizes that pandemic preparedness requires detailed and up-to-date operational plans and processes describing the different roles of staff, procedures, and uses of instruments in responding to crises (OECD 2022).  perational Process–Related Success and Challenges Figure 4.6. O Reported by Projects Areas Lesson Partnership and Flexibility in Operational Streamlining Implementation Direction Collaboration Procedures Support to Projects Success The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 (n = 120) Challenge (n = 165) Share of lessons Share of lessons in row (percent) in row (percent) 12 20 12 60 30 40 50 60 Source: Independent Evaluation Group review of project Implementation Status and Results Reports for lesson analysis. Note: The COVID-19 portfolio includes 120 successes and 165 challenges in the selected areas of the figure, extracted from 113 projects coded with those areas. “Partnership and collaboration” looks at partnerships to support implementation, information sharing, and joint analyses. “Flexibility in operation- al procedures” looks at the timely processing of new financing for the response. “Streamlining imple- mentation support to projects” looks at factors that facilitated clients to rapidly implement support in projects (appendix F). World Bank staff responded to enormous demands to deliver extraordinary support in unprecedented circumstances. Staff worked for long hours to deliver new and repurposed operations, without travel, while learning to use remote connectivity and geospatial tools and adapting to home-based work amid evolving family arrangements and stresses. Some staff suffered person- al losses; many coped with pandemic-related anxiety. Heavy work pressures stemmed not only from mounting the COVID-19 response but also from oth- er ongoing considerations: the locust response, natural disasters, debt relief efforts, the July 2020 staff realignment, other internal reorganizations (such 92 as splitting the Africa Region into two Regions), and a leadership rotation in the Human Development Practice Group. Access to surge capacity to process early health support was critical, but in- ternal mobilization of this capacity was challenging. As the crisis continued, managers in health mobilized surge capacity—using retirees, exchanging overloaded staff, and increasing the responsibilities of country office staff— to help quickly process projects. The operational process review suggests that further surge capacity to process early health support could have come from within the World Bank. Social Protection and Jobs; Education; and Health, Nutrition, and Population were processing 240 percent to 480 per- cent more projects, whereas other GPs were processing fewer projects than before. The operational process review findings also point to a challenge in the incentives to collaborate across GPs at the operational level to help pro- cess and implement emergency project support, such as a DPF or MPA led by one sector, as each GP has its unique sector focus. Gender, Citizen Engagement, and Safeguards GPs addressed gender requirements more consistently when support predat- ing COVID-19 could be adjusted. Addressing gender requirements with new support in a crisis was challenging for World Bank teams without technical assistance to help consistently incorporate gender approaches in the re- sponse. In Social Protection and Jobs, earlier analysis of gender equality was Independent Evaluation Group World Bank Group    93 applied in COVID-19 projects, including the expansion of registries with an emphasis on female-headed households and women with young children. Op- portunities to address gender in the health response were missed, and gender analysis in the Health, Nutrition, and Population response is rare. An internal review by the Gender Group of 27 MPA projects highlighted 3 projects that in- tentionally addressed gender equality (about 10 percent): projects in Pakistan and Sierra Leone defined gender activities as part of project components, and the Mongolia project included a section on gender differentials. Project teams noted a need for more technical support to implement well-developed gender approaches in health and education projects. In Uganda, Social Sustainability and Inclusion reviewed COVID-19 projects to identify opportunities to ad- dress gender, which helped GPs to plan support. Outside specific innovations, citizen engagement received limited emphasis across GPs. Some interviewees noted that the current approach to citizen engagement had limited applicability in a crisis and often was not develop- ing country systems that could be drawn on during COVID-19. For example, the citizen engagement intranet site has very few updates and little content on COVID-19. Helpful innovations used by health and education projects included SMS feedback from citizens about services. Having platforms fos- tering citizen engagement in place before the next crisis could be important because lack of trust and accountability and the weak roles of citizens were challenges to mobilizing demand for services during the COVID-19 response. The new Environmental and Social Framework was challenging amid the early months of the crisis. Extensive support was provided to World Bank teams to implement the Environmental and Social Framework, including The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 help with completing documents. COVID-19 guidance released early in the pandemic (in response to operational realities) allowed for the Environ- mental and Social Framework requirements to be met after the project was approved, although they needed to be completed before processing pro- curement of medical goods. Despite this much appreciated support, because the Environmental and Social Framework was new, interviews noted that developing the required outputs and processes involved on-the-job learning, which was challenging because clients—especially in health for processing the MPA—were overwhelmed with urgent COVID-19 demands. The outputs required from clients included the Environmental and Social Review Sum- mary, Environmental and Social Commitment Plan, Stakeholder Engagement Plan, and documents for risk-related topics, such as medical waste, social inclusion, and nondiscrimination. The World Bank used additional financing in other crises to provide flexible and fast-disbursing support, which built on existing safeguards to avoid adding new steps in a crisis. However, the use of additional financing was initially limited in COVID-19 operations because projects approved under the previous safeguards policies were required to transition to the new Environmental and Social Framework until a waiver was issued in June 2020. More streamlining of safeguard requirements for new projects in the early crisis response could have been helpful. 94 Procurement of Goods The relief stage supported emergency procurement of medical and oth- er goods, with successful efforts to expedite processes. By June 2020, five months into the response, almost 50 percent of procurement contracts sup- porting the early COVID-19 response were signed, ensuring that countries received protective gear and medical supplies and equipment. In alignment with the World Bank’s Guidance on Procurement in Situations of Urgent Need of Assistance or Capacity Constraints, procurement was supported through countries adopting accelerated procedures, World Bank–facilitated pro- curement, and collaboration with United Nations organizations. Accelerat- ed processes included allowing for retroactive financing, direct selection, electronic bidding by email, and larger up-front payments to secure goods on global markets. Although steps were not always clear, and case studies noted an opportunity to clarify guidelines on accelerated procurement procedures, procurement in COVID-19 was quick and smooth compared with past crises, helping to obtain needed medical supplies. A previous IEG evaluation of sup- port to natural disasters identified procurement as a main constraint (World Bank 2006), whereas in COVID-19, procurement was not a main challenge reported in Implementation Status and Results Reports or interviews, de- spite global supply chain difficulties. In COVID-19, countries successfully employed a variety of mechanisms to Independent Evaluation Group World Bank Group    95 procure needed goods quickly, including procurement using United Nations agencies and World Bank–facilitated procurement. This may have been difficult before the World Bank’s procurement reform in 2016. Close support from World Bank procurement specialists was key, and in some cases, senior procurement staff tracked health goods to ensure that they arrived rapidly in the intended country. Direct selection was used for almost 60 percent of contracts. There was World Bank–facilitated procurement of scarce medi- cal goods. Some World Bank teams preferred to use United Nations agen- cies for procurement, given their simplified processes, but countries that used World Bank–facilitated procurement reported that the service helped reduce the cost and speed of receiving goods where it was used. In total, the World Bank procured just under $1.9 billion of critical goods in the early response, including COVID-19 test kits, personal protective equipment, facility improvements, laboratory and medical equipment, and technology (figure 4.7). About 67 percent of the procurement of goods for the health and social response was supported by the MPA, making it a critical vehicle for emergency financing. Figure 4.7. Cumulative Procurements for Goods by Category, by Date 4,000 3,500 Cumulative procurements (count) 3,000 2,500 2,000 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 1,500 1,000 500 0 20 0 0 20 0 20 0 20 0 20 21 1 21 21 0 02 02 02 02 02 02 20 02 20 20 20 20 20 20 20 .2 .2 .2 .2 .2 .2 l. 2 g. p. ay b. ov ec n. r. n. r. b ar ar ct Ap Ap Au Se Fe Fe Ju Ju Ja M M M N O D Category Vaccine Nutritional Learning materials Essential health care Communication Biohazard waste Vehicles COVID-19 management treatment Sanitation Laboratory Technology Mask, N95 particulate Facility Medical Protective Testing kits, improvement equipment gear COVID-19 Source: Independent Evaluation Group portfolio. Note: The data include signed procurements only, which means active contracts with a signature date between February 1, 2020, and April 30, 2021, and a status of signed, under implementation, completed, or under review. Signed procurements also include active contracts with a signature date after February 1, 2020, no contract sign date, and a status of signed or under implementation. For procurements with- out a contract sign date, the sign revision date was used. The number of procurements per category was calculated by separating 2,690 individual goods procurements into components against a defined taxonomy using text analytics. The number of procurements of each good is 8,565. 96 Key challenges to procurement included the coordinated planning and tracking of goods until their arrival in health facilities, and limited emer- gency preparedness of procurement systems in countries. World Bank teams worked closely with partners to plan what was procured, but it was challeng- ing, and in some cases, goods were already procured by another partner by the time they arrived, and another item was needed more urgently. In some countries, World Bank teams helped develop systems to plan and track goods until the arrival at health facilities, which helped monitor what was pur- chased by different partners and ensured accountability of goods arriving to benefit communities. In Paraguay, the health ministry used georeferenced data to plan procurement. Tajikistan and Zimbabwe developed electronic tracking systems, including an SMS platform to verify receipt of goods. In these countries, health sector supply chains were strengthened in real time. World Bank teams also worked with the government to adopt accelerated procurement processes so they could procure goods more quickly using country systems. Such support was provided in Djibouti and by the Africa CDC and Organisation of Eastern Caribbean States regional projects. Monitoring and Reporting World Bank projects planned indicators, although timely monitoring was challenging. The review of indicators shows that countries planned indi- cators to measure 60 percent of response areas (figure 4.8). Few indicators Independent Evaluation Group World Bank Group    97 measured child welfare and social services, and measurement of community engagement was even more limited. Vaccination is measured less, mainly because vaccine projects were developed later in the response. However, less than 40 percent of indicators were tracked during the response, giv- en that project reporting is every six months. A review on the first year of the pandemic highlights that nearly every report identifies the lack of data collection as an important constraint in understanding funding, activities, and results (Johnson and Kennedy-Chouane 2021). Monitoring of indicators from new projects was especially challenging; however, the health response stands out as having good monitoring, compared with other areas because Health, Nutrition, and Population monitored the new MPA projects closely (figure 4.9).  ountry-Level Alignment of Indicators to Measure the Figure 4.8. C COVID-19 Response Ensure Health Health Risk Ensure Child Welfare Services Communication Vaccination and Social Services Protect the Poor Community Institutional Strengthening and Vulnerable Engagement of Response Percent of countries Percent of countries 25 40 25 90 60 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 80 90 Source: Independent Evaluation Group portfolio. Note: The figure shows the proportion of countries with at least one indicator in a project in their portfo- lio to monitor a planned COVID-19 thematic response area. “Ensure health services” includes indicators measuring achievements of health support in the COVID-19 response. “Protect the poor and vulnerable” includes indicator measuring social protection and informal sector improvements. “Ensure child welfare and social services” includes indicators measuring education, nutrition, and psychosocial improve- ments. “Community engagement” includes indicators measuring aspects of social cohesion and citizen engagement. “Institutional strengthening of response” includes indicators measuring system and policy improvements. The total number of indicators is 2,219 and countries is 92. Indicators frequently measured outputs, but measurement of the quality of services, social cohesion, and systems improvement was limited. Indicators often measured materials received, such as equipment, and in some cases, the delivery of the service. These data will need to be complemented by data col- lection on quality improvements and outcomes to assess the achievements of support to restructure systems (box 4.3). Moreover, there is limited measure- ment of and support to community activities, especially social cohesion. 98 Figure 4.9. Extent of Monitoring in Countries by Response Area Lending Group Ensure Health Health Risk Protect the Poor Communication Vaccination and Vulnerable Services IBRD or Blend IDA 0 20 40 60 80 00 0 20 40 60 80 100 0 20 40 60 80 00 0 20 40 60 80 100 1 1 Institutional Ensure Child Welfare Community Strengthening Lending Group and Social Services Engagement of Response IBRD or Blend IDA 0 20 40 60 80 00 0 20 40 60 80 00 0 20 40 60 80 00 1 1 1 Share of countries (percent) Level of monitoring Low Medium High Source: Independent Evaluation Group portfolio. Note: The figure reports the share of countries within each lending group and response area by level of monitoring. Level of monitoring is based on the tracking or progress share of indicators in a country response area: low (0 percent to 22 percent); medium (22 percent to 52 percent); high (52 percent to 100 percent). IBRD and blend: n = 39 countries; IDA: n = 49 countries. N = 88 countries. Regional projects are excluded from the analysis. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association. About two-thirds of countries show evidence of positive progress on tracked indicators. Early progress is most notable in IDA countries (figure 4.10), Independent Evaluation Group World Bank Group    99 where a medium to high share of tracked indicators show early progress, especially for Africa. A little more than 40 percent of all indicators assess- ing the health response (ensuring health, risk communication, vaccination, and strengthening of health systems and coordination) show early progress. There is less progress for indicators measuring results in social protection and education, with only about one-quarter showing evidence of progress. Box 4.3. Examples of Indicators Monitoring the Response For the health response, indicators focus on surveillance, case management, and essential health services. Indicators include outputs measuring numbers of equip- ment (ventilators, COVID-19 test kits, and sanitation kits) and health workers and lab staff trained, and some examples of measuring the quality, knowledge, and systems improvements: » Hospitalized patients with COVID-19 who are treated per national guidelines » Health facilities providing 75 percent of the essential package of services » Suspected COVID-19 cases diagnosed by laboratories within 24 hours » Population able to identify three key symptoms and prevention measures of The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 COVID-19 » Health workers fully vaccinated against COVID-19 » Contracting and deployment of surge capacity health workers » Hospitals with triage and isolation capacity per a quality checklist » Coordinated disease surveillance systems in place in the animal health and pub- lic health sectors for zoonotic diseases or pathogens identified as joint priorities For the social response, common themes monitored are social protection and child welfare. For the relief stage, social protection indicators often monitor outputs related to the delivery of immediate assistance, improvements in social registries and sys- tems, and in some cases, changes in beneficiaries receiving the assistance. Child wel- fare indicators include tracking children returning to school, opening of schools, and the inclusion of vulnerable groups. Examples of indicators measuring the response include the following: » Children whose learning was assessed to evaluate loss of learning during school closure » Primary schools reopened after implementation of safety plans » Coverage of hygiene promotion activities (continued) 100 Box 4.3. Examples of Indicators Monitoring the Response (Cont.) » Female students provided access to psychosocial support services » Frontline health-care workers with the knowledge to care for survivors of gender- based violence » Workdays created for women to work by local contractors » Beneficiaries receiving COVID-19–related cash assistance » Schools where parents reported improvements in learning during COVID-19 » Local governments implementing participatory planning processes » Households benefiting from shock-responsive safety net programs » Citizen engagement messages distributed via radio Source: Independent Evaluation Group portfolio. Note: Health response included ensuring critical health services, continued essential health services, health risk communication, vaccines, and health systems strengthening. Social response included ensuring social protection of vulnerable and informal workers, learning of vulnerable chil- dren, nutrition, psychosocial support, community engagement, and social systems strengthening. Independent Evaluation Group World Bank Group    101  ountries by Extent of Level of Progress on COVID-19 Figure 4.10. C Lending group Response Indicators IBRD or Blend (n = 39) IDA (n = 49) 0 0 10 10 20 20 3030 40 40 50 50 60 60 70 70 80 80 90 90 100 100 Share of countries (percent) Evidence of progress level Low Medium High The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 Source: Independent Evaluation Group portfolio. Note: The figure shows the percentage of countries by levels of evidence of progress of indicators, where levels correspond to terciles of the distribution of the evidence of progress share of total indicators across countries: low (0 percent to 16.67 percent), medium (16.67 percent to 51.76 percent), and high (51.76 percent to 100 percent). Indicators were coded based on whether the monitoring value showed no change or adverse change from the baseline or evidence of progress toward the target. The total number of countries is 88. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association. Integrated reporting of data on various areas of the World Bank’s COVID-19 response was important to discussions with the country government, World Bank teams, partners, and headquarters. World Bank country management noted that it was challenging to compile data across projects to get a full pic- ture of progress on the response for weekly or monthly meetings. Staff faced many fragmented requests from headquarters, GPs, and others to report on projects. The challenge of monitoring during the crisis reinforces the value of investing in better routine data systems for adaptive implementation of country support (discussed in chapter 3), such as geo-enabled monitoring, SMS, and dashboards, which were used in some countries in Africa and Latin America. The operational process review suggests that improving integrated reporting could benefit World Bank portfolio monitoring and policy dialogue during a crisis but also more routinely. The systems set up for monitoring the vaccine response provide an example of a coordinated approach to coun- try program monitoring that aligns with partners and strengthens country systems (Chazaly and Goldman 2021). Box 4.4 provides examples of integrat- ed reporting by World Bank GPs. 102  xamples of the Global Mobilization of Data and Knowledge Box 4.4. E Resources Compilation of countries’ social protection responses. The Social Protection and Jobs Global Practice team monitored projects supporting COVID-19 and innovations supporting social protection. Monitoring of Multiphase Programmatic Approach and health portfolio. The World Bank Health, Nutrition, and Population team closely monitored the Multiphase Pro- grammatic Approach, including indicators and areas of implementation, by country task teams. The Health, Nutrition, and Population portfolio was also closely monitored by regions to track project support for COVID-19, Contingency Emergency Response Components activation for COVID-19, and additional financing for vaccines. Education portfolio tracking. The Education Global Practice tracked projects support- ing COVID-19, including estimated financing allocations. Source: Independent Evaluation Group internal operational process review. Partnerships to Facilitate Response Partnerships between development organizations and the World Bank helped preparedness and crisis responses in countries. Partnerships devel- Independent Evaluation Group World Bank Group    103 oped by GPs and country management units before COVID-19 often pro- vided technical knowledge and financing for the health and social response. Having such relationships defined before COVID-19 helped the partners to align strategies to rapidly collaborate on COVID-19 support in countries, as other evaluations have also found (Johnson and Kennedy-Chouane 2021). Important partnerships identified in interviews, documents, and case studies included development organizations such as United Nations’ agencies, mul- tilateral development banks, bilateral donors, foundations, nongovernmen- tal organizations, and global partnership organizations (Global Partnership for Education [GPE]; Global Financing Facility; Education Technology; Gavi, the Vaccine Alliance; and the Global Partnership for Social Accountability). The World Bank engaged in four main types of partnerships to support the COVID-19 response: » Financing partnerships to provide grants within World Bank projects or aligned complementary financing support to the World Bank’s response in countries » Technical partnership activities to innovate and expand knowledge for qual- ity implementation of response activities, often supporting analytic work to understand the status of the response » Procurement and supply chain partnerships to improve the purchase and delivery of goods » Nongovernmental partnerships to support provision and feedback on services at the front line of the response At the global level, partnerships also provided guidance on emergency plans The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 and quality standards for the health response in countries. Key global part- nerships are described in box 4.5.  xamples of Global Partnerships Supporting COVID-19 Box 4.5. E Response Health response: » The Global Financing Facility (GFF) provides financing and technical assistance for essential health services with a commitment to ensure that all women, children, and adolescents can survive and thrive. GFF grants supported knowledge work and learning exchange in partner countries to prioritize and plan for continued essential health services, strengthen frontline service delivery, and meet demand for sexual and reproductive health and other lifesaving services. GFF briefs in more than 60 countries highlighted severe reductions in the provision of oral anti- biotics, vaccinations, childbirth, and family planning resulting from the pandemic. In Uganda, the GFF helped assess the communication strategy for COVID-19 health service disruption, in partnership with the Development Economics Vice Presidency. It also supported expansion of the national performance-based financing program to better address infection prevention and control in essential health services to manage waves of COVID-19. (continued) 104  xamples of Global Partnerships Supporting COVID-19 Box 4.5. E Response (Cont.) » Gavi, the Vaccine Alliance provided immediate grants to countries in response to COVID-19. The World Bank is an alliance member, sits on the board, and has agreements in place on financing and supporting countries to deliver vaccines. In Pakistan, based on its long-term relationship, Gavi, the Vaccine Alliance and the World Bank undertook joint missions to support the national immunization pro- gram to manage its COVID-19 response. In Tajikistan, the organizations partnered to provide COVID-19 vaccines to cover the first 16 percent to 20 percent of the population and financing to the United Nations and international nongovernmen- tal organizations for technical assistance and cold chain improvements. Education: » Global Partnership for Education provided a large grant program dedicated to the COVID-19 response, and projects working with these grants collaborated with consortium partners, including the United Nations Educational, Scientific, and Cultural Organization and United Nations Children’s Fund. These grants- supported schools provide distance learning, teacher training, and school reopening. Across the portfolio, focus areas included the use of technology and gender equity. In Djibouti and Uganda, grants addressed gender-based violence, remote learning, and return to school efforts. Independent Evaluation Group World Bank Group    105 » The Education Technology (EdTech) thematic group in the Education Global Practice, drawing on a global partnership, shared tools and produced research and experiences to accelerate digital learning. Using global EdTech tools in World Bank loans led to enhanced knowledge on remote learning. Technical advice was deployed to projects through a network of regional staff drawing on the EdTech Hub, supported by the World Bank, the United Nations Children’s Fund, and United Kingdom’s Foreign, Commonwealth and Development Office. Gender: » The World Bank’s Gender Group collaborated with UN Women and others in pre- paring a report on strengthening gender measures and data in the COVID-19 era. (continued)  xamples of Global Partnerships Supporting COVID-19 Box 4.5. E Response (Cont.) Citizen engagement: » The Global Partnership for Social Accountability facilitates collaboration of civil society organizations with governments and engagement of citizens to solve de- velopment problems, strengthen accountability, and improve governance. During COVID-19, the Global Partnership for Social Accountability supported monitoring of the COVID-19 response in Tajikistan through civil society organizations. Procurement: » The World Bank and United Nations agencies work in partnerships that enable joint procurement of goods and other collaborations within projects. Across more The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 than 40 countries, the World Bank procured medical equipment and supplies for the response with the Food and Agriculture Organization, the United Nations Chil- dren’s Fund, the United Nations Development Programme, the United Nations Office for Project Services, the United Nations Population Fund, the World Food Programme, and the World Health Organization (WHO). The World Bank also collaborated with WHO and the World Food Programme on the COVID-19 Supply Chain Task Force. These relationships provided a helpful procurement option to complement World Bank–facilitated procurement support. Health standards and guidance: » The World Bank worked with WHO to set standards for COVID-19 response plans; these guided the strategic areas planned and supported by partners and coun- tries globally. WHO was also the key agency to provide guidance to countries on medical goods. Sources: Independent Evaluation Group portfolio; Aslam and Rawal 2021; Rodriguez et al. 2021; Muñoz-Najar et al. 2022; World Bank Group and United Nations 2021. Global partnerships in health and education were quickly adapted to sup- port the COVID-19 response in countries. In education, working with the GPE partnership consortium (which includes World Bank, UNICEF, and the United Nations Educational, Scientific, and Cultural Organization), partners 106 coordinated technical and financing support rapidly to support the response with wide coverage of countries. Case studies highlighted the success of GPE grants to quickly expand education support, collaborating with UNICEF and other partners. In Rwanda, GPE supported approaches to remote learning, including broadcasting lessons on the radio and free SMS messages. The Education Technology team facilitated rapid sharing of technical knowledge with partners and countries to expand technology-based learning. For ex- ample, more than 30 continuity stories from around the world highlighting countries’ remote learning solutions were developed in partnerships with the Organisation for Economic Co-operation and Development, Harvard Global Education Initiative, and HundrED. The approach it employed is a model that could be expanded to other sectors. In health, the Global Financing Facility provided just-in-time knowledge support for health services and strategies. However, countries could have benefited from earlier technical knowledge to continue essential health services and for gender and risk communication. The World Bank’s partnership with Gavi, the Vaccine Alliance was key in countries such as Tajikistan, but more countries could have benefited from this type of early support on vaccines during COVID-19—a theme that also arose in IEG’s Global Program Review (World Bank 2014). Global Partnership for Social Accountability support in citizen engagement stands out for ad- dressing a key need, but the limited coverage of countries was a challenge to strengthening accountability mechanisms, trust, and participation. Procure- ment partnerships were helpful across sectors. Health guidance partnerships with WHO were important for aligning health actions in countries; neverthe- Independent Evaluation Group World Bank Group    107 less, challenges arose because of the emphasis of this guidance on the health sector and less so on the social impacts of COVID-19 (discussed in chapter 2) and of alignment on vaccines (discussed under vaccine support). Financing and technical partnerships helped expand response actions quick- ly in countries, although technical partnerships were rare. Funding partner- ships were common for coordinating support. In India, engagement with the Asian Development Bank provided an additional $500 million that followed prior actions similar to those of the World Bank’s social protection and edu- cation support. Also in India, financing for the World Bank’s health project was coordinated with the Asian Infrastructure Investment Bank. In Hondu- ras, the World Bank collaborated with the Inter-American Development Bank in financing the social response, including joint missions. In the Philippines, the Asian Development Bank and the World Bank also complemented each other by jointly financing civil works for laboratory structures and equip- ping laboratories. Technical partnerships were important for expanding COVID-19 support, although rare in countries, especially in the health sector. For example, the World Bank and UNICEF jointly expanded remote learning and psychosocial services in Djibouti, and the World Bank helped the government expand its vouchers and school feeding interventions with the World Food Programme. Collaboration with nongovernmental organizations and the private sector, even though limited, helped expand community-based implementation and use of technology. For example, in Tajikistan, the World Bank worked with the Open Society Assistance Foundation to support civil society capacities to prevent and respond to COVID-19 and with the Aga Khan Development The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 Network to train health workers. In Togo, the new Novissi social protection program helps citizens who lost their income and were pushed into poverty by the pandemic. Supported by the World Bank to undertake phone surveys, the government partnered with the University of California and the nonprof- it GiveDirectly to prioritize those most in need by using satellite imagery, mobile phone data, and nationally representative household consumption data (World Bank 2022a). In Belize, the World Bank project partners with the national bank and telecommunications company to deliver cash transfers. Beneficiaries are notified of the funds by SMS message, and they collect their payment through a network of agents. A few examples of partnerships with nongovernmental organizations also occurred. Further collaboration with nongovernmental organizations could have helped expand technical learn- ing and community activities. Case studies noted a challenge in providing technical assistance to help countries expand quality critical health services. Partnerships in this area were limited. Countries required technical learning to implement new surveillance and case management approaches. Where World Bank teams already had technical knowledge implementing critical health services, they could apply that to COVID-19. Some countries benefited from technical knowledge partnerships through projects before COVID-19. For example, the 18th Replenishment of IDA (IDA18) and the 19th Replenishment of IDA 108 committed to support pandemic preparedness plans in 25 countries (IDA 2017, 2020). During COVID-19, the Regional Disease Surveillance Systems Enhancement Project in West Africa coordinated technical learning through its cross-country exchanges, including with the Africa CDC, UNICEF, and WHO. Similarly, the Organisation of Eastern Caribbean States Regional Health Project supported technical knowledge learning with the Pan Ameri- can Health Organization and the US Centers for Disease Control and Preven- tion. Technical learning partnerships to improve preparedness may be scaled up to other countries through such regional collaboration. Pandemic Emergency Financing Facility In coordination with partners, PEF grants provided support for COVID-19 plans; however, the amounts were small, and the funding did not support just-in-time actions in the first weeks of the response. The PEF was set up in 2016 through partnership and working with the private sector, with funding provided by Australia, Germany, IDA, and Japan and also insurance cov- erage through World Bank catastrophe bonds and insurance-linked swaps (World Bank 2020c). PEF provided surge financing to IDA countries facing large cross-border disease outbreaks, such as Ebola, and to catalyze the creation of a global market for pandemic insurance instruments. The WHO pandemic declaration triggered the PEF’s insurance window, and allocations were defined in a little more than a month. All $196 million of the fund was transferred to support COVID-19 responses in 64 countries by September Independent Evaluation Group World Bank Group    109 2020 (World Bank 2020c). The timeliness of PEF resources for just-in-time use by World Bank teams was limited by the need to declare an emergency to access the funding and by the processing requirement that PEF had to be included in a World Bank financing project for recipient execution. The amount of PEF financing might have been more useful with payment divided among fewer countries. Alternatively, PEF might have been more useful and timely had it been provided to health teams as a World Bank–executed trust fund for just-in-time financing of joint ASA to inform country and partners responses. Many governments struggled with how to respond to COVID-19 and sought just-in-time ASA in health, especially diagnostics and technical assistance, to refine strategies and planned actions. Such support was critical throughout the early response for adaptive management but especially so in the early weeks and months of the pandemic. Flexibility to use PEF in this way could have increased financing for just-in-time ASA by about threefold, given the limited allocation of about $60 million to ASA. The flexibility of PEF to finance United Nations partners to facilitate coor- dination in countries and the quick processing of its grants were key assets. More than 60 percent of PEF grants were used to finance United Nations partners; the rest were processed in World Bank projects. In Honduras, the Pan American Health Organization received World Bank PEF support to adapt and equip seven health establishments for COVID-19 between July 2020 and March 2021. In Mozambique, the United Nations Population Fund received PEF resources to continue support to sexual and reproductive health services during COVID-19. In Uganda, PEF was integrated in MPA financing, allowing for faster processing over IDA credit resources, and the funding was used to procure health-related goods, training, and other sup- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 port for the national COVID-19 plan. Vaccine Support In a context of high uncertainty in the early months, the World Bank en- gaged with global partners in efforts related to vaccines but lacked an in- strument to help expedite country vaccine access. In the first months of COVID-19, Health, Nutrition, and Population convened with global partners to explore ways to help low-income countries to access vaccines when they became available on global markets (see table 4.2 for a timeline of vaccine response). In addition to competing with wealthier countries, securing ad- vance access to vaccines carried a significant risk because it was not known if vaccines would be successful—IDA countries could not assume this burden of risk without global assistance. Earlier, for the pneumococcal vaccine, the World Bank had gathered grant resources together to aggregate demand for vaccines across countries (Cernuschi et al. 2011). However, a World Bank instrument was not available for quick use in a crisis context to aggregate demand across countries for advance market commitments other than convening actors for grant resources. Another approach could have been to secure earlier commitments for donations from high-income countries, as advocated outside the evaluation period by the Multilateral Leaders Task Force on COVID-19 in June 2021. Organizing these actions would have re- 110 quired strong commitments from high-income countries to act quickly early in the response. The World Bank’s early convening with partners helped form the Access to COVID-19 Tools Accelerator partnership and coordinate support to vaccines and health systems at the country level. In June 2020, the World Bank committed to co-convene the Access to COVID-19 Tools Accelerator health systems and vaccine pillars (the vaccine pillar became Country Access to COVID-19 Vaccines [COVAX]), but in July 2020, the World Bank decided not to co-convene COVAX and focused on country-level support through the health systems pillar. The health systems pillar could be supported through the MPA and existing country relationships. The World Bank remained involved in COVAX but not as a co-convener, and it focused on the country level, where it had good access to financing and country relationships to support response actions. This decision was taken within a challenging global context marked by much uncertainty about vaccines and extensive internal discussion about how to support advance market financing for vaccines without an appropriate global-level instrument. At the country level, the focus was on the MPA financing for vaccines and engagement in country-level committees to plan for vaccines. Having a global-level instrument that allowed for advance market commitment could have helped increase the value added of the COVAX partnership to help low-income countries access vaccines one year earlier. Stronger global-level engagement Independent Evaluation Group World Bank Group    111 of World Bank expertise in COVAX early on, as was seen later in response, may have helped facilitate earlier access to global supplies, manufacturing, and risk pooling arrangements for IDA countries.  ummary Timeline of World Bank Vaccine Response Table 4.2. S Date Description February– » In early 2020, the World Bank’s Emergency Operations Center and the June 2020 Coalition for Epidemic Preparedness Innovations convened a global vaccine development task force. » In the World Bank, Health, Nutrition, and Population took a lead role with other partners (including Gavi, the Vaccine Alliance and WHO) to prepare for COVID-19 vaccines and supported working groups on vaccine sup- ply, manufacturing, and deployment. » In April 2020, this task force was folded into the ACT Accelerator—a glob- al collaboration to accelerate development, production, and equitable access to COVID-19 tests, treatments, and vaccines, of which the World Bank is a founding partner. » In June 2020, the World Bank committed to co-convene the ACT Accel- erator vaccine pillar (which later became COVAX) and health systems pillar. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 July– » The World Bank engaged with the vaccine pillar of the ACT Accelerator, December though it did not co-convene COVAX, given the uncertainty about vac- 2020 cines and extensive internal discussion about the need for an appropri- ate instrument to finance advance commitments at the global level. The World Bank focused on the health systems and response pillars, which seek to ensure that countries have the necessary technical, operational, and financial resources to translate new COVID-19 tools into national re- sponse interventions. The World Bank also focused on helping govern- ments plan for vaccines with partners in countries. » The World Bank approved additional financing in October 2020 of US$12 billion for the Multiphase Programmatic Approach to meet a critical need to strengthen systems to manage vaccines; this later expanded to US$20 billion. The financing also provided the option for countries to purchase from COVAX. » Internal leadership of the COVID-19 response on vaccines shifted from the Emergency Operations Center (led by Health, Nutrition, and Pop- ulation and other Global Practices) to Operations Policy and Country Services. » The World Bank, through the ACT Accelerator, supported consultations with countries related to strengthening health systems, participated in vaccine planning to assess national readiness, and supported the development of vaccine tracking and distribution. A joint assessment of country readiness for COVID-19 vaccines with UNICEF and WHO started in November 2020 and was published in March 2021 (World Bank 2021a). . (continued) 112 Date Description January– » From January 2021, the World Bank increased its engagement in global July 2021 partnerships on vaccines, as signaled by management speeches and in internal reporting to the World Bank Board of Executive Directors em- phasizing the role of partners in the World Bank’s response. » In February, with the first approvals of vaccine projects, the World Bank Board emphasized the need for strong cooperation with COVAX and other development partners. It also stressed the importance of a region- al perspective. » In April 2021, the safeguards guidance of the World Bank on vaccines was revised to align with WHO standards. » In June 2021, culminating from many months of discussion, the World Bank fully engaged with Africa CDC, African Export-Import Bank, and UNICEF in the Africa Vaccine Acquisition Task Team to help countries access vaccines, as a complement to COVAX. Furthermore, during this period, the Multilateral Leaders Task Force on COVID-19 was also launched that involved the International Monetary Fund, WHO, the World Bank, and the World Trade Organization; the predecessor to this was a High-Level Task Force for Vaccines that acted as a vehicle for monitoring, sharing, and coordinating information, data, and activity regarding vaccine availability. An advance market mechanism was launched with COVAX in July 2021 based on aggregated demand across countries using World Bank and other multilateral development bank financing. Sources: Independent Evaluation Group portfolio; Dalberg Advisors 2021; Van Trotsenburg 2021; WHO 2021c; World Bank 2021a, 2021b, 2021h. Note: The COVAX partnership within ACT Accelerator is for COVID-19 vaccine development, manufac- turing, and equitable access for countries. ACT = Access to COVID-19 Tools; Africa CDC = Africa Centres for Disease Control and Prevention; COVAX = Country Access to COVID-19 Vaccines; UNICEF = United Nations Children’s Fund; WHO = World Health Organization. Independent Evaluation Group World Bank Group    113 Engagement of regional organizations in COVID-19 vaccines was important for coverage in Africa. The World Bank also engaged in early discussions in 2020 with the Africa CDC to help facilitate cooperation among countries in Africa for the procurement and delivery of vaccines. Research published by the World Bank in April 2021 identified that multilateral action, including action by the African Union on vaccines that could supply enough vaccine to cover 60 percent of population by March 2022 (Agarwal and Reed 2021). It was not until June 2021, however, following months of discussion, that the World Bank announced that it was partnering to support the Africa Vaccine Acquisition Task Team of the Africa CDC with resources to help countries ac- cess vaccines, as a complement to COVAX. The formal announcement of the World Bank and Africa Vaccine Acquisition Task Team partnership was the culmination of months of work invested by World Bank management (World Bank 2021h). Earlier partnership on vaccines with the Africa CDC would have helped countries in Africa with limited capacities to act independently to aggregate demands and take earlier actions to pool efforts to procure and plan for vaccines across countries. The regional project analysis (appendix G) exposes the strong added value of support to regional organizations for en- gaging political leadership and helping countries plan and cooperate on pro- curement and technical actions for disease response, which could have been extended to vaccines. The Africa CDC partnership was key to help countries such as Mozambique, which has fully vaccinated 42 percent of its population (World Bank 2022a; box 4.6).1 Box 4.6. Vaccination in Mozambique The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 The World Bank is supporting the government of Mozambique to acquire, manage, and deploy COVID-19 vaccines through the COVID-19 Multiphase Programmatic Ap- proach project that was approved in June 2021. This project enables the government to procure vaccines against COVID-19 while supporting vaccine distribution and ad- ministration for the country’s national vaccination campaign and continuity of essential health services. Through this project, the country was among the first to sign an agree- ment with the United Nations Children’s Fund and the Africa Vaccine Acquisition Trust of the Africa Centres for Disease Control and Prevention. Consequently, financing has already been committed to acquire approximately 9.3 million single-dose COVID-19 vaccines and related supplies for Mozambique. Additionally, supported by a Global Financing Facility grant, the project seeks to maintain essential health, specifically to address disruptions in routine essential maternal, child, and adolescent health services at the primary health-care level and services for communicable diseases. Source: Independent Evaluation Group portfolio and case studies. World Bank teams supported planning for vaccines in countries in collab- oration with partners. Work through the Access to COVID-19 Tools Accel- erator at the country level provides an example of global partners aligning to coordinate planning, tracking, and diagnostics of vaccines at the country level. In countries, World Bank teams engaged in committees to plan and track vaccine support. In Djibouti, Honduras, the Philippines, and Tajiki- 114 stan, the World Bank coordinated with partners to prepare for early vaccine deployment. In Mozambique, the Philippines, and Tajikistan, the World Bank supported digital tracking systems for vaccine rollout and communication about vaccines. More early emphasis could have been put on communication plans and activities to strengthen vaccine delivery in countries. The World Bank’s MPA financing for vaccines provided prompt country-level support that coincided with the approval of the first vaccines, but it took time to develop country vaccine projects. In October 2020, the World Bank approved additional financing for vaccines of $12 billion for the MPA, which later grew to $20 billion. The financing was in anticipation of the upcoming emergency approval of the first COVID-19 vaccines by WHO (including $6 billion for IDA, of which $294.97 million was grant and $6 billion for International Bank for Reconstruction and Development countries). The first vaccines to receive emergency approval by WHO were Pfizer-BioNTech on December 31, 2020; Johnson & Johnson on March 12, 2021; and Moderna on April 30, 2021. The financing aimed to support the full vaccination of 1 billion people, or about 20 percent of country populations. The 20 percent coverage corresponded to the WHO Allocation Framework target for priority immunization that focused first on frontline health workers and caregivers, then the elderly and younger people with underlying conditions, which place them at higher risk for COVID-19 (WHO 2020a). The MPA emphasized community engagement and risk communication, which were limited in the Independent Evaluation Group World Bank Group    115 early COVID-19 response. However, the approval of MPA vaccine projects in countries took time because a second step of the World Bank Board decision was required for the first five projects—after the restructuring of a project in Lebanon to purchase vaccines (January 20, 2021), the first five MPA vaccine projects were approved by the World Bank Board in Cabo Verde, Mongolia, and Tajikistan (February 1, 2021) and in Afghanistan and Nepal (March 18, 2021). Projects of $100 million and more also required board approval; Bangladesh, Ethiopia, the Philippines, and Tunisia were approved in March 2021. After these projects, approvals in other countries increased quickly in subsequent months, with vaccine projects approved in 19 countries by April 30, 2021. Vaccine supply constraints on global markets and safeguards were chal- lenging for the first MPA vaccine projects. Without earlier advance market commitments at the global level, countries with limited resources were often unable to access global supplies of vaccines, despite the well-designed and prompt country-level financing of the MPA. Of the countries with less than 20 percent COVID-19 vaccine coverage, 89 percent are IDA recipients, and 75 percent are in Africa. Vaccine supply remained constrained well into 2021, in part as a result of manufacturing issues and the emergence of the Delta vari- ant that led to export restrictions on India’s vaccine supply. In 2021, the MPA financing for vaccines helped countries access vaccines from manufacturers and through donations, COVAX, and the Africa CDC, but supplies were often limited, and timing of receiving vaccines from different sources was difficult to manage and link to campaigns. Early use of MPA financing to procure vaccines in the first months of 2021 was also slowed by added safeguard pol- icies. The procurement of vaccines needed to follow additional World Bank guidance and safeguards beyond those of WHO. The operational process re- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 4 view and case studies (appendix C) acknowledged the rationale of caution in financing new, unproven vaccines. However, this guidance meant that in ear- ly 2021, World Bank vaccine projects could not process government requests to pool vaccine purchases with partners or often support the distribution of donated vaccine resources. After several months, the World Bank adjusted its actions (on April 16, 2021), recognizing the challenge, and aligned with WHO guidelines and Gavi, the Vaccine Alliance. A waiver was also provided for the first vaccine project to encounter this challenge in Lebanon. 116 1 COVID-19 Vaccine Deployment Tracker. https://covid19vaccinedeploymenttracker. worldbank.org/tracker.  Independent Evaluation Group World Bank Group    117 5 | Conclusions and Way Forward The quality of the health and social response was good, given considerable uncertainty in the early months. The emergency response was particularly swift in the most vulnerable countries. Among countries with medium to high vulnerability to human capital and development losses, the World Bank financed an estimated $30 billion in the early health and social response to COVID-19—about 40 percent of the World Bank’s total commitments to COVID-19 in fiscal years 2020 and 2021. Staff globally and clients in countries worked tirelessly to support the response. Although it is too early to observe outcomes, the evaluation points to prom- ising evidence of early successes, such as the expansion of critical health and social protection capacities. The World Bank used its experience from past crises to respond quickly and effectively, and teams innovated and engaged in frequent dialogue to adjust actions. Operational flexibility facilitated rapid financing for the MPA, which was critical to expanding health sup- port, and procurement was smooth, compared with past crises. World Bank country programs also drew on existing partnerships, crisis instruments, and regional projects to facilitate timely actions. Nevertheless, and notwithstanding preparedness efforts over the years, the World Bank and many client countries were not adequately prepared for the crisis. Countries with better capacities to coordinate, monitor, and deliver local services (health and education, among others), robust human capi- tal investments, and better public health preparedness were often able to address the crisis needs more comprehensively. Internal World Bank efforts were facilitated by already having operational support to human capital, gender, disease preparedness, existing data systems and partnerships, and crisis instruments in country portfolios. The evaluation findings point to the value of focusing on pandemic and cri- sis preparedness efforts in countries in the World Bank, at the regional level, and with global partners. 118   » Prioritizing support in areas to protect against human capital losses. In the early COVID-19 response, there was an important emphasis on emergency health and social protection support. This needed to be quickly complemented with support for education, maternal and child health, and women and girls, which in some countries led to a secondary crisis of health and education loss and deepening gender inequalities. Countries needed the systems to continue to deliver and ensure the quality of these services during the crisis, for example, crisis-adapted platforms, such as for telemedicine and remote learning. Having systems in place that can ensure continued access to essential health, education, and gender-based services, in addition to emer- gency social protection, is important for protecting human capital, especially in vulnerable groups. » Regional leadership and institutional capacities for crisis preparedness and crisis response. Regional organizations have an important role in con- vening leaders and technical actors in countries for policy dialogue, technical learning, cooperation, and problem-solving. Developing the capacities of re- gional organizations is key to facilitate learning and actions by countries and to expand preparedness capacities and rapid actions for crisis response. » Preparedness in countries. Institutional capacities for crisis preparedness, such as functioning coordination structures, critical health service capacities, and data systems, are important for building strong preparedness. Independent Evaluation Group World Bank Group    119 » Internal preparedness of the World Bank. Partnerships, operational readiness (tools and flexibilities), and hands-on assistance need to be in place before a crisis. Country portfolios that include crisis instruments and support for human capital allow for quick access to financing and swift support to vulnerable groups in times of crisis, while maintaining the focus on longer-term human capital development. Also important for country portfolios was drawing on repurposed projects, regional projects, ASA, grants, and trust funds to enable quick early financing. » Global alignment among partners. Global alignment and coordinated ac- tions at the global level are important for good support to countries. In the early days of the response, the World Bank could have played a more decisive role at the global level regarding vaccines and to ensure that guidance to pro- tect human capital was consistently part of the early global-level guidance on the COVID-19 response. Recommendations The findings from the evaluation inform four recommendations for ensuring stronger future preparedness. Recommendation 1. Use the World Bank’s crisis recovery efforts to strengthen the resilience of essential health and education services to en- sure that human capital is protected in a crisis. Proposed Actions The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 5 » In health, build on innovations developed during COVID-19 to help countries strengthen telehealth and other platforms for continuing essential health services in an emergency. Help countries improve the quality of frontline services, including the availability of data to inform decisions for quality improvements. Services could be improved to better manage supplies, deliver vaccines, support health workers to deliver effective care, and ensure infec- tion prevention and control measures. The availability and use of feedback from beneficiaries and coverage of vulnerable groups are also important. The World Bank could also help develop new capacities to deliver services, such as in psychosocial care. » In education, draw on evidence and innovations of the COVID-19 response to strengthen platforms for continuous learning in a crisis. Strengthen commu- nity networks that have been established to support learning. To avoid learn- ing losses, and facilitate knowledge building to uptake effective approaches to help children in and out of school catch up. Help countries develop ap- proaches that increase the reach to vulnerable groups that may have been missed by remote learning. Strengthen monitoring of beneficiary feedback on the quality of learning. Recommendation 2. Apply a gender equality lens to health and social crisis response actions across sectors. 120 Proposed Actions » Develop actions across sectors (in health, education, urban, and social pro- tection) for protecting women and girls from shocks, which can be drawn on in a crisis response. This is especially important in countries with high needs for addressing gender equality. Examples of areas to support gender equality include psychosocial support, sexual and reproductive health, income and asset accumulation, and community engagement. Recommendation 3. Help countries strengthen regional cooperation and crisis response capacities for public health preparedness. Proposed Actions » Support regional organizations to facilitate cooperation, political leader- ship, and technical learning, especially in Africa. Such support could help strengthen preparedness in countries and regional mechanisms for crisis response, facilitate financing and technical partnerships, encourage innova- tion, and expand evidence to scale up effective approaches. Regional support could also facilitate evidence-based and data tools to help countries priori- tize investments and monitor crisis response actions. » Help countries strengthen national and subnational platforms to coordinate and deliver crisis interventions, such as One Health platforms, with great- Independent Evaluation Group World Bank Group    121 er emphasis on critical health services and demand-side activities, such as citizen engagement. At the national level, invest in platforms that coordinate action and prepare various sectors to take on specific roles in crisis. At the subnational level, invest in platforms that can reach local government and communities for disease surveillance, risk communication, delivery of health and social services, and monitoring support. Recommendation 4. Build on the COVID-19 experience to strengthen the World Bank’s internal crisis preparedness so that it has the tools and proce- dures ready to respond in future emergencies. Proposed Actions » Review and expand operational flexibilities for processing new projects in crises and develop guidance on the effective use of instruments at different stages of crisis response. The World Bank could also explore innovative ways to strengthen the use of crisis instruments in countries, such as through support to communities, and expand guidance on hands-on assistance for citizen engagement and gender, learning from the provision of such support in procurement. » Expand and strengthen the World Bank’s partnerships and instruments to enable coordinated financing, advance market commitments, and technical support that will help countries strengthen crisis preparedness. The partner- ships could be at the global, regional, and country levels. They could include The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Chapter 5 technical partnerships to expand knowledge for quality implementation of preparedness activities, partnerships with nongovernment and the private sector to support community-based implementation, feedback on services and use of technology, and global partnership for aligning financing, plans, and guidance to support countries. » Strengthen tools to allow for the integrated management and frequent re- porting of monitoring data on projects in World Bank portfolios. 122 Bibliography Afrobarometer. 2021a. “COVID-19 Impact? Ugandans Grow More Discontent with Economic and Living Conditions, Afrobarometer Study Shows.” Press Release, March 30. Afrobarometer. 2021b. “Résumé des Résultats: Enquête Afrobarometer Round 8 au Sénégal, 2021.” Consortium for Economic and Social Research (CRES), Dakar, Senegal. Afrobarometer. 2021c. “Satisfaits de la Gestion de la Pandémie de COVID-19, les Sénégalais sont Sceptiques vis-à-vis des Vaccins.” Press Release, March 5. Afrobarometer. 2021d. “Ugandans Willing to Be Vaccinated Despite Doubts about Vaccine Safety, Afrobarometer Study Shows.” Press Release, June 4. Agarwal, R., and T. Reed. 2021. “How to End the COVID-19 Pandemic by March 2022.” Policy Research Working Paper 9632, World Bank, Washington, DC. Aslam, M., and Shenila Rawal. 2021. Formative Evaluation of GPE’s Support for Response to the COVID-19 Crisis. Washington, DC: Global Partnership for Education. https://www.globalpartnership.org/content/formative-evaluation- gpes-support-response-covid-19-crisis. Independent Evaluation Group World Bank Group    123 Barış, E., R. Silverman, H. Wang, F. Zhao, and M. A. Pate. 2021. Walking the Talk: Reimagining Primary Health Care after COVID-19. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/35842. Bau, N., G. Khanna, C. Low, M. Shah, S. Sharmin, and A. Voena. 2021. ”Women’s Well-Being during a Pandemic and Its Containment.” NBER Working Paper 29121, National Bureau of Economic Research, Cambridge, MA. Bautista Jr., A., D. Balibrea, and D. G. Bleza. 2020. “Knowledge, Attitude, and Prac- tice toward the Coronavirus Disease (COVID-19) Outbreak among Selected Employed People in the National Capital Region, Philippines.” Asian Journal for Public Opinion Research 8 (3): 324–50. Bhattacharya, S., and S. S. Roy. 2021. “Intent to Implementation: Tracking India’s Social Protection Response to COVID-19.” Social Protection and Jobs Discussion Paper 2107, World Bank, Washington, DC. Cernuschi, T., E. Furrer, N. Schwalbe, A. Jones, E. R. Berndt, and S. McAdams. 2011. “Advance Market Commitment for Pneumococcal Vaccines: Putting Theory into Practice.” Bulletin of the World Health Organization 89 (12): 913–18. Chazaly, C., and E. Goldman. 2021. MENA Real Time Assessment: COVID 19 Response. New York: UNICEF. Dalberg Advisors. 2021. ACT-Accelerator Strategic Review: An Independent Report Prepared by Dalberg. Geneva: Dalberg Advisors. https://www.who.int/publica- tions/m/item/act-accelerator-strategic-review. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Bibliography Dixon, M. G., M. Ferrari, S. Antoni, X. Li, A. Portnoy, B. Lambert, S. Hauryski, C. Hatcher, Y. Nedelec, M. Patel, J. P. Alexander Jr., C. Steulet, M. Gacic-Dobo, P. A. Rota, M. N. Mulders, A. S. Bose, A. Rosewell, K. Kretsinger, and N. S. Crowcroft. 2021. “Progress toward Regional Measles Elimination—Worldwide, 2000–2020.” Morbidity and Mortality Weekly Report 70 (45): 1563–69. DPME (Department of Planning, Monitoring and Evaluation), GTAC (Government Technical Advisory Centre), and NRF (National Research Foundation). 2021. South Africa COVID-19 Country Report, 1st edition. Pretoria, South Africa: Gov- ernment Communication and Information System. FAO (Food and Agriculture Organization of the United Nations), OIE (World Organi- sation for Animal Health), WHO (World Health Organization), and UNEP (Unit- ed Nations Environment Programme). 2022. “Memorandum of Understanding between the Food and Agriculture Organization of the United Nations and the World Organisation for Animal Health and the World Health Organization and the United Nations Environment Programme Regarding Cooperation to Combat Health Risks at the Animal-Human-Ecosystems Interface in the Context of the ‘One Health’ Approach and Including Antimicrobial Resistance.” Rome: FAO. https://www.fao.org/3/cb9403en/cb9403en.pdf. Gentilini, U. 2022. Cash Transfers in Pandemic Times: Evidence, Practices, and Implica- tions from the Largest Scale-Up in History. Washington, DC: World Bank. 124 GFF (Global Financing Facility). 2021. Reclaim the Gains: The Case for Investing in the Global Financing Facility 2021–25. Washington, DC: World Bank Group. Global Education Evidence Advisory Panel. 2022. Prioritizing Learning during COVID-19: The Most Effective Ways to Keep Children Learning during and Post-Pandemic. Washington, DC; London; Florence: World Bank; UK Foreign, Commonwealth, and Development Office; UNICEF Office of Research— Innocenti. Gold, J., and S. Hutton. 2020. “3 Lessons from Past Public Health Crises for the Global Response to COVID-19 (Coronavirus).” IEG Blog (blog), May 6, 2020. https://ieg.worldbankgroup.org/blog/3-lessons-past-public-health-crises- global-response-covid-19-coronavirus. Grover, S., S. Sahoo, A. Mehra, A. Avasthi, A. Tripathi, A. Subramanyan, A. Pattojoshi, G. P. Rao, G. Saha, K. K. Mishra, K. Chakraborty, N. P. Rao, M. Vaishnav, O. P. Singh, P. K. Dalal, R. K. Chadda, R. Gupta, S. Gautam, S. Sarkar, T. S. R. Sathyanarayana, V. Kumar, and Y. C. J. Reddy. 2020. “Psychological Impact of COVID-19 Lockdown: An Online Survey from India.” Indian Journal of Psychiatry 62 (4): 354–62. IDA (International Development Association). 2017. “Additions to IDA Resources: Eighteenth Replenishment—Toward 2030: Investing in Growth, Resilience and Opportunity.” Board Report 112728, World Bank, Washington, DC. Independent Evaluation Group World Bank Group    125 IDA (International Development Association). 2020. “Additions to IDA Resources: Nineteenth Replenishment—IDA19: Ten Years to 2030: Growth, People, Resil- ience.” Board Report 146261, World Bank, Washington, DC. Independent Panel for Pandemic Preparedness and Response. 2021. COVID-19: Make It the Last Pandemic. Independent Panel for Pandemic Preparedness and Re- sponse. Geneva: World Health Organization. https://theindependentpanel.org/ wp-content/uploads/2021/05/COVID-19-Make-it-the-Last-Pandemic_final.pdf. Ipsos. 2020. “Responding to COVID-19: Highlights of a Survey in Mozambique.” Ipsos, Paris. Johnson, L., and M. Kennedy-Chouane. 2021. “The COVID-19 Pandemic: How Are Humanitarian and Development Cooperation Actors Doing So Far? How Could We Do Better? Synthesis of Early Lessons and Emerging Evidence on the Initial COVID-19 Pandemic Response and Recovery Efforts.” COVID-19 Global Eval- uation Coalition, Organisation for Economic Co-operation and Development, Paris. KNOMAD (Global Knowledge Partnership on Migration and Development). 2021. “Resilience: COVID-19 Crisis through a Migration Lens.” Migration and Development Brief 34, World Bank, Washington, DC. https://www.knomad.org/ publication/migration-and-development-brief-34. Loperfido, L., M. Burgess, N. Dulieu, C. Orlassino, M. Sulaiman, and S. M. Arlini. 2020. The Hidden Impact of COVID-19 on Child Poverty. London: Save the Children The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Bibliography International. Muñoz-Najar, A., A. Gilberto, A. Hasan, C. Cobo, J. P. Azevedo, and M. Akmal. 2022. Remote Learning during COVID-19: Lessons from Today, Principles for Tomorrow. Washington, DC: World Bank. Nieves, C. D., I. Gaddis, and M. Muller. 2021. “Gender and COVID-19: What Have We Learnt, One Year Later?” Policy Research Working Paper 9709, World Bank, Washington, DC. OECD (Organisation for Economic Co-operation and Development). 2022. “First Lessons from Government Evaluations of COVID-19 Responses: A Synthe- sis.” OECD Policy Responses to COVID-19, OECD, Paris. https://www.oecd. org/coronavirus/policy-responses/first-lessons-from-government-evalua- tions-of-covid-19-responses-a-synthesis-483507d6/#contactinfo-d7e5802. Poljanšek K., L. Vernaccini, and M. Marin Ferrer. 2020. “INFORM COVID-19 Risk Index.” European Union, Luxembourg. https://publications.jrc.ec.europa.eu/ repository/handle/JRC120799. Ravelo, J. L., and S. Jerving. 2022. “COVID-19—A Timeline of the Coronavirus Out- break.” Devex, Inside Development. https://www.devex.com/news/covid-19-a- timeline-of-the-coronavirus-outbreak-96396. 126 Rodriguez, M. B., C. Cobo, A. Muñoz-Najar, and I. S. Ciarrusta. 2021. Remote Learn- ing during the Global School Lockdown: Multi-Country Lessons. Washington, DC: World Bank. Saavedra, J., C. Aedo, O. Arias, A. Pushparatnam, H. Rogers, and M. G. Bernal. 2020. Realizing the Future of Learning: From Learning Poverty to Learning for Everyone, Everywhere. Washington, DC: World Bank Group. Schneider, S. H., J. Eger, M. Bruder, and J. Faust. 2020. “Public Opinion on Inter- national Solidarity in the Coronavirus Pandemic.” DEval Policy Brief 4/2020, German Institute for Development Evaluation, Bonn. Subandoro, A. W., S. Holschneider, and J. Ruel-Bergeron. 2021. “Operationalizing Multisectoral Nutrition Programs to Accelerate Progress: A Nutrition Gover- nance Perspective.” Health, Nutrition, and Population Discussion Paper, World Bank, Washington, DC. Sumalatha, B. S., L. D. Bhat, and K. P. Chitra. 2021. “Impact of COVID-19 on Informal Sector: A Study of Women Domestic Workers in India.” The Indian Economic Journal 69 (3): 441–61. Tuppal, C. P., M. M. G. Ninobla, M. G. D. Ruiz, R. D. Loresco, S. M. P. Tuppal, I. I. Panes, R. M. F. Oducado, D. A. M. Prudencio, P. D. Vega, M. J. E. Eribal, D. V. P. Real, and M. N. T. Roa. 2021. “Knowledge, Attitude, and Practice toward COVID-19 among Healthy Population in the Philippines.” Nurse Media Journal of Independent Evaluation Group World Bank Group    127 Nursing 11 (1): 61–70. UN (United Nations). 2020. United Nations Comprehensive Response to COVID-19: Saving Lives, Protecting Societies, Recovering Better. New York: UN. UN Women. 2021. COVID-19 Rapid Gender Assessment: Gender Perspective Mozam- bique. New York: UN Women. Vancutsem, S., and A. Mahieu. 2020. Results of the Evaluation of Enabel’s Response to the COVID-19 Pandemic. Brussels: Enabel. Van Trotsenburg, A. 2021. “Leaving No Country Behind: Africa’s Pathbreaking Collective Action on Vaccines.” Voices (blog), August 6, 2021. https://blogs. worldbank.org/voices/leaving-no-country-behind-africas-pathbreaking- collective-action-vaccines. WHO (World Health Organization). 2017. “One Health.” Questions and Answers, WHO, Geneva (accessed April 6, 2022). https://www.who.int/news-room/ques- tions-and-answers/item/one-health. WHO (World Health Organization). 2020a. Guidance on Developing a National Deploy- ment and Vaccination Plan for COVID-19 Vaccines: Interim Guidance, November 16, 2020. Geneva: WHO. https://www.who.int/publications/i/item/WHO-2019- nCoV-Vaccine_deployment-2020.1. WHO (World Health Organization). 2020b. The Impact of COVID-19 on Mental, Neurological and Substance Use Services: Results of a Rapid Assessment. Geneva: WHO. WHO (World Health Organization). 2021a. COVID-19 Strategic Preparedness and The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Bibliography Response Plan. Geneva: WHO. WHO (World Health Organization). 2021b. “Second Round of the National Pulse Sur- vey on Continuity of Essential Health Services during the COVID-19 Pandemic: January–March 2021: Interim Report.” WHO, Geneva. WHO (World Health Organization). 2021c. Strategic Plan and Budget, October 2021 to September 2022: Enhancing Equity in Access to COVID-19 Tools. Geneva: WHO. https://www.who.int/publications/m/item/act-accelerator-strategic-plan- budget-october-2021-to-september-2022. WHO (World Health Organization). 2021d. “WHO COVID-19 Partners Platform.” WHO, Geneva (accessed May 1, 2021). https://covid19partnersplatform.who.int/en. WHO (World Health Organization). 2022. “Timeline: WHO’s COVID-19 Response.” WHO, Geneva (accessed March 2, 2022). https://www.who.int/emergencies/dis- eases/novel-coronavirus-2019/interactive-timeline. WHO (World Health Organization) and UNICEF (United Nations Children’s Fund). 2021. Monitoring COVID-19 Vaccination: Considerations for the Collection and Use of Vaccination Data: Interim Guidance, March 3, 2021. Geneva and New York: WHO and UNICEF. https://apps.who.int/iris/bitstream/handle/10665/339993/ WHO-2019-nCoV-vaccination-monitoring-2021.1-eng.pdf. World Bank. 2006. Hazards of Nature, Risks to Development: An IEG Evaluation of World Bank Assistance for Natural Disasters. Independent Evaluation Group. 128 Washington, DC: World Bank. https://openknowledge.worldbank.org/bitstream/ handle/10986/7001/366150Hazards0and0risks01PUBLIC1.pdf?sequence=1&is- Allowed=y. World Bank. 2012. The World Bank Group’s Response to the Global Economic Crisis: Phase II. Independent Evaluation Group. Washington, DC: World Bank. https:// ieg.worldbankgroup.org/sites/default/files/Data/reports/crisis2_full_report.pdf. World Bank. 2014. The World Bank’s Partnership with the GAVI Alliance. Global Program Review. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/wbp_ gavi_alliance2.pdf. World Bank. 2017. Crisis Response and Resilience to Systemic Shocks: Lessons from IEG Evaluations. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/reports/building- resilience.pdf. World Bank. 2019a. IDA’s Crisis Response Window: Lessons from IEG Evaluations. Independent Evaluation Group. Washington, DC: World Bank. https://ieg. worldbankgroup.org/sites/default/files/Data/reports/syn_idacrisisresponse.pdf. World Bank. 2019b. Knowledge Flow and Collaboration under the World Bank’s New Operating Model. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/reports/kfc.pdf. Independent Evaluation Group World Bank Group    129 World Bank. 2019c. Two to Tango: An Evaluation of the World Bank Group Support to Fostering Regional Integration. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/ files/Regionalintegration.pdf. World Bank. 2020a. “Citizen Engagement and Stakeholder Consultations during COVID-19.” Factsheet, November 24, World Bank, Washington, DC. https://www. worldbank.org/en/news/factsheet/2020/12/01/citizen-engagement-and-stake- holder-consultations-during-covid-19. World Bank. 2020b. “COVID-19 Strategic Preparedness and Response Program Using the Multiphase Programmatic Approach (Global COVID-19 MPA).” Project Paper PAD4185, World Bank, Washington, DC. World Bank. 2020c. “Fact Sheet: Pandemic Emergency Financing Facility.” Brief, April 27, World Bank, Washington, DC. https://www.worldbank.org/en/topic/ pandemics/brief/fact-sheet-pandemic-emergency-financing-facility. World Bank. 2020d. “Governance and Institutions COVID-19 Response Resources.” Brief, World Bank, Washington, DC. https://www.worldbank.org/en/topic/ governance/brief/governance-institutions-covid-19-response-resources. World Bank. 2020e. The COVID-19 Pandemic: Shocks to Education and Policy Responses. Washington, DC: World Bank. https://openknowledge.worldbank.org/ handle/10986/33696. World Bank. 2020f. The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19. Washington, DC: World Bank. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Bibliography World Bank. 2021a. Assessing Country Readiness for COVID-19 Vaccines: First Insights from the Assessment Rollout. Washington, DC: World Bank. World Bank. 2021b. “COVID-19: World Bank Group Support for Fair and Affordable Access to Vaccines by Developing Countries.” World Bank Group, Washington, DC. World Bank. 2021c. Crisis and Recovery: Economic and Social Monitoring from Listening to Tajikistan. Washington, DC: World Bank. World Bank. 2021d. Djibouti—COVID-19 National Panel Phone Survey, Wave 4 (CNPPS-W4). Washington, DC: World Bank. World Bank. 2021e. “Factsheet: Policy Actions for School Reopening and Learning Recovery.” World Bank, Washington, DC (accessed July 19, 2022). https://www. worldbank.org/en/news/factsheet/2021/04/30/notes-on-school-reopening-and- learning-recovery. World Bank. 2021f. “Findings for COVID-19 from the World Bank’s Support to Ad- dress Ebola Outbreaks.” IEG Lesson Library (July 28), Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2021g. “Impacts of COVID-19 on Communities in the Philippines: Results from the Philippines High Frequency Social Monitoring of COVID-19 Impacts Round 2: April 8–14, 2021.” World Bank, Washington, DC. 130 World Bank. 2021h. “World Bank and African Union Team Up to Support Rapid Vacci- nation for Up to 400 Million People in Africa.” Press Release, June 21. https:// www.worldbank.org/en/news/press-release/2021/06/21/world-bank-and-afri- can-union-team-up-to-support-rapid-vaccination-for-up-to-400-million-peo- ple-in-africa. World Bank. 2022a. “Lessons from the Review of Health and Social Innovations in the COVID-19 Pandemic Response.” IEG Lessons Library (March 29), Indepen- dent Evaluation Group, World Bank, Washington, DC. World Bank. 2022b. Women, Business, and the Law 2022. Washington, DC: World Bank. World Bank. 2022c. World Development Report 2022: Finance for an Equitable Recov- ery. Washington, DC: World Bank. World Bank. Forthcoming. Embedding Human Capital in Policy Financing: A Just-in- Time Note on the Footprint of the Human Capital Project on Development Policy Operation Design. Washington, DC: World Bank. World Bank Group. 2020a. Prioritizing Human Capital in the World Bank Operational Response to COVID-19: The First 100 Days. Washington, DC: World Bank Group. World Bank Group. 2020b. Saving Lives, Scaling-Up Impact, and Getting Back on Track: World Bank Group COVID-19 Crisis Response Approach Paper. Approach Paper. Washington, DC: World Bank Group. World Bank Group. 2021a. Prevention, Preparedness, and Response: The World Bank Independent Evaluation Group World Bank Group    131 Group’s Role in Future Crises. Washington, DC: World Bank Group. https:// devcommittee.org/sites/dc/files/download/Documents/2021-09/DC2021- 0006%20Final%20Prevention%20paper.pdf. World Bank Group. 2021b. World Bank Group Financing for Green, Resilient, and Inclusive Development—Toward a Post Pandemic Approach. Washington, DC: World Bank Group. https://devcommittee.org/sites/dc/files/download/Docu- ments/2021-09/DC2021-0007%20Final%20GRID%20paper.pdf. World Bank Group and United Nations. 2021. United Nations—World Bank Partnership in Crisis-Affected Situations: 2020 UN–World Bank Partnership Monitoring Report. Washington, DC: World Bank Group. APPENDIXES Independent Evaluation Group The World Bank’s Early Support to Addressing COVID-19: Health and Social Response Appendix A. Methodology Evaluation Questions The overarching question that the evaluation answers is: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? Three questions underlie this query: » What has been the relevance of the World Bank’s early COVID-19 response to addressing the needs of countries in saving lives and protecting poor and vul- nerable people (that is, the diagnosis, design, and tailoring of interventions The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A to country situations)? » What has facilitated or hindered implementation of the World Bank’s COVID-19 responses in countries, and how is the World Bank supporting learning and adjustments? » How well are operational processes, instruments, and partnerships support- ing the World Bank’s COVID-19 responses in countries? Evaluation Design and Framework The evaluation is designed to support learning from the World Bank’s COVID-19 response based on evidence at the country, portfolio, and corpo- rate levels. It uses a mixed methods approach that combines quantitative and qualitative evidence. To support feedback, the design split the delivery of the evaluation into modules, whose staggered delivery enabled dialogue with World Bank management and staff on interim findings as evidence emerged. The evaluation is underpinned by a conceptual framework of the stages and thematic areas of the health and social response and by a theory of action to guide the review of the quality of the World Bank’s COVID-19 response to support countries. The conceptual framework is adapted from the World Bank Group’s COVID-19 response framework (World Bank 2020b). It describes a multidimensional crisis response with interlinked health and social responses 134 across three stages: relief, restructuring, and recovery. The theory of action defines the dimensions to assess the quality of World Bank support in relation to its relevance to needs of countries, implementation and learning and ad- justment, and operational processes in support of countries. In developing the theory of action, the evaluation team drew on principles related to the science of delivery were drawn on (Asis and Woolcock 2015). Evaluation Components Table A.1 lists and describes the components of the overall evaluation de- sign. The components’ methods vary to support the triangulation of findings across the three evaluation questions, drawing on data from the country, portfolio, and corporate levels. Independent Evaluation Group World Bank Group    135 136 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A Table A.1. Evaluation Components Evaluation Component Description Case-based analysis Case studies of eight countries reviewed their projects and analytic work, and interviewed World Bank staff, government, and civil society to review the quality of the World Bank’s early COVID-19 response against the theory of action. Review of country situations: The analysis used publicly available data on indicators in areas of the conceptual framework, portfolio support types, needs, and data, and data on World Bank support to human capital in countries before COVID-19 to (i) apply machine implementation status learning cluster analysis to understand differences in support to COVID-19 across countries; (ii) assess the alignment of the World Bank’s COVID-19 support with countries’ needs; and (iii) apply decision tree analysis to understand the features of the portfolio that facilitated or hindered satisfactory project implementation in countries during the early COVID-19 response. Rapid review of evidence A structured literature review identified evidence on 50 interventions to support effective epidemic and crises responses using a rapid scoping method. Evidence was reviewed from systematic reviews and coun- try studies and benchmarked against the World Bank support to countries. Review of lessons The review synthesized lessons from past crises in areas of the theory of action, based on 170 projects where the World Bank responded to crises in the past 20 years. The lessons were then benchmarked against successes and challenges from Implementation Status and Results Reports of projects supporting the early COVID-19 response in countries. Learning on regional support Four regional projects were reviewed to assess their support to the COVID-19 response in countries. Using the outcome harvesting approach, evidence on emerging results was collected from a review of project documents and interviews of country actors and World Bank staff involved in the projects, which were then verified in consultation with the project team. Stocktaking of innovations Innovations supporting the World Bank’s COVID-19 response in countries were identified through a crowdsourcing survey of country teams, the portfolio review, and a review of innovation stories published by Global Practices. (continued) Evaluation Component Description Portfolio identification and analysis A portfolio of 253 projects and 175 advisory services and analytics supporting the first 15 months of the COVID-19 response was identified from 107 countries using a systematic process of search, delimitation and inclusion, coding and verification, and analysis. Analysis was done against the areas of the theory of action. The portfolio analysis included a review of support the Multiphase Programmatic Approach used by Health, Nutrition, and Population (HNP). Monitoring analysis Based on the portfolio of 253 projects, 2,219 indicators were identified and reviewed. Analyses of indicators examined monitoring of the early COVID-19 response. Analysis of multiphase The analysis of the MPA draws evidence from the evaluation portfolio, case studies, and innovation programmatic approach of health stocktaking to assess the MPA projects led by the HNP Global Practice. The focus is on the first year of the MPA support between April 2020 and April 2021. Review of internal processes and Interviews and documents on the World Bank response were analyzed to identify lessons from the World partnerships Bank’s COVID-19 corporate-level response across areas of the theory of action. Key aspects of the review looked at coordination and collaboration, past crisis experiences, financing instruments, operational processes, partnerships, knowledge support, and monitoring of the response. Source: Independent Evaluation Group; World Bank 2022. Independent Evaluation Group World Bank Group    137 Ensuring Validity of Findings The evaluation team took steps to reinforce consistent assessment of the quality of the World Bank’s early COVID-19 response. » The components of the evaluation addressed different levels of analysis (country, portfolio, corporate) to address the evaluation questions; the design of the protocols used for the analysis followed the theory of action and con- ceptual framework (figure A.1). » Triangulation was applied at multiple levels by cross-checking evidence sources within a given evaluation component and across components. For example, in case studies, information was extracted from interviews with country implementers and World Bank staff and document review to validate The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A findings. Further, the team triangulated findings across different evaluation components and levels of analysis by iteratively synthesizing and making sense of evidence to respond to the evaluation questions. For example, val- idating findings from the case studies using findings from the review of coun- try situations, portfolio, and internal processes and partnership review. » Findings from the analyses were validated through discussions with World Bank counterparts, such as teams in World Bank country offices and Global Practices (GPs), to interpret and review findings from the evaluation compo- nents. Consultations were also organized with World Bank counterparts to validate the evaluation’s scope and methods. » Advisers and peer reviewers provided feedback at the beginning, during, and at the end of the evaluation process, and the team followed Independent Evaluation Group (IEG) quality control processes. » The evaluation team triangulated findings across other ongoing evaluations, including the IEG evaluation of the economic response to COVID-19. Limitations Notwithstanding these steps, the following are key limitations of the evalua- tion design. 138 » Since this is an early-stage evaluation, outcomes are not assessed; rather, the focus is on the quality of the response based on the relevance of its design and whether implementation and learning processes and outputs are well positioned to support pathways to outcomes. This analysis offers learning to inform later stages of the response. » The portfolio analysis focuses on the World Bank’s early COVID-19 response between February 1, 2020, and April 30, 2021, data was extracted at various dates outlined in detail below, providing an estimated snapshot of the early time period of an evolving portfolio. The portfolio is focused on countries vulnerable to human capital and development losses, and, thus, it is most representative of these countries. To consider the dynamic nature of the re- sponse, a rapid update of the portfolio was done at the end of the evaluation to understand how support is shifting. Moreover, the portfolio estimates early support and financing in the countries included in the evaluation. Countries have a range of repurposed projects supporting COVID-19 that the evaluation may not fully capture. Moreover, the portfolio focuses on the GPs included in the evaluation, while a range of GPs supported the health and social response in countries, such as Agriculture and Food, Social Sustainability and Inclu- sion, and Transport. Further, a detailed analysis of COVID-19 financing is beyond the scope of the evaluation. » Case studies included information on the response in the country, advisory services and analytics (ASA) and project document reviews and interviews Independent Evaluation Group World Bank Group    139 with country actors and World Bank staff. The case studies were completed over about six months, between April 2021 and October 2021, due to the high number of COVID-19 cases in countries, the intensive burden of the pandem- ic on health sector personnel, and illness and personal losses of interviewees due to COVID-19. Interviews with subnational actors are limited to two case studies (Senegal and India). Challenges interviewing local actors in countries were mitigated by using publicly available secondary data on beneficiary feedback about the COVID-19 response. » The review of country situations estimates country needs before and during the COVID-19 response using publicly available data. However, data on indicators are limited to those available at the national level. Data are miss- ing for some countries and the analysis does not include data on how other partners may have addressed needs. Moreover, the findings of the cluster- ing and decision tree analyses are based on analyzed features of the early response. The response may have different features in future that would need to be considered. » A strength of the literature review is that no interventions are ruled out ex ante. However, this is a rapid review limited to evidence from systematic reviews and country studies published after January 2016. It likely misses recent studies emerging from the ongoing support to COVID-19 country responses. Moreover, the review of evidence on social protection interven- tions is limited by the focus on epidemic and crisis situations, since these are long-term interventions in countries to protect vulnerable groups, expanded in a crisis. Similarly, evidence on remote learning may be transferable from The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A noncrisis situations and would have been missed by the literature review. » The regional project review examines the support of regional projects to implementation and learning to advance country responses for COVID-19. However, broader achievements of these projects are beyond the scope of the evaluation. Moreover, while the outcome harvesting method used for this analysis provides a useful means of backward-tracing verified outcomes, selecting on outcomes may introduce biases of omission relative to which areas were and were not explored in interviews and identified in documents. To avoid this, interview questions were semistructured. » A key strength of the stocktaking of innovations is the capture of examples of how the World Bank innovated in the early COVID-19 response. However, the evaluation does not look at the effectiveness of these innovations. Moreover, the innovations identified were self-selected by World Bank teams or have been chosen for inclusion in documents that were reviewed by the evalua- tion. As a result, the analysis may have missed innovations not reported by World Bank teams. Also, what constitutes an innovation can be interpreted in different ways. » The main limitation of the corporate-level review of internal processes and partnerships is that interviews captured perspectives from the first 15 months of implementation, and the understanding of and thinking about the response evolved rapidly. 140 Figure A.1. Evaluation Design Matrix Source: Independent Evaluation Group. Note: EQ = evaluation question. Independent Evaluation Group World Bank Group    141 Overview of Evaluation Methods Case-Based Analysis The case studies provide in-depth analysis of the quality of the early COVID-19 response at the country level for all areas of the theory of action. The case studies focus on the evaluation time period (February 1, 2020, to April 30, 2021), and interviews were conducted between April and Octo- ber2021. The evaluation team selected eight countries (Djibouti, Honduras, India, Mozambique, the Philippines, Senegal, Tajikistan, and Uganda) for analysis to understand the support of multiple GPs and how early support is helping to protect human capital. The cases selected received health, social, and institutional strengthening support and have a Human Capital Index of The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A 50 percent or below. The case study analysis triangulates data from project and country documents, interviews, and secondary data on the COVID-19 situation (Yin 1999). Data were captured in Excel using a consistent protocol across countries. Interviewees include actors involved in implementation of the response, including World Bank task teams and country management, government, civil society, and development partners. Evidence was gathered and synthesized using consistent protocol through remote connections due to travel restrictions. In each country, the IEG team employed national con- sultants to facilitate country stakeholder interviews. The case studies focus on five GPs: Education; Health, Nutrition, and Population (HNP); Macro- economics, Trade, and Investment; Social Protection and Jobs; and Urban, Disaster Risk Management, Resilience, and Land (GPURL). Other GP projects were reviewed, and staff were interviewed, when recommended by country management to understand the cross-sector breadth of the health and social support in the countries. Review of Country Situations: Support Types, Needs, and Implementation Status The analysis estimates how the response has addressed needs of countries, the differing types of support to countries, and features that facilitated the satisfactory implementation of early World Bank support. The analysis uses the thematic areas of the conceptual framework—critical health services, es- 142 sential health services, risk communication, protecting poor people and vul- nerable, child welfare and social services, and cross-cutting areas of gender and inclusion—to guide data collection on indicators to estimate country sit- uations and needs at the onset of COVID-19. It also looks at data on poten- tial factors of country vulnerability (urban risk and response capacities), and data on the situation during COVID-19 (disease situation, responsiveness of the countries, and estimated changes in the social situation). Publicly avail- able indicators from secondary data sources were collected for 80 countries. For themes with multiple indicators, principal component analysis and com- posite measures were used to reduce the dimensionality of the data (Howe et al. 2008; Pirani 2014). The data on country situations was combined with portfolio data on the World Bank’s COVID-19 response, and data on World Bank support to human capital before COVID-19 from a forthcoming IEG analysis (World Bank, forthcoming). The data on country situations is from between February 2020 and April 2021. Data on human capital support between FY15 and FY20 before COVID-19 from a forthcoming Independent Evaluation Group (IEG) evaluation was also integrated in the Excel data set (World Bank, forthcoming).Three analyses were done using this integrated data set: (i) machine learning clustering analysis in Python was used to un- derstand the types of World Bank support planned among countries (Caliński and Harabasz 1974; Davies and Bouldin 1979; Handl and Knowles 2007); (ii) Stata and Excel were used to develop a heat map to assess the alignment of World Bank support with country needs and previous human capital support; Independent Evaluation Group World Bank Group    143 and (iii) decision tree analysis was conducted in Python and applied at the country level to understand the conditions facilitating and hindering sat- isfactory project implementation in the early COVID-19 response (Kam Ho 1995; Schapire 2013). The main classification feature in the decision tree was the proportion of projects with satisfactory implementation status ratings. Rapid Evidence Review The literature review synthesizes evidence on the effectiveness of health and social interventions to support epidemics and crises to better under- stand the relevance of the World Bank’s interventions in countries (Arksey and O’Malley 2005; Levac et al. 2010). As the COVID-19 evaluation design calls for a nimble, learning-oriented approach, a rapid scoping method was used to identify evidence relevant for the World Bank’s response. The review focuses on evidence from recent health crises, such as Ebola, and knowledge on COVID-19 published after January 2016. Covidence software and Excel were used to manage the review. Evidence sources are limited to systematic review studies and country studies in English from low- and middle-income countries. Key databases searched between November 15, 2020, and Febru- ary 15, 2021, include EvidenceAid, PubMed, SCOPUS, Cochrane, Campbell, 3ie, J-PAL, World Bank Development Impact Evaluation, and World Bank Open Knowledge Repository. Minimum quality standards were assured by prioritizing peer-reviewed articles. The search for evidence was conducted in two phases: the first stage was a broad search using keywords “epidem- ic” “outbreak” or “pandemic;” and the second stage used keywords related to the thematic areas of the conceptual framework of the evaluation. All The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A article titles and abstracts were manually screened, and then the full text of the remaining articles reviewed. The final search phase yielded 70 rele- vant articles, with evidence on 50 interventions relevant to different areas of the COVID-19 response framework. The coverage of these interventions in the World Bank’s portfolio was then reviewed to assess the alignment of COVID-19 support with the existing evidence base on what works to support crisis response. Review of Lessons The review of lessons helps to understand areas that facilitated or hindered implementation of the early COVID-19 response in countries, and how these compare to past crises. The analysis systematically identifies and synthesizes lessons reported in projects from crises over the past 20 years (January 2000 through December 2020) and benchmarks these against successes and chal- lenges reported in Implementation Status and Results Reports of projects supporting the COVID-19 response. The search for lessons in projects was conducted in two phases. The first stage was a broad search using keywords (“crisis” “emergency” “epidem*” “disease” AND “outbreak” ”pandemic” the second stage used keywords related to the thematic areas of the conceptu- al framework of the evaluation. In total, 256 lessons from 170 past projects were coded in Excel against the theory of action and grouped by common areas of success or challenge. The successes and challenges reported in the 144 current response were then benchmarked against past lessons to assess how the World Bank improved its crisis support. Learning on Regional Support The review of regional support focuses on the value-added by four regional health projects (Regional Disease Surveillance Systems Enhancement proj- ect, Africa Centres for Disease Control and Prevention project, East Africa Public Health Laboratory Networking, and Organisation of Eastern Carib- bean States Regional Health Project) to support country-level COVID-19 responses. These were the main regional projects identified in the portfolio and in discussion with GPs supporting COVID-19 projects. The analysis uses an outcome harvesting method (Wilson-Grau 2019), which draws on evidence from interviews with country implementers and World Bank staff involved in regional projects and project document review (such Implemen- tation Status and Results Reports and Aide Memoires) to gather detailed information in the form of verifiable outcome statements that describe early results and process milestones achieved to support countries. These state- ments focus on what the milestone was, who was involved, when and where, why it was significant, and how the project provided support. The timeline of the analysis is the evaluation period (February 1, 2020, to April 30, 2021), and interviews were conducted between April and June 2021. The findings are organized by result areas to analyze the support to COVID-19 responses Independent Evaluation Group World Bank Group    145 across regional projects. A review process with project teams verified the accuracy of the outcome statements. Stocktaking of Innovations The stocktaking of innovations seeks to understand the innovations under- taken by the World Bank in-country responses to COVID-19. Innovations were captured in more than 100 countries through the portfolio review, crowdsourcing, and reviewing documented innovations. The examples of innovations were collected between February and June 2021. First, innova- tions described in project documents and Implementation Status and Results Reports were extracted. Second, innovations were crowdsourced from World Bank task teams by inviting all Human Development Practice Group Program Leaders to complete a brief questionnaire in SurveyMonkey to capture new approaches. Sixty-eight percent of World Bank country management units re- sponded to the survey. Third, innovations were identified by reviewing stories shared in documents and databases on the World Bank COVID-19 intranet sites of GPs. The team included examples of innovations if they reflected a new approach or practice in the country and if the World Bank had a defined role in designing or implementing the innovation. Innovations also needed to fall within the scope of the COVID-19 health and social response to be included. Each innovation was coded by the following: thematic area; actors involved in its implementation (community groups, health structures, local government, multisectoral team, schools, private sector, and nongovernmen- tal organizations); whether the innovation addressed digitalization, gen- der, or monitoring and evaluation; and if it could be applied to support the The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A restructuring of systems. All innovations were coded and analyzed in Stata through descriptive statistics, similar types of innovations were grouped to identify common themes, and positive outliers were identified where innova- tions were limited and could be further expanded (World Bank 2022). Portfolio Identification and Analysis For the portfolio analysis a systematic document and data review focused on internal sources and was enriched by using external quantitative data. The portfolio covers the period of February 1, 2020, to April 30, 2021. The project list was first extracted on February 1, 2021, and then updated on May 12, 2021. Disbursement monthly snapshot data have been updated to June 1, 2021, and Implementation Status and Results Report data were extracted on November 5, 2021. The process is anchored in the thematic areas of the conceptual framework and dimensions of the theory of action to assess the quality of the World Bank’s early COVID-19 response. The methodology for the portfolio review consists of four main stages: search, delimitation and inclusion, coding and verification, and portfolio data analysis. » In the search stage, operational financing projects and ASA using COVID-19 project tags and text analytics were searched. » In the delimitation and inclusion stage, parameters were applied to align the identified portfolio with the scope of the evaluation. This limited the cover- 146 age to five GPs (Education, HNP, Macroeconomics, Trade, and Investment, Social Protection and Jobs, and the Global Practice of Urban, Resilience, and Land [GPURL]), and support to pillars 1, 2, and 4 of the COVID-19 response that were active or approved by April 30, 2021. The portfolio of ASA also covered the Poverty and Equity GP. Further, the portfolio was limited to proj- ects in countries that received fast-track financing for COVID-19 or support through regional projects (n = 270 parent projects; n = 196 ASA). The portfo- lio was also limited to include countries classified as having a medium, high, or very high vulnerability to their development achievements and human capital gains being offset by COVID-19.1 » In the coding and verification stage, the COVID-19 portfolio of projects and ASA was manually reviewed to verify the inclusion of projects and code the elements of projects related to the conceptual framework and theory of action. After coding, the final verified portfolio included 253 operational financing projects and 175 ASA in 97 countries. » In the portfolio data analysis stage, data were analyzed for learning on the evaluation questions. This involved adding data features to the final portfolio from the World Bank’s systems on disbursement, trust funds, restructuring, procurement, and implementation status. The final data set was analyzed in Excel, Stata, and Tableau software. Python was used for text analysis of pro- curement data to identify types of goods. Independent Evaluation Group World Bank Group    147 The portfolio analysis included a review of support through the Multiphase Programmatic Approach used by HNP. Monitoring Analysis As part of the portfolio-level analysis IEG also reviewed 2,219 indicators from the projects (covering the portfolio period of February 1, 2020, to April 30, 2021). These indicators were coded by pillar, thematic response area, and stage of the response (relief or restructuring), and by evidence of tracking or progress.2 Indicators were then analyzed in Stata and Tableau. Analysis of Multiphase Programmatic Approach of Health The analysis reviews the quality of early support of the MPA projects led by HNP. The analysis applies the evaluation’s theory of action (support to needs, implementation and learning, and operational processes and part- nerships). It then assesses overall progress of the first year of support of the MPA, toward achieving its objective. The analysis draws on evidence from the evaluation portfolio, case studies, regional project analysis, and inno- vation stocktaking to assess the early MPA projects. The focus is on the first year of the MPA support from when the first projects were approved in April 2020 to April 30, 2021, within the portfolio period covered by the evaluation. Review of Internal Processes and Partnerships The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A This corporate-level review sought to distill lessons and findings on how the World Bank’s COVID-19 coordination and collaboration, experience with past crises, financing instruments, operational processes, partnerships, knowledge support, digital tools, and monitoring of the response guided and supported the early COVID-19 response. IEG structured its data collection and analysis using the evaluation’s theory of action components: relevance to countries’ needs, multidimensional implementation, operational process- es, partnerships, and learning. IEG reviewed key documents on the World Bank’s response and conducted more than 90 individual or group semistruc- tured interviews with World Bank staff and managers from GPs and corpo- rate units, all regions, select board members and advisers, and partners. IEG synthesized interview and document review evidence to distill lessons and findings on: (i) factors that facilitated the response to support countries and (ii) opportunities to improve future actions. The analysis covers the 15-month evaluation period, with interviews taking place between February September 2021. 148 References Arksey, H., and L. O’Malley. 2005. “Scoping studies: toward a methodological frame- work.” International Journal of Social Research Methodology 8 (1): 19–32. Asis, M. G., and Michael. Woolcock. 2015. “Operationalizing the Science of Delivery Agenda to Enhance Development Results.” World Bank, Washington, DC Caliński, T., and J Harabasz. 1974. A dendrite method for cluster analysis. Communi- cations in Statistics 3(1): 1–27, DOI: 10.1080/03610927408827101. Davies, D. L., and D. W. Bouldin. 1979. A Cluster Separation Measure. IEEE Transac- tions on Pattern Analysis and Machine Intelligence, vol. PAMI-1 (2): 224–227, doi: 10.1109/TPAMI.1979.4766909. Handl, J., and J. Knowles. 2007. An Evolutionary Approach to Multiobjective Cluster- ing. IEEE Transactions on Evolutionary Computation, 11 (1): 56–76, February 2007, doi: 10.1109/TEVC.2006.877146. Howe, L. D., J. R. Hargreaves, and S. R. Huttly. 2008. “Issues in the construction of wealth indices for the measurement of socioeconomic position in low-income countries.” Emerging Themes Epidemiology 5, 3. https://doi.org/10.1186/1742- 7622-5-3 Kam H., Tin. 1995. “Random decision forests,” Proceedings of 3rd International Conference on Document Analysis and Recognition, 1: 278–282. doi: 10.1109/ Independent Evaluation Group World Bank Group    149 ICDAR.1995.598994. Levac, D., H. Colquhoun, and K. O’Brien. 2010. “Scoping Studies: Advancing the Methodology.” Implementation Science 5 (1): 69. Pirani, E. 2014 “Wealth Index.” In Michalos A. C. (ed.) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht. https://doi.org/10.1007/978-94- 007-0753-5_3202 Poljanšek K., L. Vernaccini, and M. Marin Ferrer. 2020. “INFORM COVID-19 Risk Index,” Publications Office of the European Union, Luxembourg (February 1, 2021). https://www.europeandataportal.eu/data/datasets/ 42dad804-af90-4eed-9a8dab8413870038?locale=e. Schapire R. E. 2013. “Explaining AdaBoost.” In Schölkopf, B., Z. L., and V. Vovk. (eds.) Empirical Inference. Berlin, Heidelberg: Springer. UN (United Nations). 2020. “United Nations Comprehensive Response to COVID-19 Saving Lives, Protecting Societies, Recovering Better.” September 2020. New York: UN. Wilson-Grau R. 2019. “Outcome Harvesting: Principles, Steps, and Evaluation Appli- cations.” Charlotte, NC: Information Age Publishing. World Bank. 2020a. The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19. Washington, DC: World Bank. World Bank. 2020b. “Saving Lives, Scaling-Up Impact, and Getting Back on Track, June 2020.” Washington, DC: World Bank. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix A World Bank. 2022. “Lessons from the Review of Health and Social Innovations in the COVID-19 Pandemic Response.” Independent Evaluation Group. Washington, DC: World Bank Group. World Bank. Forthcoming. “Embedding Human Capital in Policy Financing: A Just in Time Note on the footprint of the HCP on DPF design.” Independent Evaluation Group. Yin R. K. 1999. “Enhancing the quality of case studies in health services research.” Health services research 34 (5 Pt 2), 1209–1224. 150 1 The Inform COVID-19 Risk Index was used to categorize countries based on their vulnera- bility to development achievements being offset by the pandemic. The evaluation adjusted the index to consider the country’s human capital index given concerns surrounding losses of human capital in countries. The countries were then separated into quartiles based on their vulnerabilities to development and human capital losses (very high vulnerability, high vulner- ability, moderate vulnerability, low vulnerability). Appendix B includes a list of the countries in the portfolio by vulnerability quartile. The Inform COVID-19 Risk Index includes dimen- sions of social inclusion (such as gender inequality and poverty), economic vulnerability, governance and institutional capacity, health systems capacity, environment, and population risks (such as access to sanitation and population mobility and density) (Poljanšek, Vernacci- ni, and Marin Ferrer 2020; UN 2020; World Bank 2020a).  2 Indicators were coded based on the status of monitoring, with “no monitoring” denoting no progress data reported, “evidence of tracking” denoting that the updated value showed no change or adverse change from the baseline, and “evidence of progress” denoting the data entered reflected progress toward the target. Independent Evaluation Group World Bank Group    151 Appendix B. COVID-19 Portfolio Analysis What is the quality of the early response to COVID-19 in countries in terms of saving lives and protecting poor and vulnerable people? To help answer this question, the Independent Evaluation Group reviewed the portfolio of operational financing projects and advisory services and analytics (ASA) that responded to COVID-19, from February 1, 2020, to April 30, 2021. The anal- ysis looks at the relevance of support to needs of countries, implementation and learning, and operational processes to assess dimensions of quality. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Methodology The methodology for the portfolio review consisted of four stages to estimate the early COVID-19 portfolio for countries vulnerable to human capital and development losses: search, delimitation and inclusion, coding and verifica- tion, and portfolio data analysis (figure B.1). Figure B.1. Portfolio Identification and Analysis Process Source: Independent Evaluation Group portfolio. Note: In the coding and verification stage, one additional country is added (Grenada) that has low vul- nerability because it is covered by a regional disease prevention–focused project. World Bank data sys- tems include Business Intelligence, Standard Reports, and the Enterprise Data Catalogue. The project list was verified against project lists from the Global Practices and an internal Independent Evaluation Group database tracking COVID-19 projects. ASA = advisory services and analytics. 152 In the search stage, operational financing projects and ASA using COVID-19 project tags and text analytics were identified. This process included all proj- ects with a COVID-19 project tag or with keywords (“COVID” or “corona”) in the text of the project title, project development objective, indicators, or summary. The ASA search focused on the keywords in the title or summary text. Additional projects and ASA were identified by reviewing COVID-19 projects tracked by Global Practices (GPs), related projects identified in proj- ect documents and Implementation Status and Results Reports,1 and proj- ects that activated Contingency Emergency Response Component (CERC; 448 projects and 446 ASA).2 Using this combination of methods, new projects and repurposed projects responding to COVID-19 were identified. In the delimitation and inclusion stage, the portfolio was limited to include the following: » Projects with support in any of the three COVID-19 response pillars covered by the evaluation between February 1, 2020, and April 30, 2021—namely, saving lives, protecting poor and vulnerable populations, and strengthening institutions for recovery. » Projects and ASA in five GPs leading the support to the early COVID-19 health and social response: Education; Health, Nutrition, and Population; Macroeconomics, Trade, and Investment; Social Protection and Jobs; and Urban, Disaster Risk Management, Resilience, and Land. Projects in these GPs Independent Evaluation Group World Bank Group    153 represent about 75 percent of the early health and social response portfolio.3 The portfolio of ASA also covers the Poverty and Equity GP. » Projects in countries that received fast-track financing for COVID-19 and or support through regional disease-focused projects. The portfolio was lim- ited to include countries classified as having a medium, high, or very high vulnerability to human capital gains being offset by COVID-19.4,5 Based on these criteria, 106 countries were eligible to be included in the portfolio. The combined project and ASA portfolios covered 98 countries, of which 97 had projects, and 62 had ASA.6,7. Twenty-nine countries in the portfolio were in fragile and conflict-affected situations (FCS). Applying these steps resulted in 270 projects and 196 ASA. Figure B.2 sum- marizes the coverage of projects and ASA in the portfolio. In the coding and verification stage, the COVID-19 portfolio of projects and ASA was manually reviewed. Coding templates based on the evalua- tion framework for the health and social response and theory of action were administered through SurveyMonkey to extract information. For projects, coders reviewed Project Appraisal Documents, program documents, Im- plementation Status and Results Reports, restructuring papers, and aide- mémoire. The template coded and extracted information for each financing project, including on interventions in the health and social response, areas of institutional strengthening support, stages of support, implementing actors, beneficiaries, address of gender, support to vulnerable groups, geo- graphical targeting, and innovations. For ASA, coders reviewed concept notes, planned deliverables, and available reports. All coding included train- ing and quality checks. The template coded information on the type and con- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B tent of the ASA. After coding and reviewing any additional related projects supporting COVID-19 identified in project documents to try to maximize the coverage of repurposed projects, the final verified portfolio included 253 operational financing projects and 175 ASA across 98 countries. About 60 percent of the financing projects identified were tagged as supporting the COVID-19 response, and the remaining 40 percent of projects, often repur- posed projects, did not have a COVID-19 tag.  ummary of Country Coverage in the Project and Advisory Figure B.2. S Services and Analytics Portfolio Source: Independent Evaluation Group. Note: The portfolio covers 107 countries; Peru had ASA identified in the portfolio but not financing proj- ects. Grenada was added to the portfolio based on its coverage by the Organisation of Eastern Caribbe- an States Regional Health Project (P168539). ASA = advisory services and analytics. 154 In the portfolio analysis stage, data were analyzed for learning on the eval- uation questions. This involved adding data features to the final portfolio from the World Bank’s systems (on disbursement, trust funds, restructuring, procurement, and implementation status). The final data set was analyzed in Excel, Stata, and Tableau software. Text analytics of procurement data used Excel and Python. The portfolio covers a substantial portion of the World Bank’s early health and social response. In the 106 countries, this portfolio covers an estimat- ed 73 percent of International Development Association commitments, 75 percent of International Development Association and International Bank for Reconstruction and Development commitments, and 95 percent of the tagged commitments made by the five GPs. The analysis covers about 40 percent of all World Bank commitments made between February 1, 2020, and April 30, 2021. Other financing was for the COVID-19 economic response not in scope of this evaluation and countries not covered by the evaluation. Limitations. Based on the verification of the portfolio in case study coun- tries, it is estimated that the portfolio covers more than 90 percent of sup- port to the early health and social response to COVID-19 in the countries and GPs analyzed. Some repurposed projects are likely missed in the port- folio analysis that were not tagged, did not have keywords, or that were not referred to in related project documents. Moreover, the response continues to be dynamic, with additional projects being added monthly. The project Independent Evaluation Group World Bank Group    155 list was first extracted on February 1, 2021, and then updated on May 12, 2021. Disbursement monthly snapshot data have been updated to June 1, 2021, and Implementation Status and Results Report data were extracted on November 5, 2021. Extraction dates are important to the extent the portfolio is highly dynamic, with internal tagging of COVID-19 projects continuously changing during the evaluation period. Relevance of COVID-19 Response to Needs of Countries Scope of the Response Portfolio In response to the COVID-19 pandemic, the World Bank increased its portfo- lio in the health and social sectors, adding small and short-duration projects across countries. The portfolio covered by this evaluation is estimated at a little more than $30 billion, including $15 billion International Develop- ment Association and $14 billion International Bank for Reconstruction and Development in operational financing, $1.5 billion in trust funds, and $60 million in ASA commitments.8 It is also estimated that there was $1.84 bil- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B lion in CERC commitments from other GPs allocated to the health and social response not covered by the portfolio. The estimated new and repurposed project financing allocated to the response in the portfolio countries is in figure B.3, panel a, and the total project financing, ASA, and trust funds are in figure B.3, panel b. Compared with the 15 months before the pandemic, the five GPs increased their processing of projects and doubled their overall commitments in the countries. Health, Nutrition, and Population had almost five times more projects approved (88 compared with 18), followed by Edu- cation (51 compared with 16), and then Social Protection and Jobs (37 com- pared with 15) and Macroeconomics, Trade, and Investment (56 compared with 34), whereas Urban, Disaster Risk Management, Resilience, and Land remained static (33 compared with 38). Other GPs—such as Transport, Water, Governance, and Energy and Extractives—processed fewer projects during the same period. The median size of projects decreased from $100 million to $70 million, meaning that GPs worked hard to manage more smaller and shorter-duration projects with an average length of about 2.5 years. In Health, Nutrition, and Population, the change was greatest, with the median project size decreasing from $79 million to $23 million. Some projects have since added additional financing for later stages of the response. 156 Figure B.3. Estimated Financing Commitments to Early COVID-19 Response for Vulnerable Countries in Portfolio a. New and repurposed project financing commitments to early COVID-19 response in portfolio Independent Evaluation Group World Bank Group    157 158 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B b. Estimated project, trust fund, and ASA commitments to early COVID-19 response in portfolio Lead Global Practice, Estimated Commitments (US$, millions) Urban, Disaster Risk Health, Social Macroeconom- Management, Nutrition, and Protection ics, Trade, and Resilience, Population and Jobs Investment Education and Land Poverty and Fiscal (projects = 111, (projects = 39, (projects = 26, (projects = (projects = Equity Year Financing ASA = 42) ASA = 45) ASA = 25) 43, ASA = 23) 34, ASA = 7) (ASA = 23) Total FY20 Projects 8,146 2,891 2,619 2,914 2,565 — 19,135 (n = 199) Trust Funds 617 86 6 192 29 — 930 ASA (n = 66) 11 6 3 4 4 5 33 Subtotal 8,774 2,984 2,628 3,110 2,598 — 20,098 FY21 Projects 358 4,433 2,865 1,287 975 — 9,919 (n = 114) Trust Funds 43 145 100 250 4 — 543 ASA (n = 99) 7 4 2 1 2 5 21 Subtotal 409 4,582 2,967 1,539 981 5 10,483 Total projects 8,504 7,324 5,484 4,201 3,540 — 29,054 Total trust funds 660 231 106 442 33 — 1,473 Total PEF 0.196 — — — — — 0.196 (continued) Lead Global Practice, Estimated Commitments (US$, millions) Urban, Disaster Risk Health, Social Macroeconom- Management, Nutrition, and Protection ics, Trade, and Resilience, Population and Jobs Investment Education and Land Poverty and Fiscal (projects = 111, (projects = 39, (projects = 26, (projects = (projects = Equity Year Financing ASA = 42) ASA = 45) ASA = 25) 43, ASA = 23) 34, ASA = 7) (ASA = 23) Total Total ASA 18 10 5 5 6 10 54 Grand Total 9,182 7,566 5,596 4,648 3,579 10 30,581 Source: Independent Evaluation Group portfolio. Note: In panel a, the size of the boxes increases with the number of projects. In each box, the top number shows the number of projects, and the bottom numbers show the estimated commitments for the Global Practice. Total estimated commitments are defined as the full project commitment amounts for projects approved on or after February 1, 2020 (including additional financing), and as the sum of undisbursed balances and disbursements for projects approved before February 1, 2020 (project commitment data were retrieved on May 12, 2021). New projects (approved on or after February 1, 2020) are assumed to have 100 percent share of COVID-19 content. The total number of projects is 253. The total estimated commitments amount is US$29,054 million. Panel b reports COVID-19 estimated commitments in US$, millions, including funding by IBRD, IDA, recipient-executed trust funds, and World Bank–executed trust funds. Projects and ASA cover the evaluation period of February 2020 to April 30, 2021, and are divided by fiscal year. Table excludes 10 ASA totaling US$3.4 million led by global thematic units and the Development Research Group. PEF amounts are based on the PEF Allocations Steering Body (World Bank 2020a). Table is based on 313 projects (253 parents and 60 additional financing), and 165 ASA. — = not applicable; ASA = advisory services and analytics; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; PEF = Pandemic Emergency Financing Facility. Independent Evaluation Group World Bank Group    159 Health, Nutrition, and Population has the largest share of projects and commitments in the evaluation portfolio, followed by Social Protection and Jobs; Macroeconomics, Trade, and Investment; Education; and Urban, Disas- ter Risk Management, Resilience, and Land. By Region, the largest share of commitments is in Africa, followed by South Asia and East Asia and Pacific Regions (figure B.4, panel a), although individual project commitments in Africa are on average smaller than other Regions. Africa also has the larg- est number of countries and projects covered by the portfolio. The portfolio includes countries and projects across various levels of vulnerability (figure B.4, panel b). Regarding ASA, Social Protection and Jobs and Health, Nutri- tion, and Population combined account for about half of the ASA, estimat- ed at $10 million and $18 million, respectively. The Poverty and Equity GP accounts for about 13 percent of ASA (about $10 million). Other ASA was The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B spread across GPs.  stimated Project Financing and Overview of Evaluation Figure B.4. E Portfolio a. Projects and estimated financing by region 160 b. Projects and estimated financing by country vulnerability Source: Independent Evaluation Group portfolio. Note: In panel a, projects (number), countries (number), COVID-19 commitments (US$, millions), and COVID-19 commitments (US$, millions) are for each Region. Color shows details about the Region (group). Details are shown for projects (number), countries (number), and COVID-19 commitments (US$, millions). In panel b, projects (number), countries (number), COVID-19 estimated commitments (US$, millions), and COVID-19 estimated commitments (US$, millions) are for each vulnerability level. Color shows details about vulnerability level (group). The view is filtered on vulnerability level, which keeps high vulnerability, medium vulnerability, and very high vulnerability. Panel b excludes one country (Grenada) that is part of two regional projects (P117871 and P168539) but that is classified as having low vulnerability. The total number of projects is 253 in 97 countries. The total estimated commitments amount is US$29,054 million. AFR = Africa Region; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; SAR = South Asia. The portfolio emphasizes new projects developed for COVID-19: 68 percent of projects are new, whereas the remaining are projects repurposed to support COVID-19. South Asia and Africa introduced the highest level of reorientation of their portfolios through new and repurposed projects and ASA. Portfolio reori- entation helped identify a range of response areas quickly (figure B.5, panel a). Independent Evaluation Group World Bank Group    161 Regional project support and trust fund financing were important to sup- porting the early response. Regional projects supported the response, draw- ing on $515 million in financing and covering 23 percent of the countries. These regional projects approved before COVID-19 were able to repurpose support across the countries to expand critical health services and insti- tutional strengthening. Trust funds supported 74 percent of countries for critical health services (figure B.5, panel b) and to coordinate the response. The Pandemic Emergency Financing Facility accounts for about $196 million of trust fund support. Other notable trust funds supporting the response in- clude the Global Financing Facility, country-level pooled donor trust funds, and the Health Emergency Preparedness and Response umbrella trust fund. World Bank teams adjusted the use of previously existing trust funds to ac- count for new needs related to COVID-19.  ortfolio Reorientation in Countries for COVID-19 Figure B.5. P and Trust Funds a. Country portfolio reorientation by Region (financing projects and ASA) b. Estimated trust fund financing to COVID-19 by Global Practice The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group portfolio. Note: In panel a, reorientation is defined as the number of projects per country responding to COVID-19, including financing projects and advisory services and analytics. It has a mean value of 3.86, median of 3.0, and standard deviation of 3.13. Reorientation levels are defined as terciles. Low reorientation ≤ 3 projects, medium = 4 projects, and high ≥ 5 projects. The total number of countries is 95. In panel b, trust fund data include both World Bank–executed and recipient-executed trust funds. The total number of projects is 146. AFR = Africa Region; ASA = advisory services and analytics; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; SAR = South Asia. Countries with lower levels of preparedness potentially needing more sup- port received the highest project financing commitments per million popula- tion in the early response (figure B.6). The portfolio also emphasized support to small states and FCS. Small states received on average about $38 million per million population in the COVID-19 response, compared with about $7 million for other countries. FCS countries received $9 million per million population. This points to the efforts to support countries in challenging situations. 162  ommitments to COVID-19 Response by Country Figure B.6. C Preparedness Source: Independent Evaluation Group portfolio; preparedness indexes are from the International Health Regulations Core Capacity Index, https://www.who.int/data/gho/data/indicators/indicator-details/ GHO/preparedness (accessed February 1, 2021); and e-SPAR State Parties Annual Reporting, “State Par- ties Self-Assessment Annual Reporting on the Implementation of the International Health Regulations.” https://extranet.who.int/e-spar/#capacity-score. Note: Values were calculated by dividing the total COVID-19 commitments in each preparedness strati- fication by the total population represented in each stratification. The total number of countries is 94 and excludes regional projects. As of January 31, 2022, the World Bank’s COVID-19 portfolio continued to expand and change composition. The evaluation conducted a rapid update of the health and social response portfolio to understand how it has grown since the early the evaluation period. The COVID-19 response has expanded to include 381 projects and $60 billion in commitments across the 106 coun- tries and five GPs. Of the projects added to the portfolio, about 40 percent were approved after April 30, 2021. The remaining 60 percent are previously approved projects that have added a COVID-19 tag or adjusted their imple- Independent Evaluation Group World Bank Group    163 mentation to support COVID-19 since the early portfolio was drawn. Health, Nutrition, and Population projects represent about 25 percent of the projects in the evolving portfolio, compared with about 40 percent in the early port- folio. Agriculture and Food has expanded its coverage to be one of the five main GPs supporting the health and social response. Design and Targeting of the Support The World Bank supported most health emergency priorities in early COVID-19 response plans in countries (66 percent), although there was limited support to the continuity of essential services (figure B.7). The World Bank responded to the needs identified in country COVID-19 plans for surveillance, case management, and infection prevention and control, aligning with the World Health Organization priority areas for COVID-19. Continuation of essential health services was often not prioritized in early country planning for COVID-19. Vaccination had limited support in the early response across regions, given the emphasis of early plans on prevention and control. Figure B.7.  Alignment of World Bank Support with Country COVID-19 Plans by Region The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group portfolio; country COVID-19 plans; WHO COVID-19 Partners Plat- form. Geneva,: WHO (accessed May 1, 2021). https://covid19partnersplatform.who.int/en. Note: The analysis shows the percent of countries by Region that had World Health Organization plans in a response area and received at least one World Bank intervention in that area. The analysis is con- ducted for countries with complete data on COVID-19 plans. The total number of countries is 66. AFR = Africa Region; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; SAR = South Asia. The World Bank’s support across GPs covered both relief efforts and reforms to restructure systems (figure B.8). Support of Health, Nutrition, and Population and to FCS countries focused on the relief stage, although there is planned support to restructure systems, such as health human resource plans. Macro- economics, Trade, and Investment and Social Protection and Jobs focused on restructuring systems, such as for social protection. In Education, early support focused on remote learning, with restructuring support to reopen schools with improved safety and sanitation conditions. Urban, Disaster Risk Management, Resilience, and Land focused on restructuring support, such as to improve conditions in urban slums. Interventions to support the resilience of systems have been incorporated into ongoing efforts across GPs, where people, com- munities, systems, and assets have been prepared for shocks that could emerge from diseases, shutdowns, and income loss. The early response did not address longer-term support for preparedness capacities after COVID-19 or consider the efficiency of resource use in countries. Likewise, sustainability in the form of planning for long-term consequences in terms of services, systems, environ- 164 ment, resources, or people was limited in all areas of the response except for support led by Macroeconomics, Trade, and Investment. Support to enhance the addressing of inclusion, gender, and digitalization in projects has been strongest in the social response (figure B.8). Social Protection and Jobs projects most consistently addressed inclusion, gender, and digitalization, followed by Education and Macroeconomics, Trade, and Investment. The absence of actions to address gender is pronounced in op- erations financed under the first Multiphase Programmatic Approach (MPA) support. Inclusion aspects in a project supported increasing the access of vulnerable groups to services and other resources. In FCS country responses, there has been better emphasis on inclusion, gender, and digitalization than in non-FCS countries.  esign Elements of Project Support to Countries by Global Figure B.8. D Practice Independent Evaluation Group World Bank Group    165 Source: Independent Evaluation Group. Note: Bar size represents the percent of projects within each Global Practice that support the speci- fied stage of the response or have specified orientation of project design. Blue bars denote areas with less than 50 percent of projects. The total number of projects is 253. ”Relief” refers to whether a project includes support for the emergency stage of the COVID-19 response. “Restructuring” refers to whether a project includes support for recovery. “Resilience” looked at whether a project supported preparing people, communities, systems, and assets for shocks, such as those that could emerge from diseases, shutdowns, and income loss. “Inclusion” looked at whether a project supported increasing the access of disadvantaged groups to services and other resources. “Sustainability” looked at whether project activities supported planning for long-term consequences in terms of the management of the ser- vices, systems, resources, or people to ensure continued benefit. “Efficiency” refers to considerations for cost-effective government resource use in a constrained environment. “Digitalization” and “gender” looked at whether project interventions included any interventions to address those areas. FCS = fragile and conflict-affected situation. Institutional Strengthening Early support to institutional strengthening focused on country-level coor- dination and core public health functions to respond to COVID-19 (figure B.9, panel a). Integrating institutional strengthening from the onset of the response helped support basic capacities for the immediate crisis, with the most extensive support going to FCS and higher vulnerability countries (fig- ure B.9, panel b). Examples of support include to help governments develop COVID-19 plans and policies, strengthen laboratory and surveillance sys- tems, and provide social protection and education services. This support will need to be deepened to support recovery. Local government received limited direct institutional strengthening support outside FCS countries, although national support to COVID-19 plans intends to channel resources to subna- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B tional levels. nstitutional Strengthening in Early COVID-19 Response Figure B.9. I in Countries a. Areas of institutional strengthening support 166 b. Extent of institutional strengthening in countries Source: Independent Evaluation Group portfolio; vulnerability indexes are from the INFORM COVID-19 Risk Index (Poljanšek, Vernaccini, and Marin Ferrer 2020; World Bank 2020b; UNINFO, COVID-19 Data Portal, https://data.uninfo.org/Home/_InformRiskk [accessed February 1, 2021]). Note: In panel a, calculations are by share of countries with at least one intervention in an institutional strengthening area. The total number of projects is 253 in 97 countries; 981 interventions focused on institutional strengthening. In panel b, the extent of institutional strengthening in a country is calculated by stratifying in terciles the average percent of institutional strengthening interventions within projects. “Low” refers to countries with less than 4 percent of possible institutional strengthening interventions, “medium” between 4 and 20 percent, and “high” 20 percent and above. Institutional strengthening is defined against the areas identified in the conceptual framework of the evaluation. COVID-19 = corona- virus; FCS = fragile and conflict-affected situation. Support to Implementation and Learning in Countries Multidimensional Implementation The World Bank’s early operational financing focused on addressing the health emergency and social protection. More than 80 percent of countries Independent Evaluation Group World Bank Group    167 received project support to ensure health services (figure B.10, panel a). The health support focused on critical health services for infection prevention and control, case management, surveillance, and laboratories (figure B.10, panel b). Risk communication has also received some attention, especially in FCS countries. The emphasis on critical health services reflects the align- ment of early support with World Health Organization priority areas. In addition, about 67 percent of countries received support to protect poor and vulnerable persons (social protection and or informal economy support). Reorientation of country portfolios to cover a range of COVID-19 response areas often drew on existing projects. The widest coverage of response areas in country portfolios is seen where existing projects were repurposed to support interventions for COVID-19. Areas not well addressed in the response are social cohesion, psychosocial care, informal economy support, and citizen engagement. Figure B.10. Thematic Areas of COVID-19 Response in Countries a. Thematic response areas The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B b. Breakdown of thematic response areas Source: Independent Evaluation Group portfolio. Note: In panels a and b, calculations are by share of countries with at least one intervention in a thematic response or breakdown of area. The total number of projects is 253, covering 3,204 interventions coded for the evaluation. The number of countries used as the base is 106. COVID-19 = coronavirus. 168 Most early support focused on engaging national ministries, with less sup- port of subnational government and targeting of specific population groups (figure B.11). Few countries have multisectoral coordination teams or proj- ects supporting community groups. Targeting of specific vulnerable groups was limited and more common in FCS countries and in Africa than other Re- gions. By GP, Education has had the most disaggregated response engaging parents and adolescents. Health, Nutrition, and Population engaged health structures and essential frontline workers through national plans. Social Protection and Jobs stands out for its focus on women and girls and vulnera- ble groups.  ountry Actors Delivering Coronavirus (COVID-19) Figure B.11. C Project Support a. Project implementing actors Independent Evaluation Group World Bank Group    169 b. Project beneficiaries Source: Independent Evaluation Group portfolio. Note: The bars indicate the percent of projects within each Global Practice that had at least one World Bank intervention that targeted the implementing actors or beneficiaries. Analysis covered 97 countries and 253 coded projects. FCS = fragile and conflict-affected situation. Implementation Status of Project Support Across GPs, about half of the projects supporting COVID-19 have satisfactory or better implementation progress ratings (figure B.12). Social Protection and Jobs shows slightly greater early implementation progress, which may relate to the extent of work done before COVID-19 to develop social protec- tion systems and thus the readiness of this sector to respond to the crisis. Projects have lower implementation ratings in countries that had more than 40 weeks of community spread of COVID-19 per the World Health Organiza- tion classification, suggesting implementation is challenging when countries experience a peak in cases. Figure B.12. Project Implementation Progress by Global Practice The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group portfolio. Note: The figure excludes one project from the Macroeconomics, Trade, and Investment Global Prac- tice. New projects with no implementation progress rating to date were excluded. The extraction date for Implementation Status and Results Reports is November 5, 2021. The total number is 205 projects. 170 Use of Advisory Services and Analytics to Guide Response ASA support cuts across thematic areas of the COVID-19 response, with more than 90 percent focused on institutional strengthening of systems, pol- icy, and services and 21 percent focused on social protection (figure B.13).  dvisory Services and Analytics Support to COVID-19 Figure B.13. A Response by Theme Independent Evaluation Group World Bank Group    171 Source: Independent Evaluation Group portfolio. Note: Because each ASA could address multiple response areas, the total percent is greater than 100; the total number is 175 in 62 countries and 13 regional units with ASA. Regional units include Africa, Andean countries, Caucasus, Central America, East Africa, Europe and Central Asia, Latin America and the Caribbean, Middle East and North Africa, multiregional, Southern Africa, Western Africa, Western Balkans, and the world. ASA = advisory services and analytics. Globally, just under 60 percent of countries have had ASA support for COVID-19. The emphasis on ASA varies by Region (figure B.14). South Asia was the only Region where all countries undertook some form of ASA. In Africa and FCS countries, more than 60 percent of countries undertook at least one ASA (although individual commitment amounts of ASA were often small). Other Regions had lower ASA coverage.  xtent of Advisory Services and Analytics Coverage in Figure B.14. E Countries by Region The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group portfolio. Note: The analysis includes only country-level ASA and excludes regional and global ASA. Figure includes 62 countries with ASA and 44 without ASA. The total number is 106 countries. AFR = Africa Re- gion; ASA = advisory services and analytics; EAP = East Asia and Pacific; ECA = Europe and Central Asia; FCS = fragile and conflict-affected situation; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; SAR = South Asia. ASA was used most often for diagnostic analysis (more than 90 percent of ASA; figure B.15). Social Protection and Jobs undertook the highest num- ber of ASA, followed by Health, Nutrition, and Population. After diagnostic analysis, ASA provided support through policy analysis (67 percent of ASA), monitoring the impact of COVID-19 (63 percent of ASA), and hands-on technical assistance (61 percent of ASA). Macroeconomics, Trade, and In- vestment supported most ASA to influence policy. Across GPs, less-covered areas of ASA were knowledge sharing (42 percent), knowledge generation to document experiences (29 percent), and operational research to identify new evidence on effectiveness (28 percent). 172  ypes of Advisory Services and Analytics Support for Figure B.15. T COVID-19 Source: Independent Evaluation Group portfolio. Note: Bar size represents the percent of ASA in Global Practices by the various type. Given that ASA can have multiple purposes, the amounts add up to more than 100 percent. The blue bars represent ASA types less than 50 percent. The total number is 175 in 62 countries, and 13 regional units with ASA. Regional units include Africa, Andean countries, Caucasus, Central America, East Africa, Europe and Central Asia, Latin America and the Caribbean, Middle East and North Africa, multiregional, Southern Africa, Western Africa, Western Balkans, and the world. “Other” are analytical products requested by Global Themes, country management units, and the Development Research Group. ASA = advisory services and analytics. Operational Processes in Support of Countries Coordination across Global Practices to Support Implementation Independent Evaluation Group World Bank Group    173 Development policy financing (DPF) and crisis instruments encouraged collaboration across GPs: 68 percent of DPFs and 52 percent of crisis in- struments had GP collaboration, often led by Macroeconomics, Trade, and Investment and Urban, Disaster Risk Management, Resilience, and Land. By Region, South Asia and Middle East and North Africa had the greatest GP collaboration on projects (more than 70 percent of countries had GP collabo- ration on projects, compared with 50 percent or less in other Regions). By GP, Health, Nutrition, and Population and Education had limited collaboration on projects, compared with other GPs (figure B.16). There are opportunities for further collaboration on the MPA, with just 28 percent of projects work- ing with another GP. Box B.1 describes examples of GP collaboration.  ercent of Collaboration in Global Practice Projects Figure B.16. P for COVID-19 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group portfolio. Note: The analysis looks at collaboration in parent and additional financing projects. The total number is 253 parent projects and 60 additional financing. Urban, Resilience, and Land = Urban, Disaster Risk Management, Resilience, and Land.  xamples of Global Practice Collaboration in the COVID-19 Box B.1. E Response » In India, Social Protection and Jobs and Health, Nutrition, and Population col- laborate to provide a health insurance plan for health workers delivering care to patients with COVID-19. » In Madagascar, Urban, Disaster Risk Management, Resilience, and Land; Health, Nutrition, and Population; and Social Protection and Jobs collaborate to reha- bilitate and equip health centers to deliver COVID-19 services; facilitate social distancing and hygiene services, including public handwashing stations and sani- tation for public transport; and provide cash transfers and cash-for-work activities. Also, Education, Governance, Social Protection and Jobs, and Health, Nutrition, and Population collaborate to improve learning support in schools, governance of education in emergencies at all levels, and water, sanitation, and hygiene in schools and to provide grants to community teachers. (continued) 174  xamples of Global Practice Collaboration in the COVID-19 Box B.1. E Response (Cont.) » In West Africa, Health, Nutrition, and Population and Agriculture collaborate to train One Health community agents in community-based surveillance and re- sponse. » In Uzbekistan, Agriculture; Energy; Finance, Competitiveness, and Innovation; Poverty and Equity; and Macroeconomics, Trade, and Investment collaborate in a development policy loan to enhance economic inclusion and social resilience in response to COVID-19 by increasing targeted support to vulnerable households. » In Nepal, Urban, Disaster Risk Management, Resilience, and Land; Governance; Sustainability and Inclusion; Transport; and Water collaborated to strengthen the institutional and fiscal capacities of municipalities for continued service delivery during COVID-19, including support to labor-intensive public works for individuals from poor and vulnerable households. Source: Independent Evaluation Group portfolio. Mix of Instruments Supporting Implementation A mix of instruments was used to deliver health and social support for Independent Evaluation Group World Bank Group    175 COVID-19 (figure B.17). The MPA and regional projects led by Health, Nutrition, and Population were the main support to the early health response and to a less- er extent, CERCs and repurposed projects. Countries in Latin America and the Caribbean used more DPFs, CERCs, and repurposed projects than other Regions. This may reflect experience with crisis response. About 17 percent of projects in the portfolio were restructured to support the early health and social response. There was limited use of additional financing between March and June 2020, with about 2 percent of projects receiving additional financing. Additional fi- nancing increased after June 2020, when a waiver was available for applying the Environmental and Social Framework safeguards (increasing to about 21 percent of the early response portfolio). More than 75 percent of additional financing is associated with new projects for COVID-19. Overall, the early response had a high use of new projects. Box B.2 provides examples on the use of instruments. Figure B.17. Use of a Mix of Instruments to Support the Response The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group portfolio. Note: The total number is 253 projects in 97 countries. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component; DPL = development policy loan; FY = fiscal year; IPF = investment project financing; MPA = Multiphase Programmatic Approach; PforR = Program-for-Results. Box B.2. Examples of Instrument Uses in the Response Development policy financing: The Colombia COVID-19 crisis response development policy financing undertook prior actions that responded to the emergency while also helping to restructure systems. The prior actions included: » Definition of a costed basket of health services and technologies to attend to pa- tients infected by COVID-19, which allowed the allocation of additional resources to health insurance companies and health-care providers. » Expansion of the main database for targeting social programs to facilitate relief response cash payments, which can be used in future emergencies. (continued) 176 Box B.2. Examples of Instrument Uses in the Response (Cont.) » Accelerated implementation of the value-added tax refund program targeted at the poorest who are already receiving cash transfers. Investment project financing: In Pakistan, a repurposed investment project financing in the education sector supported the emergency response by procuring personal protective equipment, sanitization, and other hygiene equipment for technical insti- tutions in Punjab. By contrast, the Khyber Pakhtunkhwa Human Capital Investment project combined emergency and restructuring needs by developing community en- gagement and feedback systems, including the implementation of a communication strategy for positive healthy behaviors and lifestyle. The project was approved in June 2020 and started disbursing in September 2020. Source: Independent Evaluation Group portfolio. Estimated Disbursement of Financing to Support the Response The World Bank disbursed an estimated 38 percent of COVID-19 commit- ments in the financing portfolio based on data up to June 1, 2021, with the first data on disbursement in March 2020, immediately when the COVID-19 crisis was declared (figure B.18). Early disbursement in the first months Independent Evaluation Group World Bank Group    177 of the pandemic reached 69 countries and steadily increased in fiscal year (FY)21. Following the announcement of the pandemic at the end of FY20, the World Bank’s COVID-19 portfolio grew quickly, disbursing just under $3.3 billion between March and June (7 percent went to FCS countries, and 69 percent went to high and very high vulnerability countries). In FY21, the pace of disbursement slowed, increasing each quarter by an average of about $1.5 billion and reaching 86 countries (including 25 FCS countries). Cumu- lative disbursement throughout the early response was about $11 billion (11 percent went to FCS and 64 percent to high and very high vulnerability countries). Emergency instruments and repurposed projects supported rapid early dis- bursement. Most MPA projects were approved by FY20. Excluding those that have yet to disburse, the MPAs took an average of about two months to make first disbursements. New DPFs started to disburse in May 2020 and made up the largest share of disbursements. This emphasizes the value of quick-dis- bursing emergency instruments, such as CERC and catastrophe deferred drawdown option, for crisis response. Moreover, existing regional projects and repurposed project support disbursed quickly, pointing to the impor- tance of having relevant investments before the crisis hit. About 50 percent of new projects disbursed within two months, about 80 percent within five months, and others lagged beyond. Figure B.18 Cumulative Disbursement of Early Support to COVID-19 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Source: Independent Evaluation Group review based on monthly Standard Reports data, extracted on June 1, 2021. Note: There are 92 countries with available disbursement information, with 86 of them showing monthly disbursements during the period for projects in the financing portfolio, in addition to three regions with disbursement information for regional projects (Eastern Africa, Western Africa, and the Organisation of Eastern Caribbean States countries). The total number of country units is 89. Disbursements are adjusted with the share of COVID-19 response content estimated in the coding of each project by the Independent Evaluation Group. New projects (approved on or after February 1, 2020) are assumed to have 100 percent share of COVID-19 content. The numerator for the percent of monthly disbursements is the cumulative disbursements reported for projects in the evaluation portfolio up to that month. The denominator is the cumulative approved project commitments in the portfolio up to the same month. The total estimated commitments amount is $29,054 million. The World Bank disbursed fastest in countries with at least moderate levels of preparedness (figure B.19). By the end of May 2021, moderate prepared- ness countries accounted for 65 percent of country disbursements made since March 2020 in the evaluation portfolio. They were followed by low preparedness countries with about one-third of disbursements; the pace of disbursements for low preparedness countries was relatively slow until September 2020, suggesting it took about six months for the World Bank to 178 deliver on its commitments under these circumstances. Figure B.19 Cumulative Disbursement by Preparedness Source: Independent Evaluation Group portfolio review based on monthly Standard Reports data, ex- tracted on June 1, 2021. Note: There are 92 countries with available disbursement information, with 86 of them showing monthly disbursements for projects in the financing portfolio. Data on vulnerability come from the INFORM COVID-19 Risk Index, adjusted to include the Human Capital Index (Poljanšek, Vernaccini, and Marin Ferrer 2020; World Bank 2020b; UNINFO, COVID-19 Data Portal, https:/ /data.uninfo.org/Home/_Inform- Riskk [accessed February 1, 2021]); data on preparedness are from WHO (2017, 2021a). Regional projects are excluded from the figure because preparedness data are not available for Regions. The total num- ber of countries is 86. COVID-19 = coronavirus. Early in the pandemic, disbursement reached countries quicker in Africa, although the overall amount was low; the key support was from Urban, Disaster Risk Management, Resilience, and Land and Health, Nutrition, and Population, with other GPs adding support by May 2020 to support the crisis (figure B.20). Between March and June 2020, disbursement of financ- ing in Africa reached 31 countries (78 percent of countries), increasing to 38 countries in FY21. The amount of financing to Africa increased in August 2020. Early financing to Latin America and the Caribbean also reached most Independent Evaluation Group World Bank Group    179 countries. Figure B.20 Cumulative Disbursement of Project Support by Region Source: Independent Evaluation Group portfolio review based on monthly Standard Reports disburse- ment data, extracted on June 1, 2021. Note: The total number of country units is 89. AFR = Africa Region; EAP = East Asia and Pacific; ECA = The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Europe and Central Asia; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; SAR = South Asia. Procurement of Goods and Services Most of the procurement of goods (52 percent) focused on providing COVID-19 testing and laboratory equipment and protective gear, including masks (figure B.21). More limited procurement supported medical equip- ment (12 percent); facility improvements for quarantine, infection pre- vention and control, and patient care (9 percent); technology (7 percent); sanitation (5 percent); medication (4 percent); and other items such as vehi- cles, waste management, communication and learning materials, and nutri- tional products (about 11 percent combined). Vaccines account for less than 1 percent of early procurement, increasing later in the response. The types of goods procured over the period remained consistent with the focus on consumables. World Bank–facilitated procurement assisted countries with difficulty to procure medical supplies when there was a client request. In the early response, World Bank–facilitated procurement accounted for about 4 percent of all procurement of goods. 180 Figure B.21 Cumulative Procurement of Goods by Type Source: Independent Evaluation Group portfolio. Note: Data include procurement contracts signed between February 1, 2020, and April 30, 2021. Pro- curements were estimated through text analytics process that reviewed each line of procurement data to identify different types of goods (N = 3,977). COVID-19 = coronavirus. Goods is the largest procurement category for the COVID-19 response, with about $2.6 billion of support to countries across GPs. Procurement contracts Independent Evaluation Group World Bank Group    181 rapidly increased by June 2020 and incrementally afterward; by March 2020, 3 percent of the total contracts were signed, and by June 2020, 38 percent were signed, mostly focused on health-related goods. The main instrument undertaking procurement was the MPA, with a significant contribution by CERC. The MPA, CERC, and repurposed projects account for just under 85 percent of all procurement. Direct selection was used for 59 percent of the contracts. Approximately 41 percent of countries procured services from United Nations agencies. References Poljanšek K., L. Vernaccini, and M. Marin Ferrer. 2020. “INFORM COVID-19 Risk Index,” Publications Office of the European Union, Luxembourg (February 1, 2021). https://www.europeandataportal.eu/data/datasets/ 42dad804-af90-4eed-9a8dab8413870038?locale=e. World Bank. 2020a. “Fact Sheet: Pandemic Emergency Financing Facility.” Brief, April 27, World Bank, Washington, DC. https://www.worldbank.org/en/topic/ pandemics/brief/fact-sheet-pandemic-emergency-financing-facility. World Bank. 2020b. The Human Capital Index 2020 Update: Human Capital in the Time of COVID-19. Washington, DC: World Bank. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B 182 1 The identification of related projects identified to be supporting the response in project documents and Implementation Status and Results Reports was done as a second stage during the project coding. However, this step was important to help identify repurposed projects in the country portfolio.  2 Projects with activated Contingency Emergency Response Component (CERC) in the 106 countries covered by the portfolio were identified through the portfolio identification and the Global Facility for Disaster Reduction and Recovery dashboard in May 2021 and June 2021. Twenty-six CERCs were included in projects covered by the portfolio. An additional 29 CERCs were activated in projects in other Global Practices to support the response.  3 The remaining 25 percent of projects not included in the portfolio analysis are in the Ag- riculture and Food, Governance, Social Sustainability and Inclusion, Water, and Transport Global Practices.  4 The INFORM COVID-19 Risk Index was used to categorize countries based on their vulner- ability to development achievements being offset by the pandemic. The evaluation adjusted the index to consider the country’s human capital index, given concerns surrounding losses of human capital in countries. The countries were then separated into quartiles based on their vulnerabilities to development and human capital losses (very high vulnerability, high vulnerability, moderate vulnerability, and low vulnerability). The INFORM COVID-19 Risk Index includes dimensions of social inclusion (such as gender inequality and poverty), eco- nomic vulnerability, governance, and institutional capacity, health systems capacity, environ- ment, and population risks, such as access to sanitation and population mobility and density Independent Evaluation Group World Bank Group    183 (Poljanšek, Vernaccini, and Marin Ferrer 2020; World Bank 2020b; UNINFO, COVID-19 Data Portal, https://data.uninfo.org/Home/_InformRiskk [accessed February 1, 2021]).  5 Very high vulnerability countries in the project portfolio are Afghanistan, Benin, Burkina Faso, Burundi, Cabo Verde, Cameroon, the Central African Republic, Chad, the Comoros, the Democratic Republic of Congo, the Republic of Congo, Ethiopia, The Gambia, Guinea, Guinea-Bissau, Lesotho, Liberia, Madagascar, Malawi, Mali, Mauritania, Mozambique, Niger, Nigeria, São Tomé and Príncipe, Sierra Leone, the Solomon Islands, Somalia, South Sudan, Togo, Uganda, and Zambia in Africa; Kiribati, the Marshall Islands, and Papua New Guinea in East Asia and Pacific; Haiti in Latin America and the Caribbean; and the Republic of Yemen in Middle East and North Africa (37 countries). High vulnerability countries in the project portfolio are Angola, Cambodia, Côte d’Ivoire, Eswatini, Gabon, Ghana, Kenya, Rwanda, Senegal, and Tanzania in Africa; Indonesia, the Lao People’s Democratic Republic, Myanmar, the Philippines, Timor-Leste, Tuvalu, and Vanuatu in East Asia and Pacific; Guatemala, Honduras, and Nicaragua in Latin America and the Carib- bean; Tajikistan in Europe and Central Asia; Djibouti, Iraq, and Lebanon in Middle East and North Africa; and Bangladesh, India, Maldives, Nepal, and Pakistan in South Asia (29 coun- tries). Moderate vulnerability countries in the project portfolio are the Seychelles in Africa; Fiji, Tonga, and Vietnam in East Asia and Pacific; Albania, Belarus, Bosnia and Herzegovina, Geor- gia, Moldova, Türkiye, Ukraine, and Uzbekistan in Europe and Central Asia; Belize, Bolivia, Colombia, Dominica, the Dominican Republic, Ecuador, El Salvador, Jamaica, Panama, Peru, St. Lucia, St. Vincent and the Grenadines, Suriname, and Trinidad and Tobago in Latin Amer- ica and the Caribbean; Jordan, Morocco, and Tunisia in Middle East and North Africa; and Bhutan and Sri Lanka in South Asia (31 countries). The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix B Low vulnerability countries in the project portfolio are Grenada in Latin America and the Caribbean. Although Grenada does not meet the vulnerability criterion to be included in the list of eligible countries, it is covered by a regional disease-focused project (P168539) and thus is added to the portfolio as an exception (one country). Countries from the eligible 106 (those with moderate vulnerability or higher) that were not covered in the portfolio of projects or advisory services and analytics are Algeria, Azerbaijan, Botswana, the Federated States of Micronesia, Namibia, Palau, Thailand, South Africa, and República Bolivariana de Venezuela.  6 Countries covered by advisory services and analytics are Angola, Burkina Faso, Burundi, Cameroon, the Central African Republic, Chad, the Comoros, the Democratic Republic of Congo, the Republic of Congo, Côte d’Ivoire, Ethiopia, Gabon, The Gambia, Ghana, Guinea, Kenya, Malawi, Mali, Mauritania, Nigeria, Rwanda, São Tomé and Príncipe, Senegal, Sierra Leone, Somalia, South Sudan, Tanzania, Uganda, and Zambia in Africa; Cambodia, Fiji, Indo- nesia, the Lao People’s Democratic Republic, Myanmar, Papua New Guinea, the Philippines, and Vietnam in East Asia and Pacific; Bolivia, Colombia, the Dominican Republic, Ecuador, El Salvador, Guatemala, Haiti, Honduras, Panama, and Peru in Latin America and the Caribbean; Djibouti, Iraq, Morocco, and the Republic of Yemen in Middle East and North Africa; Albania, Tajikistan, and Türkiye in Europe and Central Asia; and Afghanistan, Bangladesh, Bhutan, India, Maldives, Nepal, Pakistan, and Sri Lanka in South Asia (62 countries). 184 7 Nine countries from the eligible 106 did not have identified project or advisory services and analytics in the early response: Algeria, Azerbaijan, Botswana, the Federated States of Micro- nesia, Namibia, Palau, Thailand, South Africa, and República Bolivariana de Venezuela.  8 An in-depth analysis of coronavirus (COVID-19) commitments and financing allocations is outside the scope of the current evaluation. The evaluation provides an estimate from avail- able data on the portfolio for the time period, countries, and Global Practices covered by the analysis.  Independent Evaluation Group World Bank Group    185 Appendix C. Case Study Findings This appendix presents the main findings for eight case study countries: Djibouti, Honduras, India, Mozambique, the Philippines, Senegal, Tajikistan, and Uganda. Methodology The evaluation team selected countries for case studies to understand the support of multiple Global Practices (GPs) and how early support to the COVID-19 response is helping to protect human capital. The selected coun- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C tries have a Human Capital Index of 50 percent or below and received health, social, and institutional strengthening support for COVID-19. In addition, the countries have differing levels of vulnerability to losing their devel- opment gains due to COVID-19, and levels of preparedness for the crisis. These criteria helped the team understand how existing capacities may have enabled a country response. Other considerations for country selection included innovations to enrich learning, mix of instruments in the country portfolio, number of projects in the portfolio supporting COVID-19, coverage of fragile and conflict-affected situations, and different regions and popula- tion sizes. These criteria resulted in 41 eligible countries, which were dis- cussed with operational counterparts to select the eight case countries. The case studies examined at the quality of World Bank support to the early COVID-19 response. Data collection and analysis were organized using to the areas of the theory of action (needs of countries, support to implementa- tion and learning in countries, and operational processes and partnerships) and conceptual framework to assess the quality of the response and how the- matic areas and stages of support were carried out (Yin 1999). Although the analysis focused on support provided by the GPs covered by the evaluation, other GP staff were interviewed when recommended by the country man- agement to understand the cross-sector breadth of support in the countries. Evidence sources reviewed included: 186 » World Bank projects and advisory services and analytics (ASA) supporting the health and social response, identified with country teams, including project documents, Implementation Status and Results Reports, Aide Memoires, and knowledge outputs.1 » Interviews with actors involved in implementation of the response, including World Bank task teams and country management, government, civil society, and development partners. » Information on the response in the country, such as national plans and sec- ondary data on the COVID-19 situation. Data collection was done remotely due to travel restrictions, with all team members trained to follow the same case study protocol in Excel to ensure systematic data collection across countries. Case took place over about six months, between April 2021 and September 2021 One team member served as a coordinator working across countries to ensure consistency. In each country, the Independent Evaluation Group team relied on national consul- tants to provide country contextual knowledge and facilitate country stake- holder interviews. Quality of World Bank Support The case study countries show a medium to high implementation support Independent Evaluation Group World Bank Group    187 across areas assessed for a quality COVID-19 response, with opportunities for improvement in some areas (table C.1). All countries have supported na- tional COVID-19 plans and iteratively adjusted their response, often through informal meetings and exchanges. Consistent strong dialogue with govern- ment supported the response, though the extent of cross-sector collabora- tion and involvement of nongovernmental actors varied among countries. In terms of operational processes and partnerships, some countries deployed a wider mix of instruments and had greater GP and partner collaboration. The addressing of gender and inclusion needs attention, with countries having limited support in this area. Some countries had stronger support of moni- toring systems, knowledge sharing, and ASA. Table C.3, later in this appen- dix, details the responses in each country. 188 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Table C.1. Summary of Quality of Responses in Case Countries Theory of action areas Djibouti Honduras India Moz. Philipp. Senegal Tajik. Uganda Support to needs of countries Alignment with national COVID-19 plans Tailoring to needs and priorities of coun- tries and building on capacities Use of knowledge work to guide needs Address gender inequality Integration of institutional strengthening support Address digitalization Support to implementation and learning Dialogue with government on implementation Knowledge sharing and promotion of innovation Involvement of nongovernment in implementation Coordination of response Implementation status of interventions, and reach to beneficiaries (continued) Theory of action areas Djibouti Honduras India Moz. Philipp. Senegal Tajik. Uganda Iterative adjustment of implementation Operational processes and partnerships Internal collaboration and coordination across GPs Partnerships to support response Mix of instruments and streamlined processes Monitoring of the response High Medium Low support support support in or no support in the the area identified in area the area Source: Independent Evaluation Group case study analysis. Note: The areas of support reviewed are based on the theory of action. The estimated level of support in each area is based on data collection synthesized from the case study protocol completed for each country through document review, interviews, and consultations with country teams. Validation meetings helped ensure consistency in synthesizing findings across countries. Independent Evaluation Group World Bank Group    189 Quality of Response: Support to Needs of Countries (Relevance) Alignment with COVID-19 Plans The World Bank supported response plans in different sectors based on country circumstance. In all countries the World Bank aligned with coun- try COVID-19 plans to address health emergency needs and provide social protection to vulnerable groups. Support to address other response areas varied. Five countries also supported remote learning and reopening of schools (Djibouti, Honduras, India, Senegal, and Uganda). Djibouti, Senegal, and Uganda focused on refugee or displaced communities. Senegal had a multisector response, including to urban water and sanitation and digital The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C strategies. In India, support aligned with national- and state-level response plans. In Mozambique, support emphasized urban water and sanitation and essential services. In the Philippines, support emphasized community en- gagement and social cohesion. In Uganda, support emphasized sector plans, including in water, electricity, social affairs, and agriculture. Tailoring Early COVID-19 Support to Country Needs and Priorities Support was limited for essential health services, risk communication, social services, and citizen engagement, and for strategies for restructuring sys- tems for recovery. In Djibouti, support to essential health services has been limited, and project support to nutrition was diverted by the COVID-19 re- sponse. In Honduras, needs related to mental health, gender-based violence, maternal and child health services, and citizen engagement were not prom- inent in early support. In India, support addressed most needs in health and education, but there was limited support to strengthen systems other than in social protection. In the Philippines, health support was newly developed during COVID-19. Mozambique stands out for its focus on essential health services and preventing a secondary health crisis through diverting resourc- es to address the urgent needs arising from COVID-19 restricting access to health services. Senegal stands out for its support in developing prepared- ness capacities that were applied in the response. In Tajikistan, longer-term 190 preparedness capacity is a challenge, and early support to essential health services and risk communication is limited. In Uganda, longer-term support is needed to strengthen systems, expand digitalization, reinforce citizen engagement, integrate public-private sector service, and enhance prepared- ness strategies, and to address the impact of COVID-19 on girls. Use of Knowledge Work to Guide Needs ASA products have been important to monitor the situation, inform needs, and collaborate with partners. Having quality ASA in place before COVID-19 helped countries support dialogue on reforms. During COVID-19, ASA supported just-in-time analyses and information to guide the response (box C.1). Access to trust fund financing and technical support were important to finance just-in-time ASA and facilitate implementation. Box C.1. Use of ASA in Case Study Countries » Country assessments and monitoring informed the response. This included sup- port to mobile surveys on the socioeconomic and gender impacts of COVID-19, poverty assessments, food security assessments, studies on refugees, assess- ments of health workers, and beneficiary monitoring. » Analyses of health systems informed thinking on how to strengthen prepared- ness, health information, vaccine rollout, and essential health services. In India, Independent Evaluation Group World Bank Group    191 Honduras, and Senegal, prior analyses on disease preparedness and surveil- lance guided actions. In Uganda, support of the Global Financing Facility ensured resources and technical expertise to assess needs to improve health information systems and maternal and child health services. » Analyses of education, social protection, and crisis systems are informing imme- diate actions and longer-term thinking. In India, the social protection response built on knowledge work undertaken over 10 years. In the Philippines, the work previously done on community crisis instruments has been critical. (continued) Box C.1. Use of ASA in Case Study Countries (Cont.) » Just-in-time ASA helped solve implementation bottlenecks in collaboration with partners. In India, just-in-time ASA was conducted with the transport sector on the logistics of oxygen. In Mozambique, ASA sought to understand the impact of the pandemic on the private sector. In Uganda, ASA on water utilities and risk com- munication informed collaborative responses. Source: Independent Evaluation Group case study analysis. Note: ASA = advisory services and analytics Building on Foundational Capacities The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C The focus on human capital before COVID-19 meant that many countries had a relevant portfolio of World Bank support to address the crisis’ urgent needs. Having prior projects focusing on human capital meant in some coun- tries, almost every project and ASA adjusted its implementation to address COVID-19, with minimum restructuring; in other countries, previous efforts on human capital helped support a focus on new projects. In Honduras, Mo- zambique, and Senegal, these efforts were coordinated across GPs to syner- gize areas of support, building on foundational capacities where there was dialogue before COVID-19. Addressing Stages, Institutional Strengthening, and Gender Equality The addressing of gender and inclusion has been fragmented. Health and education support was often national, with limited targeting of vulnerable groups. In all countries social protection support targeted gender and vulner- able groups, though the extent varied in each country. Djibouti tailored its support to slums and used gender assessments. In Mozambique, cash transfers were gender-sensitive, prioritizing school retention and enrollment for women and vulnerable girls. Both Mozambique’s Senegal’s health response empha- sized women and children. Projects in Uganda include plans to address gen- der-based violence, and to rebuild maternal health services and engage girls. 192 In all countries, early COVID-19 support started a process to build more resilient systems, which demands continuation (box C.2). Sustainability will require additional support to restructure policies and systems. Box C.2. Support to Build Resilience in COVID-19 Response » In health, resilience was supported through critical health services for disease prevention and control. This support included improving surveillance, laborato- ries, infection prevention and control (IPC), facilities, and human resource ca- pacities. In some contexts, this support helped develop supply chains. Countries such as Djibouti have new capacities, given their limited preparedness before COVID-19. In India, the health response strengthened the network of public and private laboratories and testing systems. In Senegal, preparedness capacities established before COVID-19 are being further strengthened. » In education, resilience was supported by developing remote learning capaci- ties. Infrastructure and the curriculum have been improved to engage students and parents, and teachers’ capacities have been developed to support learning continuity. A few countries developed strategies to use new digital capacities to restructure systems. » In social protection, resilience was supported by strengthening systems. In India, prior actions helped consolidate state-level social protection systems and support migrant laborers. In the Philippines, support to expand the social pro- Independent Evaluation Group World Bank Group    193 tection system improved the capacity to respond to crises. In Tajikistan, early investments enabled the national rollout of the first comprehensive national social protection plan. Source: Independent Evaluation Group case study analysis. Note: IPC = infection prevention and control. Addressing Digitalization The digitalization of services was well integrated in the social response and less so in the health response. In education, countries supported television, radio, and online pedagogy resources for student learning. In Honduras, support included a package prepared for children and parents to follow up on television and radio classes. India’s support included a digital platform for teacher training. In Uganda, education support included SMS messag- ing to parents. In India, the Digital Infrastructure for Knowledge Sharing or DIKSHA platform engaged communities and teachers in creating and sharing relevant content for distance and remote learning. In health, there was sup- port to health information systems, contact-tracing applications, and digital surveillance. Mozambique, the Philippines, and Tajikistan supported digital tracking systems for vaccine rollout. In Tajikistan, health sector support included information hotlines, SMS texts to citizens in remote areas, online third-party monitoring, and electronic supply chain management. In social protection, countries supported expanding digital beneficiary databases and payment systems. India and the Philippines strengthened their national The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C identification systems, with links to digitalized payments for social bene- fits, social registry data on vulnerable groups, and data on migrant laborers. Djibouti supported an online platform for tracking food vouchers. Several countries used geo-enabling technology to gather monitoring data through smartphones. In Senegal, development policy financing (DPF) helped ad- vance a strategic agenda for digital infrastructure. Support to Implementation and Learning in Countries Dialogue with Government on Implementation Frequent policy dialogue supported robust responses, especially where it built on long-standing relationships in sectors. In Honduras, India, the Philippines, and Tajikistan, the social protection response built on years of policy analysis (as many as 10 years); while in Uganda, the social protec- tion dialogue is newly evolving. In Honduras, India, and the Philippines, COVID-19 initiated new policy dialogue in the health sector. In other coun- tries, the health sector dialogue was well-established, though experience delivering critical health services was limited. In Mozambique, the dialogue focused on ministries responsible for health, education, urban, and social protection to move forward quickly. In Uganda, dialogue similarly focused on specific ministries. In Senegal, the response built on a long-standing policy dialogue across government. COVID-19 intensified the importance of this di- 194 alogue for health and education. Fragmented communication and coordina- tion across sectors were challenges in countries and at subnational levels. In Tajikistan, the health project financed an adviser to support the COVID-19 response, which was a successful strategy to facilitate communication and coordination. Knowledge Sharing and Promotion of Innovation In each country, innovations addressed implementation challenges. For example, in Djibouti, networks of parent groups addressed challenges of reaching communities; in India, the engagement of women’s organizations addressed gaps in the availability of personal protective equipment; in Sen- egal, the One Health coordination mechanism helped with multisectoral engagement; and in Tajikistan, engagement with nongovernment actors to monitor the COVID-19 response helps to improve accountability. Knowledge sharing was limited in the case study countries. Djibouti stands out for using knowledge sharing to develop its response between education and social pro- tection. Senegal had knowledge sharing through its regional project support. Coordination of Response High-level leadership was important to support a rapid early response. Half of the countries featured strong leadership at the prime minister or presi- dent level. In Senegal, One Health multisectoral coordinating bodies estab- Independent Evaluation Group World Bank Group    195 lished before COVID-19 enabled rapid action with presidential participation. In Djibouti, high-level leadership ensured a swift coordinated response plan across sectors. In India, it enabled a quick response to the first COVID-19 wave, with a mix of interventions across sectors. In Honduras, despite cen- tral leadership, coordination and communication across sectors limited the response. Although Mozambique had a proactive early response, it has been difficult to maintain focus on COVID-19 given urgent simultaneous crises. The Philippines government responded quickly with an interagency task force, though the initial anchoring of the crisis as a health emergency slowed the response. The Tajikistan government was initially slow to respond, though the response accelerated with changes in ministerial leadership. In Uganda, implementation was overseen by a national task force headed by the prime minister and sector-level committees, reaching to subnational levels. However, changes in government limited leadership on COVID-19. Few countries had established structures to engage civil society and com- munities. Social protection had the widest engagement of nongovernment actors, since programs were organized to involve social agents, the infor- mal sector, farmers, and local civil society in communities and slum areas. Education had collaboration with groups, such as parent-teacher associa- tions. The Philippines supported community-based emergency instruments. In Tajikistan, the Global Partnership for Social Accountability supported third-party monitoring of the COVID-19 response by a civil society consor- tium. In Senegal and Uganda, structures established for nutrition helped with nongovernmental and community engagement. In health across coun- The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C tries, some support for community health workers was evident. India and Uganda had some support to private sector health services. Several countries plan to engage civil society in vaccine monitoring. Implementation Status of Interventions and Reach of Beneficiaries Countries frequently cited early results of World Bank support (table C.2). These results included procurement of medical supplies and equipment and the delivery of social protection benefits. Countries reported success in up- dating social protection systems, including expanding digitalization. A range of countries reported successes in implementing distance education tools, knowledge and skill building, and risk communication (though the extent of risk communication was often limited). Some countries also reported suc- cesses in improving health infrastructure and digitalized information sys- tems to deliver services and coordinate the response. 196 Table C.2. Examples of Early Results Reported in the Case Studies Country Example results reported Djibouti Strengthened national plan; medical supplies and equipment; case man- agement tracking, training of health workers, and guidelines; food vouch- ers and cash transfers; isolation facilities; digital platforms in education and network of parent associations Honduras Medical supplies and equipment; equipped laboratories for testing; skill building of health workers in IPC; food support for vulnerable households; virtual platform for preschool learning; early procurement of vaccines; and new partnership with health sector to support health preparedness and services India Medical supplies and equipment; expanded testing capacity; mobilization of community health workers for risk communication; emergency cash transfer and food support; and digitalization of learning for children Mozambique Medical supplies and equipment; cash transfers and system improve- ments; and continued support of essential health services during the crisis Philippines Medical supplies and equipment; digitalization of social protection pay- ments; new implementation of community projects to support COVID-19; early procurement of vaccines; and new partnership with health sector for preparedness response Senegal Medical supplies and equipment; improved laboratory testing capaci- ty; isolation facilities; cash transfers to all households in social registry; strengthened One Health approach for multisector coordination; nutrition messaging to communities; expanded country digitalization strategy; and digital infrastructure in education Tajikistan Medical supplies and equipment; risk communication; expansion of cash assistance payments; digital systems to track coordination of support, Independent Evaluation Group World Bank Group    197 supply chain, and vaccines in health sector; and early support to vaccines Uganda Medical supplies and equipment; financing of services in health facilities; expanded laboratory capacity; water service improvements; behavior change communication; national social protection registry; expanded farm voucher; digital platforms in education and networks of parents and teachers; and nutrition messaging Source: Independent Evaluation Group. Note: IPC = infection prevention and control. Country surveys show that some countries have made progress in reaching beneficiaries in areas where the World Bank works, but challenges remain. The main achievements were noted in implementing preventive measures, risk communication, and social protection. For example, in Djibouti and the Philippines, high levels of vaccine acceptance were reported. Djibouti, India, the Philippines, Senegal, and Tajikistan identified early achievements in the coverage of social protection. Challenges still need to be addressed in all countries, the most widespread are access to essential care services, the livelihoods of informal workers, and negative social cohesion. In Djibouti, Mozambique, Tajikistan, and Uganda, surveys reported ongoing access to care issues were felt more severely by vulnerable population groups. In Sen- egal, disparities in awareness were reported, with women, noncity dwellers, less educated, younger, and poorer populations less informed. Negative im- pacts on the livelihoods of informal workers were reported in Djibouti, India, Mozambique, the Philippines, and Uganda. In Djibouti, India, and Mozam- bique, respondents highlighted negative mental health issues. In India and Mozambique, female respondents especially marked mental health issues. In Honduras, Mozambique, and the Philippines a significant number of children The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C were unable to access virtual schooling or learning due to issue related to the internet, equipment, and teachers. In Senegal, Mozambique, and Ugan- da, distrust of government is an issue (Afrobarometer 2021a, 2021b, 2021c, and 2021d; Bau et al. 2021; Bautista, Balibrea, and Bleza 2020; Bhattacharya and Roy 2021; Grover et al. 2020; IPSOS 2020; Sumalatha 2021; Tuppal et al. 2021; WHO 2021; World Bank 2020; World Bank 2021a, 2021b, and 2021c; UNWomen 2021). Iterative Adjustment of Implementation Across countries, frequent meetings facilitated the review of progress, problem-solving and implementation adjustments. Frequent meetings with project and global teams pinpointed issues and identified ways to address them. In Djibouti, the package of services delivered by social agents was adapted to include COVID-19 messages. In Honduras, weekly exchanges were used to adjust project implementation plans and procurement plans. In other countries, frequent coordination meetings among project teams and virtual supervision support helped adjust implementation to address emerg- ing needs. Uganda added COVID-19 training support for the private sector and assessed risk communication on challenges identified in the COVID-19 communication strategy. 198 Quality of Response: Operational Processes and Partnerships Internal Coordination to Support Implementation GPs collaborated on instruments. In Tajikistan, the COVID-19 Emergency Project led by the Health, Nutrition, and Population (HNP) GP, collaborat- ed with the Social Protection and Jobs GP, and with the Social Sustainabil- ity and Inclusion (SSI) and Governance GPs for third-party monitoring of the response. In India and Mozambique, Contingency Emergency Response Components (CERC) were used to redirect resources across sectors. In the Philippines, HNP and Digital Development collaborated to digitalize systems and with SSI to support stakeholder consultations. Also, in the Philippines, the Beneficiary FIRST (Fast, Innovative, and Responsive Service Transforma- tion) Social Protection Project led by Social Protection and Jobs collaborated with Education; Finance, Competitiveness, and Innovation; HNP; and SSI to expand digital cash grants for vulnerable families with children. In Uganda, SSI supported review of the portfolio to integrate gender and inclusion. DPFs engaged multiple GPs in policy actions, such as for child policy, health mate- rials, informal sector engagement, and social protection. GPs collaborated on the catastrophe deferred drawdown options (CAT DDO) in Honduras. Collab- oration with the Water GP has been crucial to support sanitation in slums, schools, health facilities, and public areas. Where there was no defined instru- Independent Evaluation Group World Bank Group    199 ment for collaboration, GP coordination of support in countries was limited. Partnerships to Support Response Well-functioning partnerships active before the pandemic adapted and made important contributions to the response. Responses in all case countries used different competencies of partners through preexisting coordination mechanisms. In Djibouti, partner collaboration enabled project teams to address needs, such as psychosocial care with UNICEF and food vouchers with World Food Programme. The World Bank in Honduras collaborated with the Inter-American Development Bank in the social response, including joint missions. In Tajikistan and Honduras, procurement was undertaken with the United Nations Office for Project Services (UNOPS). In Tajikistan, frag- mented coordination was supported by developing a web-based platform to track donor support and help improve partnership. In Uganda, a trust funded staff member in the World Bank office supports donor coordination, and ASA helped align the World Bank and partners on common strategies, such as for risk communication and water service improvements. Having the support of EdTech, the Global Financing Facility (GFF), WHO, and Gavi, the Vaccine Alliance, in a country portfolio often supported quick actions to expand sup- port for COVID-19. Mix of Instruments and Streamline Processes Countries combined instruments to support a more agile response. These include crisis instruments (CERC and CAT DDO), repurposed projects, DPFs, The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C regional projects, Program-for-Results (PforR) financing, new projects, and to some extent, additional financing, though the main use of that instrument has been to replenish project support disbursed for the crisis (box C.3). Box C.3. Use of Instruments in Case Study Countries » Crisis instruments supported immediate response. In Honduras and Mozambique, emergency instruments helped leverage resources for multiple emergencies. In Mozambique, for example, the CERC of an urban sanitation project supported surveillance and infection prevention and control (IPC), and an emergency cy- clone response project was repurposed for case management, IPC, risk commu- nication, and social protection. The Philippines activated the community-based Disaster Response Operations Modality supported by a World Bank project. Across countries, challenges in using emergency support included the limited coverage of crisis instruments in the country portfolio, and the requirement for government to announce the emergency before implementation. » Adjusting projects to address COVID-19 needs supported a cross-sector re- sponse and helped set a course for reforms to restructure systems. The urgency of the COVID-19 response helped adjust projects to accelerate actions on key reforms, such as to expand social protection, strengthen local service delivery, and improve water and sanitation conditions in schools and slums. In India, seven education projects were repurposed to respond to COVID-19. (continued) 200 Box C.3. Use of Instruments in Case Study Countries (Cont.) » In Senegal, several projects were synergized to support social protection and food security. In Uganda, water, agriculture, and governance support was ad- justed, with AF to expand support to farmers and assure the continuation of local services. » The MPA led by HNP helped orient health project support and procured supplies for response plans. The MPA framework provided flexible guidance to organize support in health. MPA financing supported supplies and critical health services rather than policy guidance or other aspects outlined. In most countries, the first round of MPA financing was processed quickly, with AF later for vaccination. In Uganda, the timeline of the MPA was delayed, raising questions about the effi- ciency of processing new project support in a crisis. » Regional projects supported early action to coordinate the response. In Senegal, support of the Regional Disease Surveillance Systems Enhancement (REDISSE) project and WHO has been critical for immediate actions to coordinate the re- sponse, which were later synergistic with the MPA. Having this preexisting support meant structures and capacities were in place to respond more quickly. » DPFs and PforR supported urgent fiscal needs. DPFs were important in India, Mo- zambique, Senegal, and Uganda for COVID-19 response plans, continuation of lo- cal services, informal workers, agricultural inputs, health supplies, and social pro- Independent Evaluation Group World Bank Group    201 tection. During the election period in Uganda, the DPF was passed by parliament in about three months. In India, PforR were used to agree on disbursement-linked indicators to orient the education and health response at the state level. » PEF provided small grant support to address urgent needs. The PEF was used to finance medical supplies and equipment, as many countries were concerned about borrowing more resources. In Senegal, PEF funds channeled through UNICEF and WHO provided timely support to subnational COVID-19 plans. In Uganda, the PEF provided immediate resources to the national COVID-19 plan, despite delays of parliamentary approval of the MPA. (continued) Box C.3. Use of Instruments in Case Study Countries (Cont.) » Trust funds assured timely support. Trust fund support helped education proj- ects in Djibouti, Honduras, India, Mozambique, Senegal, and Uganda, for remote learning and school reopening. Source: Independent Evaluation Group case study analysis. Note: AF = Additional Financing; CERC = Contingency Emergency Response Components; DPF = de- velopment policy financing; IPC = infection prevention and control; MPA = Multiphase Programmatic Approach; PEF = Pandemic Emergency Financing Facility; PforR = Program-for-Results Financing. Countries had delays in early support to vaccination. Djibouti, Honduras, the Philippines, and Tajikistan coordinated with partners to prepare for early The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C vaccine deployment, and all countries have plans to support vaccines. World Bank policy limited the ability of project teams to respond quickly to vaccine plans. For example, project teams could not procure nonapproved vaccines, which limited collaboration in task forces to support vaccine deployment. Also, the interest of government in lending rather than grant support for vaccines varied across countries. Emergency procedures accelerated procurement and financing of activities. This included using direct contracting, e-bidding, shorter bidding periods, retroactive processes, United Nations partner procurement, and World Bank–facilitated procurement (BFP)—though guidelines were not always clear. Countries procured a large quantity of medical supplies early in the crisis, and orders took several months due to global shortages. Most gov- ernments used their own national procurement procedures. In Djibouti, the government adopted accelerated emergency guidelines for medical procure- ment, though this was not done in all sectors. In India and the Philippines, national procedures were assisted by close tracking by the chief procurement officer. In Mozambique and Senegal, projects collaborated to spread pro- curement across projects. However, in Senegal, despite dedicated efforts led by the president, cost fluctuations and limited global availability of items made procurement challenging. In Honduras, India, and Mozambique, BFP was used to procure items not easily available. However, BFP often required multiple contracts and complicated logistics. In Honduras, BFP support re- 202 quired coordination with the World Food Programme to purchase and deliver personal protective equipment from China, and the United Nations Office for Project Services provided logistical support. In India, the HNP and Transport GPs partnered on transportation logistics for oxygen. Another challenge in countries was the slow delivery of centrally procured items to subnational areas and audits on receipt. Safeguards were challenging given the need to apply new guidelines amid a crisis. In most countries, the learning curve for a new project to apply the new Environmental and Social Framework was high, despite templates and extensive handholding from safeguard teams. It was a labor-intensive process for staff and government who were already overwhelmed. More- over, structures were not well organized for required stakeholder consul- tations. The engagement of government experts to support the safeguards’ implementation in new projects was challenging, with ministries also overwhelmed by COVID-19 and in some cases delayed project support and procurement of goods. Monitoring of the Response Limited communication with subnational levels and monitoring capacity of countries constrained decisions. World Bank systems require formal report- ing every six months, limiting the usefulness of those reports in an emergen- cy context. Other challenges include the limited capacity of governments to Independent Evaluation Group World Bank Group    203 measure results and the availability of real-time data on COVID-19 cases. To address these, World Bank teams coordinated closely with partners to mon- itor health aspects of the response and met with the government weekly to discuss bottlenecks and review progress. Meetings were often with national counterparts since lack of mission travel limited communication with subna- tional actors. Some countries financed data to monitor the response. For ex- ample, Djibouti supported iterative beneficiary monitoring of education and social protection projects. In Tajikistan and Uganda, geo-enabled monitoring and supervision tools facilitated remote monitoring and project supervision of nutrition support. 204 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Table C.3. Synthesis of COVID-19 Responses in Case Study Countries Support to needs of countries Support to implementation and learning Operational processes and partnerships Djibouti Alignment with plans Leadership and coordination Country program coordination In April 2020, Djibouti launched its National Soli- A high-level committee led by the prime minis- The World Bank linked authorities to technical darity Pact framing the COVID-19 response. The ter provided guidance and led the vaccination experts and shared global experiences. Early government also put in place a preparedness committee, and sectors have separate COVID-19 in the crisis, the focus was on critical sectors to response plan. The World Bank’s support aligns plans. However, early in the pandemic commu- cushion the economic downturn and pro- with national plans and sector-specific plans, nication was unclear, and ministries had limited tect vulnerable populations. The focus of the including for distance learning and refugee experience to respond. In health, a multisectoral World Bank strategy on human capital before communities. committee met daily to weekly, as did technical COVID-19 meant there was a relevant portfolio Tailoring to needs and priorities committees of health structures. in health, social protection, education, urban, Immediate support expanded quarantine and Building on evidence and past lessons poverty, finance, and social development that testing centers. Health capacity is low, and In Djibouti, support followed WHO guidance and could be adapted. there was limited experience in providing critical aligned with available evidence. Greater capacity Partnerships services. Support to essential health services for consistent communication and coordination Though needing clarification in the initial has been limited, and project support to nutri- may have strengthened the early response, response donor coordination with the govern- tion was diverted by the COVID-19 response. and a stronger focus on gender across sectors ment has been present from early in the crisis UNICEF and WHO led vaccination response with and the continuity of essential health services. drawing on long-standing relations between World Bank support. Institutional strengthening of preparedness, par- UN agencies and the World Bank. For example, Use of knowledge work ticularly health systems will be important going joint support was provided to develop and cost ASA products have been adapted to help assess forward. the national plan and good collaboration with and monitor the COVID-19 situation and share Policy dialogue United Nations partners, including organized examples of experiences from other countries. The World Bank and government had strong support for vaccination led by UNICEF and This includes surveys monitoring the socioeco- dialogue on the response. Early in the crisis, WHO. In education, the Global Partnership for nomic impact, knowledge exchanges, a poverty dialogue focused on potential strategies for Education supported a coordination committee, assessment, and an analysis of health system mitigation, and sharing of experiences of other and there is collaboration with UNICEF on a strengthening for universal health coverage. countries. Then, dialogue shifted to tracking domestic resource mobilization strategy and to actions and assessing COVID-19 impacts. assess learning needs. World Food Programme (WFP) and the World Bank collaborated on food vouchers. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Djibouti Addressing resilience, inclusion, sustainability, Iterative adjustments Mix of instruments and gender Most projects and ASA in the portfolio were The portfolio included repurposed projects, Resilience was supported by expanding health adapted to address COVID-19. World Bank CERC, MPA, PEF, and AF. CERC provided timely facilities for patient care, aggressive efforts to teams met weekly with the government to financing to address the just-in-time needs of identify cases, and support to remote learning review progress, adapt implementation, and government without restructuring and heavy ne- and psychosocial care of children and teach- solve problems. For example, in social protec- gotiations. The MPA framework provided flexible ers. World Bank support focused on refugees, tion, the package of services was adapted to guidance. MPA supported financing of critical displaced persons, and vulnerable populations include COVID-19 messages and distribute hy- health services—other areas of the MPA frame- in slum areas. In social protection, the social giene kits. The urban project was also adapted work were not possible. The PEF also supported registry reached female-headed households. to address COVID-19 in slum areas. urgent medical supplies and equipment. Education support helped public schools, Involvement of nongovernment Monitoring with a focus on girls and children with special The education sector developed a partner- The projects used indicators aligned with the needs. Surveys on COVID-19 collected gen- ship agreement working with local authorities WHO global monitoring framework for COVID-19. der-relevant data. and nongovernmental groups, such as par- An ASA led by the Poverty GP is supporting Addressing digitalization ent-teacher associations and local civil society iterative beneficiary monitoring of education All sectors moved training and communica- groups. Social protection engaged with social and social protection projects to see if support is tion online. World Bank activities supported: agents and community actors. Response plans reaching beneficiaries. A challenge is the capacity (i) mobile phone survey methods; (ii) an online in health included some support to community of the government to measure results. platform for tracking food vouchers; and (iii) health workers. television, radio, and online student learning, tablets, internet access, and online pedagogy resources. Health has less support for digitali- zation. (continued) Independent Evaluation Group World Bank Group    205 206 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Djibouti . Procurement Procurement guidelines and direct agreements accelerated processes. Retroactive procure- ment moved activities forward when funds were not immediately available but could later be reimbursed. Accelerated emergency guidelines for procurement by the government’s medical and materials center enabled the purchase of key items, such as medical and laboratory equipment and supplies. The World Bank’s response included procuring school kits, tablets, and materials for distance learning. Safeguards For new projects, the learning curve for applying the new Environmental and Social Framework was high. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Honduras Alignment with plans Leadership and coordination Country program coordination In Honduras, the World Bank response fully The government up to the level of the president The World Bank repurposed its existing country aligned with the government’s plans, including adopted a national response plan for COVID-19 portfolio and prepared new operations to strengthening surveillance; laboratory support; in February 2020. The response was facilitated swiftly support the government response. case management and treatment; IPC; vaccine by prior ASA and project support to improve These adjustments facilitated a rapid, holis- procurement and deployment; emergency preparedness. The national plan defines the tic, people-centered response involving all cash transfers; and remote learning. responsibilities, procedures, and multisectoral GPs. National actors appreciated the technical Tailoring to needs and priorities and subnational coordination mechanisms for soundness and speed of the support. To adjust World Bank support was well tailored to the the response. Nevertheless, coordination and the support, meetings were held frequently needs and priorities articulated by the govern- communication have been challenging. with the operations manager, program leaders, ment. However, mental health, GBV, maternal Building on evidence and past lessons task teams, and government implementers. and child health services, and citizen engage- The COVID-19 support built on crisis response Mix of instruments ment support were not prominent in early experience especially in social sectors. The The adjusted country portfolio drew on CERCs, support. response focused on interventions proven to which were repurposed to tackle the impact Use of knowledge work work and were anchored in policy dialogue, with of hurricanes in addition to COVID-19. Health ASA products before and during COVID-19 ASA playing a key role in informing projects. developed an emergency MPA, which was have been critical to inform the response and Strengthening support to communication and quickly approved in April 2020. The project was provide longer-term direction for restructuring coordination, essential services, and citizen and complemented by AF to purchase and deploy systems. They include ASAs on emergency community engagement could benefit future vaccines. A CAT DDO was also approved in preparedness; adaptive safety nets; education crises. April 2020, facilitating multisectoral coordina- service delivery; early childhood develop- tion across GPs. PEF funds, channeled through ment; and public expenditure reviews cover- PAHO, supported just-in-time infrastructure ing health, education, and social protection. improvement, medical and lab equipment and Though the response missed opportunities to supplies, and telehealth support establish just-in-time ASA. (continued) Independent Evaluation Group World Bank Group    207 208 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Honduras Addressing resilience, inclusion, sustainability, Policy dialogue Monitoring and gender This social response was built on a long-stand- The government prepared plans, progress Although investments aimed to secure ing policy dialogue with the government, reports, and evaluations. World Bank staff in equipment, supplies, and vaccines, Honduras which deepened through daily exchanges. health and other sector teams met weekly focused on improving surveillance, laborato- ASA informed this dialogue and the World with the government to address bottlenecks. ries, and social protection systems to increase Bank’s CAT DDO was critical to policy reform. In Monitoring was often done informally, given the resilience. The response is bringing new ways health services, COVID-19 helped renew policy limited capacity to monitor the response in real to deliver education, health services, and dialogue for the first time in years. The support time. social assistance, improving sustainability and established a trust with government counter- Procurement inclusion. The nutrition program engaged rural parts. Policy dialogue evolved from emergency Meetings helped proactively track procure- communities in inclusive and efficient ways, measures to a dialogue aimed at assuring better ment support of projects and World Bank–fa- and cash transfers support women. Howev- health systems. In education, the dialogue sup- cilitated procurement (BFP), which was used er, the addressing of gender and inclusion in ported quick project adjustments. to procure PPE and medical equipment. The health has been limited. Partnerships World Bank contracted UNOPS and WFP to Addressing digitalization Coordination and collaboration in between the provide logistical support in the transportation, Digitalization has been supported particularly in World Bank and Inter-American Development storage, and distribution of PPE. BFP process- the social sector, where the World Bank helped Bank in the social sector has been good, with es and logistics became more complex than Honduras establish a more transparent and regular meetings and joint missions and feed- expected. effective payment mechanism. Health and ed- back. However, in health, collaboration was more Safeguards ucation digitalization represented only a small limited. Support to government was though The emergency MPA project followed the new part of the support aimed to improve digital separate lines although there is a platform for safeguards framework, which was labor-inten- infrastructure. high-level cooperation in health that includes the sive for the staff, particularly since November Pan American Health Organization (PAHO), UN- 2020 when they had to manage the impact of AIDS, UNICEF, Food and Agriculture Organization hurricanes and COVID-19. (FAO), WFP, United States Agency for Internation- al Development (USAID), European Union and the Inter-American Development Bank. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships India Alignment with plans Leadership and coordination Country program coordination All areas of the health and social response align The Indian government responded quickly to blunt The country response was coordinated through with national plans and policies. Support was also the first COVID-19 wave, undertaking interventions the targeted design of three projects in health, provided at the state level through ongoing peer in the social protection, education, and health social protection, and micro, small, and medium learning. The health response funded the national sectors. Under the leadership of the prime minister, enterprises. Two CERC activations by Urban, Di- COVID-19 containment plan by supporting testing, the country undertook the world’s largest national saster Risk, Resilience, and Land (GPURL) projects tracing, tracking, and establishment of intensive lockdown and announced a relief package of about provided emergency health support. State-level care units and isolation centers. The World Bank’s 10 percent of gross domestic product. The coun- health responses benefited from the national support was flexible enough to adapt and was try put in place a national response plan focused COVID-19 program. A shared ASA between health highly appreciated by the Ministry of Health and on cluster containment, testing, tracking, tracing, and transport focused on logistics for oxygen. The Family Welfare (MOHFW). The DPFs aligned with and social distancing. A group of ministers under education response undertook adaptations with government plans to provide social support. The the minister of health served as the apex body for limited input from other GPs. The support provided World Bank also supported micro, small, and policy decisions, providing direction to the states. for staff members was praised across GPs as quick medium enterprise liquidity to help maintain work- Several coordination committees were formed at and supportive of staff requirements working force human capital. the state and national level. Health messaging was across the entire World Bank Group. Country and Tailoring to needs and priorities a challenge, as contradictory messaging emanat- headquarters GPs had good coordination, but The World Bank’s health response met emergency ed from different national agencies, while several the inability to travel restricted coordination with needs and supported evolving priorities, building religious festivals and political campaigns hindered communities. on previous good client relations. The MOHFW re- needed social distancing. Mix of instruments ported being satisfied with the emergency loan and Building on evidence and past lessons The country deployed a range of instruments for flexibilities introduced. Crisis response and restruc- Social protection used accumulated knowledge crisis support and restructuring needs. To avoid turing needs were met through the social protection over a 10-year period to define the response. The hitting the single borrower limit and free funds, support. The health support focused on the crisis but health sector drew on their experience of imple- the World Bank team canceled and restructured also contributed to restructuring. The main challeng- menting tuberculosis and HIV testing and tracking commitments. Seven education projects were es in the health response related to risk communica- systems and an analysis of health system challeng- repurposed to respond to COVID-19. Education tion, gender targeting, essential health services, and es in India. Education drew on lessons and experi- introduced new PforR. The health response was community health workers. In state projects, services ence generated by extensive prior engagement in driven through the MPA and incorporated two new for treatment for other diseases, such as cancer, education and shared GP-level knowledge where investment project financing (IPF), with state-level declined as services focused on COVID-19. A clear required to provide technical assistance. projects adapted to respond. The social protec- strategy to address learning losses from the school tion response was implemented through a DPF. closures has yet to emerge. CERCs were activated in GPURL. (continued) Independent Evaluation Group World Bank Group    209 210 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships India Use of knowledge work Policy dialogue Monitoring ASA and knowledge work were important in The influence of policy dialogue during the pan- Monitoring of the response is continuous and is the response. The SPJ response was built on demic was found to be responsive where there being used within the World Bank and in engaging knowledge work undertaken over 10 years. Since were strong prior relationships with the government. with partners. The Poverty GP and SPJ work with implementation of prior actions, they have used The country office has a close working relationship Centre for Monitoring the Indian Economy, who monitoring information from the Centre for Moni- with the Department of Economic Affairs within the undertake frequent surveys to track COVID-19 toring the Indian Economy to track and communi- Ministry of Finance and good relationships in social effects. The World Bank corporate-level moni- cate changes in social protection coverage. HNP protection. In the health sector, previous dialogue toring efforts have been less useful for informing undertook knowledge work during the pandemic had focused on the state level and was limited to adaptations. The project monitoring for the MPA at state and national levels. Some prior HNP work work on tuberculosis and HIV/AIDS at the national was initially too complicated and restructured at the state level supported the health response, level. The Department of Economic Affairs helped about 10 months into the project. Most indicators and surveillance and testing of tuberculosis open the space for dialogue and lending by helping for the social and health responses show achieve- and HIV informed the response. The education to coordinate with the MOHFW. Consequently, as ment above target. response did not target individual ASA, using engagement restarted, technical support was pro- GP-level knowledge and existing dialogue to vided to respond to COVID-19. In states where there provide inputs on lessons. No ASA were identified was a good prior relationship, the World Bank’s on risk communication or assuring health services. advice was taken up faster, for example, in defining The Transport GP undertook an ASA related to the cost structure for government health insurance for logistics of oxygen to help resolve bottlenecks. COVID-19, and in Maharashtra and Kerala, especial- ly on the oxygen supply. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships India Addressing resilience, inclusion, sustainability, Partnerships Procurement and gender The World Bank worked well with development Tensions arose with procurement processes within India’s response contributed to developing the partners. Asian Development Bank (ADB) lent an the World Bank and with the client. To implement resilience of the health system, with limited additional $500 million to the health response and the MPA quickly it was agreed that India could use support to inclusion, sustainability, and gender. supported the government’s social protection pack- national procurement mechanisms and the World Resilience has been supported by developing age using prior action, such as the World Bank’s. Bank provided ongoing support to the govern- a network of public and private laboratories and ADB also replicated the STARS education project. ment through consultants and local procurement testing systems, developing intensive care units Asian Infrastructure Investment Bank provided specialists. Tensions arose with the client in the capacities, and supporting new labs for genomic support to the World Bank’s health project. There MPA over the use of a “Make in India” clause and sequencing. Prior work with women’s self-help was ongoing coordination with UNICEF and WHO of eligible expenditures; these were resolved groups and small and medium enterprises better at the country level. with Country Management Unit guidance. BFP enabled their mobilization to support large-scale Involvement of nongovernment proved useful for oxygen, where quick turnaround procurements of COVID-19 PPEs and other Innovations resulted from support to the private periods were required. The client reported that no critical support. Education support helped project sector and limited involvement of civil society other multilateral development bank was able to beneficiaries better enter the labor market or organizations in the health and social protection provide this support. It was suggested that a PforR retrain. Prior actions in the social protection DPF response. In implementing the MPA, extensive or DPF would have been better options for the helped consolidate state-level systems and reach consultations were not held. The education sector emergency response to avoid the procurement migrant laborers. The social protection response has had ongoing engagement of civil society. For tensions. did include prior actions directed at environmen- example, in the Nagaland, the client used existing tal sustainability in climate-resilient public works. relationships with community structures to deepen Gender considerations were not included in the understanding and buy-in. The World Bank used health response. In social protection, women were public-private partnerships to catalyze innovation in targeted to receive benefits. the health sector and the biopharma arena, which supported the development of the first COVID-19 DNA vaccine. (continued) Independent Evaluation Group World Bank Group    211 212 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships India Addressing digitalization Iterative adjustments Safeguards Increased levels of digitalization were embraced All areas of the COVID-19 response made imple- TTLs reported that implementing safeguards were across the health and social responses. The social mentation adjustments. The MPA was restructured cumbersome and suggested that it could be sim- protection response supported the increased use in April 2021 to increase allocation to the crisis re- plified for the MPA. Stakeholder engagement was of digitalized social registries. In education, the re- sponse, while investment project financing projects reported to have targeted messaging on the proj- sponse supported the shift to online teaching and repurposed uncommitted funds to COVID-19. The ect, rather than genuine consultation. Civil society an online platform that offers teacher training. Few education response restructured and reallocated engagement was reported to not have occurred in provisions for the privacy of data and measures funding to adjust to the digital learning environment. the MPA. The MPA used the government’s redress to strengthen cybersecurity were identified in any Social protection undertook one DPF program mechanisms, but evidence from existing projects World Bank project reviewed. The government is and is starting a second program. Beyond these shows that grievance redress mechanisms (GRMs) drafting legislation for data privacy and cybersecu- formal adjustments, the World Bank regularly met do not always function well at the state level. The rity, but concerns were voiced on whether citizens with other partners and the client to identify areas review of GRM for the health project found there are informed and protected. to adapt. It was reported that uncertainty about re- was no reporting of cases, suggesting the need for quirements and waivers delayed some implemen- improvements in accountability and GRM. tation. CERC components are now routinely written into projects for future adjustments. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Mozambique Alignment with plans Leadership and coordination Country program coordination World Bank support fully aligned with govern- Although the government took the initiative and re- A portfolio of projects designed for emergency ment plans in the national strategy response to sponded early, it has been difficult to maintain focus response existed and was adapted to address COVID-19. Support aimed at strengthening the as the country has faced multiple urgent crises. The COVID-19. Weekly management meetings held capacity of the health and social response sys- national COVID-19 response plan was launched with program leaders coordinated the World tems. A notable challenge was providing sufficient in March 2020, though there was limited support Bank support. Engaging all country team mem- resources, such as cash transfers and support for in place to coordinate its implementation. Donors, bers helped with coordination across GPs. CERCs small enterprises, to allow the population to com- rather than government, have the coordination required coordination between the task teams of ply with social distancing requirements while also and implementation of the response. However, the the source project and of HNP to program funds. maintaining economic activity to sustain society. health, social affairs, education, and finance sectors Good collaboration also existed between the Ed- Tailoring to needs and priorities have plans for COVID-19 support. ucation and Water GPs to deliver WASH interven- A significant concern for the country was avoiding Building on evidence and past lessons tions for schools. a secondary health crisis linked to diverted atten- The early response built on lessons learned from Mix of instruments tion and resources to COVID-19. The country had responding to the cyclones, which occurred two CERC components across the country portfolio, significant health needs and deepened develop- years earlier and presented a similar acute cri- mainstreamed in many World Bank projects, were ment concerns about areas such as malaria, HIV/ sis. The response included most health support rapidly activated. For example, reallocation of AIDS, malnutrition, and maternal and child health. recommended by WHO. The vaccine rollout will funds from the cyclone response and recovery To mitigate this, the World Bank used its Primary build on global lessons, such as the use of digital project provided early financing for health and so- Health Care Strengthening Program, including AF, technologies to increase demand, reduce vaccine cial protection. The implementation of other ongo- and the COVID-19 Strategic Preparedness and hesitancy, and engage communities in the monitor- ing projects was accelerated to meet needs. The Response Project to focus on ensuring continuity ing of the vaccine rollout. Coordination helped en- COVID-19 Response DPF was used to meet fiscal of essential health services. Nevertheless, there sure that pressing priorities for essential health care needs. The $2 million PEF funds played a minor was a sense that support to COVID-19 negatively received continue prioritization. However, COVID-19 role in supporting the health response. The MPA affected essential services. received disproportionate attention and resources facilitated the response, and the second phase of compared with other urgent priorities. the MPA provided an adaptable framework for the vaccine purchasing and deployment project. The instrument mix provided a robust World Bank re- sponse, and given familiarity with CERC, no major challenges occurred. (continued) Independent Evaluation Group World Bank Group    213 214 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Mozambique Use of knowledge work Policy dialogue Monitoring The country response did not significantly draw In the absence of a centrally organized govern- There is no overall monitoring of the COVID-19 on ASA in designing and supporting its pandemic ment response, the World Bank led sector-specific response, while projects report on specific activi- response. A just-in-time knowledge work was responses in consultation with relevant ministries ties. The government’s monitoring is concentrated initiated to assess the pandemic’s impacts on the and other development partners. Support focused in the health response, where the World Bank private sector and the effectiveness of govern- on health, education, urban, and social protection supports efforts to strengthen the country’s health ment measures to respond to those impacts. sectors. The response built on existing sector rela- management information system to monitor case Addressing resilience, inclusion, sustainability, tionships and accelerated the pace and direction of management of COVID-19 patients and adverse and gender measures that were already undertaken before the reactions to vaccination. The social response sought to address gender pandemic. Procurement needs with cash transfers prioritizing women and Partnerships Mozambique relied on BFP to purchase about WASH (water, sanitation, and hygiene) interven- Coordination among development partners used $9 million of respiratory equipment, diagnostic tions targeting vulnerable girls, especially the existing platforms, which support a crisis response equipment, and PPE. Delivery of centrally pro- rural poor, to increase their school retention and group of donors (African Development Bank, IMF, cured items to the provinces was often slow and enrollment. It also strengthened social protection World Bank, Canada, United Kingdom, Ireland, complicated. Implementation was hampered by systems to reach vulnerable beneficiaries. The United States, European Union, and the Nether- the global supply crisis, though World Bank pro- health response prioritized maintaining maternal lands) that meet monthly with the government on curement proved faster than other government or and child health services. The challenge was in cyclone recovery. The International Community United Nations agency efforts. terms of scale rather than focus, as needs are vast COVID-19 task force led by the British High Com- Safeguards beyond available resources. missioner was also established. Sector working The implementation of safeguards in ongoing Addressing digitalization groups enabled a division of labor among donors. projects had unintended results. For example, the Digitalization has been supported across the main The coordination and collaboration have been suc- Urban Transformation Project required resettle- areas of the response: in health through support cessful; however, the effectiveness is constrained ment of some residents to new locations. for digital surveillance and digitalization of vaccine by the government’s response limitations. rollout; in social protection through digitalization of money transfers; and in education through strengthening digitalized distance learning. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Mozambique Involvement of nongovernment Civil society supports vaccines by monitoring deployment and tracking vaccine delivery. Nongov- ernment groups will support the implementation of communication campaigns and behavior change interventions. The greater engagement of com- munity groups is key to providing accountability for government resources. Health support involved CHWs. Iterative adjustments Frequent coordination meetings among World Bank teams and virtual supervision support enabled learning and adaptation. Ongoing implementation faces two challenges: the low capacity of sub- national areas to monitor and plan the response and inadequate data to learn from and adapt the economic response. (continued) Independent Evaluation Group World Bank Group    215 216 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Philippines Alignment with plans Leadership and coordination Country program coordination World Bank support aligned with the national The government responded quickly with ambi- There was good collaboration within and across response plan, particularly pillars 1 and 2, which tious plans to control COVID-19 and provide social World Bank sectors. For example, the Digital includes support for health and social protection. protection to large segments of the population. An Development and Poverty GP collaborated in The plan was designed in collaboration with the interagency task force led the response and consti- the study of COVID-19 impacts, and the SSI GP health sector. The World Bank did not support tuted 34 government agencies. Originally, the crisis supported the health sector in conducting its first other sectors, such as education, as this was was considered a health emergency with the health stakeholder consultations. The Beneficiary FIRST not an explicit priority. Efforts aimed to fill needs sector overseeing the task forces and plans. How- (Fast, Innovative, and Responsive Service Transfor- where government support was insufficient in the ever, it was quickly seen that to mobilize a response mation) project, led by SPJ, collaborated with SSI, COVID-19 plan, and provide guidance, structural, with robust actions across multiple sectors, central HNP, and Finance, Competitiveness, and Innova- and financial support to expansion of the govern- government leadership was needed and moved to tion for the payment systems and digital agenda ment’s response. the Department of Finance. The pandemic support and health and with education on cash transfers. Tailoring to needs and priorities accelerated and deepened interagency coordina- Mix of instruments The World Bank launched its first project in tion. In social protection, a portfolio of relevant proj- health since 2013 to support government needs. Building on evidence and past lessons ects was adapted and accelerated to support On the social response, the government largely The World Bank drew on lessons learned and the government’s cash transfer program. A key relies on its own resources. The World Bank’s structures developed in the wake of natural disas- early support was the community-based DROM social protection support informed the design of ters, most notably Typhoon Haiyan in 2013. The early activated in the KALAHI project to support critical the government’s ambitious expansion of cash tranche of World Bank financing for health was built health services, cash transfers to casual laborers transfers. Support to local government through the on the DROM introduced into World Bank lend- who lost their jobs, and community engagement. World Bank’s disaster response instrument was ing operations. Moreover, the strong emphasis on The support built on the World Bank’s commu- flexible to allow implementation of their own priori- community response aligns with global evidence. nity-driven development activities to mobilize ties—the capacity of local government officials to The social protection response projects also built community engagement, social cohesion, and apply the instrument was important. World Bank on knowledge products to design support most local government emergency response planning. policy limited responsiveness to the plan for vac- effectively. These helped in the design of the govern- DROM is a process adjustment for emergencies cination, given restrictions on purchasing vaccines ment’s social protection response and provided a that expedites approval, processing, and release that were not approved by the World Bank–. The platform for identifying beneficiaries for the expand- of project funds to support local subprojects. The client did not request the World Bank’s support in ed program. Developing procurement pathways with second phase of the MPA project was critical for education. expedited processes for emergencies may facilitate financing vaccine procurement. Lack of clear a more efficient response in the future communication and complex processes led to a slower and suboptimal response. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Philippines Use of knowledge work Policy dialogue Monitoring The World Bank drew on knowledge work done The long-standing policy dialogue in the social The MPA included support to track indicators for previously to inform its social protection and protection sector facilitated a robust and rapid vaccination rollout online. Household surveys community development activities. This included response, but the lack of policy and financing fed into the Beneficiary FIRST project to provide impact evaluations and guidance notes on tech- engagement in the health sector for more than 10 real-time information on the impact of COVID-19 nical topics. Recent and ongoing ASA supporting years required the reestablishment of dialogue and on households in the poorest areas. the response include a poverty and inequality engagement strategy. Close dialogue with the De- Procurement assessment and a survey monitoring the impact of partment of Health has been reestablished through Initially, the familiarity of the health sector with COVID-19 on households. The health project also regular meetings. A key challenge is the devolved World Bank procurement processes slowed the supported a vaccine readiness assessment. and fragmented nature of government in health response. Tracking procurement by the chief Addressing resilience, inclusion, sustainability, and social protection. procurement officer addressed this challenge. and gender Partnerships Later, vaccine procurement was much improved, Support for laboratories, isolation facilities, and Regular partner coordination processes in the with the World Bank fast-tracking the first vaccine equipment, along with strengthening digitalization health sector are in place to ensure the comple- procurement contract and the chief procurement and support to the national identification system, is mentarity of support. Though the initial focus on office conducting market engagement to orient expected to improve service delivery capacity for health slowed cross-sector coordination. The social vaccine manufacturers. Other challenges were health and social protection, and hence increase protection sector has multiple partners providing communication across partners to coordinate resilience and build sustainability. Social protection support, including ADB, Australia’s Department of procurement of items, such as COVID-19 testing expansion to vulnerable households and migrant Foreign Affairs and Trade and UNICEF. The de- machines and kits. BFP was undertaken to procure workers addresses inclusion, as does the explicit volved nature of the government with fragmenta- PPE and ventilators. targeting of women, who make up more than 85 tion of implementing agencies sometimes made Safeguards percent of cash recipients. dialogue and decision-making slow and complicat- In health, the guidelines were too labor-intensive ed. The World Bank is leading the social response and not well adapted for the crisis setting, leading partner coordination. Coordination was smooth, to procurement delays. but finding avenues for collaboration or joint action is often challenging with many partners providing support in similar areas. (continued) Independent Evaluation Group World Bank Group    217 218 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Philippines Addressing digitalization Involvement of nongovernment The pandemic response facilitates the implemen- Civil society organizations have been engaged tation of the government’s digital transformation in the implementation of project support for agenda in social protection by supporting: (i) the COVID-19, especially in the implementation of com- national identification system; (ii) digital payment munity-based DROM in communities. Community tools, such as mobile money, and the use of leadership developed in response to the typhoon digital identification and verification systems for was crucial in ensuring community engagement cash transfers; and (iii) fast-tracking long-standing for risk mitigation and maintaining social cohesion reforms of the business sector on digitalization. In during the pandemic. health, the World Bank will support digital tracking Iterative adjustments and monitoring of vaccine delivery. Weekly meetings between the government and World Bank task team helped identify issues and devise necessary actions to facilitate project implementation. Useful adjustments to World Bank projects have been made by fast-tracking activities and adjusting the scope of projects. Senegal Alignment with plans Leadership and coordination Country program coordination The World Bank’s response fully aligned with the The government took the initiative in implement- The Country Partnership Framework facilitated government’s plans, for surveillance; labs, testing, ing a COVID-19 response plan from early 2020, a rapid, holistic, people-centered, multisectoral and treatment; IPC; risk communication; safety which was multisectoral, included all the strategic response, which involved all GPs in the portfolio. net cash transfers; school reopening; urban and interventions recommended by WHO, and coor- National actors appreciated the technical sound- peri-urban water and sanitation; and displaced dinated by an established One Health Council and ness and speed of the support. Frequent meetings persons. The main challenge was early support to multisectoral coordinating bodies and committees, with country staff helped adjust project and ASA finance vaccines. extending to the district level. The response was fa- support and encouraged cross-project learning. cilitated by the active involvement of the president Though, the overwhelming workload took its toll in the One Health Council, and prior ASAs and proj- on staff. ect support to improve emergency preparedness. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Senegal Tailoring to needs and priorities Building on evidence and past lessons Mix of instruments The World Bank support was well tailored to Support built on experience with the Ebola A portfolio of relevant projects and ASAs already needs and priorities articulated by the govern- outbreak, with greater emphasis on consistent existed, which was adapted to address COVID-19, ment. However, more guidance and experience communication of policy to subnational levels, risk and projects drew on CERC. Projects synergized on communication and behavior change strategy communication, psychosocial care, and community support to interventions. The Health GP developed effectiveness was needed, and psychosocial sup- engagement. an emergency MPA, the COVID-19 Response port was not prominent in the response. Policy dialogue Project, in two weeks, complementing the existing Use of knowledge work Senegal’s response built on a long-standing policy health project, the REDISSE project, and sup- ASA before and during COVID-19 informed the dialogue with the government, which deepened port of the GFF. The DPF facilitated multisectoral World Bank’s response, including project restruc- though daily exchanges to advise on COVID-19 pri- coordination across GPs to support the COVID-19 turing and longer-term directions. This included orities. Dialogue and policy reform were supported national plan and address its financing gap. PEF ASA on emergency preparedness; water and through ASAs and the World Bank’s DPF. In health, funds of $1.5 million, channeled through UNICEF sanitation; adaptive safety nets; essential health the dialogue intensified the sector’s embrace and WHO, covered subnational plans services; food security assessment and monitoring of a multisectoral approach. In education, dia- Monitoring in COVID-19; education service delivery indicators; logue supported plans for immediate actions and The COVID-19 model of rapid preparation and youth employment and informal sector; digital strengthened sectoral coordination and institutional multiple adjustments during implementation led development; COVID-19 impact monitoring; and capacity. to creative working. The response focused on public expenditure reviews. ASA for behavior participatory monitoring and evaluation (M&E) and change communication would have helped. joint learning and used the MPA results framework. The government proactively prepared plans, prog- ress reports, and evaluations. On project approval, World Bank staff set up an implementation task force that met weekly to address bottlenecks. (continued) Independent Evaluation Group World Bank Group    219 220 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Senegal Addressing resilience, inclusion, sustainability, Partnerships Procurement and gender There was good coordination and joint learning Regular meetings helped track procurement and Senegal’s response is characterized by a strong between the World Bank and partners to plan and accelerated procedures. A large portion of medical focus on prevention by improving surveillance, risk track national and decentralized COVID-19 plans. supplies were procured in the first month of im- communication, laboratory and social protection The policy reform matrix for the DPF offered a plementation; and more supplies were procured systems, and resilience for future crises. Moreover, strong example of partner collaboration. Partners for a later wave of the pandemic. Procurement of the response is generating new capacities to de- provided technical and financial support in line with masks was spread across projects. The govern- liver education, health services, social assistance, their comparative advantages. Though challenges ment used its own procurement procedures for and health insurance. The response focused on arose in the cross-sectoral response where the the medical supplies; their suppliers did not have women and children’s vulnerability, women’s delineation of roles, responsibilities, and mandates fast delivery times. The government opted out of empowerment, and maternal and child health ser- was unclear. using World Bank–facilitated procurement, which vices. The leveraging of the nutrition program has Involvement of nongovernment the World Bank assessed, in retrospect, as the been an efficient way to engage communities. The nutrition program engaged the nongovern- more cost-effective option. Addressing digitalization mental sector in the response. Civil society groups Safeguards The World Bank was well prepared to support the and nongovernmental actors provided care for The MPA emergency project followed the new strengthening and scale-up of digitalization due suspected cases outside hospitals; sensitized safeguards framework, which was labor-intensive to ASAs undertaken before the crisis. Digitalization populations about social distancing; communicated for staff. Qualified consultants trained on the new has been supported in health, social protection, health and nutrition messages; targeted and distrib- framework accelerated the government’s prepa- and education, and was the focus of the DPF, uted food assistance; and strengthened community ration of documentation. The existing safeguard including digital infrastructure, e-governance, and leaders’ capacities. specialists in government were not trained. digital banking. A stand-alone operation focused Iterative adjustments on digitalization is being prepared. Health and other sector teams met weekly with ministries and key implementers of projects, which created an iterative process for resolving challeng- es and supported ongoing learning. Examples of problem-solving included the decision to recruit contractual staff to surge resources for patient care and the provision of computing and video equip- ment to communicate between health offices. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Tajikistan Alignment with plans Leadership and coordination Country program coordination The World Bank’s support aligned with the na- Initially the government was slow to respond, Initial World Bank support focused on health and tional COVID-19 response, which divided needs though it accelerated with changes in ministerial social protection, including donor coordination among donors. In health, plans called for support leadership. Tajikistan declared itself COVID-19 free and dialogue to advise the government. Fund- to critical health services and a national task force in January 2021, which curtailed COVID-19 response ing support for the response was almost entirely for risk communication. The government and efforts, though health strengthening continues. through the MPA project. The country program partners coordinated on a vaccination plan. Social Coordination with donors to address government focused on a small range of interventions in a few protection support aligned with sector develop- needs was fragmented, but improved when the sectors, with coordination led by the country office. ment plans. health ministry was supported to develop a web- The HNP focal point was important in coordinating Tailoring to needs and priorities based platform to track donor support. health-related interventions and support. The early health response addressed critical Building on evidence and past lessons Mix of instruments health services, while longer-term preparedness The support aligns with WHO guidelines. Global Tajikistan was one of the first countries to develop capacity is a challenge. There was limited early Partnership for Social Accountability (GPSA) support MPA emergency financing in health, and AF for support to ensure continuity of essential health could be important to engage communities in vaccines. It also repurposed social protection proj- services and vaccine communication. It may be monitoring the response. ect support and ASAs. The PEF was not used, as important to assess emerging needs in education, Policy dialogue the deadline was before the government’s official such as for psychosocial support, which were not The response built on long-standing dialogue in announcement of the pandemic. The corporate part of requested support. health, social protection, and finance. In the first framework for the global MPA delayed approval of Use of knowledge work phase of the response, the World Bank provid- the first MPA and AF. ASA was used to diagnose and monitor the ed daily guidance to help develop the national Monitoring situation and to inform coordination on response response plan. The MPA project financed an adviser World Bank staff and the government frequently strategies. For example, the vaccine readiness to support coordination of the response. In social checked in to review the COVID-19 situation. For assessment framework tool was applied through a protection, early dialogue supported expansion social protection, the management information coordinated effort of government and partners. of cash transfers and adoption of a national social system setup under the social protection project en- assistance plan. Ongoing dialogue is supporting abled electronic monitoring of cash transfer receipt. expansion of performance-based financing support Geo-enabled monitoring and supervision tools were to strengthen health service delivery. useful for remote monitoring. An ASA on mobile en- gagement introduced an innovative system for SMS communications nationwide, which has been used for risk behavior and vaccine messaging. (continued) Independent Evaluation Group World Bank Group    221 222 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Tajikistan Addressing resilience, inclusion, sustainability, Partnerships Procurement and gender An existing coordination committee, chaired by The first MPA was largely used to procure medi- The early investments of the World Bank helped the World Bank, and the COVID-19 Response cal equipment and supplies. The use of BFP was improve health facilities and enabled the national Group coordinated donors. Government leader- considered, but a single contract was signed with rollout of Tajikistan’s first comprehensive national ship of donors weaker than needed strengthening, UNOPS for almost all procurement needs, and social protection plan, including a social registry resulting in imperfect information on who was doing limited procurement of medical supplies was done for decision-making on payment eligibility and what and where there are limitations and overlaps. locally. The country manager enlisted the support a comprehensive management information A World Bank partnership with Gavi, the Vaccine of the president and Ministry of Finance to provide system. Social protection specifically targeted Alliance helped share information for early actions oversight in the initial phases of the response. female-headed households with children, though on vaccines. Safeguards gender was not strongly addressed in other as- Involvement of nongovernment Waivers of safeguards requirements helped approve pects of the response. There is limited involvement of nongovernment in new projects, though all requirements needed to Addressing digitalization project support, though there are plans to include be completed to start. Safeguard advisers provided Health support addressed information hotlines training of community volunteers for vaccine mes- needed handholding. A central quality review team and SMS text messaging. There was support for saging. GPSA supported capacity development and reviewed document. using digital methods for information gathering execution of third-party monitoring of the response and monitoring. The country developed an elec- by a consortium of civil society organizations tronic aggregation system for vaccination data at Iterative adjustments the district level and electronic system to manage Based on learning during the pandemic, the AF of health supply chains. the Tajikistan Health Services Improvement Project will expand performance-based financing and capacity building for primary health-care services. Moreover, the World Bank will continue to build on social protection reforms. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Uganda Alignment with plans Leadership and coordination Country program coordination The World Bank supported all pillars of the Uganda developed a comprehensive national The country office coordinated the World Bank’s COVID-19 national response plan and financed COVID-19 response plan overseen by a multi- response, which engaged most sectors. Projects coordination at national and subnational levels. sectoral task force headed by the prime minister were adjusted across GPs, which was easier to do The response reinforced the Uganda National and sector-level committees. A similar multisector where there were long-standing relationships with Social Protection Policy and aligned with sector structure was established reflected at the regional the government. The DPO was the most collab- response plans. and district levels, with varied success. Overall lead- orative instrument. The social development team Tailoring to needs and priorities ership of the response has been strong in sectors, supported all teams to address social inclusion and The response supported relief needs in health, building on experience with Ebola, though national gender equality. though risk communication, essential health election activities delayed new projects. Mix of instruments services, and vaccine support were not strong in Building on evidence and past lessons Activation of CERC in health was immediate based the early response. In the social response, support The support aligns with WHO guidance for the on experience of Ebola, but this was the only addressed public schools. Community engage- national response plan in health. Key areas to be CERC in the portfolio. Other early support was from ment support focused on health volunteers, strengthened are surveillance, community activities, adjustments to projects, AF, GPE, PEF, DPO, and nutrition, parent groups, and farmers. Institutional essential services, and risk communication. ASA. The DPO offered multisector support to ensure strengthening support was provided, there has Policy dialogue basic utilities and expand social registration, support been support to local governments. Challenges In education, health, water, and social development, agriculture inputs, and procure medical supplies. are longer-term capacity building of surveillance, frequent and close dialogue with the government Decoupling of the PEF grant from the MPA avoided citizen engagement, public-private sector service on the CERC and MPA has enabled adjustments to delays of parliamentary approval of credit. Trust funds integration, education and health systems, and previous World Bank projects to support COVID-19, allowed for immediate actions. Parliament passed preparedness coordination. ensured continued support for local government the DPF in about three months, signifying the urgen- services, and encouraged reforms. cy attached to this financing. (continued) Independent Evaluation Group World Bank Group    223 224 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Support to needs of countries Support to implementation and learning Operational processes and partnerships Uganda Use of knowledge work Partnerships Monitoring ASA informed discussions with the government The World Bank participated in the donor coordi- Frequent check-ins with project implementers and collaboration with partners and identified nation group since before COVID-19. Having a trust enabled an exchange on successes and ways to strategies for COVID-19. The World Bank support- funded staff in the World Bank office to facilitate address bottlenecks, while few indicators were ed a range of ASA on water utilities; economic up- donor coordination has been critical. The sector formally tracked on COVID-19 support. Lockdowns dates; COVID-19 impacts; results-based financing groups from before COVID-19 facilitated coordina- minimized physical monitoring of projects unless in the health sector; health information systems; tion and collaboration. Moreover, the World Bank electronic systems were already in place. Geo-en- risk communication, and essential services sup- participates in most of the committees for the abled monitoring has been useful in agriculture for ported by GFF; a vaccine readiness assessment; response strategy and provides technical guidance. remote project supervision. Studies have served as and the impact on refugees. For example, the World Bank supported ASA for the a means for monitoring the impact of the pandemic. Addressing resilience, inclusion, sustainability, risk communication strategy in collaboration with A persistent challenge has been the availability of and gender USAID. Collaboration with UNICEF in WASH has real-time data on COVID-19 cases. The World Bank supported access to essential been key to delivering rapid support for education. Procurement health services by women and children. Sup- Overall collaboration with health and other sectors The CERC and PEF have largely been used to pro- port to laboratories built systems capacity for and partners could be strengthened to improve cure medical equipment and supplies. Projects used disease testing. The redesign of the COVID-19 multisectoral support. EdTech provided collabora- their existing procurement arrangements, adopting communication strategy should help better reach tion for the remote learning response, and GFF for emergency guidelines to allow for flexible and accel- vulnerable groups, especially youth. Agriculture health services and risk communication erated procedures, including the use of e-bidding, support is improving access to nutritious food and Involvement of nongovernment direct contracting, shorter bidding periods, retroactive quality inputs. Support helped strengthen local The World Bank supported training of private health financing, and increased advanced payment. Close government services. Support to refugee and host service providers in COVID-19 case management, support of procurement specialists was important. communities targeted vulnerable households and which was a challenge in the early response. Safeguards women. The DPF helped improve systems for so- With ministries overwhelmed by COVID-19, engag- cial protection and resilience of the private sector. ing an expert to apply the new safeguards frame- In education, support helped distance learning work was impossible. Applying a new framework and school reopening, with a focus on children was labor-intensive given the many emergency with special needs. demands. (continued) Support to needs of countries Support to implementation and learning Operational processes and partnerships Uganda Addressing digitalization Iterative adjustments Support included building the capacity of the Most projects adopted new standard operating statistics agency to conduct phone surveys, health procedures to ensure safe implementation of information systems support, and geo-enabling activities. For example, training and community technology to gather monitoring data. Radio and mobilization activities were adjusted to limit social television disseminated more information. Remote contact. Weekly check-in on project implementa- learning in education has been critical. The DPF tion resulted in adjustments. For example, COVID-19 supported local governments with internet and training support was added to include the private data. sector, and the assessment of risk communication was done based on challenges identified in the communication strategy. Source: Independent Evaluation Group case study analysis. Note: The response details on each country are based on data collection synthesized from the case study protocol completed for each country through document review, interviews, and consultations with country teams. Internal validation meetings were also held to help ensure consistently in synthesizing findings across countries. ADB = Asian Development Bank; AF = additional financing; AfDB = African Development Bank; ASA = advisory services and analytics; BMGF = Bill and Melinda Gates Foundation; CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Components; CHW = community health worker; DPF = development policy financing; DPO = development policy operation; DROM = Disaster Response Operations Modality; FIRST = Fast, Innovative, and Responsive Service Transformation; GFF = Global Financing Facility; GP = Global Practice; GPE = Global Program for Education; GPSA = Global Partnership for Social Accountability; GPURL = Global Practice of Urban, Resilience, and Land; GRM = grievance redress mechanisms; HNP = Health, Nutrition, and Population; IMF = International Monetary Fund; IPC = infection prevention and control; IPF = investment project financing; JICA = Japan International Cooperation Agency; MOHFW = Ministry of Health and Family Welfare; PAHO = Pan American Health Organization; PEF = Pandemic Emergency Financing Facility; PforR = Program-for-Results Financing; PPE = personal protective equipment; SPJ = Social Protection and Jobs; SSI = Social Sustainability and Inclusion; TTL = task team leader; UN = United Nations; UNICEF = United Nations Children’s Fund; UNOPS = United Nations Office for Project Services; WFP = World Food Programme. Independent Evaluation Group World Bank Group    225 Bibliography Afrobarometer. 2021a. “COVID-19 Impact? Ugandans Grow More Discontent with Economic and Living Conditions, Afrobarometer Study Shows.” Press Release, March 30. Afrobarometer. 2021b. “Résumé des Résultats: Enquête Afrobarometer Round 8 au Sénégal, 2021.” Consortium for Economic and Social Research (CRES), Dakar, Senegal. Afrobarometer. 2021c. “Satisfaits de la Gestion de la Pandémie de COVID-19, les Sénégalais sont Sceptiques vis-à-vis des Vaccins.” Press Release, March 5. Afrobarometer. 2021d. “Ugandans Willing to Be Vaccinated Despite Doubts about The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Vaccine Safety, Afrobarometer Study Shows.” Press Release, June 4. Bau, N., G. Khanna, C. Low, M. Shah, S. Sharmin, and A. Voena. 2021. ”Women’s Well-Being during a Pandemic and Its Containment.” Working Paper 29121, National Bureau of Economic Research, Cambridge. Bautista Jr., A., D. Balibrea, and D. G. Bleza. 2020. “Knowledge, Attitude and Practice toward the Coronavirus Disease (COVID-19): Outbreak among Selected Em- ployed People in the National Capital Region, Philippines.” Asian Journal for Public Opinion Research 8 (3): 324–50. Bhattacharya, S., and S. S. Roy. 2021. “Intent to Implementation: Tracking India’s Social Protection Response to COVID-19.” Social Protection and Jobs Discussion Paper 2107, World Bank, Washington, DC. Grover, S., S. Sahoo, A. Mehra, A. Avasthi, A. Tripathi, A. Subramanyan, A. Pattojoshi et al. 2020. “Psychological Impact of COVID-19 Lockdown: An Online Survey from India.” Indian Journal of Psychiatry 62 (4): 354–62. Ipsos. 2020. “Responding to COVID-19: Highlights of a Survey in Mozambique.” Ipsos, Paris. Sumalatha, B. S., L. D. Bhat, and K. P. Chitra. 2021. “Impact of COVID-19 on Informal Sector: A Study of Women Domestic Workers in India.” The Indian Economic Journal 69 (3): 441–61. 226 Tuppal, C. P., M. M. G. Ninobla, M. G. D. Ruiz, R. D. Loresco, S. Mae P. Tuppal, I. I. Panes, R. M. F. Oducado et al. 2021. “Knowledge, Attitude, and Practice toward COVID-19 among Healthy Population in the Philippines.” Nurse Media Journal of Nursing 11 (1): 61–70. WHO (World Health Organization). 2021. Second round of the national pulse survey on continuity of essential health services during the COVID-19 pandemic: Janu- ary-March 2021. Interim Report April 23. World Bank. 2020. Honduras—COVID-19 High-Frequency Phone Surveys (HFPS) 2020, Waves 1–3. Washington, DC: World Bank. World Bank. 2021a Crisis and Recovery: Economic and Social Monitoring from Lis- tening to Tajikistan. Washington, DC: World Bank World Bank. 2021b. Djibouti—COVID-19 National Panel Phone Survey, Wave 4 (CNPPS-W4). Washington, DC: World Bank. World Bank. 2021c. Impacts of COVID-19 on Communities in the Philippines: Results from the Philippines High Frequency Social Monitoring of COVID-19 Impacts Round 2: April 8–14, 2021. Washington, DC: World Bank. UN Women. 2021. COVID-19 Rapid Gender Assessment: Gender Perspective Mozam- bique. New York: UN Women. Yin R. K. (1999). “Enhancing the quality of case studies in health services research.” Independent Evaluation Group World Bank Group    227 Health services research 34 (5 Pt 2): 1209–1224.Analysis of Country Support Types, Needs, and Implementation Status. 1 Djibouti projects reviewed: Integrated Slum Upgrading Project (P162901); Integrated Slum Upgrading Project AF (P172979); Toward Zero Stunting in Djibouti (P164164); Inte- grated Cash Transfer and Human Capital Project (P166220); Integrated Cash Transfer and Human Capital Project AF (P174566); COVID-19 Response (P173807); COVID-19 Response AF (P174675); Education Emergency Response to COVID-19 (P174128); Expanding Opportuni- ties for Learning GPE (P166059). ASA reviewed: Health System Strengthening for Universal Health Coverage and COVID-19 Response (P175615); Programmatic Poverty Work (P174572). Honduras projects reviewed: Disaster Risk Management Project CERC (P131094); Corridor Second Food Security Project (P148737); Social Protection Integration (P152057); First Fiscal Sustainability DPF (P155920); Integrating Innovation for Rural Competitiveness in Honduras IRM (P158086); Transparency Accountability and Results in Honduras (P161696); Integrating Innovation for Rural Competitiveness in Honduras II CERC (P168385); AF Social Protection The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Integration (P175718); Early Childhood Education Improvement Project (P169161); DRM De- velopment Policy Credit with a CAT DDO (P172567); COVID-19 Emergency Response Project (P173861); AF for Honduras COVID-19 Emergency Response Project (P176015); Improving Access and Governance in Health (P176532). ASA reviewed: Central America Early Childhood Development (P169033); ECD and MCH (P169202); Adaptive Social Protection: Strengthening Cash Transfers for Post Disaster Response (P1704337); Better Knowledge and Analytics to Support Governments’ Response to COVID-19 in LAC (P174597); Public Expenditure Review (P175145); Pandemic Preparedness and Response (P175274); Public Health Preparedness Assessment (P175552); CA-SPL Systems for a more Inclusive and Resilient Recovery in the Aftermath of COVID-19 (P175631); Exploratory Grant: Honduras Health (GFF support). India projects reviewed: India COVID-19 Emergency Response and Health Systems Pre- paredness Project​(P173836); Accelerating India’s COVID-19 Social Protection Response ; Strengthening Teaching-Learning and Results for States Program (P173943​and P174027)​ ; Nagaland: Enhancing Classroom Teaching and Resources​(P172213​ (P166868)​ ); Micro, Small ); Higher Education Quality Improve- and Medium Enterprises Emergency Response​(P174292​ ment Project (P150394); Technical Education Quality Improvement Project III (P154523); Uttarakhand Workforce Development Project (P154525); Jhelum and Tawi Flood Recovery Project and CERC (P154990); Nai Manzil – Education and Skills Training for Minorities (P156363); Skills Strengthening for Industrial Value Enhancement Operation (P156867); Skill India Mission Operation (P158435); Odisha Higher Education Program for Excellence and Equity (P160331); Strengthening Teaching-Learning and Results for States (P166868); Nagaland: Enhancing Classroom Teaching and Resources (P172213); Gujarat Outcomes for 228 Accelerated Learning (P173704); Mizoram Health Systems Strengthening Project (P173958); Andhra Pradesh Health Systems Strengthening Project (P167581); Meghalaya Health Systems Strengthening Project (P173589); Tamil Nadu Health System Reform Program (P166373). ASA reviewed: India PMJAY & Universal Health Coverage (P171432); India Technical Assistance for Health, Social Protection and Economic Response to COVID-19 (P174418); India 21st Century Health System – Health Financing Service Delivery and Public Health (P175882); Ac- celerating Direct Benefit Transfers in Low Income States (P158289); Strengthening Capability of Social Protection Delivery Chains (P167256); Knowledge sharing on innovative social pro- tection systems (P166658); India Social Registry and Socioeconomic Caste Census (P161831); Combating Poverty: Role of Safety Nets (P149391). Mozambique projects reviewed: COVID-19 Response DPO (P174152); Urban Development and Decentralization Project (P163989); Cyclone Idai and Kenneth Emergency Recovery and Resilience Project (P171040); Maputo Urban Transformation Project (P171449); Economic Linkages for Diversification (P171664); Improving Learning and Empowering Girls in Mozam- bique (P172657); Social Protection Project – Third AF and COVID-19 Response (P174783); COVID-19 Strategic Preparedness and Response Project (P175884); AFR RI-Southern Africa Tuberculosis and Health Systems Support Project (P155658); Primary Health Care Strength- ening AF (P168314); Urban Sanitation Project (P161777). Philippines projects reviewed: KALAHI-CIDSS National CDD-Additional Financing and COVID-19 Response (P161833); Promoting Competitiveness and Enhancing Resilience to Natural Disasters Sub-Programs 2 DPF (P170914); COVID-19 Emergency Response Project (P173877); COVID-19 Emergency Response Project AF (P175953); Beneficiary FIRST (Fast, Innovative, and Responsive Service Transformation) Social Protection Project (P174066); Independent Evaluation Group World Bank Group    229 Emergency COVID-19 Response DPF (P174120). ASA reviewed: Philippines: The Future of Jobs (P173234); Strengthening Support for the Pantawid Program and Philippines Social Protection Systems (P173380); Real Time Monitoring of COVID-19 Impacts in the Philippines (P174356); Poverty & Inequality in the Philippines: Past, Present & Perspectives for the Fu- ture (P174861); Strengthening Local Health Systems for UHC (P175650). Senegal projects reviewed: Regional Disease Surveillance Systems Enhancement (REDISSE) (P154807); Investing in Maternal Child and Adolescent Health (P162042); COVID-19 Re- sponse (P173838); COVID-19 Response AF (P175992); Quality and Equity of Basic Education (P133333); Quality and Equity of Basic Education AF (P163575); Investing in the Early Years for Human Development (P161332); Safety Net (P133597) and Safety Net AF (P173214); Youth Employability (P167681); Multisectoral Structural Reforms DPO3 (P170366); Multisectoral Structural Reforms Supplemental Financing DPO3 AF (P173918); Equitable and Resilient Recovery (P172723); Saint-Louis Emergency Recovery and Resilience (P166538); Saint-Louis Emergency Recovery and Resilience AF (P170954); Municipal Solid Waste Management Proj- ect (P161477); Rural Water Supply and Sanitation (P164262); West Africa Agricultural Pro- duction Program (P153419); Sahel Irrigation Initiative Support (P154482); WAAPP Support to Groundnut Value Chain (P158265); Agriculture Livestock and Competitiveness (P164967). ASA reviewed: Poverty Monitoring (P164474); UHC and Pandemic Preparedness (P164017); Public Expenditure Review (P170349); Digital Development (P171740); Adaptive Social Protection (P174074); Poverty Assessment (P173204); Strengthening Data and Knowledge on Poverty in Sub-Saharan Africa (P172791); Water Security in Senegal (P172233); Education Service Delivery Indicators; and analytic work supported by the Global Financing Facility to assess basic services quality and continuity; Agriculture Sector: food security assessment in COVID context. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix C Tajikistan projects reviewed: Social Safety Net Program (P122039); Health Systems Improvement (P126130) Emergency COVID-19 (P173765). ASA reviewed: Listening to Tajikistan (P171950); Strengthening Tajikistan’s Health System for Women, Children and Adolescents (P172002); KTF: Mobile Engage (P173327); Agri-food sector and Public Expen- diture Review (P174499); Third Party Monitoring of Tajikistan Emergency COVID-19 Project (P175904). Uganda projects reviewed: Uganda Reproductive, Maternal and Child Health Services Im- provement Project CERC and AF (P155186, P174163); COVID-19 Economic Crisis and Recov- ery DPF (P173906); COVID-19 Emergency Education Response Project (P174033); Secondary Education Expansion Project (P166570); COVID-19 Response and Emergency Preparedness Project (P174041); Laboratory Networking Project (P111556); Building Resilient Commu- nities (P173818); Third Northern Uganda Social Action Fund (P149965); Multisector Food Security and Nutrition and AF (P149286, P176878); Skills Development Project (P145309); Intergovernmental Fiscal Transfers Program and AF (P160250, P172868); Integrated Water Management and Development Project (P163782); Irrigation for Climate Resilience Project (P163836); Development Response to Displacement Impacts Project (P152822); Agriculture Cluster Development Project (P145037). ASA reviewed: Strengthening Social Protection Systems in Uganda (P175018); Continuity of Essential Health Services (GFF); Uganda Umbrel- la Authorities (Global Water Security Program of Water GP Trust Fund); Monitoring COVID-19 Impacts (P175978); Uganda Economic Update (P174884); COVID-19 health service disrup- tions (P176692).  230 Appendix D. Review of Country Situations: Analysis of Country Support Types, Needs, and Implementation Status This appendix analyzes the World Bank’s early health and social response in countries on its, support to COVID-19 needs and satisfactory implementation. Data and Methodology The analysis combined publicly available data on indicators with the eval- uation’s portfolio data to estimate how well COVID-19 responses aligned with the health and social needs of countries, and to examine features of the portfolio that facilitated or hindered satisfactory implementation. Public- ly available data on indicators related to country situations were used to estimate the needs for each area of the health and social response framework for COVID-19 (table D.1). Indicators were identified to estimate the needs of countries before the COVID-19 response and during the early response, and categorized in quartiles by reviewing the spread of data across countries, Independent Evaluation Group World Bank Group    231 with the best situation the fourth quartile and the worst situation the first quartile.1 For areas with multiple indicators, principal component analysis and composite measures were constructed for each area (Howe et al. 2008; Pirani 2014).2 The portfolio data included information on support in health, social protection, child welfare and social services, community engagement, institutional strengthening, operational instruments, implementing ac- tors, and project implementation status from between February 2020 and April 2021. Data on human capital support between FY15 and FY20 before COVID-19 from a forthcoming Independent Evaluation Group (IEG) evalua- tion was also integrated in the Excel data set (World Bank, forthcoming). Table D.1. Indicators Related to Country Situations Area Country-level indicators Situation before COVID-19 Preparedness Laboratory capacity—laboratories set up to identify IHR 2017 capacities infectious agents and hazards to deliver Human resources capacity—trained persons in place for critical health public health emergencies, including nurses, midwives, services for IHR 2017 physicians, epidemiologists, laboratory specialists, and epidemics communication specialists, among other expertise. Surveillance capacity—systems for rapid detection and IHR 2017 response to public health risks Health Access to health care score—captures health expenditures, Inform services vaccinations, health personnel per population, and maternal 2020 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D capacity mortality Risk communication capacity—systems to communicate Risk commu- risks and promote community resilience to cope with a IHR 2017 nication public health emergency Socioeconomic vulnerability score—captures Human Capital Inform Index, Multidimensional Poverty Index, Gender Inequality 2020 Protect Index, income inequality, development assistance, and poor and remittances vulnerable Vulnerable groups score—captures refugees, displaced Inform persons, women and girls subjected to violence, and 2020 persons with HIV, malaria, tuberculosis, neglected tropical diseases, and undernourishment and dietary adequacy Population in multidimensional poverty, age 0 to Oxford Child welfare 9—captures acute deprivations in health, education, 2010–18 and social and living standards services UNSD Birth registration—coverage of birth registration 2001–19 Gender Inequity score—considering Gender Inequality Index and Inform equality income inequality 2020 COVID-19 vulnerability score—composite measure based on Community 10 indicators: air transport, tourism, IHR capacity, points of Inform vulnerability entry, access to cities, road density, literacy, mobile cellular 2020 subscriptions, internet use, and trust in government Inform awareness score—captures literacy, mobile cellular Inform Digitalization subscriptions, and internet use 2020 Urban COVID-19 hazard and exposure score—composite measure health risk based on eight indicators: population density, urban pop- Inform ulation, household size and type, access to sanitation and 2020 drinking water, and open defecation (continued) 232 Area Country-level indicators Situation before COVID-19 Health epidemic Health capacity specific to epidemic response—average of Inform response 13 IHR core capacity score and operational readiness index3 2020 capacity Situation during early COVID-19 response Average stringency of prevention measures—composite Government measure based on four indicators of prevention: gathering Oxford responsive- restrictions, testing policy, contact tracing, facial coverings, 2020–21 ness averaged between February 2020 and April 2021 Average community transmission—transmission Disease WHO classification of the spread of COVID-19, averaged between situation 2020–21 February 2020 and April 2021 COVID-19 poverty increase projection—regional estimate of World increase in poverty (less than $1.90 per day per capita) Bank Social in 2021 from baseline projections before the pandemic 2020 situation Average economic support—measure of income support Oxford services and debt relief 2020–21 Total school closures—total weeks of full or partial school UNESCO closures between February 2020 and April 2021 2020–21 Source: Lakner et al. 2020; Thomas et al. 2021; Poljanšek, Vernaccini, and Marin Ferrer 2020; UNESCO 2021; UNSD 2021; WHO 2021. Note: Inform = Inform COVID-19 Risk Index; IHR = International Health Regulation; Oxford = Oxford COVID-19 Government Response Tracker; UNESCO = United Nations Educational, Scientific and Cultural Organization; UNSD = United Nations Statistics Division; WHO = World Health Organization. Having constructed an integrated country-level data set, the following Independent Evaluation Group World Bank Group    233 analyses were conducted using Python, Stata, and Excel, with visualizations in Tableau: » Machine learning clustering analysis to understand the types of World Bank support planned among countries (Caliński and Harabasz 1974; Davies and Bouldin 1979; Handl and Knowles 2007). The algorithm reviewed the port- folio data and clustered countries that showed similar patterns of support in addressing country situations.4 Principal component analysis was applied across the clusters, reducing the variables to produce meaningful groupings of data for interpretation. The assessment tested three clustering models.5 The final analysis used a hierarchical clustering algorithm that was compared with other tested algorithms.6 » Heat map of the alignment of World Bank support with country needs and previous human capital support (table D.3). The heat map shows the level to which the World Bank responded to country needs before COVID-19, and the extent that having previous human capital support aided in addressing of country needs during COVID-19. The analysis looked at countries with indicators in the bottom two quartiles, in terms of having a defined need be- fore COVID-19, and the proportion of these needs that were addressed by the support of World Bank projects in the country portfolio. » Decision tree analysis at the country level to understand portfolio features that facilitated or hindered satisfactory project implementation in the early COVID-19 response (Kam Ho 1995; Schapire 2013). The outcome variable used to construct the decision tree analysis was the proportion of projects The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D focused on COVID-19 in a country portfolio with satisfactory (satisfactory or highly satisfactory) implementation progress ratings. The decision tree was constructed using AdaBoost, which identifies a series of trees after testing several models and uses a tenfold cross-validation.7 The final features in the model were selected using the backward feature selection, a step-by-step approach. Finally, a set of eight features that produced the best score were selected to identify those that were important to satisfactory project imple- mentation in countries. Types of Support Planned by the World Bank in Countries The machine learning clustering analysis suggests that prioritizing World Bank support to needs was more challenging in some countries. The anal- ysis identified five clusters of countries (figure D.1) that can be grouped into three types based on the level to which World Bank support prioritized country needs. All countries had core World Bank support to expand critical health services and social protection. Better prioritization of World Bank support to address needs is seen in countries with higher average epidem- ic response capacities before COVID-19 and capacities to deliver essential health services, and early government responsiveness to put in place preven- tion measures, which is also supported by findings from case studies con- 234 ducted for the evaluation. Case studies suggest a progressive prioritization of the response in some countries to better address needs and reach vulnerable groups. World Bank support was challenging to prioritize in countries with lower health services capacities, slower average early government respon- siveness, and extensive needs to address the health and social threats of COVID-19. » About 11 percent of countries fall into clusters that had high government responsiveness or previous preparedness to coordinate and deliver critical health services, and focused World Bank support to prioritize needs. About 8 percent of countries (cluster 1) undertook a focused response with a higher intensity of interventions on laboratories, vaccination, and social cohesion, drawing on government leadership and previous crisis experience. Two percent of countries (cluster 2) developed a multisectoral response with increased intensity of engagement across levels of government and reach to the community to address a range of needs. These countries also undertook more advisory services and analytics relative to other countries to inform needs, and had some preparedness to deliver critical health services be- fore COVID-19. One percent of countries (cluster 3) had a high focus on the social response with a high degree of reorientation of the country portfolio to address needs across sectors, while responding to the high impacts of the COVID-19 crisis. In all three clusters, governments had medium to high aver- age responsiveness to act on COVID-19 measures, and medium to high aver- Independent Evaluation Group World Bank Group    235 age capacity to deliver essential and critical health services before COVID-19. » About 53 percent of countries (cluster 4) had high average capacities to deliver health services before COVID-19, and focused World Bank support on priorities in a few areas to address needs. These countries often had fewer pre–COVID-19 needs. However, they also often faced a higher early impact of the COVID-19 crisis and may face increasing needs in the future. » About 36 percent of countries (cluster 5) had extensive needs and limited capacities to deliver health services, making prioritizing support to address needs challenging. These countries often had low levels of human capital and extensive health and social development needs before COVID-19. The key in these countries was protecting against development losses, and early govern- ment responsiveness to meet prevention needs was often low. Case studies from the evaluation suggest opportunities to progressively prioritize support to better focus on needs and vulnerable groups. Even with pre–COVID-19 preparedness there were often vast needs across sectors, and limited capac- ities to deliver health services. Regional project support and multisectoral coordination may help countries to strengthen their response capacities (as seen in Senegal), though reinforcing health systems to deliver essential health services may be an important part of preparedness in these countries.  ypes of World Bank COVID-19 Support to Countries Figure D.1. T The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D Source: Independent Evaluation Group. Note: Cluster 1 (purple) = focused response. A cluster of seven countries: Republic of Congo, Djibouti, Ghana, Honduras, Kenya, Sierra Leone, and Togo. Cluster 2 (gold) = comprehensive multisector re- sponse. A cluster of two countries: Senegal and Pakistan. Cluster 3 (blue) = high social support, institu- tional strengthening, and responsiveness of government. India is the sole country in this cluster. Cluster 4 (green) = high-capacity countries. Large cluster of 48 countries: Albania, Bangladesh, Belarus, Bhutan, Bolivia, Bosnia and Herzegovina, Cabo Verde, Cambodia, Colombia, Comoros, Dominica, Ecuador, El Salvador, Eswatini, Fiji, Gabon, Georgia, Guatemala, Haiti, Indonesia, Iraq, Jordan, Kiribati, Lao People’s Democratic Republic, Lebanon, Maldives, the Marshall Islands, Moldova, Morocco, Myanmar, Nepal, Nicaragua, Panama, Philippines, São Tomé and Príncipe, the Solomon Islands, Sri Lanka, St. Lucia, Suriname, Tajikistan, Trinidad and Tobago, Tunisia, Türkiye, Tuvalu, Ukraine, Uzbekistan, and Vietnam. Cluster 5 (orange) = broad support with high needs. Main cluster of 32 countries: Afghanistan, Angola, Benin, Burkina Faso, Burundi, Cameroon, Central African Republic, and Chad, Côte d’Ivoire, Demo- cratic Republic of Congo, Ethiopia, The Gambia, Guinea, Lesotho, Liberia, Madagascar, Malawi, Mali, Mauritania, Mozambique, Niger, Nigeria, Papua New Guinea, Rwanda, Somalia, South Sudan, Tanzania, Timor-Leste, Uganda, Vanuatu, Republic of Yemen, and Zambia. Boldface indicates that the country is a case study in the evaluation. N = 90 countries. ASA = advisory services and analytics; ISR = Implementa- tion Status and Results Report. 236 Findings of Country Needs Analysis The World Bank’s support for COVID-19 addressed most emergency needs related to critical health services, coordination, social protection, and digita- lization, with challenges in other areas (figure D.2a). Challenges arose in ad- dressing needs related to essential health services, community engagement, and urban public health support. About 45 percent of countries addressed needs to a high level (figure D.2b). Figure D.2. Alignment of Project Support with Needs of Countries a. Alignment of World Bank country support to needs by response area b. Extent of alignment of World Bank country support to needs Source: Independent Evaluation Group portfolio. Independent Evaluation Group World Bank Group    237 Note: Figure a shows the percent of countries with needs in the bottom two quartiles that received at least one World Bank intervention in that area. Interventions are based on the analysis of 203 projects coded for the evaluation in 89 countries that had data on needs and World Bank support. Red shad- ing indicates that needs were addressed in less than 50 percent of countries. Gray shading indicates that needs were addressed in 50 percent or more of countries. Figure b shows the overall alignment of country support by quartile. Very high = more than 80 percent alignment between World Bank interventions in the portfolio and needs; high = 66.6 to 80 percent alignment; low = 33.3 to 66.6 percent alignment; very low = 0 to 33.3 percent alignment. Analysis of Previous Support to Human Capital Countries that previously had high levels of support to human capital more consistently addressed health and social needs during the early response. About 79 percent of countries in the portfolio had high or very high levels of World Bank support to human capital in health, social protection, or education before COVID-19. About 52 percent of these countries had early COVID-19 responses that addressed health and social needs at high or very high levels, compared with 46 percent of these countries that had low or very low previous support to human capital (figure D.3). Having developed this support before COVID-19 helped prepare the countries to flexibly respond to needs related to the crisis.  ddress of COVID-19 Response Needs Based on Prior Figure D.3. A The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D Human Capital Support Source: Independent Evaluation Group needs analysis. The human capital data were coded as part of World Bank forthcoming IEG assessment. Note: The extent of human capital support before COVID-19 is based on interventions in projects before COVID-19, before February 1, 2020, in the Health, Nutrition, and Population; Social Protection and Jobs; and Education Global Practices.8 Interventions during COVID-19 are defined as the quantity of interven- tions per country in an area in the highest two quartiles of their distribution across countries. A country need is defined as the baseline indicator of an area in each country falling in the bottom two quar- tiles of the indicator’s distribution across countries. Extent of support to interventions to address needs during COVID-19: Very low: 0 percent – 33 percent, Low: 33 percent – 67 percent, High: 67 percent – 80 percent, Very high: More than 80 percent. Extent of human capital support before COVID-19: Very low: 0 projects – 1 projects, Low: 2 projects – 3 projects. High: 4 projects – 7 projects, Very high: 8 projects – 20 projects. N = 78 countries. Support to Satisfactory Implementation at a Country Level The decision tree analysis identifies a mix of country portfolios features that appear important to satisfactory implementation progress (satisfactory or highly satisfactory). The importance of each feature in the decision tree is proportional to the average decrease in impurity, or the explanatory val- ue-added by the variable to correctly predict the classification of the imple- 238 mentation status of a project. Features on higher nodes in the decision tree have more explanatory value. All features are independently important, but together they make it more likely for projects to have a satisfactory imple- mentation progress. The key features important to satisfactory implementa- tion progress are shown in table D.2 in the order of their explanatory value as predictors in the model. The coverage of these features in country portfo- lios points to opportunities for improving future World Bank crisis respons- es, especially in terms of better support to monitoring, citizen engagement, essential health services, gender, and urban health, which were less frequent in countries. The findings also point to the value of investing in prepared- ness in countries with greater needs and lower capacities such that they can better equip for crisis response. Figure D.4 shows that only about 40 percent of countries had high or very high coverage of most of these features in their support to the COVID-19 response. Table D.2.  Country Portfolio Features Facilitating Satisfactory Implementation of Early COVID-19 Health and Social Response in Countries Feature of country support Findings of decision tree Monitoring and evidence of Countries with a higher frequency of World Bank support early progress (Country cover- to monitoring of the COVID-19 response (in the top two age: 74 percent had indicators quartiles) and early evidence of progress were more likely monitored, and 64 percent had to have projects with satisfactory implementation prog- indicators that showed early ress: 30 percent of explanatory value, with an additional Independent Evaluation Group World Bank Group    239 evidence of progress) 10 percent if there was early evidence of progress. In early evidence, the key was having indicators that could provide routine information on project implementation. Notably, this support was important for all countries. Support to citizen engagement, Countries with World Bank support to COVID-19 in essential health services, and citizen engagement, essential health services, and laboratories (Country coverage: laboratories were more likely to have projects with 43 percent, 41 percent, and 71 satisfactory implementation progress: each intervention percent) added an explanatory value of 12 percent. Notably, this support was important for all countries irrespective of their needs or situation before COVID-19. This finding is consistent with the case studies and evidence from the literature, which point to the importance of having health service capacity in place during a crisis, and demand-side engagement of communities. Moreover, the importance of access to laboratory testing is seen in past lessons and the regional project analysis. (continued) Feature of country support Findings of decision tree Address gender equality This finding suggests that World Bank support to (Country coverage: 64 percent gender in the early COVID-19 response was important had support to address gender for all countries, but especially in countries with greater equality needs in the early needs related to gender equality. Better addressing COVID-19 response) of gender equality needs in the country portfolio (top quartile) made it more likely to have projects with satis- factory implementation progress: 10 percent explanato- ry value was added. Address urban health risks The findings suggest that in countries with higher urban (Country coverage: 24 percent health risks targeted support in this area was important in had urban support in the early the COVID-19 response. Better addressing urban health COVID-19 response) risks for the spread of COVID-19 in a country (top two quar- tiles) made it more likely to have projects with satisfactory implementation progress during the early COVID-19 response: 8 percent explanatory value was added. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D Preparedness capacities to The findings suggest that supporting critical health deliver critical health services services was important among countries with lower (Country coverage: 49 percent preparedness. Countries better prepared to deliver had better preparedness to critical health services before COVID-19 (top quartiles) delivery critical health services were more likely to have projects with satisfactory im- (top two quartiles); 91 percent plementation status: 6 percent explanatory value was of countries with lower pre- added by having preparedness capacities in critical paredness to delivery critical health services before COVID-19. health services (bottom two quartiles) received support in the early COVID-19 response to address this need) Source: Independent Evaluation Group decision tree analysis. Note: Percentages reported for critical health services, gender inequality, and urban health risk measure the extent to which World Bank support in the respective area was aligned with a country’s needs in that area. A need is defined as the underlying needs variable in an area falling in the bottom 50 percent of its distribution across countries. IHR = International Health Regulations. 240  overage of Features Supporting Satisfactory Figure D.4. C Implementation in Countries Source: Independent Evaluation Group portfolio. Note: The levels relate to quartiles of the share of features supporting satisfactory implementation in countries. Very low = 0 to 37.5 percent; low = 37.5 to 50 percent; high = 50 to 75 percent; very high = more than 75 percent. N = 97 countries. Independent Evaluation Group World Bank Group    241 242 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D Table D.3. Abridged Heat Map of Pre–COVID-19 Needs and COVID-19 Response in Countries Human capital Country Reorientation COVID Innovations Baseline needs support before and region for COVID support for COVID COVID Essential Critical Assure child Human capital Extent of Protect the Alignment Reported Country health health welfare and support before portfolio vulnerable with needs innovations services services social services COVID-19 adjustment East Africa regional average: High High High - worst (1st better (3rd worse (2nd worst (1st Angola Very high Very low High Yes quartile) quartile) quartile) quartile) worst (1st worst (1st worst (1st worst (1st Burundi Low Low Low Yes quartile) quartile) quartile) quartile) worse (2nd worst (1st worse (2nd worse (2nd Comoros Low High Low Yes quartile) quartile) quartile) quartile) Congo, worst (1st better (3rd worst (1st worst (1st Democratic Very high High Very high Yes quartile) quartile) quartile) quartile) Republic of worst (1st best (4th worst (1st worst (1st Ethiopia Very high Very high High Yes quartile) quartile) quartile) quartile) worst (1st worst (1st worse (2nd worse (2nd Kenya Very high Very high Very high Yes quartile) quartile) quartile) quartile) worst (1st worst (1st worst (1st worst (1st Lesotho Low Very low Very high Yes quartile) quartile) quartile) quartile) worst (1st worst (1st worst (1st worst (1st Madagascar Low High Very high Yes quartile) quartile) quartile) quartile) better (3rd better (3rd worst (1st worst (1st Malawi Very high Very low Very low Yes quartile) quartile) quartile) quartile) (continued) Human capital Country Reorientation COVID Innovations Baseline needs support before and region for COVID support for COVID COVID Essential Critical Assure child Human capital Extent of Protect the Alignment Reported Country health health welfare and support before portfolio vulnerable with needs innovations services services social services COVID-19 adjustment Mozam- better better (3rd worst worst Very high High Very high Yes bique 3rd quartile) quartile) (1st quartile) (1st quartile) worse better (3rd worst worst Rwanda Very high High Very high Yes (2nd quartile) quartile) (1st quartile) (1st quartile) São Tomé worse worst (1st better better and Very low High Low No (2nd quartile) quartile) (3rd quartile) (3rd quartile) Príncipe worse worst (1st worst Somalia (2nd Very low Low High No quartile) (1st quartile) quartile) South worst worst (1st worst worst Very low Low Very low No Sudan (1st quartile) quartile) (1st quartile) (1st quartile) worse worst worst (1st worst (1st Tanzania (2nd Very high High Very low Yes (1st quartile) quartile) quartile) quartile) worse worst worst worst Uganda (2nd High Very high Very high Yes (1st quartile) (1st quartile) (1st quartile) quartile) worse worst (1st worst worst Zambia Very low High Very high Yes (2nd quartile) quartile) (1st quartile) (1st quartile) (continued) Independent Evaluation Group World Bank Group    243 244 The World Bank’s Early Support to Addressing COVID-19: Health and Social Response  Appendix D Human capital Country Reorientation COVID Innovations Baseline needs support before and region for COVID support for COVID COVID Essential Critical Assure child Human capital Extent of Protect the Alignment Reported Country health health welfare and support before portfolio vulnerable with needs innovations services services social services COVID-19 adjustment West Africa regional average: High High High - worst (1st worst (1st worse (2nd worst (1st Benin High Low Very high Yes quartile) quartile) quartile) quartile) worse Burkina worst (1st worst (1st worst (1st (2nd quar- Very high High Very high Yes Faso quartile) quartile) quartile) tile) better (3rd better (3rd better (3rd worse (2nd Cabo Verde Low Very low Low No quartile) quartile) quartile) quartile) worst (1st better (3rd worst (1st worst (1st Cameroon High High Low Yes quartile) quartile) quartile) quartile) Central worst (1st worst (1st worst (1st worst (1st African Very low High Very high Yes quartile) quartile) quartile) quartile) Republic worst (1st worst (1st worst (1st worst (1st Chad High Low Low Yes quartile) quartile) quartile) quartile) Congo, worst (1st worst (1st worst (1st worse (2nd Low High Low Yes Republic of quartile) quartile) quartile) quartile) Côte worst (1st best (4th worst (1st worst (1st Very high High High Yes d’Ivoire quartile) quartile) quartile) quartile) (continued) Human capital Country Reorientation COVID Innovations Baseline needs support before and region for COVID support for COVID COVID Essential Critical Assure child Human capital Extent of Protect the Alignment Reported Country health health welfare and support before portfolio vulnerable with needs innovations services services social services COVID-19 adjustment worse (2nd worst (1st worse (2nd worse (2nd Gabon Low Very low Very low No quartile) quartile) quartile) quartile) Gambia, worst (1st worst (1st worse (2nd worst (1st High High Low Yes The quartile) quartile) quartile) quartile) worse worse (2nd worse (2nd worse (2nd Ghana (2nd quar- Very high High Very high Yes quartile) quartile) quartile) tile) worst (1st best (4th worst (1st worst (1st Guinea High High High Yes quartile) quartile) quartile) quartile) worst (1st best (4th worst (1st worst (1st Liberia Very high Low Very high Yes quartile) quartile) quartile) quartile) worst (1st worst (1st worst (1st worst (1st Mali High Low Low Yes quartile) quartile) quartile) quartile) worst (1st worst (1st worse (2nd worst (1st Mauritania Very low High Very high Yes quartile) quartile) quartile) quartile) worst (1st better (3rd worst (1st worst (1st Niger High Low High Yes quartile) quartile) quartile) quartile)