Approach Paper Preparedness for Epidemics and Pandemics: Evaluation of the World Bank Group’s Support June 3, 2025 1. Background and Context 1.1 This evaluation focuses on the preparedness of the World Bank Group, including the World Bank and the International Finance Corporation (IFC), to support epidemic- and pandemic-related health emergencies. A health emergency is defined as an extraordinary event that is determined to constitute a public health risk to other states through the international spread of disease and to potentially require a coordinated international response (WHO 2005). While health emergencies can include natural disasters or humanitarian crises that impact the health of populations, the evaluation centers on the preparedness—including prevention, detection, and readiness—to act quickly in health emergencies whose origins or sources are related to infectious disease outbreaks and antimicrobial resistance. 1.2 Globally, the increasing risk of outbreaks and epidemics that could escalate into pandemic health emergencies underscores the need for countries to be better prepared. The risk of emerging diseases is increasing by more than 6 percent annually, and the number of disease outbreaks with the potential to spread has grown to several hundred per year since 2000 (World Bank 2022c). The likelihood of another pandemic causing mortality and morbidity similar to or potentially worse than COVID-19 in the next 25 years is estimated to be nearly 70 percent (World Bank 2024a). Linked to this, there are growing threats, for example, related to biosecurity and antimicrobial-resistant pathogens that are becoming increasingly difficult to treat. Technology also presents new opportunities for addressing risks. 1.3 A strong development and economic rationale exists for investing in preparedness to reduce the risk of outbreaks and epidemics. As demonstrated by the COVID-19 pandemic, future pandemics could negatively affect development priorities outlined in the Sustainable Development Goals and social and political stability, which could reverse hard-won development gains. Investing in preparedness actions could help reduce the social and economic impacts of future epidemics and pandemics. The cost of preparedness is estimated to be less than 1 percent of the devastating financial toll of a pandemic. For example, the costs of COVID-19 are estimated to be over US$13 trillion, and the economic and social cost of the Ebola outbreak in West Africa in 2014 is estimated at US$53 billion (Huber et al. 2018; IMF 2022; World Bank 2022c). The Democratic Republic of Congo’s 10th Ebola outbreak, between 2018 and 2020, cost 1 approximately US$1.2 billion, more than the entire health budget for the country (Zeng et al. 2023). The Bank Group estimates that antimicrobial-resistant pathogens could result in US$1 trillion in extra health care costs by 2050 and up to US$3.4 trillion in annual gross domestic product losses by 2030 (Jonas et al. 2017). Although investing in preparedness carries an up-front cost, when done effectively, it can institutionalize capacities to de-risk investments in countries. 1.4 According to the Global Health Security Index—which measures 195 countries’ preparedness and capacity to respond to epidemics and pandemics—all countries are dangerously unprepared for epidemics and pandemics, with less developed countries being particularly vulnerable because of their weak systems. During COVID-19, the coordination and communication of decentralized actions to prevent and manage disease risks in communities was a key challenge (Bell and Nuzzo 2021; World Bank 2022d, 2024a). COVID-19 showed that countries with weaker capacities—for example, in surveillance, community protection, governance, environmental measures, health service delivery, and the supply of vaccines and medicines—were ill-equipped to respond effectively (GPMB 2024; World Bank 2022d). In Africa, for example, over 50 reported zoonotic outbreaks in 2022 had the potential to escalate into significant epidemics. The Global Preparedness Monitoring Board (2023) emphasizes the importance of prioritizing World Bank support to less-prepared countries that have a higher risk of infectious diseases and could become sources of future pandemics. The World Bank Group’s Efforts Toward Health Emergency Preparedness 1.5 Health emergency preparedness for future epidemics and pandemics is one of the Bank Group’s global challenge areas and a top priority within its crisis response framework. The crisis response framework recognizes that infectious disease–related emergencies cannot be treated as surprises, especially in a rapidly changing world (World Bank 2022b). In response, the Health Emergency Prevention, Preparedness, and Response (HEPPR) Global Challenge Program, which aims to reduce the risk of future health emergencies, was launched in May 2024 (World Bank 2024a). This program seeks to address the recurring cycle of “panic and neglect” that often follows epidemics and pandemics, when political commitment and financing for continued efforts wane. Moreover, not being sufficiently prepared for crises hits poor people and the most vulnerable the hardest. 1.6 Since COVID-19, the Bank Group has focused on scaling up its support to ensure preparedness for epidemics and pandemics across countries. These efforts emphasize developing essential One Health and public health capacities, as well as strengthening health systems and ensuring access to countermeasures. The Bank Group organizations—including the World Bank, IFC, and the Multilateral Investment 2 Guarantee Agency—are collaborating to help countries build more resilient health systems, including by strengthening emergency-ready health services, public health services, and medical supply chains. Scaled-up efforts of the Bank Group across countries emphasize actions across sectors; use of data, knowledge, and technology; collaboration with partners, including greater involvement of the private sector; and multilevel actions at the regional, cross-border, country, and decentralized levels to help strengthen institutional capacities and outcomes to enhance preparedness for epidemics. 1.7 A mix of financing instruments supports the scale-up of epidemic and pandemic preparedness. These instruments include the Pandemic Fund; International Development Association and International Bank for Reconstruction and Development financing for analytic work; IFC investments and advisory support; the Health Emergency Preparedness and Response Umbrella Program; the Global Environment Facility’s support for One Health; regional programs, such as the Africa Centres for Disease Control and Prevention and an expanding network of regional projects focused on One Health that involve multiple sectors; and country diagnostics to prioritize preparedness actions. In 2024, the World Bank introduced a Crisis Preparedness and Response Toolkit to facilitate rapid financing during crises. One purpose of the toolkit is to increase the amount of ex ante financing available in a Bank Group country portfolio, which could be activated or drawn on in a crisis, including a health emergency. The recent Independent Evaluation Group (IEG) COVID-19 evaluation found that swift financing proved challenging during COVID-19, often due to the limited flexibility to move around and redirect financing to support emergency needs (World Bank 2022d). The results and sustainability of ongoing interventions, partnerships, private sector engagement, and cross-sector collaboration are critical to ensuring that scaled-up efforts build on what works and support institutional capacities for better preparedness and resilience in future epidemics and pandemics. 1.8 The ongoing scale-up of Bank Group preparedness support builds on decades of experience, albeit on a smaller scale. World Bank support over the years has included efforts for river blindness, HIV/AIDS, dengue fever, severe acute respiratory syndrome, avian influenza, Middle East respiratory syndrome, Ebola, Zika virus, COVID-19, and other diseases. Support has often focused on response for control of an epidemic, but initiatives across sectors since the early 2000s have also strengthened systems for preparedness. For example, projects have included support for public health institutes, veterinary services for animal health, wildlife disease monitoring, water disease monitoring, workforce development, disease surveillance systems, health service system capacities to address disease risk, and supply chains for drugs and vaccines, although the efforts have been fragmented. A series of regional projects have supported preparedness, such as the East Africa Public Health Laboratory Networking Project 3 (2010–21), 3A-West Africa Disease Surveillance (2012–17), Southern Africa Tuberculosis and Health Systems Support Project (2016–24), and Regional Disease Surveillance Systems Enhancement (2016–23). The 18th Replenishment of the International Development Association supported projects to develop pandemic preparedness plans and strengthen multisectoral health emergency preparedness (Glassman et al. 2018; Tichenor and Sridhar 2017). Figure 1.1 shows a timeline of Bank Group support for epidemics and how this support has aligned with the global events and the actions of other development partners. 1.9 Private sector engagement also plays a critical role in epidemic and pandemic preparedness. In some developing countries, the private sector constitutes up to 80 percent of health service capacities. It fosters innovations like digital solutions that can enhance the effectiveness and reach of a response. The private sector can also play a role in research and development, manufacturing, and distribution of medical countermeasures, as well as in livestock and food industries, which are critical for One Health and preventing antimicrobial resistance (World Bank 2024a). Examples of how IFC advisory support has contributed to preparedness include efforts to improve the quality of veterinary and health services, laboratories, food systems, and supply chains, such as for active pharmaceutical ingredients for vaccines. Examples of IFC investment projects that could help preparedness include support to strengthen the manufacturing and distribution of human and animal vaccines, health diagnostics, and personal protective equipment. 4 Figure 1.1. Timeline of Epidemics and Pandemics and Selected Actions Taken by the World Bank Group and Development Partners Sources: Benton 2001; Global Preparedness Monitoring Board 2023; International Working Group on Financing Preparedness 2017; Jonas et al. 2017; Sejvar 2003; Stratton et al. 2019; WHO 2021, 2023; WOAH n.d.; World Bank 2012, 2013, 2016b, 2018a, 2022a, 2022d, 2024b, 2024c. Note: CDC = Centers for Disease Control and Prevention; CEPI = Coalition for Epidemic Preparedness Innovations; GAVI = Global Alliance for Vaccines and Immunisation; GEF = Global Environment Facility; GFF = Global Financing Facility; HEPPR = Health Emergency Prevention, Preparedness, and Response; IDA = International Development Association; IFC = International Finance Corporation; JEE = Joint External Evaluation; SARS = severe acute respiratory syndrome; TB = tuberculosis; WHO = World Health Organization; WOAH = World Organisation for Animal Health. 5 1.10 The expansion of Bank Group support emphasizes core One Health and public health capacities for sustained epidemic preparedness. These capacities are needed in areas such as veterinary services, surveillance systems, community protection, and environmental measures, all of which are fundamental to managing disease risks. One Health gained prominence during the response to severe acute respiratory syndrome in the early 2000s and has been integral to Bank Group support since then, including in the Global Program for Avian Influenza Control and Human Pandemic Preparedness and Response (2006–12), which coordinated veterinary and public health actions. The Bank Group developed an operational framework for One Health in 2018. Since the One Health approach is relatively new within the Bank Group, it is expected that most of the Bank Group’s support for epidemic and pandemic preparedness has consisted of stand- alone sectoral operations, with only limited emergence of a new cohort of multisectoral One Health lending. A global action plan was developed to strengthen One Health in 2022 (FAO et al. 2022). The concept of One Health underlies the design of the HEPPR Global Challenge Program and the Bank Group’s crisis response framework (World Bank 2022b, 2024a). 2. Purpose and Audience 2.1 The evaluation’s purpose is to assess the Bank Group’s support for epidemic preparedness, which aims to help client countries prevent, detect, and ensure readiness for infectious disease outbreaks. This evaluation will provide insights into the effectiveness of past and ongoing preparedness efforts supported by the World Bank, specifically examining what works and why. These findings can inform scale-up to enhance efforts of the World Bank’s HEPPR Global Challenge Program. The evaluation will focus on the support of the World Bank and IFC. Given that the Multilateral Investment Guarantee Agency has a relatively small and predominantly new portfolio in this area, it will not be included in the evaluation. 2.2 The primary audiences for the evaluation are the Bank Group’s Board of Executive Directors and managers and operational staff supporting preparedness efforts across sectors. The secondary audiences include client countries that urgently need to increase preparedness capacities as well as development partners and practitioners engaged with the Bank Group in supporting client countries. 2.3 The evaluation team will collaborate closely with key Bank Group entities engaged in supporting country preparedness. These entities include the HEPPR Global Challenge Program; the Pandemic Fund; and staff involved in projects, partnerships, knowledge work, and other aspects of support in the client countries. To maximize the evaluation’s operational relevance, the team will maintain ongoing communications and 6 exchanges with these entities to discuss and share early findings. This continuous dialogue will facilitate the interpretation of evidence and help inform ongoing work. Conceptual Underpinning 2.4 Strengthening preparedness for epidemics and pandemics is a multisectoral agenda. Effective prevention, detection, and readiness to respond require coordinated action across human health, animal health, agriculture, the environment, and other sectors to reduce the risk of outbreaks, limit disease spread and ensure health systems are better equipped to manage future threats. The logic of the conceptual framework emphasizes that investing in scaling up core preparedness capacities—including One Health and public health services—as well as strengthening health systems and ensuring access to countermeasures, contributes to building sustained capacity and resilience against epidemics. Expected outcomes include stronger multisectoral policies and systems, improved disease prevention and detection, enhanced leadership and decision-making to mitigate risks, and readiness to scale up services rapidly to control disease spread. In the long term, support to countries aims to sustainably develop systems and policies that reduce the frequency and severity of outbreaks and epidemics, increase resilience, save lives, and lower risks and costs to society. 2.5 The One Health approach, an integrated, multidisciplinary strategy that recognizes the interconnectedness of human, animal, and environmental health, is essential for scaling core preparedness capacities. It promotes collaboration among various sectors—such as public health, veterinary medicine, environmental science, and agriculture—to prevent, detect, and ensure readiness to respond to health threats that affect humans, animals, and ecosystems. This approach emphasizes diseases caused by a pathogen spilling over from animals into humans, as these diseases are the leading cause of disease outbreaks with pandemic potential. Figure 2.1 highlights the risks of disease spread in communities due to the environment, agriculture, water, animals, and health care that One Health aims to reduce. Exposure to pathogens has increased with environmental changes in land use, migration, urbanization, and deforestation, among other challenges countries face. High-risk populations such as wildlife traders and agriculture and health care workers may be the first groups exposed to the pathogens, which can then spread wider in communities. Diseases can also spread through contaminated water systems, agriculture waste, and food, as well as when environmental measures are inadequate to protect against disease exposure. Health care systems are crucial to care for infected people in communities, but they can become transmission points if they lack adequate capacity for case identification, reporting, and infection control. Antimicrobial-resistant pathogens, which are driven by the misuse of antibiotics in agriculture and health care, complicate the treatment of disease cases and 7 efforts of the health care system to halt the spread of disease in communities (Magouras et al. 2020; UNEP 2023; World Bank 2010, 2018a, 2022c, 2023a). Figure 2.1. One Health Risks of Disease Spread Sources: Independent Evaluation Group, adapted from Magouras et al. 2020; UNEP 2023; World Bank 2010, 2018a, 2022c, 2023a. 2.6 The conceptual framework aligns with key Bank Group strategies and global initiatives on pandemic preparedness. Specifically, it is informed by Bank Group strategies, including multisector frameworks to operationalize One Health and the HEPPR Global Challenge Program (Global Preparedness Monitoring Board 2023; World Bank 2018a, 2022a, 2023a, 2024a; see figure 2.2 and box 2.1). The alignment of the conceptual framework with the approach of the HEPPR Global Challenge Program and the Pandemic Fund allows for learning from the evaluation to inform ongoing operational work as well as drawing on past evidence. The long-term perspective of the evaluation also allows for learning about changes in Bank Group support to areas of the conceptual framework over time, including since the COVID-19 pandemic. 8 Figure 2.2. Conceptual Framework of Epidemic and Pandemic Preparedness Guiding the Evaluation ous s s s ors or roup Global challenge Air borne, fecal borne, Health, agriculture, s ru s program food borne, vector environment, ater, World Bank lending Financial intermediaries borne, ater borne, and finance, digital, urban and analytics and trust funds zoonotic diseases, and IFC investment and antimicrobial resistance o r p ro advisory January 2000 through Crisis financing June 2024 r p r ss or p p r r s r o r Prevention Detection Readiness Response Recovery Emergency period or s ss o pr p r ss r ou r sur s p s s r s I STIT TIO A STRE GTHE I G for better policies, regulation and standards, systems, and multisector leadership and action plans Strengthening national and Strengthening national and Strengthening national and regional capacities for cross regional health system region pharmaceutical sectoral coordination for disease capacities for preventing capacities to manufacture prevention, detection, and outbreaks from spreading to MCM for preventing, readiness humans and delivering diagnosing, and treating One Health net orks and emergency services hen diseases in both animals and platforms needed humans One Health human resource Community based and Diversification of the capacities veterinaries, health primary health care systems production landscape for orkers, laboratory orkforce Epidemic ready primary MCM Farm safety practices, animal health care ith proper Optimizing MCM supply vaccination and disease control, infrastructure, diagnostic chain procurement, storage, appropriate use of antibiotics, tools, clinical protocols to logistics and distribution animal food trade markets and maintain service uality Investing in research and food surveillance and safe Referral systems to manage development of ne handling practices potential outbreaks technologies and treatments Wildlife management, habitat Reinforcing infection preservation and conservation prevention and control strategies practices AMR risk and vector controls in Strengthen community health ater, soil and other areas that orkers and frontline may spread disease emergency health orkforce WASH, aste management, ater Digital health solutions e treatment health, telemedicine Early arning and surveillance Monitoring of risk factors and systems, disease reporting continuity of regular health net orks service delivery aboratory net orks ithin and across countries timely diagnosis and information sharing Biosafety and biosecurity standards Public health risk communication r ross oo u r rs ps s or o s o p op r Enhanced One Health Enhanced infection Enhanced systems for orkforce control e uitable access to Enhanced coordination Enhanced systems for countermeasures in an Enhanced surveillance and health service delivery in epidemic laboratories an epidemic and reaching Enhanced biosafety vulnerable groups Enhanced environmental Enhanced monitoring of measures risk factors and uality Food safety Community engagement and risk communication Enhanced multisector Enhanced disease Enhanced leadership Enhanced readiness policies and systems prevention and and decisions to expand services detection mitigate risks uickly ess fre uent and intense outbreaks and epidemics; greater resilience to disease risks; lives saved; and reduced risk and cost of health emergencies Source: Independent Evaluation Group. Note: AMR = antibiotic resistance; IFC = International Finance Corporation; MCM = medical countermeasures; WASH = water, sanitation, and hygiene. 9 Box 2.1. Explaining the Evaluation’s Conceptual Framework The conceptual framework posits that preparedness support for epidemics and pandemics helps countries develop sustained institutional capacities to prevent, detect, and ensure readiness to respond to infectious disease outbreaks and threats. It delineates three main support areas, intervention areas, and four change pathways for the scale-up of preparedness, as well as expected results—that is, intermediate outcomes, outcomes, and impacts: Support Areas Support across the three areas is assumed to be tailored to the country’s political and economic context, existing capacities, and infectious disease risks, and may involve public, private sector, and other actors, such as from civil society and communities. 1. Scaling core preparedness capacities. These capacities form the backbone of a country’s health emergency preparedness to prevent the emergence and spread of infectious disease outbreaks (including zoonotic, vector-borne, foodborne, and waterborne diseases, as well as antimicrobial-resistant pathogens) and enable early detection of disease spillover risks through a cross-sectoral approach that integrates One Health and public health perspectives. Strengthening these capacities helps minimize the impact of infectious diseases and enhances public health security. The following are examples of intervention areas: • Strengthening coordination and leadership: Establishing robust frameworks for collaboration among various sectors such as health, agriculture, environment, and water (including the private sector) to ensure an integrated approach to disease prevention. Developing policies and protocols that facilitate seamless coordination and communication between different sectors to preemptively address potential health threats. • One Health networks and platforms: Promoting the One Health approach, which recognizes the interconnectedness of human, animal, and environmental health. Creating networks and platforms that facilitate planning, integration of actions across sectors, monitoring, and information sharing and collaborative efforts among veterinarians, health workers, environmental scientists, and other stakeholders. • Core human resource capacities: Building and enhancing the skills and knowledge of veterinarians, frontline community animal health workers, human health workers, and laboratory workforces to effectively prevent and manage diseases. Providing training and resources to ensure these professionals are equipped to handle emerging health threats. Timely deployment of workers and teams to investigate outbreaks in humans and animals. • Farm safety practices and animal health: Implementing farm safety practices to minimize the risk of disease transmission from animals to humans. Improving outreach of veterinary services and promoting the use of animal vaccination and disease control measures to prevent outbreaks at the source. • Appropriate use of antibiotics: Encouraging the responsible use of antibiotics in animal husbandry to prevent the development of antimicrobial resistance. Educating farmers and veterinarians on best practices for antibiotic use. 10 • Animal food trade markets and food surveillance and safety: Monitoring and regulating animal food trade markets to ensure safe handling practices and prevent the spread of zoonotic diseases. Establishing food surveillance systems to detect and address potential health risks in the food supply chain. • Wildlife management and habitat preservation: Implementing strategies for wildlife management and habitat preservation to reduce the risk of zoonotic diseases and prevent disease spillover from animals to humans. Protecting natural habitats to maintain ecological balance and prevent the encroachment of human activities that may lead to disease outbreaks. • Antimicrobial resistance risk and vector controls: Addressing antimicrobial resistance risks through targeted interventions and surveillance of antimicrobial resistance in humans, animals, and the environment (water, soil, and other areas) that may spread disease. Strengthening regulation and stewardship programs for the use of antimicrobials. Implementing vector control measures to reduce the transmission of diseases by insects and other vectors. • Water, sanitation, and hygiene (WASH); waste management; and water treatment: Promoting water, sanitation, and hygiene practices to prevent the spread of infectious diseases. Ensuring proper waste management and water treatment to maintain a healthy environment. • Early warning and surveillance systems: Developing and maintaining early warning systems to detect potential health threats before they escalate. Establishing surveillance networks to monitor disease patterns and provide timely information for preventive actions. • Laboratory networks and timely diagnosis: Strengthening public and private laboratory networks within and across countries to ensure timely diagnosis and information sharing. Enhancing laboratory capacities to detect and identify pathogens quickly and accurately. • Biosafety and biosecurity standards: Developing and implementing biosafety and biosecurity standards to prevent accidental release of pathogens and ensure safe handling of infectious materials. Educating health workers and laboratory personnel on best practices for biosafety and biosecurity. • Public health risk communication: Developing effective public health risk communication strategies to inform and educate the public about preventive measures. Ensuring transparent and timely communication to build trust and encourage community engagement in disease prevention efforts. Intermediate outcomes in this support area include enhanced workforce capacities to address animal, environmental, and human health risks; coordination of aligned multisectoral actions; surveillance and laboratory networks; biosafety; food safety (emphasizing zoonotic disease risks); environmental measures (such as sanitation and vector control and efforts to limit waterborne disease); and community engagement and risk communication. 2. Safe and scalable care for emergencies. This area focuses on strengthening national and regional health system capacities for preventing outbreaks from spreading to humans and delivering emergency services when needed. Community-based and primary health care systems serve as a population’s first point of contact with the health system, offering the community a chance to detect, report, and react to an outbreak in a timely manner. Inadequate preparedness can lead to rapid spreading of disease and have consequences for vulnerable groups if essential services are diverted or shut down. The following are examples of intervention areas: 11 • Strengthening national and regional health system capacities: Enhancing the ability of health systems to detect and respond to outbreaks promptly. Developing protocols and systems for rapid identification and reporting of infectious diseases. • Community-based and primary health care systems: Strengthening community health systems to ensure early detection of health threats at the local level. Training primary health care providers to recognize and report signs of emerging diseases. • Epidemic ready primary health care: Equipping primary health care facilities with the necessary infrastructure, diagnostic tools, and clinical protocols to detect and manage disease cases. Ensuring that primary health care systems are prepared to maintain service quality during an epidemic. Deployment of workforce surges during a public health event including contractual agreements with private health providers. • Referral systems to manage potential outbreaks: Establishing efficient referral systems to ensure timely and appropriate response to detected health threats. Facilitating the transfer of patients and information between public and private facilities and different levels of the health system. • Reinforcing infection prevention and control practices: Implementing robust infection prevention and control measures in health care facilities —including quarantine and containment measures—to reduce pathogen transmission and contain health threats. Training health workers on best practices for infection control to prevent the spread of diseases. • Strengthening community health workers and frontline emergency health workforce: Enhancing the capacity of community health workers and frontline emergency health workforce to detect and respond to health threats. Providing training and resources to ensure these workers can effectively identify and report emerging diseases. At the community level, efforts may strengthen community health worker networks by mobilizing and training community health workers to serve as frontline responders in detecting and reporting potential disease cases. Community health workers can educate communities about disease prevention, recognize early signs of outbreaks, and ensure timely referrals to health facilities. Engaging local communities in early detection efforts helps them identify potential outbreaks before they spread. • Digital health solutions: Developing and using digital health solutions (e-Health, telemedicine) to improve the detection and monitoring of health threats and to facilitate remote diagnosis and reporting. • Monitoring of risk factors and continuity of regular health service delivery: Establishing systems to monitor risk factors and ensure the continuity of regular health services during an epidemic. Using data and technology to track health trends and detect potential outbreaks. Intermediate outcomes in this support area include enhanced infection control, clinical standards, and systems to expand services in an outbreak or epidemic and maintain and monitor the quality of services. 3. Equitable access to medical countermeasures. This area focuses on strengthening national and regional capacities for financing, manufacturing, procurement, and delivery of medical devices and supplies (for example, equipment and technology such as ventilators, diagnostic tests, protective equipment, drugs, and vaccines) for preventing, diagnosing, and treating diseases in both animals and humans—with a focus on managing health emergencies—through coordinated efforts among stakeholders at all levels. Countries need to be ready to access lifesaving medical supplies such as vaccines, diagnostics, ventilators, personal protective 12 equipment, and oxygen devices to control an outbreak or epidemic. The following are examples of intervention areas: • Strengthening national and regional pharmaceutical capacities: Enhancing the ability of countries and regions to manufacture medical countermeasures (MCM) for preventing, diagnosing, and treating diseases in both animals and humans. • Diversification of the production landscape for MCM: Reducing dependency on a limited number of suppliers by diversifying the production landscape for MCM by providing financial incentives and promoting public-private partnerships to leverage private sector expertise and resources in MCM production. Such efforts can promote commercial viability and sustainability while strengthening the policy and regulatory environment for enhanced public-private collaboration, capital mobilization, and harmonization of regulatory frameworks. • Optimizing MCM supply chain: Improving the efficiency and effectiveness of the MCM supply chain, including procurement, storage, logistics, and distribution. The development, strengthening, and coordination of pooled procurement mechanisms at both the global and regional levels may also help ensure timely and equitable access to medical countermeasures. Such interventions could include creating mechanisms that can mobilize financing in advance (ex ante agreements) and surge resources when needed, targeting key aspects of the value chain. • Investing in research and development of new technologies and treatments: Fostering innovation in the development of new technologies and treatments for diseases affecting both animals and humans. Allocating funds for research and development projects focused on developing new MCM technologies and treatments. Encouraging collaborations between research institutions, universities, and pharmaceutical companies to drive innovation. Intermediate outcomes in this support area include enhanced policies and systems to access medical countermeasures, including research and development; manufacturing capabilities; access to essential medicines; supply chains; and regulations. Change Pathways Across the three support areas, there is an emphasis on change path ays to scale up and help develop the institutional capacities of policies, services, systems, and multisector arrangements and contribute to outcomes. Change path ays are flexible and can be applied individually or in combination, tailored to the specific context of each country. • r u s or o s are intended to combine effective activities from different sectors —such as health, agriculture, environment, and the private sector—to help countries enhance prevention, detection, and response capacities. • oo o o s are intended to help countries share information for decisions; target, coordinate, monitor, and evaluate actions; and provide platforms and expertise to strengthen systems and services. Kno ledge initiatives are also intended to enhance collaboration and learning to support spillover and the implementation of actions across and ithin countries. • u p op r o s at the regional, national, and local levels are intended to help strengthen coordination; develop an integrated architecture to address disease risks across country borders and remote geographies; and facilitate changes in behaviors and practices among people, such as leaders, health care orkers, environmental experts, farmers, and community groups. 13 • r rs ps are intended to help enhance and align technical kno ledge and provide implementation support and financing to scale up preparedness results across countries. Outcomes Expected outcomes include stronger multisectoral policies and systems, improved disease prevention and detection, enhanced leadership and decision-making to mitigate risks, and readiness to scale up services rapidly to control disease spread. Impacts Health emergency preparedness efforts across the three support areas ultimately contribute to the goals of reducing the frequency and intensity of outbreaks and epidemics, increasing systems’ resilience, averting pandemics, saving people’s lives, and reducing risks and costs for society. Source: Independent Evaluation Group. 3. Strategic Positioning of the Evaluation 3.1 This evaluation is part of the IEG effort to enhance the Bank Group (World Bank and IFC) learning to improve results in critical global challenge areas. Other ongoing evaluations focus on climate change and biodiversity. This evaluation adds to previous IEG efforts to learn from Bank Group support related to epidemics and pandemics, regional projects, health systems, partnerships, and private sector engagement in health, including support to tuberculosis, HIV/AIDS, Ebola, and COVID-19 (Mullen 2005; World Bank 2009a, 2009b, 2011, 2013, 2016a, 2018b, 2019, 2021, 2022d). Findings from past evaluations are summarized in box 3.1. A past portfolio review of World Bank lending for communicable disease control also informs the evaluation (Martin 2010). Box 3.1. Past Independent Evaluation Group Evaluation Findings on Preparedness for Epidemics • Human resource strengthening. Past evaluations emphasized the need to establish decentralized workforce scale-up mechanisms before outbreaks and provide needed compensation. • Multisector coordination. Strategic coordination was often strong, but implementation- level coordination across sectors was challenging. The key was focusing on sectors with the highest potential impact and clear implementation roles. • Laboratories. It was important to strengthen existing laboratory systems rather than building new infrastructure, as well as to consider maintenance, sample transport, networks to decentralized areas, and technical capacity needs. • Disease surveillance. While past support has expanded epidemiological surveillance in countries, scale-up was slow, coverage of risk groups was limited, and data collection was inconsistent. Countries needed stronger capacity to track disease risks through community- level networks. 14 • Community engagement and risk communication. Engaging nongovernment actors facilitated outreach to risk groups, although outcomes were rarely tracked. During the COVID-19 pandemic, weak community engagement and misinformation hindered the response. During the Ebola outbreak, strengthening local media and leaders was key for communication. During the avian influenza outbreak, community engagement of farmers was key to controlling disease spread. • Systems for health service delivery in an epidemic. Improving capacity for continued service delivery is a recommendation from the Independent Evaluation Group’s COVID-19 evaluation and a lesson from the World Bank Group’s support during the Ebola outbreak. • Supply chain strengthening. Efforts to ensure quality assurance, harmonize procurement rules, and involve regulators helped address this area, but a sustained and coordinated approach to strengthen systems’ access to medical supplies quickly was lacking. • Regional engagement. Successful regional efforts supported institutions, facilitated cross- country spillover and resource sharing, and had clear accountability frameworks. Regional coordination was important for ensuring information sharing and access to high-security facilities. The COVID-19 evaluation recommended better regional engagement for strengthening preparedness. • Private sector engagement. Several past Independent Evaluation Group evaluations have recommended better engagement with the private sector to enhance preparedness and health service delivery. • Monitoring and leadership. Maintaining political support is challenging. Monitoring both implementation and policy outcomes, measuring quality, and building evaluation capacity were important. A shift from output to outcome tracking was needed. • Partnerships. Technical partnerships helped enhance implementation and scale-up. Clear division of partner roles and strong links to country-level work were key. • Sustainability. Past Independent Evaluation Group evaluations emphasized a more strategic, evidence-based approach focused on building sustainable capacity and institutions for preparedness rather than isolated interventions. Sources: Mullen 2005; World Bank 2009a, 2009b, 2011, 2013, 2016a, 2018b, 2019, 2021, 2022d. Added Value of the Evaluation 3.2 Consultations emphasize the areas in which the evaluation can inform the future actions and results of Bank Group support for preparedness for epidemics and pandemics. The evaluation team consulted with Bank Group staff and experts at different levels in various departments, including Verticals, the Pandemic Fund, country teams, IFC, and partners. The following areas identified in these consultations frame the design of the evaluation: • Reviewing the extent to which the Bank Group support applies a One Health approach. Assess whether the Bank Group’s financial support and advisory 15 services and analytics (ASA) adequately address foundational areas of One Health to develop core capacities that prevent future disease risks. • Identifying factors that enable or constrain countries’ scaling up of preparedness. Assess factors that could help or hinder ongoing support to scale up preparedness achievements in countries and inform course corrections. Pathways for scale-up involve the enhancement of actions across sectors; the use of data, knowledge, and technology; multilevel actions at the regional, cross- border, country, and local levels; and actions involving partners and the private sector. • Reviewing the monitoring and evaluation of ongoing support. Assess the arrangements in place and whether the Bank Group is on track to measure needed results. • Reviewing how well the Bank Group engages with different actors in its support for preparedness. Examine the successes and challenges of partnerships, private sector engagement, and cross-sector collaboration and how well they support efforts to expand countries’ preparedness for epidemics. • Assessing the results and sustainability of past interventions. Examine the outcomes of past interventions and the extent to which they were successful in supporting institutional change. Identify whether ongoing efforts incorporate past learning to address challenges. • Assessing the Bank Group’s readiness to rapidly finance clients in an emergency. Examine changes in client access to rapid response financing in Bank Group country programs since COVID-19. 4. Evaluation Questions and Coverage or Scope Evaluation Questions 4.1 The evaluation’s overarching question is, How well has the Bank Group (World Bank and IFC) supported preparedness for epidemics and pandemics in client countries? 4.2 Two main evaluation questions emphasize this overarching question: • To what extent has the Bank Group’s support for preparedness been relevant to addressing client country needs and applying a One Health approach that integrates different sectors? 16 o 1.1. To what extent has the Bank Group’s support for preparedness been relevant to the client country’s disease risks, frequency of outbreaks, and gaps in core capacities? o 1.2. To what extent has the Bank Group’s support for preparedness included One Health interventions that are aligned with evidence of effectiveness or proven best practices? o 1.3. To what extent has the Bank Group’s support engaged the private sector to enhance preparedness for epidemics and pandemics? o 1.4. To what extent has the Bank Group set up tools or mechanisms since the COVID-19 pandemic that would allow countries to quickly and flexibly access financing in the event of an epidemic-related health emergency? • To what extent have the Bank Group’s efforts contributed to enhancing preparedness for epidemics and pandemics at the country and regional levels? o 2.1. How well have the results of preparedness efforts been measured? o 2.2. To what extent have intermediate outcomes been achieved in countries and regions, and how sustainable are the outcomes of past efforts? o 2.3. What factors have facilitated or hindered the implementation, performance, and scale-up of preparedness support? Evaluation Scope 4.3 The scope of this evaluation is determined by four key parameters: conceptual boundaries, disease focus, institutional coverage, and time coverage. 4.4 The conceptual boundaries of the evaluation’s framework, shown in figure 2.2, will define the analytic scope. This evaluation will examine Bank Group support (World Bank and IFC) that aligns with the preparedness capacities for preventing, detecting, and building readiness to respond to infectious disease risks. In terms of portfolio scope, this means that emergency response and recovery projects are included if they contribute to strengthening preparedness capacities, such as through enhanced disease surveillance, policies, epidemic service-delivery systems, or manufacturing and distribution of medical countermeasures. The evaluation will concentrate on the three support areas, interventions areas, change pathways, and results—both intermediate outcomes and to the extent possible outcomes—described in the conceptual framework. The evaluation will not assess the contribution of the Bank Group to impacts. While the evaluation portfolio encompasses multisectoral support across health, agriculture, 17 environment, water, finance, urban, and digital sectors (including support to public and private sector) aligned with the conceptual framework, it will not be limited to projects that integrate and apply a One Health lens. This recognizes that most of the Bank Group’s support in pandemic and epidemic preparedness has often consisted of stand- alone sectoral operations. To manage the scope of the evaluation and maintain focus, certain areas are excluded. For instance, social protection interventions are excluded because they are addressed in a separate forthcoming IEG evaluation on adaptive social protection. Economic interventions—such as those related to insurance and financial risks and financial services—are not covered by the evaluation since they do not directly impact disease spillover prevention, early detection, or outbreak containment, but response efforts which are out of scope. 4.5 The evaluation will focus on the Bank Group’s support for preparedness against epidemic- and pandemic-related health emergencies. It will exclude health emergencies resulting from natural disasters (such as earthquakes), humanitarian crises, and environmental toxin exposure (for example, chemical spills). The evaluation will cover projects with activities aimed at preventing, detecting and ensuring readiness for infectious disease risks, including zoonotic, vector-borne, foodborne, and waterborne diseases, as well as antimicrobial-resistant pathogens. Projects without a clear link to reducing infectious disease risks—such as those focused on water sanitation, primary health care, livestock management, climate change, or biodiversity support for forestry or land degradation—will be excluded unless they explicitly include activities for outbreak prevention, detection or building readiness. Similarly, food security projects will be excluded unless they address zoonotic disease risks because this topic will be covered in a forthcoming IEG evaluation. Since IEG has conducted thematic evaluations on the Bank Group’s support for AIDS, tuberculosis, avian influenza, and COVID-19, projects covered in those evaluations will be excluded. However, relevant past findings will be referenced where applicable. Projects implemented after the time frame of previous IEG evaluations will be included. 4.6 The evaluation will focus on the contributions of the Bank Group institutions— the World Bank and IFC—to strengthening epidemic and pandemic preparedness capacities in countries and regions. For the World Bank, this area includes International Development Association and International Bank for Reconstruction and Development lending, as well as ASA projects, whereas for IFC, it encompasses investment projects and advisory services. Given that the Multilateral Investment Guarantee Agency has a relatively small and predominantly new portfolio in this area, it will not be included in the evaluation. In its portfolio and partnership analysis, the evaluation will also examine relevant initiatives from the Health Emergency Preparedness and Response Umbrella Program, the Pandemic Fund, and the Global Environment Facility. 18 4.7 The evaluation will cover approximately 20 years of epidemic preparedness support to address both long-term impacts and recent developments. The timeline will span from January 2000 through June 2024 for the World Bank portfolio and from January 2005 through June 2024 for the IFC portfolio. This extended timeline will enable the evaluation team to assess how past support has built preparedness capacities over time in areas of the conceptual framework to look at the evolution of Bank Group support (World Bank and IFC). The evaluation will assess both closed and active projects to provide insights into the effectiveness of Bank Group support to inform the current portfolio and ongoing efforts by the HEPPR Global Challenge Program to scale up preparedness. For closed projects, the evaluation will look at the results achieved— both intermediate outcomes and outcomes—and, where possible, will look at how past investments have contributed to greater resilience and reduced epidemic risks. In countries with recent active support, the evaluation will assess the early success of change pathways toward desired results. 5. Evaluation Design and Components Evaluation Design 5.1 The following approaches anchor the evaluation design. 5.2 Theory-based design. The evaluation adopts a theory-based design grounded in the results logic of the evaluation’s conceptual framework for preparedness for epidemics. The conceptual framework will guide all data collection and analysis protocols to collate evidence from the various evaluation components and answer the evaluation questions. 5.3 Focus on enabling and constraining factors to inform scale-up of preparedness. Across the evaluation components, the evaluation team will make an effort to identify and understand enabling and constraining factors that can help inform ongoing efforts to scale up preparedness in countries and regions through the change pathways in the conceptual framework. 5.4 Mixed methods design and triangulation. The evaluation will adopt a mixed methods approach, combining quantitative and qualitative data to provide a comprehensive and nuanced understanding of the complexities involved in multisectoral efforts to enhance preparedness. This approach will allow for an assessment of both the effectiveness of the Bank Group’s efforts (World Bank and IFC) in preparedness and the mechanisms behind its success or failure. Triangulating evidence from different evaluation components will strengthen the validity of findings, enhance the cross-verification of results, and minimize biases that could arise from relying on a 19 single method. The evaluation design matrix and diagram in appendix A (see table A.1 and figure A.1) illustrate how evidence from these different analytic components will be triangulated to answer the evaluation questions. 5.5 Regional and country focus. The evaluation design will emphasize the integration of regional and country-level support, by looking at both levels across its analyses. This focus was identified based on the importance of delivering results for client countries and the growing emphasis on the need for a regional architecture to support cross-country collaboration. Evaluation Components 5.6 Portfolio review and analysis (PRA). The PRA will identify World Bank lending projects and ASA, as well as IFC investments and advisory services in countries. The PRA’s identification protocol will use text analytics and draw on a taxonomy of key concepts and their corresponding keywords and phrases derived from the conceptual framework. The evaluation team will validate the identified portfolio via consultations with Bank Group counterparts. The portfolio is characterized into “core and “noncore” projects. Core projects have a strong focus on preparedness areas defined in the evaluation’s conceptual framework (as evidenced in their objectives, titles, or descriptions) and peripheral or noncore projects also include preparedness activities but not as the main focus. Due to the high volume of projects in the portfolio, the evaluation will concentrate its detailed analysis on core projects, although both categories will be reviewed to assess the relevance of the Bank Group’s support to countries and regions (evaluation question 1). The PRA will include the following dimensions: • Analysis of support areas, consistency, timeliness, and intensity. The portfolio analysis will assess areas where the Bank Group has supported preparedness for epidemics and pandemics and estimate the intensity, duration, coverage, and timeliness of support in countries and regions. When possible, the analysis will unpack the support areas in the evaluation’s conceptual framework, interventions supported in countries using different Bank Group instruments (including World Bank investment project financing, development policy financing, ASA, trust funds, and IFC investments and advisory services), change pathways, and intended intermediate outcomes. The intensity (number of projects, instruments, financing amounts, and interventions supporting preparedness), duration (years of support), coverage of different support areas of the conceptual framework, and timeliness (for example, of disbursement and in supporting past epidemics) will be assessed to examine the level of support over time in a country and region. 20 • Review of results measurement. The PRA will assess how well the World Bank has measured and achieved results from its preparedness support across conceptual framework areas.1 To this end, the evaluation will first conduct a rapid background literature review of key considerations and measures to assess preparedness for epidemics and map them to the evaluation’s conceptual framework. The PRA will use this as a benchmark to assess the extent to which the World Bank is using meaningful and comprehensive measures to assess its health preparedness results. This analysis will examine—among other results— indicators’ achievement rates for closed projects and how well intermediate outcomes and outcomes are measured, including metrics contributing to the Bank Group scorecard indicator of people benefiting from strengthened capacity to respond to health emergencies (World Bank 2024b). Key informant interviews will complement the review to help the evaluation team understand factors that facilitate and constrain results measurement. • Factors affecting implementation, results, and scale-up. The PRA will apply the Delivery Challenges in Operations for Development Effectiveness taxonomy, as used in IEG’s Results and Performance of the World Bank Group 2023 (World Bank 2023b), to identify common factors affecting the implementation and scale-up of preparedness support and results in projects. Text from Implementation Completion and Results Reports (ICRs) on closed projects will be analyzed to uncover both positive and negative factors. The evaluation may also leverage supervised machine-learning models developed for the Results and Performance of the World Bank Group 2023 to automate parts of the analysis. The findings will be complemented by evidence from case studies and the staff survey. 5.7 Alignment analyses. Alignment analyses will combine PRA evidence with secondary data sources (for example, a client country’s epidemic risks, frequency of outbreaks, and preparedness capacities), literature on One Health effectiveness, and literature on private sector roles to assess the relevance of the Bank Group’s efforts to support preparedness for epidemics and pandemics in countries. • Alignment of support for different country situations. The evaluation will assess the relevance of World Bank and IFC support in addressing the needs of countries with different epidemic risks, governance and social readiness, and frequency of outbreaks. The analysis will leverage secondary data to estimate 1A critique of health preparedness measurement has been its heavy focus on compliance with International Health Regulations and less emphasis on the measurement of outcomes (Fan and Smitham 2023). 21 countries’ preparedness needs, drawing on sources such as the INFORM Epidemic Risk Index, which considers environment, animal, and human health risks; Notre Dame data on countries’ governance and social readiness for managing emerging climate and disease risks; and data on disease outbreak news to estimate the frequency of disease events (European Commission 2020; Torres Munguía 2022; University of Notre Dame 2023). Governance and social readiness indicators consider political stability; corruption; regulatory quality; rule of law; inequalities; and access to technology, innovation, and education (Chen et al. 2015). Factors such as leadership and technology access affected countries’ response capacities during COVID-19 (World Bank 2022d). Other sources may include data on preparedness capacities and veterinary service capacities. External data will be visualized through a heat map and overlapped with PRA evidence to assess the extent to which there has been support for countries and regions with disease hot spots, higher epidemic risks, and lower governance and social readiness to address emerging disease risks. The evaluation will also look at the extent to which recent support post–COVID-19 has become more aligned with countries’ situations. • Alignment of support involving the private sector. The evaluation will assess private sector support in the portfolio against evidence from the literature on successful private sector roles or arrangements to support preparedness for epidemics. First, the team will conduct a rapid literature review of systematic reviews and case studies to extract evidence on private sector roles in support areas and intermediate outcomes in the conceptual framework. Second, the team will use the literature as a benchmark to assess private sector support in IFC and World Bank projects that involve the private sector. Finally, the team will synthesize the findings visually to map the extent to which the Bank Group has had relevant private sector support for health preparedness in the portfolio. Key informant interviews will complement the analysis to enable the team to understand areas of private sector support in the portfolio. 5.8 Review of emergency financing readiness. The analysis will examine the extent to which the World Bank’s Crisis Preparedness and Response Toolkit is helping to fill gaps to enhance the readiness for financing during an epidemic. While it is still early in the implementation of the Toolkit, the evaluation can inform ongoing efforts to enhance ex ante tools for financing readiness. During COVID-19, the Bank Group faced challenges in ensuring portfolio-level flexibility so countries could rapidly access financing to expand health services, purchase medical supplies, and coordinate support across sectors and partners (World Bank 2022d). The analysis will review the inclusion of ex ante financing tools in Bank Group country portfolios since COVID-19 (such as 22 contingency financing rapid response options and deferred dropdown options in investment project financing and development policy lending) and inquire about adjustments made to the country portfolio to ensure readiness for timely and coordinated action in a future epidemic, including any IFC support to enhance ex ante readiness. The analysis will explore the extent to which country management perceives itself as better prepared to handle epidemic-related emergencies and avoid the pitfalls encountered during the COVID-19 response. The analysis will focus on a sample of countries with higher epidemic risks as per the INFORM Epidemic Risk Index (European Commission 2020)—for which baseline information on the World Bank portfolio is available from IEG’s 2022 COVID-19 evaluation (for example, on the availability of contingency emergency response components, International Development Association regional crisis window financing, and funds from the Pandemic Emergency Financing Facility)—to assess the enhanced ex ante readiness of the portfolio. The data collection protocol will consist of (i) a rapid review of ex ante financing tools available in country portfolios for epidemic readiness since COVID-19 and (ii) structured interviews with Bank Group senior operations staff in selected countries to help the evaluation team understand how the portfolio has been adjusted to facilitate readiness to address an epidemic-related emergency in those specific contexts, and opportunities to further enhance ex ante readiness. Criticisms of the Bank Group include concerns about whether country programs can meaningfully increase available rapid financing tools for emergencies and address the need for partnership and flexibility in different country contexts (Stefan and DeGrauw 2024). 5.9 Case studies. The case studies will use contribution analysis to assess Bank Group support to specific (intermediate) outcomes in selected countries. This protocol will assess the Bank Group (World Bank and IFC) achievements to contribute to selected intermediate outcomes in the conceptual framework and aims to understand constraints and enabling factors in implementation, as well as change pathways. Cases will be selected based on a purposive sampling strategy with a stratification component based on countries’ epidemic risk, and social and governance readiness (European Commission 2020; Torres Munguía 2022; University of Notre Dame 2023). The outcome areas for analyzing contributions include enhanced surveillance, laboratories, biosafety, systems for health service delivery, and access to medical countermeasures. The case studies will prioritize countries with at least one closed project and a higher density of support in projects and analytic work, increasing the likelihood of identifying evidence of outcomes. The case study protocol will involve a review of project documents (such as World Bank aide-mémoire and ICRs and IFC completion reports, among others); available evidence on results (such as from indicators, evaluation reports, and field visits); and key informant interviews with World Bank and IFC teams, clients, and other stakeholders involved in the intervention, such as partners or laboratory associations. 23 The data collection for the cases will follow a consistent protocol to build a database that documents the findings for analysis. 5.10 Documentation of outcomes from closed support in countries. The analysis will identify and document outcomes contributed to by closed World Bank and IFC projects, with a focus on intermediate outcomes that strengthen institutional capacities for preparedness. Outcome harvesting will be used to systematically code information on new actions, policies, systems, practices, and other changes contributed to by projects, as well as how the outcomes were supported in a country context. The coding will target selected countries and intermediate outcomes defined in the conceptual framework, emphasizing areas of One Health and health service strengthening. The data collection will involve reviewing completion documents from ASA and closed projects, including ICRs, ICR Reviews, and evaluation reports (where they exist), as well as Expanded Project Supervision Reports assessing IFC project performance. A sample of documented outcomes describing key changes to improve preparedness will be validated through structured feedback from key informant interviews with task teams and clients and through requests for additional evidence of the achievements (where available) and the sustainability of the changes in the country. The harvested outcome information will be compiled into a data set that will enable the evaluation team to identify areas of operational success and areas where results have been limited, and more attention may be needed. 5.11 Partnership analysis. The evaluation will review World Bank and IFC partnerships supporting preparedness in client countries through different modalities, including the Pandemic Fund, Global Environment Facility, and Health Emergency Preparedness and Response Umbrella Program. The focus will be on active partnerships to inform the ongoing portfolio. The evaluation team will validate the list of partnerships in our sample via consultation with Bank Group counterparts to ensure that the main partnerships that support countries are covered. The goal of the partnership analysis is twofold: (i) understand how the Bank Group works with partners, and (ii) uncover the key characteristics that make for good partnerships, with a focus on the extent to which these partnerships benefit clients and enable the achievement of key outcomes in the evaluation’s conceptual framework. The evaluation will map partnership features, activities, and enabling factors based on comparative analysis and interviews. First, the evaluation will assess the partnership landscape, focusing on coverage of high-risk countries, key outcomes, and modalities of partnering. Using key informant interviews and coding of documents, the evaluation team will compile a database for comparative analysis of partnerships’ features. This evidence will be triangulated with informant interview evidence to assess the reasons for potential gaps in coverage. Second, to understand the activities’ benefits for client countries and 24 their contributions to preparedness results, the evaluation team will use a structured questionnaire via interviews with a sample of task teams implementing partnership activities in different regions. The final output will consist of a matrix mapping the findings on the partnerships reviewed and factors facilitating or hindering the success of the partnership activities. The data on partnerships will also be combined with country situation data collected for the alignment analysis so the evaluation team can consider the extent to which partnerships prioritize countries with higher epidemic risks and coordinate financing with other donors. 5.12 Staff survey. A staff survey will be conducted to understand factors that have facilitated collaboration and that have constrained or facilitated implementation, scaling up, and the success of World Bank and IFC health preparedness interventions in countries. The survey will also ask respondents about access to knowledge and technical expertise to support implementation. The survey will target task team leaders who worked on projects and analytic tasks related to the support areas in the conceptual framework. The survey will be tested to ensure the questions are clear, and responses will be reviewed iteratively so the evaluation team can adapt strategies to increase the feedback. 5.13 Multivariate analysis. If data and time permit, the evaluation will explore a multivariate analysis (either a regression or a classification model) at the country or project level, combining data from the PRA and alignment analyses, among others to understand factors associated with the successful implementation and achievement of outcomes by Bank Group projects that support preparedness. 5.14 Key informant interviews. Throughout the evaluation, interviews will be conducted with experienced staff from the World Bank, IFC, and relevant experts to help the evaluation team interpret findings. These insights will be essential in contextualizing the results within the evolving landscape of preparedness for epidemics, ensuring the analysis reflects the latest context and current knowledge. Limitations 5.15 Table A.1 details the limitations of each evaluation component. The more recent support to preparedness post–COVID-19 is unlikely to have strong evidence of results. Also, the success of preparedness can be challenging to measure, as it depends on the country being tested by an epidemic. However, by looking at support over the past 20 years in locations where epidemics have occurred, the evaluation can offer important lessons for the successful scale-up of current support efforts by the HEPPR Global Challenge Program. Moreover, it is expected that the portfolio of private sector support will be limited and have less evidence of results. To mitigate this challenge, the PRA includes a literature review of models of private sector engagement, which can help 25 benchmark Bank Group support involving the private sector and inform opportunities to scale up such support. The evaluation will also ask about the complementarities between support of the World Bank and IFC in the key informant interviews. 6. Quality Assurance Process 6.1 The evaluation will follow IEG’s quality assurance processes to ensure the rigor and usefulness of its design, analyses, and findings. Processes include internal review processes, data collection protocols for all components, and review by the IEG methods team. Four independent peer reviewers will also review the evaluation design, methodologies, and the final report: Alice Norton, head of the Policy and Practice Research Group in the Pandemic Sciences Institute at the University of Oxford; Chikwe Ihekweazu, deputy executive director of the WHO Health Emergencies Programme; Jennifer Lasley, senior programme coordinator at the World Organization for Animal Health; and Michele de Nevers, director of sustainability programs for the Haas School of Business at the University of California, Berkeley, and former World Bank environment sector director. Peer reviewers will be consulted at key stages throughout the evaluation process for review and feedback. 6.2 The evaluation will complement IEG’s regular quality assurance through informal engagements with technical advisers and an advisory panel. Reantha Pillay from Resolve to Save Lives will serve as an adviser in areas of safe and scalable emergency health services and One Health. Zelalem Tadesse, from the Food and Agriculture Organization, will be an adviser on agriculture and environment sector practices for One Health. In addition, to enable the evaluation team to gather feedback on early findings and interpret key messages from the evaluation, virtual workshops will be held with an advisory panel of experts identified from different sectors and regions. 7. Expected Outputs, Outreach, and Tracking 7.1 The evaluation timeline is FY 2025–26, with the evaluation report review by IEG and World Bank management and the Committee on Development Effectiveness discussions expected in FY26. 7.2 Throughout the evaluation, IEG will continue its consultative engagements with World Bank and IFC staff and external experts to shape the evaluation methods and interpret findings from the evaluation. Participants in consultative engagements will vary by topic. For example, technical consultations on case study analyses may be held with task team leaders and managers engaged in country programs. 26 8. Resources 8.1 The task team leaders of the IEG evaluation are Jenny Gold and Mercedes Vellez. Core team members for the evaluation are Harsh Anuj, Ananda Ghose, Giuseppe Iarossi, Epimaque Nsanzabaganwa, Xiaoxiao Peng, Santiago Ramirez Rodriguez, Karima Saleh, Diana Stanescu, and Yezena Yimer. The work will be conducted under the guidance of Sabine Bernabè (Director General, Evaluation), Theo David Thomas (director, Human Development and Economic Management), Timothy Johnston (manager), and Estelle Raimondo (methods adviser). 27 Bibliography Bell, Jessica A., and Jennifer B. Nuzzo. 2021. Global Health Security Index: Advancing Collective Action and Accountability Amid Global Crisis. Global Health Security Index. https://ghsindex.org/report-model. Benton, Bruce. 2001. “The Onchocerciasis (Riverblindness) Programs Visionary Partnerships.” Africa Region Findings and Good Practice Infobriefs 174, World Bank. Chen, C., I. Noble, J. Hellmann, J. Coffee, M. Murillo, and N. Chawla. 2015. “University of Notre Dame Global Adaptation Index: Country Index Technical Report.” University of Notre Dame. https://gain.nd.edu/assets/254377/nd_gain_technical_document_2015.pdf. European Commission. 2020. “INFORM Epidemic Risk Index” (database). European Commission. Accessed October 10, 2024. https://drmkc.jrc.ec.europa.eu/inform- index/INFORM-Risk/INFORM-Epidemic-Risk-Index. Fan, Victoria, and Eleni Smitham. 2023. “The Pandemic Fund’s Results Framework: Early Reflections and Recommendations.” Center for Global Development (blog), April 28. https://www.cgdev.org/blog/pandemic-funds-results-framework-early-reflections-and- recommendations. FAO (Food and Agriculture Organization), UNEP (United Nations Environment Programme), WHO (World Health Organization), and WOAH (World Organisation for Animal Health). 2022. One Health Joint Plan of Action (2022–2026): Working Together for the Health of Humans, Animals, Plants and the Environment. WHO. https://www.who.int/publications/i/item/9789240059139. Glassman, Amanda, Brin Datema, and Amanda McClelland. 2018. “Financing Outbreak Preparedness: Where Are We and What Next?” Center for Global Development (blog), November 9. https://www.cgdev.org/blog/financing-outbreak-preparedness-where-are- we-and-what-next. GPMB (Global Preparedness Monitoring Board). 2023. GPMB Monitoring Framework for Preparedness: Technical Framework and Methodology. World Health Organization. https://www.gpmb.org/reports/m/item/gpmb-monitoring-framework-full. GPMB (Global Preparedness Monitoring Board). 2024. The Changing Face of Pandemic Risk: 2024 Report. World Health Organization. https://www.gpmb.org/reports/m/item/the- changing-face-of-pandemic-risk-2024-report. Huber, Caroline, Lyn Finelli, and Warren Stevens. 2018. “The Economic and Social Burden of the 2014 Ebola Outbreak in West Africa.” Journal of Infectious Diseases 218 (Suppl 5): S698– S704. https://doi.org/10.1093/infdis/jiy213. IMF (International Monetary Fund). 2022. World Economic Outlook Update, July 2022: Gloomy and More Uncertain. IMF. https://www.imf.org/en/Publications/WEO. 28 International Working Group on Financing Preparedness. 2017. From Panic and Neglect to Investing in Health Security: Financing Pandemic Preparedness. World Bank. http://hdl.handle.net/10986/26761. Jonas, Olga, Alec Irwin, Franck Berthe, et al. 2017. Drug-Resistant Infections: A Threat to Our Economic Future. World Bank. Mackenzie, John S., and Martyn Jeggo. 2019. “The One Health Approach—Why Is It So Important?” Tropical Medicine and Infectious Disease 4 (2): 88. https://doi.org/10.3390/tropicalmed4020088. Mackenzie, John S., Moira McKinnon, and Martyn Jeggo. 2014. “One Health: From Concept to Practice.” In Confronting Emerging Zoonoses, edited by Akio Yamada, Laura H. Kahn, Bruce Kaplan, Thomas P. Monath, Jack Woodall, and Lisa Conti. Springer Tokyo. https://doi.org/10.1007/978-4-431-55120-1_8. Magouras, Ioannis, Victoria J. Brookes, Ferran Jori, Angela Martin, Dirk Udo Pfeiffer, and Salome Dürr. 2020. “Emerging Zoonotic Diseases: Should We Rethink the Animal–Human Interface? Frontiers in Veterinary Science 7: 582743. https://doi.org/10.3389/fvets.2020.582743. Martin, Gayle H. 2010. “Portfolio Review of World Bank Lending for Communicable Disease Control.” Working Paper 2010/3, World Bank. Mullen, Patrick. 2005. Review of National HIV/AIDS Strategies for Countries Participating in the World Bank’s Africa Multi-Country AIDS Program. World Bank. Sejvar, James J. 2003. “West Nile Virus: An Historical Overview.” Ochsner Journal 5 (3): 6–10. Stefan, Cristina, and Anna DeGrauw. 2024. “Expanding the Deck or Just Reshuffling? What the Crisis Preparedness and Response Toolkit Could Mean for IDA Countries.” Centre for Disaster Protection (blog), March 18. https://www.disasterprotection.org/blogs/expanding- the-deck-or-just-reshuffling-what-the-crisis-preparedness-and-response-toolkit-could- mean-for-ida- countries#:~:text=The%20risk%20of%20having%20a,from%20its%20current%208.3%25%2 0baseline. Stowell, Daniel, and Richard Garfield. 2021. “How Can We Strengthen the Joint External Evaluation?” BMJ Global Health 6 (5). https://doi.org/10.1136/bmjgh-2020-004545. Stratton, J., E. Tagliaro, J. Weaver, et al. 2019. “Performance of Veterinary Services Pathway Evolution and One Health Aspects.” Revue Scientifique et Technique 38 (1): 291–302. https://doi.org/10.20506/rst.38.1.2961. Tichenor, Marlee, and Devi Sridhar. 2017. “Universal Health Coverage, Health Systems Strengthening, and the World Bank.” BMJ 31 (358): 3347. https://doi.org/10.1136/bmj.j3347. 29 Torres Munguía, Juan Armando. 2022. “A Global Dataset of Pandemic- and Epidemic-Prone Disease Outbreaks.” Figshare data set. https://doi.org/10.6084/m9.figshare.17207183.v2. UNEP (United Nations Environment Programme). 2023. Bracing for Superbugs: Strengthening Environmental Action in the One Health Response to Antimicrobial Resistance. UNEP. https://www.unep.org/resources/superbugs/environmental-action. University of Notre Dame. 2023. “ND-GAIN Index” (Database). Accessed October 10, 2024. https://gain.nd.edu/our-work/country-index/download-data. WHO (World Health Organization). 2005. International Health Regulations. 3rd ed. WHO. WHO (World Health Organization). 2021. “UNICEF, WHO, IFRC and MSF Announce the Establishment of a Global Ebola Vaccine Stockpile.” News Release, January 12, 2021. https://www.who.int/news/item/12-01-2021-unicef-who-ifrc-and-msf-announce-the- establishment-of-a-global-ebola-vaccine-stockpile. WHO (World Health Organization). 2023. WHO South-East Asia Regional Strategy for the Prevention and Control of Nipah Virus Infection 2023–2030. WHO. https://www.who.int/publications/i/item/9789290210849. WOAH (World Organisation for Animal Health). n.d. “International Standards.” Accessed February 26, 2025. https://www.woah.org/en/what-we-do/standards. World Bank. 2009a. Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population: An Evaluation of World Bank Group Support Since 1997. Independent Evaluation Group. World Bank. World Bank. 2009b. “The Stop Tuberculosis Partnership.” Global Program Review 4 (1). Independent Evaluation Group. World Bank. World Bank. 2010. People, Pathogens, and Our Planet. Volume 1: Towards a One Health Approach for Controlling Zoonotic Diseases. World Bank. https://documents1.worldbank.org/curated/en/214701468338937565/pdf/508330ESW0whi t1410B01PUBLIC1PPP1Web.pdf. World Bank. 2011. “The Global Fund to Fight AIDS, Tuberculosis, and Malaria, and the World Bank’s Engagement with the Global Fund.” Global Program Review 6 (1). Independent Evaluation Group. World Bank. World Bank. 2012. People, Pathogens and Our Planet. Volume 2: The Economics of One Health. World Bank. http://hdl.handle.net/10986/11892. World Bank. 2013. Responding to Global Public Bads: Learning from Evaluation of the World Bank Experience with Avian Influenza 2006–13. Independent Evaluation Group. World Bank. World Bank. 2016a. Public-Private Partnerships in Health: World Bank Group Engagement in Health PPPs. World Bank. 30 World Bank. 2016b. “World Bank Provides $150 Million to Combat Zika Virus in Latin America and the Caribbean.” Press Release, February 18, 2016. https://www.worldbank.org/en/news/press-release/2016/02/18/world-bank-provides-150- million-to-combat-zika-virus-in-latin-america-and-the-caribbean. World Bank. 2018a. One Health: Operational Framework for Strengthening Human, Animal, and Environmental Public Health Systems at their Interface. World Bank. https://documents1.worldbank.org/curated/en/703711517234402168/pdf/123023- REVISED-PUBLIC-World-Bank-One-Health-Framework-2018.pdf. World Bank. 2018b. World Bank Group Support to Health Services: Achievements and Challenges. Independent Evaluation Group. World Bank. World Bank. 2019. Two to Tango: An Evaluation of World Bank Group Support to Fostering Regional Integration. Independent Evaluation Group. World Bank. https://ieg.worldbankgroup.org/evaluations/regional-integration. World Bank. 2021. Findings for COVID-19 from the World Bank’s Support to Address Ebola Outbreaks. Independent Evaluation Group. World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Topic/COVID19_LessonsFromEb ola.pdf. World Bank. 2022a. Change Cannot Wait: Building Resilient Health Systems in the Shadow of COVID- 19. World Bank. https://openknowledge.worldbank.org/server/api/core/bitstreams/a560d0b2-b2dc-5800- 84a3-0ac0b29477cc/content. World Bank. 2022b. Navigating Multiple Crises, Staying the Course on Long-Term Development: The World Bank Group’s Response to the Crises Affecting Developing Countries. World Bank. World Bank. 2022c. Putting Pandemics Behind Us: Investing in One Health to Reduce Risks of Emerging Infectious Diseases. World Bank. http://hdl.handle.net/10986/38200. World Bank. 2022d. The World Bank’s Early Support to Addressing COVID -19: Health and Social Response. An Early-Stage Evaluation. Independent Evaluation Group. World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/Covid-19-health- and-social-response.pdf. World Bank. 2023a. Preventing, Preparing for, and Responding to Disease Outbreaks and Pandemics: Future Directions for the World Bank Group. World Bank. http://hdl.handle.net/10986/38521. World Bank. 2023b. Results and Performance of the World Bank Group 2023. Independent Evaluation Group. World Bank. http://hdl.handle.net/10986/40886. World Bank. 2024a. Global Challenge Program: Enhanced Health Emergency Prevention, Preparedness, and Response. Approach Paper. World Bank. 31 World Bank. 2024b. New World Bank Group Scorecard FY24–FY30: Driving Action, Measuring Results. World Bank. http://documents.worldbank.org/curated/en/099121223173511026/BOSIB1ab32eaff0051a2 191da7db5542842. World Bank. 2024c. “Proposed Changes to IBRD/IDA Operational Policies to Enhance the World Bank Crisis Preparedness and Response Toolkit.” Board Paper, World Bank. Zeng, Wu, Hadia Samaha, Michel Yao, et al. 2023. “The Cost of Public Health Interventions to Respond to the 10th Ebola Outbreak in the Democratic Republic of the Congo.” BMJ Global Health 8 (10): e012660. 32 Appendix A. Evaluation Design Matrix Table A.1. Evaluation Design Matrix Information Key Questions Required Information Sources Data Collection Methods Data Analysis Methods Limitations Question 1. To hat extent has the Bank Group’s support for preparedness been relevant to addressing client country needs and applying a One Health approach that integrates different sectors? 1.1. To hat extent Information on the Information from internal systems on A portfolio identification protocol or o o s s o suppor r s The alignment analysis has the Bank extent to hich Bank Bank Group support analytic ork, including text analytics and o ss ss uses national level Group’s support Group support projects, IFC investments and validation techni ues ill be used s ill synthesize information data and may mask for preparedness World Bank and IFC advisories bet een FY 2000 and June to identify relevant projects and on preparedness support at the inade uate been relevant to has focused on FY24 for the World Bank and bet een analytic ork in World Bank and country and regional levels. It ill subnational level the client country’s countries ith higher FY05 and June FY24 for IFC. IFC portfolios. assess types of Bank Group support targeting. disease risks, epidemic risks and External data sources ill include the External data ill be compiled by in countries and regions, looking at fre uency of lo er capacities to I FORM Epidemic Risk Index country and year to create a heat the consistency in terms of Data gaps may limit outbreaks, and respond to health European Commission 2020 , otre map on country epidemic risks, continuity and evolution of support time series analysis. gaps in core emergencies and has Dame data on countries’ governance hotspots, governance, social over time, financing amounts, capacities? been timely in and social readiness to address readiness, and other information, timeliness, ratings, and intensity. providing this emerging risks niversity of otre including variations over time, s s o suppor o support. Dame 2023 , and outbreak reports here possible. The analysis ill r ou r s u o s ill Torres Munguía 2022 . Other external cover IDA and IBRD countries ith map the Bank Group’s portfolio data sources may include information available external and portfolio against the heat map data to assess on country preparedness, donor data. the coverage of support in countries financing, veterinary service capacities, ith different levels of epidemic risk and health systems. and preparedness capacities and the extent to hich there has been consistent and speedy support for hot spots in countries and regions over time. 1.2. To hat extent Information on the Information from systematic revie s Protocol for rapid scoping s s o suppor or Evidence of the has the Bank World Bank and IFC’s on effective interventions to support literature revie aligning ith the or p s ill map effectiveness of One Group’s support support, through intermediate outcomes and outcomes conceptual frame ork. The One Health intervention areas in the Health interventions for preparedness different sectors, to for One Health, ith a focus on findings from the literature ill be Bank Group portfolio based on may be limited. applied a One evidence based One articles since 2008, hen the Bank used to create an evidence map, evidence from the literature. Health approach Health interventions Group first addressed One Health. hich ill be used as a aligned ith in countries. 33 Information Key Questions Required Information Sources Data Collection Methods Data Analysis Methods Limitations evidence of World Bank and IFC portfolio of benchmarking tool to assess One Project documents effectiveness or projects and analytic ork supporting Health support in the portfolio. may lack details on proven best the One Health area of the conceptual Protocol to identify One Health interventions. practices? frame ork. interventions in the Bank Group portfolio. 1.3. To hat extent Information on ho Evidence from synthesis articles and Protocol for rapid scoping of s s o suppor The private sector has the Bank ell the Bank Group case studies on private sector models literature on private sector o pr s or ill portfolio of support Group’s support World Bank and IFC to support areas of the conceptual interventions to help enhance map private sector support in the may be limited. engaged the has integrated frame ork. public sector preparedness in portfolio against evidence from the private sector to private sector World Bank and IFC projects areas that align ith the literature and summarize findings on enhance models to support supporting the private sector in areas conceptual frame ork. private sector support in the preparedness for health preparedness. of the conceptual frame ork. Protocol to code support for the portfolio in a visual representation. epidemics and Information on the private sector in the portfolio and K or r s ill pandemics? use of the Crisis map the support areas against the complement the analysis to Preparedness and conceptual frame ork and models understand areas of private sector Response Toolkit. from the literature. support in the portfolio. 1.4. To hat extent Country level External data on a country’s epidemic The data collection protocol ill o r Country portfolio data has the Bank portfolio information risks to sample countries compiled in consist of a rapid revie of the r ss ill take stock of changes may not reflect very Group set up tools on tools added to uestion 1.1 . Evidence on ex ante country portfolio to identify in access to ex ante financing tools recent changes real or mechanisms country portfolios for financing tools from the Crisis changes in the availability of ex since COVID 19 in Bank Group time data , making it since the COVID 19 epidemic readiness, Preparedness and Response Toolkit ante financing tools since COVID country portfolios for epidemic harder to assess pandemic that including crisis and other adjustments made to the 19 to ensure epidemic readiness, readiness. ongoing adaptations. ould allo instruments in the current country portfolio and as ell as structured intervie s K or r s ill be The revie might miss countries to World Bank’s Crisis comparison information from IEG’s ith Bank Group senior operations conducted ith Bank Group senior ne developments in uickly and flexibly Preparedness and COVID 19 evaluation, such as on staff in countries on ex ante operations staff in countries ith countries that are just access financing in Response Toolkit, availability of contingency emergency readiness support of both World high epidemic risks to understand beginning to the event of an and perceptions of response components, IDA regional Bank and IFC ith high epidemic their readiness for a future epidemic implement or adapt epidemic related the extent of crisis indo financing, and funds risks to assess changes to the emergency and the changes made to the tool kit post– health emergency? readiness of country from the Pandemic Emergency portfolio since COVID 19, and the portfolio since COVID 19. COVID 19. management, gaps Financing Facility for rapid response opportunities to further enhance The focus on specific in readiness. World Bank 2022 . readiness. high risk epidemic countries may limit generalizability to all Bank Group countries. 34 Information Key Questions Required Information Sources Data Collection Methods Data Analysis Methods Limitations Question 2. To hat extent have the Bank Group’s efforts contributed to enhancing preparedness for epidemics and pandemics at the country and regional levels? 2.1. Ho ell have Indicators measured Evidence from global reports on Protocol for a rapid literature or o o r o r su s Indicators may not be the results of by World Bank indicators to measure health revie of indicators to measure sur ill revie project ell documented. preparedness projects that address preparedness outcomes in the different aspects of health indicators at the country level to efforts been health preparedness conceptual frame ork. preparedness. examine ho ell they measure measured? and the extent to Indicator data from internal systems Protocol to code the indicator data outcomes and different levels of the hich they measure for closed and active projects in the against the conceptual frame ork results chain, hether the intended outcomes measurement has improved over World Bank portfolio. and the literature and combine it of the support in the Evidence of constraining and ith data on the country situation time, and hether measurement country. from uestion 1. and use of indicators is better in facilitating factors of measuring countries ith more intense or results. Evidence on facilitating and consistent support or regional constraining factors to projects. measurement ill be dra n from intervie s and case studies. K or r s ill be conducted ith task team leaders of selected projects to complement the revie so the evaluation team can understand factors that constrain and facilitate results measurement. 2.2. To hat extent Information on ho World Bank portfolio and country Contribution analysis ill be used s s u s ill analyze the Ongoing projects may have intermediate the World Bank and situation data from uestion 1 ill be for the case studies. The protocol contribution of support in countries only have early outcomes been IFC support has used to select a purposive sample for for data collection ill include to specific intermediate outcome. evidence of the achieved in contributed to case studies focused on five selected revie ing project documents and The intermediate outcome areas for success of change countries and impacts, outcomes, intermediate outcome areas of the analytics ork such as aide analyzing contributions ill include strategies and regions, and ho and change conceptual frame ork about 30 mémoire and ICRs, PCRs, among surveillance, laboratories, biosafety, intermediate sustainable are the path ays to develop cases, or 6 per area . To make it more others ; available evidence on systems for health service delivery, outcomes. outcomes of past preparedness. likely that results can be observed, the results against a specific theory of and access to medical efforts? cases ill focus on countries ith at change for each intermediate countermeasure procurement. The Information on least one closed project, and be outcome elaborated based on the case ill include assessing outcome achievement spread across countries ith differing measurement literature see constraints and facilitating factors may vary for different epidemic risks, and social and uestion 2.1 above ; and and change path ays, and evidence partnerships. governance readiness see data on intervie s ith task teams, clients, of contributions to outcomes, country situations for uestion 2.1 . and partners. against a theory of change specific Fe countries have ist of key partners supporting health The protocol for assessing to the intermediate outcome area. had consistent support preparedness ith the Bank Group in partnerships ill include a first The findings of all cases ill be over time to countries. Information on their phase of intervie s ith partners compiled into a database for and document revie . The second analysis. 35 Information Key Questions Required Information Sources Data Collection Methods Data Analysis Methods Limitations coverage of countries, intended phase ill use a structured K or r s ith task strengthen outcomes, support areas, and uestionnaire to collect feedback teams, clients, and partners involved preparedness. processes used to measure results and from a sample of task teams on in the intervention ill complement prioritize support. External data on the success of partnerships. The case studies. health and donor financing from information collected ill be r rs p s s ill assess the OECD n.d. and GHDE n.d. . compiled into a database for contribution of the partnership Indicators from closed projects. analysis and combined ith activities to outcomes in countries. information on the portfolio and The final output ill be a matrix Project documents, including ICRs, ICRRs, PPARs, XPSRs, PCRs, and other external data on donor financing assessing features of partnerships reports from closed projects in the and country situations see supporting the World Bank portfolio, uestion 1 . as ell as external data on country evaluation portfolio. Achievement data on portfolio situations and donor financing. Portfolio data to identify and select indicators. K or r s ill countries ith closed projects and analytic ork for outcome A protocol for outcome harvesting complement the partnerships documentation, emphasizing to extract and code information on analysis to identify partnership intermediate outcomes that features and reasons underlying outcomes related to One Health and health service strengthening. strengthen institutional capacities potential gaps in coverage. for preparedness in selected or o o r o r su s countries and areas of the sur ill include assessing conceptual frame ork. The the achievement of indicators. protocol ill include revie ing o u o o ou o s from results information in project closed projects in the portfolio ill documents and for substantive be compiled in a data set for visual outcomes in a country context, analysis. collecting structured feedback from t o or three individuals task teams and clients ho are kno ledgeable about the results for validation and learning about the sustainability of the achievement. 2.3. What factors Information on Data on the project portfolio, Data from across the evaluation u r s s at the country Constraints may differ have facilitated or constraints and including project ratings and data on ill be integrated for multivariate and project level ill assess factors based on the country’s hindered the enabling factors country situations and outbreak modeling analysis. associated ith successful project context. implementation, related to change events from uestion 1 , ill be achievements in countries. performance, and path ays for scale combined ith data on outcomes and or o o s s ill code factors Multivariate analysis scale up of up, outcomes, and indicator achievement. affecting implementation and scale does not establish 36 Information Key Questions Required Information Sources Data Collection Methods Data Analysis Methods Limitations preparedness sustainability that Constraints and enabling factors The portfolio analysis ill use the up of preparedness interventions causality, as it is support? affect or hinder described in ICRs of closed World DeCODE taxonomy applied in the from closed projects. susceptible to issues scale up. Bank projects. Results and Performance of the sur s ill be conducted to like omitted variable Constraints and enabling factors World Bank Group 2023 World help the evaluation team understand bias. experienced by task teams. Bank 2023 to identify common factors that have facilitated The validity and factors positively or negatively collaboration and constrained or reliability of the survey affecting the implementation of enabled scaling up and the success results ill depend on World Bank projects that address of Bank Group health preparedness response rates. preparedness. interventions in countries. The survey ill be distributed to World Bank and IFC staff leading projects and analytics. Source: Independent Evaluation Group. Note: DeCODE = Delivery Challenges in Operations for Development Effectiveness; GHDE = Global Health Data Exchange; IBRD = International Bank for Reconstruction and Development; ICR = Implementation Completion and Results Report; ICRR = Implementation Completion and Results Report Revie ; IDA = International Development Association; IEG = Independent Evaluation Group; IFC = International Finance Corporation; OECD = Organisation for Economic Co operation and Development; PCR = Project Completion Report; PPAR = Project Performance Assessment Report; XPSR = Expanded Project Supervision Report. 37 Figure A.1. Evaluation Design Diagram 20 year IFC and World Bank portfolio projects, or o o r s s looking at support areas, advisory services, and indicators ; rapid literature measurement, and factors affecting implementation and revie of indicators results u s o To hat extent has Secondary data on country epidemic risks and s s looking at the portfolio support to the Bank Group s support and preparedness capacities; evidence maps on One country situations, One Health, and the private sector preparedness been relevant to Health and private sector models addressing client country needs and applying a One Health Information on crisis response tools in Bank o r r ss of portfolio Group country portfolios; intervie s ith senior approach that integrates different staff in countries sectors? Sample of countries ith support in outcome Relevance to country situations, One Health, and s s u s that use contribution analysis to assess private sector for uick financing in an epidemic areas; protocol for case studies, including client results in key outcome areas intervie s Completion reports from sampled countries; o u o o ou o s from closed support protocol for outcome harvesting u s o To hat extent have Structured mapping of information on r rs p s s to understand factors of success partnerships from intervie s and document the Bank Group s efforts and outcomes revie contributed to enhancing Survey protocol to collect data on constraints sur to understand factors facilitating and preparedness for epidemics and and facilitating factors of World Bank Group hindering implementation, results, and scale up pandemics at the country and support regional levels? Management, staff, partners, and external actors K or r s to inform findings across Measurement and achievement of outcomes, involved in preparedness support methods sustainability, factors facilitating or hindering outcomes, scale up Data from across the evaluation methods u r ss o p or to understand factors associated ith successful project achievements Source: Independent Evaluation Group. Note: IFC = International Finance Corporation. 38 References European Commission. 2020. “INFORM Epidemic Risk Index” (database). European Commission. Accessed October 10, 2024. https://drmkc.jrc.ec.europa.eu/inform- index/INFORM-Risk/INFORM-Epidemic-Risk-Index. GHDE (Global Health Data Exchange). n.d. “ Development Assistance for Health Database 1990– 2022.” Accessed October 10, 2024. https://ghdx.healthdata.org/record/ihme- data/development-assistance-health-database-1990-2022. OECD (Organisation for Economic Co-operation and Development). n.d. “DAC1: Flows by Sector and Donor.” OECD Data Explorer data set. https://www.oecd.org/en/data.html. Torres Munguía, Juan Armando. 2022. “A Global Dataset of Pandemic- and Epidemic-Prone Disease Outbreaks.” Figshare data set. https://doi.org/10.6084/m9.figshare.17207183.v2. University of Notre Dame. 2023. “ND-GAIN Index.” Notre Dame Global Adaptation Initiative data set. Accessed October 10, 2024. https://gain.nd.edu/our-work/country- index/download-data. World Bank. 2022. The World Bank’s Early Support to Addressing COVID -19: Health and Social Response. An Early-Stage Evaluation. Independent Evaluation Group. World Bank. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/Covid-19-health- and-social-response.pdf. World Bank. 2023. Results and Performance of the World Bank Group 2023. Independent Evaluation Group. World Bank. http://hdl.handle.net/10986/40886. 39