Strengthening pandemic preparedness and response through integrated modelling Strengthening pandemic preparedness and response through integrated modelling Strengthening pandemic preparedness and response through integrated modelling ISBN (WHO) 978-92-4-009088-0 (electronic version) ISBN (WHO) 978-92-4-009089-7 (print version) ISBN (OECD) 978-92-6-433273-7 (electronic version) © World Health Organization, Organisation for Economic Co-operation and Development and International Bank for Reconstruction and Development/The World Bank, 2024 Some rights reserved. This work is available under the Creative Commons Attribution-NonCommercial- ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc- sa/3.0/igo). Under the terms of this licence, you may copy, redistribute and adapt the work for non-commercial purposes, provided the work is appropriately cited, as indicated below. In any use of this work, there should be no suggestion that the World Health Organization (WHO), the Organisation for Economic Co-operation and Development (OECD) and/or the International Bank for Reconstruction and Development/The World Bank (The World Bank) endorses any specific organization, products or services. The use of the WHO, OECD, or The World Bank logo is not permitted. If you adapt the work, then you must license your work under the same or equivalent Creative Commons licence. If you create a translation of this work, you should add the following disclaimer along with the suggested citation: “This translation was not created by the World Health Organization, (WHO), the Organisation for Economic Co-operation and Development (OECD) and/or the International Bank for Reconstruction and Development/The World Bank (The World Bank). WHO, OECD and The World Bank are not responsible for the content or accuracy of this translation. The original English edition shall be the binding and authentic edition”. Any mediation relating to disputes arising under the licence shall be conducted in accordance with the mediation rules of the World Intellectual Property Organization (http://www.wipo.int/amc/en/ mediation/rules). Suggested citation. Strengthening pandemic preparedness and response through integrated modelling. Geneva: World Health Organization, the Organisation for Economic Co-operation and Development and/or the International Bank for Reconstruction and Development/The World Bank; 2024. Licence: CC BY-NC-SA 3.0 IGO. Cataloguing-in-Publication (CIP) data. CIP data are available at http://apps.who.int/iris. Sales, rights and licensing. To purchase WHO publications, see https://www.who.int/publications/ book-orders. To submit requests for commercial use and queries on rights and licensing, see http:// www.who.int/about/licensing. Third-party materials. If you wish to reuse material from this work that is attributed to a third party, such as tables, figures or images, it is your responsibility to determine whether permission is needed for that reuse and to obtain permission from the copyright holder. The risk of claims resulting from infringement of any third-party-owned component in the work rests solely with the user. General disclaimers. For the OECD, this work is published under the responsibility of the Secretary- General of the OECD. The findings, interpretations, conclusions and opinions expressed and arguments employed in this publication do not necessarily reflect the official views of the Member States of WHO, the Member countries of the OECD, or The World Bank, its Board of Executive Directors, or the governments they represent. The names and representation of countries and territories used in this joint publication follow the practice of the WHO. The designations employed and the presentation of the material, as well as any data included in this publication do not imply the expression of any opinion whatsoever on the part of WHO, OECD, and/or The World Bank concerning the legal status of or sovereignty over any country, territory, city or area or of its authorities, or concerning the delimitation of frontiers or boundaries nor to their name. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement. The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by WHO, OECD and The World Bank in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. All reasonable precautions have been taken by WHO, OECD and/or The World Bank to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO, OECD and/or The World Bank be liable for damages arising from its use. Contents Acknowledgements    ix Abbreviations    xi About this guide    xii Key messages    xiv 1. Understanding the challenges of policy-making during epidemics and pandemics    1 1.1 Policy-making in a changing and uncertain environment    2 1.1.1 Understanding the complexity of pathogen transmission and disease severity    2 1.1.2 Identifying optimal policy responses in the face of uncertainty    3 1.2 Managing intense political and time pressure    5 1.2.1 Establishing whole-of-government and whole-of-society approaches    5 1.2.2 Identifying the best available evidence    6 1.2.3 Overcoming disciplinary silos    7 2. Understanding the role of integrated modelling in pandemic preparedness and response    11 2.1 A common framework for epidemiological and economic considerations in policy-making    12 2.1.1 Capturing a wider set of factors influencing the course and impacts of epidemics    12 2.1.2 Capturing key basic factors that drive differential impacts across populations    14 2.1.3 Formalizing ways to weigh the benefits and costs of various policies    14 2.2 A tool to compare the magnitude and uncertainty of effects of various policies    15 2.2.1 Designing and optimizing epidemic or pandemic control policies    15 2.2.2 Evaluating crisis-related economic policies    18 2.2.3 Making explicit assumptions about key uncertain elements    18 iii 2.3 A tool to help strengthen pandemic preparedness and response in policy-making    19 2.3.1 Building, using and refining integrated models through an iterative process    19 2.3.2 Populating, calibrating and validating integrated models    21 3. Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response    24 3.1 Initiative 1: Formally incorporate integrated modelling into policy-making    25 3.1.1 Institutionalizing integrated modelling to inform policy-making    25 3.1.2 Assigning clear modelling-related responsibilities    26 3.1.3 Establishing procedures on the production of adequate integrated models    27 3.1.4 Creating an ecosystem of key data and data sources for policy-making    28 3.2 Initiative 2: Establish and maintain active communications    32 3.2.1 Creating formal communications channels for before, during and after epidemics    32 3.2.2 Communicating policy questions and how integrated modelling can help    32 3.2.3 Facilitating interactions at all stages of new model development    35 3.2.4 Creating active networks of policy-makers and modelling groups    35 3.2.5 Increasing integrated modelling literacy and communication across society    41 3.3 Initiative 3: Match the appropriate integrated model with the right question    42 3.3.1 Building base-modelling structures before epidemics and pandemics or during interepidemic or pandemic times    42 3.3.2 Building capabilities to determine how to match the appropriate models with the right policy questions    43 3.3.3 Ensuring integrated modelling can adapt to policy priorities during epidemics and pandemics    43 iv 3.3.4 Creating procedures and systems for comparative integrated modelling    44 3.4 Initiative 4: Build local integrated modelling production capacity    44 3.4.1 Establishing networks that facilitate cooperation and collaboration between integrated modelling groups    45 3.4.2 Incorporating integrated modelling and real world decision-making into educational programmes    47 3.4.3 Creating new interdisciplinary modelling positions along with appropriate incentives to fill those positions    49 Concluding remarks    51 References    52 Annex 1 Methods and declarations of interest    64 Annex 2 COVID-19 integrated modelling    66 Annex 3 Key data sources used in COVID-19 integrated models    69 Annex 4 A framework to contextualize the four initiatives proposed to enhance the production and use of integrated modelling    74 Annex 5 Ten key principles to help effective communication about modelling    80 v List of figures Fig. 1. Integration of epidemiological and economic considerations through in-person contacts, workforce productivity and behavioural change due to infection avoidance    13 Fig. 2. Iterative process to build, use and refine integrated models    20 Fig. 3. Four-step approach to engage policy-makers in model development and evolution    34 Fig. 4. Collaboratory, an interactive knowledge-sharing space for the epidemic and pandemic intelligence community    41 Fig. A2.1. Simplified representation of COVID-19 integrated modelling    66 Fig. A4.1. A framework of country types along two dimensions: the degree to which modelling is currently used to inform policy-making and the current availability of modelling and integrated modelling    76 vi List of tables Table 1. Projected health and short-term economic costs across all scenarios and additional GDP loss per death averted for each scenario compared to the next-less-stringent scenario    16 Table 2. Steps in integrated modelling and how policy-makers and those supporting them can be involved    36 Table A4.1. Summary of short-term activities by country type to formally incorporate integrated modelling into policy-making (Initiative 1)    75 Table A4.2. Summary of short-term activities by country type to establish and maintain active communications (Initiative 2)    77 Table A4.3. Summary of short-term activities by country type to match the appropriate integrated model with the right policy question (Initiative 3)    78 Table A4.4. Summary of short-term activities by country type to build local integrated modelling production capacity (Initiative 4)    79 Table A5.1. Ten key principles to help effective communication about modelling    80 vii List of boxes Box 1. PHSM    3 Box 2. A decision framework to inform policy-making during the COVID-19 pandemic    6 Box 3. Modelling the COVID-19 pandemic at the Bank of Italy    8 Box 4. Integrated modelling to inform pandemic control strategies in Norway    10 Box 5. Illustrative application of integrated modelling in a lower-middle-income country    16 Box 6. The G20 Joint Finance and Health Task Force for pandemic prevention, preparedness and response    17 Box 7. Approaches to calibrate the locally developed Tekanelo integrated model in South Africa    23 Box 8. Calibration of an existing integrated model to inform policy-making in Sri Lanka    29 Box 9. The FASSSTER data ecosystem and modelling platform in the Philippines    30 Box 10. The Integrated Outbreak Analytics (IOA) initiative    31 Box 11. A four-step approach to engage policy-makers during model development and evolution in Argentina    33 Box 12. Sustained communication and collaboration between local and international modelling institutes, policy-makers and the WHO: the experience of the Philippines’ FASSSTER team    38 Box 13. Collaboratory, an initiative of the WHO Hub for Pandemic and Epidemic Intelligence    40 Box 14. The MIDAS network    46 Box 15. Key success factors in translating modelled evidence into effective policy-making and policy impact    48 viii Acknowledgements The World Health Organization (WHO), the Organisation for Economic Co- operation and Development (OECD) and the World Bank gratefully acknowledge the many individuals and organizations who contributed to the development of this guide. The development of this guide was coordinated by Edith Patouillard of the WHO Department of Health Financing and Economics, who also led the writing, together with: Tessa Tan-Torres Edejer (WHO Department of Health Financing and Economics), Ece Özçelik and Michele Cecchini (OECD Health Division, Directorate of Employment, Labour and Social Affairs), Sarah Alkenbrack (the World Bank, Health Nutrition and Population Practice) and Andrew Burns (the World Bank, Macroeconomics and Fiscal Management Global Practice). WHO, OECD and the World Bank thank external contributors who provided substantial contributions, reviews and suggestions, making this report possible: Bruce Y Lee (City University of New York Graduate School of Public Health & Health Policy), Victoria Fan (Center for Global Development), Raffaele Vardavas (RAND Corporation), Mark Blecher (National Treasury, South Africa), Sarah Pallas (Centers for Disease Control and Prevention, United States of America), Jeanette Vega (Pronova Technologies, Chile), Aditya Goenka (University of Birmingham), Indrani Gupta (Institute of Economic Growth, India), Amanda Glassman (Center for Global Development), Lacina Balma (African Development Bank) and Marcia Castro (Harvard TH Chan School of Public Health). Thanks go as well to those who have reviewed institutional-level illustrations of the production and use of integrated modelling for policy-making: Gesine Meyer-Rath (University of Witwatersrand), Maria Regina Justina E Estuar and Elvira de Lara-Tuprio (Ateneo de Manila University), Adolfo Rubinstein (Center for Implementation and Innovation in Health Policies, Institute for Clinical Effectiveness and Health Policy (IECS), Argentina), Sabina Marchetti, Alessandro Borin, Andrea Brandolini, Francesco Paolo Conteduca and Giuseppe Ilardi (Bank of Italy), Steinar Holden (University of Oslo), Ingrid Hjort (BI Norwegian Business School), Gunnar Rø and Marissa Leblanc (Norwegian Institute of Public Health), and Brittany Hagedorn (Bill & Melinda Gates Foundation). Additional reviewers of specific sections of the report include Katharina Hauck and Patrick Doohan (Abdul Latif Jameel Institute for Disease and Emergency Analytics, Imperial College London), Matteo Pianella (Stockholm University), Sofía Jijón (French National Centre for Scientific Research, CNRS), Pietro Angelo Manfredo Francesco Manfredi (University of Pisa), Alberto d’Onofrio (University of Trieste), Matteo Galizzi (London School of Economics) and Hazel Squires (University of Sheffield). Acknowledgements ix WHO expresses gratitude to WHO technical staff and consultants at headquarters, regional and country offices for their inputs and reviews that helped improve the quality of specific sections: Elena Altieri (Behavioural Insights unit), Brett Archer and Marie-Amelie Degail Chabrat (Department of Surveillance Systems), Tim Chadborn (Behavioural Insights unit), Ogo Chukwujekwuo (WHO Regional Office for Africa), Lorcan Clarke (Department of Health Security Preparedness), Julia Fitzner (Department of Pandemic and Epidemic Intelligence Systems), Graham Harrison (WHO Country Office of the Philippines), Raymond Hutubessy (Department of Immunization, Vaccines and Biologicals, and Secretariat of the G20 Joint Finance and Health Task Force), Nirmal Kandel (Department of Health Security Preparedness), Ramona Ludolph (Department of Epidemic & Pandemic Preparedness and Prevention), Andrew Mirelman (Department of Health Financing and Economics), Sara Pedron (Department of Epidemic & Pandemic Preparedness and Prevention), Claudia Pescetto (WHO Regional Office for the Americas), Alaka Singh (WHO Country Office for Sri Lanka), Ryoko Takahashi (Department of Epidemic & Pandemic Preparedness and Prevention), Chelsea Taylor (WHO Regional Office for the Western Pacific), Katelijn Vandemaele (Department of Epidemic & Pandemic Preparedness and Prevention) and Chatura Wijesundara (WHO Country Office for Sri Lanka). WHO, OECD and the World Bank are grateful to Bruce Aylward (WHO Division Universal Health Coverage/Life Course), Francesca Colombo, Mark Pearson and Stefano Scarpetta (OECD Directorate of Employment, Labour and Social Affairs) and David Wilson (the World Bank, Health Nutrition and Population Practice) for their oversight and review. WHO would like to thank for their funding support the Department of Foreign Affairs, Trade and Development of Canada; the United States Agency for International Development; the Federal Ministry of Health of Germany; and the Directorate-General for International Partnerships of the European Commission. x Strengthening pandemic preparedness and response through integrated modelling Abbreviations DRC Democratic Republic of the Congo EVD Ebola virus disease FEVR framework assessing economic and health vulnerabilities and risks GDP gross domestic product IECS Instituto de Efectividad Clínica y Sanitaria IOA integrated outbreak analytics ISS Istituto Superiore di Sanità JFHTF Joint Finance and Health Task Force MIDAS Models of Infectious Disease Agents Study MOH ministry of health OECD Organisation for Economic Co-operation and Development PHE public health emergency PHSM public health and social measures SAGE Scientific Advisory Group on Emergencies SARS severe acute respiratory syndrome WHO World Health Organization Abbreviations xi About this guide Who is the audience of this guide? This guide was prepared for policy-makers at all levels of government and those who support them, such as technical units, experts and scientific advisory groups and teams responsible for collecting and managing various sources of national and subnational data. The guide is also relevant to the many stakeholders who support evidence-informed policy-making, including academic institutions with expertise in epidemiology, health systems, economics, and social and behavioural sciences; modelling groups; bi- and multilateral agencies; nongovernmental organizations; and funding institutions. Why was this guide developed? The emergence of a novel pathogen with epidemic or pandemic potential gives rise to significant uncertainty about how it will spread and cause disease, whether available control measures can slow its spread and mitigate impacts, and how people will react to policy responses and the risk of infection. Combined with medical countermeasures, public health and social measures (PHSM) can help reduce transmission and mitigate disease severity. However, the implementation of PHSM can have unintended negative social and economic consequences. Thus, alongside a high degree of uncertainty, policy-making must consider both health and socioeconomic impacts of alternative epidemic or pandemic response strategies. Integrated epidemiological–macroeconomic modelling (“integrated modelling”) refers to a wide range of interdisciplinary models that merge transmission drivers, health systems, health outcomes, and economic and other sectoral considerations into a common analytical framework and can explore multiple response policy scenarios in a changing and uncertain environment. Typically, integrated models consist of an epidemiological model that simulates the dynamics of the transmission of a pathogen combined with a macroeconomic model that simplifies the structure of the economy and simulates its functioning. Integration between epidemiological and economic systems captures how changes in the epidemiology may be influenced by changes in the economic systems and vice versa. Integrated modelling can inform policy-making about the benefits and drawbacks of alternative response strategies, notably when stringent measures may be warranted to limit transmission and mitigate morbidity and the risk of overwhelming health systems. Integrated modelling offers a way to explicitly xii Strengthening pandemic preparedness and response through integrated modelling consider key trade-offs inherent to policy-making before, during and after epidemics and pandemics. They can be used to inform a wide range of policy considerations, including protection of health, health systems’ response capacity, education of young generations, capacity of populations to make a living, and the overall economic security and well-being of society. Integrated modelling can therefore help facilitate context-specific, equity-oriented and evidence-informed policy-making. Finally, integrated modelling can strengthen epidemic and pandemic preparedness by encouraging intersectoral and interdisciplinary collaborations between key stakeholders and helping to identify the data needed for better understanding of the complexity of epidemics and pandemics. Several factors may, however, make it difficult to maximize the value of integrated modelling. Historically, many disciplines have remained silos, impeding the development of truly interdisciplinary models. In addition, in many countries, mathematical modelling is not commonly used to inform policy-making. Finally, during epidemics or pandemics that generate both health and socioeconomic crises, policy objectives across sectors may seem to be in conflict, generating additional challenges for intersectoral collaboration and the implementation of optimal policies for society. In this context, and building on earlier joint work (1), the World Health Organization, the Organisation for Economic Co-operation and Development and the World Bank have partnered to produce this guide on integrated modelling, with the hope that it will support epidemic and pandemic preparedness efforts and future policy responses. Annex 1 describes the methods that were used to inform the development of the guide. How is this guide organized? This guide starts by describing the complex challenges faced by policy-makers in the context of epidemics and pandemics. Next, it makes a strong case for the adoption of modelling-based approaches to inform policy-making, with an emphasis on integrated models that merge epidemiological and economic considerations in a changing and uncertain environment. The guide concludes by proposing four initiatives that countries can consider to improve the development, refinement and use of integrated models as part of their efforts to strengthen epidemic and pandemic preparedness and response strategies. About this guide xiii Key messages During epidemics and pandemics, there is often great uncertainty about how biological, epidemiological, social, behavioural, economic and other contextual factors interact and influence the transmission of a pathogen and the impacts of various combinations of PHSM and medical countermeasures (2–10). Combined with intense political and time pressure, these uncertainties can amplify the tensions inherent in policy-making, especially when people’s lives and livelihoods, two interdependent societal objectives, are at stake (11, 12). Mathematical modelling can play an important role in policy development because it helps to overcome the inherent difficulties in projecting the likely outcomes of alternative response policies in a rapidly changing environment (13–15). Historically, modelling in many disciplines has remained in silos, limiting the ability of models to paint a comprehensive picture to inform policy-making. For instance, epidemiological modelling alone cannot account for the potential economic cost of an epidemic and possible control measures, whereas macroeconomic modelling does not consider important factors that influence the spread of a pathogen. Given these limitations, integrated epidemiological–macroeconomic modelling (integrated modelling) has emerged in recent years as an interdisciplinary approach to support evidence-informed policy-making (16–20). Using a joint analytical framework, integrated modelling considers interdependent policy objectives during epidemics and pandemics. These include reducing transmission1 and excess deaths, mitigating hospital admission overload, reopening schools and minimizing the costs of service closures on society. Thus, integrated modelling can enhance the understanding of transmission mechanisms and influencing factors, and systematically simulate the potential health and economic outcomes of various policy options under a range of assumptions about key variables. Integrated modelling also brings transparency in the way trade-offs inherent to policy-making are considered in a context of intense political and time pressure. Finally, building capacities in integrated modelling before and between epidemics or pandemics can contribute to better preparedness through enhanced intersectoral and interdisciplinary collaborations, and thus better guide policy-making (21, 22). 1 In this guide, the focus is on modelling human-to-human transmission of pathogens via contacts. xiv Strengthening pandemic preparedness and response through integrated modelling To enhance the production and use of integrated modelling in policy-making, this guide proposes four initiatives and associated activities for before, during and after epidemics and pandemics: Initiative 1: Formally incorporate integrated modelling into policy-making. Activities include i) institutionalizing the use of modelling in policy-making; ii) giving government units clear responsibilities for modelling; iii) establishing procedures on the use of models; and iv) creating an ecosystem of key data and data sources. Initiative 2: Establish active communication between the various actors. Activities include i) creating formal communication channels between policy-makers and modelling groups; ii) communicating policy questions and associated scenarios and what models can do; iii) establishing systems to facilitate interactions at all stages of model development; iv) creating active networks of policy-makers and modelling groups; and v) establishing systems to increase integrated modelling literacy and communication of activities and results across society. Initiative 3: Match the policy questions of interest to the appropriate integrated models. Activities include i) building base-modelling structures during interepidemic times based on key policy questions of interest; ii) building capabilities to determine how to match the appropriate models with the right policy questions; iii) encouraging the refinement of integrated models to ensure they can adapt to policy priorities; and iv) creating procedures and systems for comparative modelling. Initiative 4: Build greater awareness and understanding in integrated modelling. Activities include i) establishing networks for collaboration between integrated modelling groups; ii) incorporating integrated modelling and real world decision-making into educational programmes; and iii) creating interdisciplinary modelling positions with adequate incentives. Key messages xv Initiatives and activities should be tailored to context and prioritized based on available resources, funding prospects and return-on-investment considerations. As circumstances may change, a flexible implementation process should be established, alongside a system to monitor implementation progress and impacts. xvi Strengthening pandemic preparedness and response through integrated modelling 1. Understanding the challenges of policy-making during epidemics and pandemics This section starts by presenting the complex set of interacting factors that influence the spread and impacts of epidemic- and pandemic-prone pathogens and the effects of response policies. It then describes how these interacting factors complexify policy-making. This section demonstrates the need for both intersectoral and interdisciplinary collaborations to inform policy-making when epidemics or pandemics put two socially important objectives at risk, namely people’s lives and livelihoods. Key messages • There is great uncertainty as to how a complex array of interacting biological, epidemiological, social, economic, behavioural, cultural and environmental factors influence the spread of a pathogen, its ability to cause severe disease and overwhelm health systems, and the effect of response policies. • Intense time pressure, resource limitations and the pressing need to identify and incorporate the best available evidence across siloed disciplines further complicate evidence-informed and inclusive policy-making. • Decisions about whether to begin, maintain, scale-up, scale-down or change response activities should be informed by evidence on the benefits, drawbacks and differential impacts of alternative policy responses across various sectors and populations, and should consider various sources of uncertainties. • Intersectoral and interdisciplinary collaborations can help policy-makers balance multiple societal objectives and optimize the impacts of policies across the range of outcomes at stake. 1. Understanding the challenges of policy-making during epidemics and pandemics 1 1.1 Policy-making in a changing and uncertain environment When a novel epidemic or pandemic-prone pathogen emerges, policy-makers must navigate changing and uncertain circumstances. To face such unfamiliar situations and navigate the increasing complexity of their role, policy-makers must learn about the wide array of interacting factors that cross multiple temporal and spatial dimensions and that influence the spread and potential impacts of an epidemic or pandemic-prone pathogen. Policy-makers must also identify optimal policy decisions about whether and how to introduce, maintain, scale up, change or phase out response activities under a large degree of uncertainty. 1.1.1 Understanding the complexity of pathogen transmission and disease severity The ability of a pathogen to spread in a population and cause severe illnesses is influenced by many biological, epidemiological, social, economic, cultural and environmental interacting factors The epidemic potential of an infectious pathogen lies in its ability to spread in a human population. This ability is promoted or hindered by a range of factors, including, for example, the pathogen’s mode of transmission (such as respiratory, sexual, skin) and the routes of contact between infected and susceptible individuals (airborne, direct contact, vector) (2). Pathogens with different modes of transmission and different contact routes have different abilities to spread in a population. More transmissible variants can also emerge and enhance the pathogen’s ability to spread among humans. Human susceptibility can be influenced by age, general health status and immunity level from potential prior infection or vaccination. Cultural, psychological, environmental and other contextual factors also matter (3). For example, people living or working in a crowded environment or close to others may be more susceptible to infections from an airborne pathogen. Individuals’ perception of risk and their ability to complete a task, such as taking several precautionary measures to avoid infection and their fear and misperceptions about an unfamiliar disease, also influence their susceptibility (4–6). New variants can also increase human susceptibility if they possess enhanced capacity to evade pre-existing immunity in humans. Disease severity also depends on many factors at different levels, including pathogen characteristics, pathogen load in the infectious contact, the immune protection of the infected person, the infected person’s general health status, the types and timing of the care received and the availability and cost of health care. 2 Strengthening pandemic preparedness and response through integrated modelling 1.1.2 Identifying optimal policy responses in the face of uncertainty Policy-makers are faced with difficult trade-offs and distributional considerations Policy-makers must choose from an increasing number of response options while dealing with a large degree of uncertainty. A combination of public health and social measures (PHSM) (Box 1), vaccinations, diagnostics and effective therapeutics is needed for an effective response (23). Box 1.  PHSM PHSM can contribute to reducing infections and severe diseases and protecting health systems by: • reducing in-person contacts between individuals or groups of people (that is, reducing physical interactions – for example, by modification of services such as schools); and • making contacts safer (for example, by promoting individual protection measures such as face masks and hand washing, or physical environment measures such as ventilation). PHSM have been referred to as “complex interventions in complex systems”, with several interacting components and numerous contextual factors influencing their choice, design, implementation and impacts. Various aspects must be considered when introducing, implementing or adjusting PHSM: • the type of intervention: physical distancing, school-based measures, international travel measures, for example; • the level of intervention stringency: ranges from recommending actions and informing about options through to implementing mandates that eliminate choices; • the target population and implementation settings: schools, points of entry and workplaces, for example; • the sector(s) of implementation: health, education, labour and trade, for example; • the level of implementation: international, national, subnational, community, individual; and • the timing of introduction, scale-up and phase-out of PHSM. Sources: Norris et al. (24), Rehfuess et al. (25) and Michie et al. (26) 1. Understanding the challenges of policy-making during epidemics and pandemics 3 The COVID-19 pandemic has shown how the spread and control of a pathogen can have broad reverberations, not only on health and the health sector but also on labour, education, tourism and hospitality sectors, among others (27). In addition, within each sector, different population groups can be affected differently, with larger negative consequences on those with greater health and/or socioeconomic vulnerabilities (8). In some cases, PHSM can have positive intended outcomes (such as decreased transmission, reduced morbidity and mortality) as well as positive unintended consequences (such as reduced traffic injuries and air pollution due to reduced population movement). Conversely, PHSM may have unintended negative effects for the health and socioeconomic well-being of individuals and societies (for example, reduced production, consumption, employment and income due to service closures; reduced educational attainment and learning opportunities due to school closings; increased social and gender inequality due to an increased care burden for women; increased violence against women and vulnerable people) (8, 28). Disadvantaged groups that have limited financial resilience to begin with and workers with lower levels of education, especially younger ones and women, may experience disproportionate socioeconomic consequences in addition to the health risk posed by the epidemic itself (29). In this complex environment, it is important that the benefits and burden of alternative response strategies are systematically compared to inform policy-making during the different phases of an epidemic (8). Such evaluation is important when the risks posed by an epidemic and alternative response policies are high. Policy-making must consider the uncertainty in how individuals respond to epidemics and pandemics During epidemics or pandemics, many types of behaviours and changes in behaviours arise, including use of health care services, engaging in economic and social activities and responding to policies. Each behaviour and change in behaviour is influenced by many factors, so each varies according to the context in which it occurs (9, 30). Perception of the risk associated with an epidemic or pandemic, beliefs about the effectiveness of preventive measures, and trust in government and other institutions are often highly influential (8, 10). People differ in their preferences and ability to access and process information and are often affected by personal values and beliefs. People are often prone to biases, think intuitively and act while being in emotional states. People are often highly influenced by social norms and the availability of social support and, correspondingly, may often act in a prosocial manner during an epidemic or pandemic, which is important for infection control and social cohesion (8, 31–33). Finally, the physical environment can have great influence on epidemic- or pandemic-related behaviours. Physical factors include, for example, the availability of protective equipment and handwash; the availability of space to physically distance at home, at work or in places for socializing; and options for ventilation. Understanding behavioural drivers by integrating 4 Strengthening pandemic preparedness and response through integrated modelling behavioural science when evaluating the benefits and costs of alternative response activities can help in the design of more effective policies (8, 34–36), including during a public health emergency (10, 37). Policy-making can also benefit from having information about the degree of uncertainty regarding the benefits and costs of various policies and behavioural responses (36, 38). 1.2 Managing intense political and time pressure During an epidemic or pandemic that generates both health and socioeconomic crises, policy objectives may seem to be in conflict while there is a pressing need for decisions to be made. 1.2.1 Establishing whole-of-government and whole-of-society approaches Policy-makers must make timely decisions while considering their potential differential impacts across various sectors and populations When policy-makers aim to balance multiple societal objectives across different population groups and sectors, joint activities performed by diverse ministries and public administrations while involving additional stakeholders (such as civil society and nongovernmental organizations) can inform decision-making processes (11, 12, 21, 39–45). This approach can also help acknowledge trade-offs in a more transparent manner, and support policy-makers make decisions about how various outcomes should be weighted based on their value to society and how to explicitly consider factors related to the distribution of costs and benefits within a population (14) (Box 2). However, these approaches may take time to establish and are best established as part of epidemic and pandemic preparedness efforts (41). 1. Understanding the challenges of policy-making during epidemics and pandemics 5 Box 2.  A decision framework to inform policy-making during the COVID-19 pandemic During the COVID-19 pandemic, policy-makers were faced with complex decisions about how to sustain both the lives and livelihoods of all members of society and protect the most vulnerable in both the short and the long term. To support policy-makers, WHO and OECD proposed a decision framework that can be used by national and subnational decision-making bodies tasked with informing or choosing implementation and adjustment of COVID-19 response strategies. The framework is rooted in WHO guidance on considerations for implementing and adjusting PHSM and offers a systematic and stepwise approach (like an “impact inventory approach” (11)) to weigh PHSM options against their wider impact on societies. The framework (1) starts from the health dimension, with an assessment of the epidemiological situation and health system capacity and potential PHSM. The framework then considers other dimensions of importance to a given society, such as economic and equity dimensions, while other considerations of importance may be added according to the context. The framework can be developed using quantitative and qualitative data through concerted dialogue and deliberation among a broad range of stakeholders, including representatives of vulnerable and underrepresented groups who may be affected by the pathogen and alternative response interventions. Sources: WHO (1, 23) 1.2.2 Identifying the best available evidence Several factors can make it difficult to rapidly identify useful quality evidence to inform policy-making Evidence-informed decision-making entails identifying, appraising and mobilizing the best available evidence for safe and effective policies (46). At the start of an epidemic or pandemic, a lack of information may prevent a full understanding of the complex systems at play and of the impact of different policy measures in different contexts (47). Surveillance systems are essential to responding to public health events and guide planning and preparation for 6 Strengthening pandemic preparedness and response through integrated modelling future events. However, linkages between multiple data collection systems may be limited and there may not be enough data specialists available during an emergency (42, 47). In addition, data collection through routine surveys and primary research studies may be disrupted. Secondary research such as systematic, rapid, scoping reviews can help summarize the available evidence to guide the policy response. As an epidemic or pandemic evolves, more information and data are collected, and reviews may soon become outdated. Limited time, human and financial resources may constrain the conduct and update of these reviews. The available evidence may also be skewed towards certain settings, and its applicability away from the context must be assessed and judged carefully. Finally, while the pivotal roles of data and evidence for effective policy have been extensively documented, it is recognized that other influential and sometimes conflicting factors can arise, including the political context, economic interests, institutional constraints and citizen values (46). 1.2.3 Overcoming disciplinary silos Each discipline thinks about a situation in its own way and represents and understands the world differently Mathematical models are used in many disciplines to simplify complex real world situations. In a rapidly changing environment, they can help overcome the inherent difficulties in predicting likely outcomes of alternative response policies. Traditionally, different disciplines tailor their models to different questions and objectives (48). For instance, response policies to the COVID-19 pandemic, including PHSM and medical countermeasures in several countries, were informed by both epidemiological and macroeconomic modelling. Epidemiological models were used to simulate the natural course of the epidemic in different settings by incorporating key epidemiological factors (for example, population mixing patterns, contact rates) that determine the dynamics of infection transmission. These models were also used to compare the impacts of different combinations of PHSM and medical countermeasures on the transmission of SARS-CoV-2 and COVID-19 clinical outcomes. Macroeconomic models were used to simulate the functioning of economies in different settings by describing the sets of agents or sectors and their interactions and behaviours. This allowed evaluation of the impact of infections and disease-related deaths on economic outcomes and of crisis-related policies intended to mitigate negative economic and social consequences, including gross domestic product and unemployment. Although often disciplines remain silos, maintaining different and separate languages, terminologies, approaches and cultures, few policy questions during large-scale epidemic outbreaks are purely epidemiological or economic questions. 1. Understanding the challenges of policy-making during epidemics and pandemics 7 Policy-making requires evidence from the integration of different disciplines Joint work between disciplines can help policy-makers consider the complexity of epidemics and the multiple perspectives and dimensions involved, and provide innovative and relevant information for policy-making (14, 49) (Box 3). However, interdisciplinary work continues to prove a real challenge (49). Working with and between disciplines requires constant explanation, adaptation and scientific readjustment from all researchers involved (49). For successful articulation of various disciplines, notably epidemiology, social sciences and humanities, three prerequisites have been identified, including mutual questioning of scientific stances and research environments; awareness of researchers’ requirements linked to their disciplinary affiliations; and joint elaboration of research, implying a constant flow between different types of knowledge (49, 50) (Box 4). However, during epidemics and pandemics, there may not be enough time to bridge these divides. Thus, rapid and effective collaboration under pressure requires advanced preparation and planning so that different disciplines can establish a common language and better understand the interactions across specialities before an epidemic (1, 22, 35, 47). Box 3.  Modelling the COVID-19 pandemic at the Bank of Italy In the early stages of the COVID-19 pandemic, the Bank of Italy research department established an internal task force to monitor the global spread of the virus, to analyse its economic impacts, and to assess policies to counter the crisis. To better understand epidemic and health trajectories, an admittedly unfamiliar topic for central banks’ researchers, the Bank of Italy established a collaborative partnership with public health experts and epidemiologists from the Italian National Institute of Health (Istituto Superiore di Sanità, ISS) and the Bruno Kessler Foundation (FBK). This interaction allowed Bank of Italy researchers to develop epidemiological models that could be used for scenario analyses. The partnership revealed two main challenges to effective cooperation between economists and health experts. First, communication was hampered by different approaches to data analysis and modelling. Second, priorities understandably differed: specifically, the economists’ need of timely, if incomplete, information for prompt economic policy decisions had to be balanced against the scientists’ need for a rigorous approach to modelling the epidemic. Moreover, confidentiality issues initially restricted the exchange of data between the institutions. continued 8 Strengthening pandemic preparedness and response through integrated modelling Box 3.  continued As the COVID-19 pandemic evolved, the close interconnection between health and economic outcomes called for an integrated modelling approach. The first step was to gather more precise information on mitigation strategies to assess their potential impact on contagion and economic activity. A detailed database of PHSM was constructed, with information at regional and local levels on closures of different types of businesses and schools. These data were then used to elaborate a synthetic national indicator of the intensity of restrictions. The second step was to work closely with ISS and FBK and leverage the expertise acquired by the Bank of Italy to develop a realistic model that tracked the regional evolution of adaptive restrictions on epidemic modelling. This resulted in an enriched epidemiological model with an added component that would reproduce the rule-based mechanism adopted by the Italian Ministry of Health in November 2020, which assigned each region to a restriction tier on a weekly basis. The flexibility of that component allowed prompt adjustment of rules over time. Although accounting for regional differences was essential for accuracy, the focus was on the overall national picture. This model enabled timely comparison of alternative policy mechanisms, and of counterfactual scenarios to disentangle the impact of specific epidemic factors, vaccination rollout and policy provisions. The final step was to feed the model outcomes into econometric forecasting tools to improve projections on the economic activity in Italy. The results from these analyses regularly informed the top management and senior policy staff at the Bank of Italy. This experience revealed the importance of being able to quickly adjust the economists’ modelling tools to account for unexpected and extraordinary non-economic factors, and the value of establishing an effective cooperation with experts from other disciplines (public health experts, in this case) to build on varied expertise and accumulated knowledge. Sources: Aprigliano et al. (51), Conteduca & Borin (52) and Marchetti et al. (53) 1. Understanding the challenges of policy-making during epidemics and pandemics 9 Box 4.  Integrated modelling to inform pandemic control strategies in Norway During the COVID-19 pandemic, the Norwegian government appointed an expert committee to evaluate the economic consequences of the pandemic and the interventions used to control infections. This group was led by Steinar Holden, professor in economics from the University of Oslo. In addition to academic members from different fields of economics, the group also included a wide range of expertise from local government, hospitals, central health authorities and the Ministry of Finance and Statistics Norway. Among other things, the committee developed an integrated economic evaluation model that was combined with an epidemiological model by the Norwegian Institute of Public Health and used to conduct a cost–benefit analysis of various COVID-19 pandemic response strategies. The aim was to estimate the economic and social impacts of PHSM – notably, quarantine and isolation and vaccination strategies – on gross domestic product (GDP), productivity and welfare, and compare these to the direct pandemic health impacts, such as the loss of quality-adjusted life years for people with and without COVID-19. The methodology was new and innovative, and developed in an iterative approach for updating and improving the framework for each of the four assessments conducted. The integrated analysis provided insights about the overall control strategy, specific interventions, and the importance of local versus national interventions in Norway. The collaboration between economists and infectious disease modellers was seminal and yielded valuable insights academically, for the public debate and for the government. These insights would not have been possible if each field worked independently. This sentiment was echoed by the government-appointed commission to evaluate the response to the COVID-19 pandemic. Their findings highlighted that interdisciplinary collaboration between health and economics professionals enriched the depth of strategic insights for managing the pandemic. This collaboration also underscored the practical and policy-making applications of mathematical modelling. Sources: Koronautvalget (54) and Norwegian Directorate of Health (55) 10 Strengthening pandemic preparedness and response through integrated modelling 2. Understanding the role of integrated modelling in pandemic preparedness and response This section describes how modelling can help policy-making amid a high degree of uncertainty, with an emphasis on integrated models that merge epidemiological and macroeconomic systems. It, then, demonstrates how integrated models can support policy-making before, during and after epidemics and pandemics. Next, the section turns its attention to the main challenges that may undermine efforts to produce and use integrated modelling to inform policy-making. Key messages • During health emergencies such as epidemics or pandemics, policy-makers must make timely decisions to introduce, keep, scale up, change or scale down response activities while faced with many uncertainties and complex interactions between a wide array of factors. • The integration of epidemiological and macroeconomic models in a shared analytical framework can help untangle these interactions and serve as a tool to understand which policy options may be more robust to safeguard population lives and livelihoods in the face of uncertainty. • Integrated modelling offers flexible analytical frameworks that can inform policy-making in different phases of epidemics and pandemics: – before or between epidemics and pandemics: to help design and plan strategies for epidemic and pandemic preparedness and response; – during epidemics and pandemics: to refine policy questions, predict health and economic outcomes of alternative response options, and to evaluate preferred strategies; and – at the end of an epidemic or pandemic: to consolidate the learnings of what went right and what went wrong with the response to help inform future preparedness and response. 2.  Understanding the role of integrated modelling in pandemic preparedness and response 11 • Integrated modelling requires an iterative process and data ecosystems that allow close coordination and communication across stakeholders from multiple sectors and disciplines. 2.1 A common framework for epidemiological and economic considerations in policy-making Integrating epidemiological and macroeconomic models in a shared analytical framework can help represent and understand the key interactions at play within and between epidemiological and economic systems. 2.1.1 Capturing a wider set of factors influencing the course and impacts of epidemics Integrating epidemiological and macroeconomic models in a shared analytical framework can help represent the interactions at play within and between epidemiological and economic systems Fig. 1. illustrates how epidemiological and economic components may be integrated. For additional information on COVID-19 integrated models specifically, see Annex 2 and Bonnet et al. (16). Typically, integrated models include several key components: • an epidemiological component that simulates the dynamics of the transmission of a pathogen in response to changes in epidemiological parameters and projects outcomes in terms of number of infections, deaths and hospital admissions; • a macroeconomic component that simplifies the functioning of the economic system and projects outcomes such as changes in GDP, income, employment, production and consumption levels; and • an integration approach through which the epidemiology (such as the number of infected cases) is influenced by the economy (for example, contacts in the workplace or during consumption), and/or the economy (workforce productivity, consumption) is influenced by the epidemiology (Fig. 1). Integration can be done through: – in-person contacts, which may change due to the epidemiological or health status of individuals or groups of individuals; for example, through isolation or hospitalization, or due to mandatory PHSM such as stay-at-home orders and closing services; – workforce productivity, which is affected by the epidemiological or health status of individuals or groups of individuals; for example, due to isolation, 12 Strengthening pandemic preparedness and response through integrated modelling hospitalization or deaths, or due to mandatory PHSM that reduce contacts, such as stay-at-home orders and closing services; and – voluntary changes in behaviours, which are due to infection avoidance (such as reduction in in-person contacts) with negative economic but positive health consequences (for example, reduced consumption but reduced infections), or conversely, with limited or no adherence to mandated policies that hurt health outcomes but may reduce economic harm. Fig. 1.  Integration of epidemiological and economic considerations through in-person contacts, workforce productivity and behavioural change due to infection avoidance Population differentiated by human contact drivers (e.g. age and occupation such as retired adults, working adults, school and preschool children) Pressure on Epidemiologic outcomes health system (e.g. infections in workplaces, schools, (e.g. communities) hospital bed occupancy) Contacts Workers’ Behaviours at work, productivity consumption places, schools Economic outcomes (e.g. GDP, consumption, unemployment, savings) Note: From top to bottom: underlying population groups, epidemiological outcomes and health system constraints, possible integration channels between epidemiology and economy, and economic outcomes. Arrows show directions of relationships. In the context of integrated modelling, “behaviours” are changes in contacts during consumption and work that result from people’s perceptions of the risk of infection in their community or from their infection status. Contacts at work, consumption places and schools result directly from public policies that intend to reduce transmission by changing contact patterns and making contact safer. Sources: based on Goenka & Liu (56) and Haw et al. (20) 2.  Understanding the role of integrated modelling in pandemic preparedness and response 13 2.1.2 Capturing key basic factors that drive differential impacts across populations Some of the key basic factors that influence the susceptibility of a person to infection, severe illness, death and negative socioeconomic consequences can be accounted for in integrated models In COVID-19 integrated models, epidemiological heterogeneity has been considered in terms of age (children, adults), location (schools, workplaces, communities and so on), workplaces (such as people working in high- and low-infection-risk sectors) or sectors, which may be defined by contact rates, size of the employed workforce, or some other factors. Economic heterogeneity has been represented in terms of different goods, sectors, labour market participation (depending on age, sex and educational status), labour productivity, consumption preferences, and the geographical location of production and consumption. Economic heterogeneity that is driven by the epidemiology has also been accounted for; for example, when susceptible and infected people are assumed to differ in their consumption behaviour and labour market participation because of varying infection avoidance behaviours, isolation or hospitalization. Additional sources of behavioural heterogeneity among population groups may include the extent to which protective measures (mask wearing, hand washing, ventilation and so on) are employed and whether economic support policies are accessed (57). 2.1.3 Formalizing ways to weigh the benefits and costs of various policies Some integrated modelling helps formalize approaches to balance the inherent trade-offs of policy-making In the case of epidemic control policies, integrated models can be used for scenario modelling or short-term projections (weeks, months). In this case, they focus mainly on comparing epidemic-related health and economic outcomes, and do not consider the potential unintended health consequences of response policies (mental health conditions, delayed diagnosis of chronic diseases, disrupted preventive and curative treatments of infectious diseases and so on). In addition, these integrated models focus on short- and medium-term outcomes without considering the longer-term economic impacts of epidemic response policies that are traditionally of interest to conventional macroeconomic models. Other integrated models can help understand the longer-term economic and health trajectories of an epidemic or pandemic based on current conditions and trends, and assumptions about the future. 14 Strengthening pandemic preparedness and response through integrated modelling The precision of longer-term models tends to decline with the length of the projection, so policy simulations and sensitivity analysis using these models tend to focus on the change in outcomes under different policy or external assumptions. These longer-term integrated models tend to be more aggregated than shorter-term models and therefore may not address operationally relevant issues for policy-makers, such as where or in which sectors will an epidemic or outbreak get bigger. Longer-horizon integrated models can help determine how much should be invested in preparedness in the first place, and in the various components and values of preparedness (58, 59). 2.2 A tool to compare the magnitude and uncertainty of effects of various policies Integrated models provide a helpful framework within which the benefits and drawbacks of alternative response strategies on health and economic dimensions can be systematically compared. They can also provide important information about the relative certainty of effects of various policy options. 2.2.1 Designing and optimizing epidemic or pandemic control policies Integrated models can be used to compare the health and economic impacts of alternative strategies and identify pandemic preparedness and response policies that maximize benefits and minimize costs During the COVID-19 pandemic, integrated modelling focused on the immediate consequences of alternative policies, including averted infections and income loss within no more than a year. It was used to project the health and economic impacts of measures that were intended to limit the number of disease-susceptible people (such as vaccination), the number of severe illnesses (such as vaccination and therapeutics) and/or that increase the capacity of the health systems (such as setting up field hospitals). Integrated modelling was also used to identify optimal control policies that can maximize health benefits and other socially desirable outcomes – for example, protecting health system responses and maintaining in-person education – while minimizing economic losses as measured by GDP and employment (60) (Box 5). More recently, integrated modelling has been instrumental to evaluate the role of investing in pandemic preparedness in reducing health and economic losses associated with potential future pandemics caused by one of four respiratory pathogens responsible for the COVID-19, Spanish flu, SARS and swine flu epidemics (59) (Box 6). 2.  Understanding the role of integrated modelling in pandemic preparedness and response 15 Box 5.  Illustrative application of integrated modelling in a lower-middle-income country The Jameel Institute, Imperial College London calibrated their integrated model to a lower-middle-income country to produce short-term projections of the trajectory of the COVID-19 epidemic and associated GDP loss in that country compared to 2019 pre-pandemic levels. They modelled four scenarios. Each assumed a different level of sectoral service closures in the 35 sectors of the selected country’s economy, alongside other measures that were in place to help reduce transmission, including test-and-trace, physical distancing, limits to social gatherings, wearing of facemasks and vaccination. Table 1 illustrates the modelled outputs of this exercise, including the projected health and economic costs of different scenarios, each assuming a different level of PHSM stringency. It also compares each scenario with the next-less-stringent scenario in terms of the incremental cost per additional death averted. For instance, under a low-stringency scenario, the number of deaths could reach nearly 25 800 at a cost of US$ 0.17 billion, while under a mid-to-low scenario, the number of deaths would be reduced to nearly 12 200 at a cost of US$ 0.73 billion. This would be equivalent to 13 600 deaths averted at a total potential cost of US$ 0.56 billion, or an additional estimated cost of around $41 000 per death averted under the mid-to-low scenario compared to a low-stringency scenario. Table 1.  Projected health and short-term economic costs across all scenarios and additional GDP loss per death averted for each scenario compared to the next-less-stringent scenario Estimated GDP Incremental GDP loss per Scenario (policy loss, billion additional death averted response stringency Number of US$ (% of pre- compared to next-less- level) deaths pandemic GDP) stringent scenario (US$) Low stringency 25 783 0.17 (0.2%) NA (no. 4) Mid-to-low 12 181 0.73 (0.9%) 41 097 stringency (no. 3) Mid-to-high 4 708 2.23 (2.6%) 200 990 stringency (no. 2) High stringency 2 526 3.47 (4.2%) 571 036 (no. 1) NA: not applicable; the least stringent scenario is the comparator Sources: WHO (61, 62), Haw et al. (63) and Doohan et al. (64) 16 Strengthening pandemic preparedness and response through integrated modelling Box 6.  The G20 Joint Finance and Health Task Force for pandemic prevention, preparedness and response In October 2021, a G20 Leaders meeting in Rome, Italy, established a Joint Finance and Health Task Force (JFHTF) aimed at enhancing finance–health dialogue and cooperation at the global level on issues related to pandemic prevention, preparedness and response. As outlined in the G20 Bali Leader’s Declaration under Indonesia’s G20 Presidency in November 2022, the Task Force was mandated to continue developing coordination arrangements between finance and health ministries, and share best practices and experiences from past finance–health coordination in order to develop joint responses and policy actions to future pandemics. In shaping G20’s future strategies to better understand economic vulnerabilities and risks from pandemics and how to mitigate them, the development of a framework assessing economic and health vulnerabilities and risks (FEVR) from pandemics was initiated by the G20 JFHTF in collaboration with global institutions like WHO, World Bank, International Monetary Fund and the European Investment Bank. Integrated modelling has been instrumental in the G20 FEVR initiative to explore hypothetical pandemic scenarios caused by one of seven respiratory pathogens, each with a disease profile informed by past epidemics. It was used to consider the trade-offs in health, social and economic outcomes for alternative response strategies inspired by policies chosen by countries during the COVID-19 pandemic. Finally, integrated modelling was used to evaluate the role of investing in pandemic preparedness in reducing health and economic losses associated with potential future pandemics caused by one of four respiratory pathogens responsible for the COVID-19, Spanish flu, SARS and swine flu epidemics. Sources: WHO et al. (59), Ministero dell’Economia e delle Finanze (65), United States Government (66) 2.  Understanding the role of integrated modelling in pandemic preparedness and response 17 2.2.2 Evaluating crisis-related economic policies Other interventions can be evaluated, such as crisis-related economic policy (such as income support, subsidies, cash and in-kind transfers) that help reduce the immediate socioeconomic burden of an epidemic, protect the most vulnerable populations, and encourage protective behaviours from individuals and communities In integrated models with a longer time horizon, the future costs of crisis-related economic policies (including increased debt levels, higher inflation) and future fiscal consolidation can also be brought into play. For instance, sectoral closures generate important economic changes in employment, incomes and demand, such as loss of employment in high-contact sectors, and reduction in demand for high-contact goods and services. In-crisis economic policies – such as changes in unemployment schemes, wage and other subsidies to firms, and loan and eviction forbearance policies – have important short-term effects such as dampening negative economic impacts. However, significant longer-term effects such as increased indebtedness translate into future fiscal tightening, and pent-up demand translates into inflationary pressures that have economic consequences and require responses by policy-makers. 2.2.3 Making explicit assumptions about key uncertain elements By varying assumptions around the various uncertainties, integrated models can help understand which policy options may be more robust in the face of uncertainty Integrated models can help account for the uncertainty about the virulence of a novel pathogen and its transmission mechanisms, the behavioural reactions of the population to infections and the effectiveness of policy responses to slow its spread and limit its negative health and socioeconomic consequences. Thus, they can help inform policy-makers about the potential health, social and economic impacts of different interventions. They can also clarify which results depend on factors that may change over time or differ by location, such as pathogen transmissibility, population health status, immunity and socioeconomic vulnerabilities (38). As more information becomes available, the models can be updated, and the uncertainty of potential outcomes narrowed. 18 Strengthening pandemic preparedness and response through integrated modelling 2.3 A tool to help strengthen pandemic preparedness and response in policy-making Preparing an integrated base-model structure and identifying key data needs before an epidemic or pandemic can support preparedness efforts and future policy responses. 2.3.1 Building, using and refining integrated models through an iterative process Integrated modelling involves building and validating models against real life observations, testing different policies, generating results and refining existing models and scenarios Most applications of integrated modelling and modelling in general allow for many repeats of this cycle over weeks, months or even years (67) (Fig. 2). Only through this iterative process can modelling groups – with the support of policy-makers, experts and advisers – build robust representations of reality that can then be used to project what could happen in fictional scenarios. A collaborative team of policy-makers (or their support teams) and of modelling groups from epidemiological, economic and behavioural sciences should thus be part of a preparedness strategy and be ready to be deployed when a pathogen with epidemic or pandemic potential emerges. The actual process of building the integrated model can help the stakeholders better understand the relevant systems and interactions at play, raise important questions and guide data collection (see section 2.3.2) (67). An iterative approach can also help ensure that adequate integrated modelling capacity exists to respond to an epidemic or pandemic if it occurs Without opportunities to gain experience, modelling groups do not necessarily understand or anticipate the needs and preferences of policy-makers and their experts and advisers. This also means that policy-makers and those who support them should clearly describe the support they need, their societal objectives and range of policy questions and associated scenarios (11, 47, 67, 68). Improved communication and collaboration between these stakeholders can improve the quality and utility of models and the decisions they support (14, 67), and so contribute to better preparedness. 2.  Understanding the role of integrated modelling in pandemic preparedness and response 19 Fig. 2.  Iterative process to build, use and refine integrated models Develop initial computational model Conduct runs/experiments with the model Elucidate Identify data Test different important gaps, guide and policies and factors and prioritize data interventions relationships collection Design, modify and Design, plan, modify implement other and implement studies policies and interventions Update model Generate results, data, insights and other new information Note: This figure shows how modelling should be an iterative process of continuously refining a model. From top to bottom: the model is initially developed and periodically updated; runs and experiments are conducted, providing insights and results that can help elucidate important factors and relationships, identify data gaps, guide and prioritize data collection, and test different policies and interventions. Finally, the model outputs can, in turn, guide and further generate the design of new studies, policies and interventions, which can then lead to more information and insights for the model. Source: based on Lee et al. (67) 20 Strengthening pandemic preparedness and response through integrated modelling The role of integrated modelling should not end when an epidemic or pandemic ends The end of an epidemic or pandemic is also a key time to consolidate lessons from the response, which can inform future preparedness and response. Modelling can shed light on what could have happened in the past had circumstances and policy decisions been different. Comparing what occurred with these modelled scenarios can also show the value of improving epidemic preparedness and responses for the future. In the rare case in which the pathogen has somehow been eliminated, modelling can guide how to prevent the pathogen from returning. If, on the other hand, the pathogen persists at a less virulent level, modelling can help guide the design and implementation of policies to prevent the pathogen from causing an epidemic or a pandemic and can support the recovery of systems and communities to minimize lasting impacts (67). Integrated modelling is one tool to aid decision-making by policy-makers Models can provide decision-making support by showing what could happen under various circumstances and can also better elucidate the systems, their mechanisms and major drivers. However, ultimately, policy-makers must make the final decisions. Furthermore, as an epidemic evolves, the objectives of a policy response may change from reducing transmission risk, excess morbidity and mortality and the risk of overwhelming the health system (when the epidemic is not under control), to minimizing social and economic damages while keeping transmission under control to protect health system capacity. Thus, it is helpful for the policy-makers or those who support them to not only understand modelling but also be involved in the modelling process to ensure that relevant societal objectives and policy options are considered. 2.3.2 Populating, calibrating and validating integrated models Integrated modelling can inform policy-making about the data needed to refine their understanding of the situation When models are available before the start of an epidemic, the existing models can be used in real time as the epidemic or pandemic evolves (69–71), even when data are scarce and little is known about a novel pathogen or when new variants of a pathogen may occur. As more data become available, modelling can be used to refine policy questions and responses – for example, to simulate alternative measures and estimate their impacts and costs. Once such studies and data collection yield more insights and data, the model and data sets can be updated, leading to more cycles of refining the model, studies, data collection and insights. This iterative process can help move towards better understanding and refinement of the questions of interest, the model, the results and the potential uses of the model (Fig. 2). 2.  Understanding the role of integrated modelling in pandemic preparedness and response 21 Each epidemic or pandemic is different and the data needs for different integrated models are different Integrated models start with similar data needs for epidemiological and macroeconomic models. For integration, localized economic activity and data about transmission drivers (such as the social, demographic and economic characteristics of a population that drive infection spread) are needed at the same level of disaggregation at various points in time (20, 72) (Box 7). In addition, the data needed for integrated modelling depends on pathogen characteristics and on how it may impact economic activity (20). A summary of the sample key data used by COVID-19 integrated models is available in Annex 3. As shown in Fig. 1, COVID-19 integrated models that consider changes in human-to-human contact rates require data on the numbers and types of in-person contacts, disaggregated by age, socioeconomic status, employment status, economic sector and type of workplace. However the availability of these data is currently extremely limited (20). Thus, to parameterize the interface between epidemiology and economics in the case of a respiratory pathogen like COVID-19, alternative data from contact surveys, people’s mobility trends and transaction patterns have been used, each with their advantages and disadvantages (20). For instance, COVID-19 integrated models that considered changes in population behaviours due to COVID-19 and associated control measures generally made assumptions about how classes of individuals may behave differently depending on their age, employment status and infection status, with individuals of the same group generally assumed to behave the same way (57). Other models that incorporated endogenous individual behaviours were mainly theoretical, because data for the calibration of individual human behaviours are limited (36). Data needs also depend on which individual preferences are represented in the model and on assumptions about how individuals make decisions to take a specific course of action (that is, weigh up the benefits and costs) to maximize their preferences in the model (20). Tools are available to help countries plan for the collection of social and behavioural data – ideally by having adaptable protocols and funding in place prior to an epidemic so that data can be collected early to parameterize models (33). 22 Strengthening pandemic preparedness and response through integrated modelling Box 7.  Approaches to calibrate the locally developed Tekanelo integrated model in South Africa In 2021, researchers from three South African universities (Pretoria, Cape Town and the Witwatersrand) came together under the auspices of the South African COVID-19 Modelling Consortium (SACMC) to integrate two existing models into a new model called Tekanelo (Pedi for “balance”) to estimate the combined effect of COVID-19 and future vaccination strategies on South African lives and livelihoods. One of the two existing models was the National COVID-19 Epidemiological model, developed by the University of Cape Town with assistance from SACMC, which had been used since April 2020. The model was used to forecast COVID-19 cases, hospital admissions and deaths, and support decision-making by the South African Government. The second existing model was a macroeconomic model (the UPGEM Generalized Equilibrium model) developed by the University of Pretoria, which had been used to represent health and other shocks to the South African economy before the COVID-19 pandemic. During the integration of these two models, two challenges emerged. First, there were no locally relevant data to inform contact patterns by age and occupation in South Africa. Age contact matrices based on international data were used in the epidemiological component, but analysts were aware that they could not be used to inform the economic outputs because they would likely not match the work contact matrix of the South African workforce. Instead, the modelling team used assumptions based on changes in productivity during earlier waves of infection and signals from policy-makers to develop realistic scenarios in the macroeconomic model. Second, there were no locally relevant, representative measures of PHSM during the earlier waves of the epidemic in South Africa. As a solution, the modelling team used a proxy for the combined impact of nationally mandated restrictions and the population’s adherence to them by drawing on data from observed changes in population behaviour in response to mortality changes in previous COVID-19 waves. From this, a range of scenarios was modelled to measure the impact of future vaccination in the face of new, more transmissible variants. Source: SACMC (73) 2.  Understanding the role of integrated modelling in pandemic preparedness and response 23 3. Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response Building on previous sections, this section proposes four initiatives that policy-makers can consider to improve the production and use of integrated models to support decision-making under a high degree of uncertainty. Under each initiative, activities that can be tailored to context and implemented by policy-makers, their support teams, modelling groups and other stakeholders as part of pandemic preparedness are also proposed. Key messages • Integrated modelling can be a valuable tool for policy-making, but the extent to which modelling is used to inform policy-making and the capacity to produce integrated models vary across countries. • Four initiatives that can enhance the production and use of integrated modelling for policy-making before, during and after epidemics include: – formally incorporate integrated modelling into policy-making – establish active communication between the different actors – match the policy questions of interest to the right integrated models – build local integrated modelling production capacity. 24 Strengthening pandemic preparedness and response through integrated modelling • Countries may prioritize the implementation of initiatives and tailor associated activities based on contexts and funding availability. • A system to monitor implementation progress and evaluate impacts can help adapt activities when needed. 3.1 Initiative 1: Formally incorporate integrated modelling into policy-making For modelling to be properly employed, it must be formally incorporated into policy-making before, during and after epidemics or pandemics. Under this initiative, four activities are proposed. Each activity may be tailored to each country context depending on the extent to which modelling is used to inform policy-making in the country (see Annex 4 Fig. A4.1 and Table A4.1). 3.1.1 Institutionalizing integrated modelling to inform policy-making Expert teams or units can be built to help incorporate integrated modelling into policy-making These teams or units can be composed of experts who can review and validate integrated models, communicate the technical details of these models and present results to policy-makers or their scientific advisory bodies. In this way, expert teams and units can play a crucial role in bridging the gap between experts who focus on modelling and policy-makers who may not have the technical expertise in modelling or the time for frequent and sustained interactions with modelling groups. Expert terms may be working within governments and in close collaboration with modelling groups in academic institutions and may conduct modelling themselves Expert teams can help identify policy questions and facilitate their translation into modelling and other analytical tools. They can also liaise with units involved in designing surveillance systems for collecting and gathering various types of data (from fields like epidemiology, economics and behavioural sciences) (1), and in response operations to support the iterative refinement of analyses (47). Also, modelling outputs should not stand alone but be triangulated with empirical and contextual information to provide both the correct interpretation of modelled outputs and best inform policy responses. Expert teams can support open and inclusive policy-making 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 25 These teams may work closely with scientific advisory groups to form a larger team that can provide high-level guidance to policy-makers about the need to integrate multiple sources of evidence and about the importance of integrated modelling to inform policy-making. Expert teams can also facilitate a transparent dialogue among scientists and advisers from different disciplines, policy-makers and the public by being receptive to feedback and inputs, including those about public preferences (74). They can help communicate which key elements are the most uncertain, what uncertainties mean (what is at stake and which risks are worth taking most seriously) and how to deal with uncertainties – for example, how the legitimate interests of different parties were assessed and weighted during policy-making. Administrative procedures can facilitate close collaboration between policy-makers, government technical teams, advisory groups, modellers and the public before, during and after epidemics or pandemics Individuals involved in policy-making may change roles over time, so relationships and informal interactions built up over time may not persist. To aid continuity, administrative or legal procedures may be developed, documented and passed to the next administrations, as is done in other areas (75). These procedures can specify the authorities and mandates – that is, the scope of work of each stakeholder group participating in policy-making, which may span the continuum of evidence synthesis, modelling and recommendations. These procedures can also aim at establishing and sustaining teams from multiple disciplines to routinely work together before, during and after an epidemic (1, 76). Within a country, these procedures could be passed from national to subnational units and adapted for use within states and provinces or counties. They could also be passed from country to country as a starting point for consideration. Even though individual governments may differ in structure, there are general principles, processes and systems that can be applied. 3.1.2 Assigning clear modelling-related responsibilities Government units can be assigned clear responsibilities in modelling before, during and after epidemics The COVID-19 pandemic underlined the pressing need to bring together various sectors to address complex and transdisciplinary societal challenges and overcome departmentalism and siloed work to increase policy relevance and effectiveness (74, 77, 78). Government units composed of individuals with expertise in various disciplines can be formed and assigned clear responsibilities in modelling, with responsibilities distributed according to the skills and expertise of the individuals forming these units. Government units can be matched to modelling groups 26 Strengthening pandemic preparedness and response through integrated modelling Matching these units to modelling groups can address the challenge that modelling groups face when identifying who to engage with in government before, during and after epidemics. This matching process can also help the modelling be more relevant to policy-making and thus more impactful. Such matching should be done by government units seeking modelling support, with possible guidance from scientific advisory groups. At the same time, procedures and agreements for such matching must allow for appropriate scientific independence and transparency to mitigate undue influence of governmental units over how models are developed and what results are generated and communicated to policy-makers. 3.1.3 Establishing procedures on the production of adequate integrated models Reviewing modelling undertaken during recent epidemics and pandemics from the perspective of epidemiology and social sciences can shed light on how the respective disciplines have addressed specific policy questions Reviews of modelling in different disciplines can help improve the understanding of the factors addressed by each discipline. This includes using real world examples to identify gaps that no discipline is covering, and areas that are complementary between disciplines (76). These reviews can support a better understanding of how modelling in different disciplines can result in different assumptions, findings and recommendations and encourage mutual understanding of the limitations and opportunities of modelling in the respective disciplines (76, 79). These reviews can be undertaken by research groups, with the support of other stakeholders, to draw on the literature and on recent country case studies. Issue calls for proposals to adapt existing models or build new integrated models Reviews can help understand how modelling and integrated modelling helped or could have helped during recent epidemics and pandemics, and in which phases and situations they may be employed or better employed. Results from these reviews can thus facilitate policy-makers or those who support them to work with modelling groups to maximize the usefulness of modelling for policy-making (80). When adequate models exist, calls may be issued for proposals to adapt the existing models to specific policy questions. When adequate models do not exist, calls for proposals to build new models may be issued. 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 27 3.1.4 Creating an ecosystem of key data and data sources for policy-making It is important to construct a rapidly and readily accessible data ecosystem Although models can be quite useful even in the absence of good-quality data, better data can improve model accuracy, reliability and validity. As good data are an essential part of evidence-informed decision-making (47), linking modelling and data surveillance systems or other data collection approaches can facilitate feedback loops (21, 47, 74) (Boxes 8, 9 and 10). For example, if they know which data are needed by modelling, on-the-ground response teams can adjust their data collection approaches accordingly (47). Data from small-scale initiatives can be useful to develop and test the base structure of an integrated model while standards for larger-scale data collection are being developed (76). At the same time, the data ecosystem can facilitate the rapid communication of model results to guide which data are collected by surveillance systems, commissioned studies and other methods (76). A regional or global dimension to evidence generation and synthesis can help build data ecosystems Few countries have the capacity to employ official expert evidence review committees such as the Scientific Advisory Group for Emergencies in the United Kingdom and the Ministerial Advisory Committee in South Africa (81, 82). Efforts to develop data systems could thus be linked to broader initiatives such as the Health Data Collaborative, which aims to build on existing data systems and processes by leveraging technical and financial resources from all sectors and investing in cross-programme aspects of data and measurement (83). Data-sharing agreements can support the development and use of data ecosystems Data-sharing agreements between governments, other stakeholders and modelling groups can be put in place and be ready to be deployed as soon as a pathogen with epidemic potential emerges (14, 76, 84). Building or expanding open-source model and code repositories (85) that can focus on the needs of integrated modelling would increase the pool of data sources and support more efficient collaboration and ongoing iteration and innovation (76). Collaborative efforts between disciplines, and sufficient time and resources, are needed to build new data ecosystems or expand existing ones. Integrated models can benefit from increased availability of data on behaviours and how they vary among different groups of the population and over time 28 Strengthening pandemic preparedness and response through integrated modelling Future developments in modelling could augment and enrich integrated models by including a more granular account of the heterogeneity of the psychological, social and environmental influences on behaviours that have important effects on epidemiological and economic outcomes. Such developments will require behavioural and social scientists to design data collections on multiple behaviours from multiple population groups. Populating this variety of behavioural model parameters will require estimates coming from a combination of nationally representative surveys, online and laboratory experiments and field experiments across different countries and over time. Incorporating social, economic and behavioural data into outbreak investigation protocols can also enable the timely collection of data. For example, data on the adoption of and adherence to transmission-reducing behaviours and the factors that facilitate or hinder these behaviours can provide information on the socioeconomic and behavioural drivers of an outbreak. These data can inform context-specific assumptions and parameters for integrated modelling during an outbreak response (76). At the country level, data from past epidemic phases may provide some insight into how these factors played out and inform integrated modelling. But sensitivity analysis may still be required to deal with this major source of uncertainty. Box 8.  Calibration of an existing integrated model to inform policy-making in Sri Lanka In Sri Lanka, a variety of PHSM to suppress the transmission of SARS-CoV-2 were deployed, including an island-wide curfew, bans on public gatherings, school closures and air travel restrictions. During the COVID-19 pandemic, the country suffered severe economic consequences, with the year-on-year real GDP contracting by 3.6% in 2020 compared to an average annual growth of around 5.3% in the previous decade. In 2021, a central policy question arose in Sri Lanka related to the optimal duration of the combined control measures to safeguard population health while minimizing the detrimental economic effects. To address this question, Sri Lanka partnered with a modelling group at Imperial College London to adapt and calibrate an existing integrated model – DAEDALUS – to the Sri Lankan epidemiological and economic contexts and policy questions. When the integrated modelling exercise started, Sri Lanka had already put in place mechanisms to ensure relevant data were routinely collected and disseminated. The National COVID-19 Surveillance System had been developed to ensure designated hospitals provided daily COVID-19 information, equipment requirements and laboratory data (86). continued 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 29 Box 8.  continued The Epidemiology Unit within the Ministry of Health and Indigenous Medical Services produced daily reports on the evolution of the epidemic in the country. In addition, the Ministry and the WHO country office coordinated to gather the specific data required for the calibration of the integrated model (including hospital occupancy rates, age-structured case fatality rates and vaccine delivery capacity), supplemented by multisectoral collaborations spanning multiple government agencies and international partners to collect additional data. The Department of Census and Statistics provided data on the demographic profile of the population. The Central Bank of Sri Lanka shared information on economic activity over time, the Asian Development Bank provided economic data for 35 sectors (such as sector-specific input–output tables) and the World Bank and United Nations Country Office provided estimates of the proportion of the population working from home. Sources: Doohan et al. (64), Haw et al. (63) and IMF (87) Box 9.  The FASSSTER data ecosystem and modelling platform in the Philippines The Feasibility Analysis for Syndromic Surveillance using Spatio-Temporal Epidemiological Modeler project (FASSSTER) began in 2016 as a government-funded project that aimed to test the design, development and adoption of a web-based disease surveillance and scenario-based modelling platform for the Philippines. The disease surveillance system was designed to extract data from a variety of data sources including electronic medical records, hospital information systems, weather data and social media data, which are among a variety of data sources needed for the computation of model parameters. The FASSSTER 2016 version of the platform contained localized epidemiological models for dengue, measles and typhoid, which were tested in the Western Visayas region of the country. In March 2020, led by the Ateneo de Manila University team, the FASSSTER platform was extended to epidemiological COVID-19 models and configured to accommodate the needs of the Philippine Government’s decision-making body, the Inter-Agency Task Force for the Management of Emerging Infectious Diseases, for data-driven and evidence-based decisions. In 2021, with the support of Imperial College London, the FASSSTER COVID-19 model was further extended to a COVID-19 integrated epidemiological–macroeconomic model. Sources: based on FASSSTER Ateneo de Manila University (88) and Department of Health, Republic of the Philippines (89) 30 Strengthening pandemic preparedness and response through integrated modelling Box 10.  The Integrated Outbreak Analytics (IOA) initiative The IOA initiative mobilizes multiple actors and organizations under the overall coordination of ministries of health (MOHs) to work together before, during and after a public health emergency (PHE). It uses a comprehensive approach to data analytics and interpretation for decision-making during PHE response. Although IOA was initially developed to respond to outbreaks, many of its principles and approaches can be applied to other PHEs. IOA achieves this by bringing together a multidisciplinary and multisectoral group of partners that collaboratively contribute to the generation, use and communication of actionable intelligence, all under the coordination of the MOHs. This entails assisting MOHs in: • identifying relevant national, regional and international partners that will support data management, interpretation and codevelopment of recommendations; • supporting capacity-building and capacity development of MOHs, national public health agencies, national emergency medical teams, clusters, partners, and local stakeholders in countries and regions of interest; • facilitating information sharing among collaborating partners; and • sharing and operationalizing guidance, methods and tools before, during and after the PHE. IOA as an approach was used for the timely generation of integrated and actionable evidence during the 2018–2020 Ebola virus disease (EVD) outbreak in the Democratic Republic of the Congo (DRC). It was then used in other outbreaks in DRC including COVID-19, other EVD outbreaks, measles and plague, and a standing IOA cell was set up in the MOH in 2021. Subsequent successful experiences beyond DRC include PHE responses in Guinea (EVD), the Republic of the Congo (COVID-19), Ghana (Marburg), Uganda (Sudan virus disease), and recently in Haiti (standing IOA cell) where MOHs and donors have requested IOA to be deployed. Sources: Carter et al. (45), Social Sciences Analytics Cell (90–92), Integrated Analytics Cell (93–97), WHO (98) and Bedford (99) 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 31 3.2 Initiative 2: Establish and maintain active communications The COVID-19 pandemic highlighted the importance of policy-making approaches that engage all relevant stakeholders including academia, the media, the private sector and communities (77, 100). Because every actor has their own experience, perspectives, jargon, processes, systems and body of knowledge, time must be taken to achieve more shared understanding (101, 102). In addition, modelling should not be produced by distant groups or networks that sit far away from the response (14), because this may lead to poor insights into the context in which an epidemic is occurring. Under this initiative, five activities are proposed. Each activity may be tailored to each country context depending on the extent to which modelling is currently used to inform policy-making (see Annex 4 and Table A4.2). 3.2.1 Creating formal communications channels for before, during and after epidemics Formal procedures and agreements can be developed to describe how policy-makers, those supporting and modelling groups can engage and collaborate before, during and after epidemics Formal procedures and agreements can indicate the need for policy-makers to clearly define policy questions and associated scenarios and to allow appropriate amounts of interaction and time with modellers to iterate over the questions and results. They can include working together to develop, test and calibrate integrated modelling base structures that can then be more rapidly adapted into different pathogen-and-emergency specifics. Building a basis of familiarity and mutual understanding as well as model structures before an epidemic means that once a pathogen with epidemic potential emerges, new models do not have to be developed from scratch. These procedures could also concern co-creating research questions with modellers and working towards more evolved modelling methodologies that integrate socioeconomic and behavioural disciplines into emergency preparedness and response (76). 3.2.2 Communicating policy questions and how integrated modelling can help It is crucial for policy-makers to define their policy questions and the alternative scenarios to be tested and for modelling groups to regularly communicate about their models 32 Strengthening pandemic preparedness and response through integrated modelling Modelling cannot only provide answers but also help identify the right questions. Therefore, close and sustained communication between policy-makers, government technical teams and modelling groups, epidemiologists, economists and behavioural scientists can help shape lists of questions, data needs and model structure. Modelling groups within and across disciplines should sustain active communications with policy-makers or the relevant government units as the role and capabilities of models and data availability evolve. Other stakeholders – such as international organizations, research and policy centres or consortiums – may also encourage communication across government sectors and between modellers and researchers from different disciplines. They can also provide guidance and other resources to policy-makers and government units about what modelling can and cannot do before, during and after epidemics and on the need for interdisciplinary and intersectoral collaborations in the context of complex systems and interventions (74) (Box 11). Box 11.  A four-step approach to engage policy-makers during model development and evolution in Argentina Before the COVID-19 pandemic, Argentina already had an interdisciplinary network of modellers that worked across various research platforms. The Institute of Clinical Effectiveness and Health Policy (Instituto de Efectividad Clínica y Sanitaria – IECS) – a non-profit academic institution affiliated with the University of Buenos Aires, focusing on health policy research – is one example of an Argentinian research platform that mobilized quickly and built an integrated model that provided outputs to support policy-makers during the pandemic. From the start, IECS aimed to build a multisectoral advisory board that would inform the development and evolution of the integrated model. The advisory board was composed of policy-makers who were involved in decisions around the use of PHSM and vaccination strategies in Argentina. Its members ranged from government representatives at national, provincial and municipal levels to national legislators and high-level executives of private and social security insurance agencies. Continuous dialogue between the IECS modellers and the advisory board members played a crucial role in validating the model and strengthened the development and evolution of the model as policy options changed throughout the course of the pandemic. The IECS team developed a four-step approach to engage with policy-makers (Fig. 3). The team started out by identifying and contacting relevant policy-makers across sectors. continued 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 33 Box 11.  continued They looked for policy-makers willing to help select and refine the policy questions and scenarios that would be modelled. A total of 30 policy-makers, ranging from mayors of large and mid-size cities to members of the Senate, were identified and invited to provide feedback on a set of policy scenarios that explored topics like relevance to policy-makers, prioritization across scenarios, additional policy questions and scenarios that could be included in future model iterations. After receiving feedback, and once the modelling framework had been developed, an interactive policy workshop was organized by the IECS team to present the model and share results generated for the various scenarios identified by policy-makers. The interactive policy workshop proved to be an important step in the development and evaluation of the integrated model. During the workshop, policy-makers recommended that the next iteration of the IECS model would benefit from exploring the social impact of PHSM, such as poverty and job losses. Importantly, policy-makers expressed interest in interacting with the model and identifying individuals who would be trained in its use, and suggested that the IECS team extend their model to other countries in the region (103). Fig. 3.  Four-step approach to engage policy-makers in model development and evolution 1 • Identify relevant policy-makers across sectors • Develop preliminary modelling scenarios to be included in a policy survey (e g  use of PHSM) • Invite policy-makers to participate in the policy survey 2 • Analyse survey results 3 • Run an interactive session with relevant policy-makers 4 • Train a selected group of policy-makers and analysts in the use of the model Source: Rubinstein et al. (104) 34 Strengthening pandemic preparedness and response through integrated modelling 3.2.3 Facilitating interactions at all stages of new model development Establishing written guidelines and procedures can give policy-makers and modelling groups more fair and equal access to each other Table 2 shows how policy-makers can be involved in all aspects of integrated model development and use. This would enhance the “pull” aspects of model development, so that modellers get more guidance on what is needed. If formal and established communication systems are lacking, only those modellers with existing personal connections with policy-makers may have access to and guidance from policy-makers. This may mean that the most appropriate models are not being developed and used. Written procedures can be shared internationally, so that countries can learn from each other. 3.2.4 Creating active networks of policy-makers and modelling groups Networks of policy-makers and modelling groups from different disciplines can be created to inform policy-makers on the types of modelling expertise available inside and outside their country National, regional and global lists of modelling groups and modelling capabilities can be assembled to inform the development of such networks. These lists can include modellers’ training, expertise in integrated epidemiological–macroeconomic methodologies and types of models developed, and experience including working with policy-makers. Because not all models are equally able to address a given policy question, the directory must contain information that will clearly help identify who can do what is needed at the time. Going beyond country borders can extend the pool of modelling groups, models and policy questions to allow policy-makers and modellers to learn from each other and help countries build their own capacities. Information on past and current contractual arrangements between modelling groups, government departments and/or international organizations can help inform the development of the network (Box 12). 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 35 Table 2.  Steps in integrated modelling and how policy-makers and those supporting them can be involved 36 How policy-makers Dangers of not having Special considerations for low- and Step Description can be involved policy-makers involved middle-income countries Establishing Determining relevant Communicate key questions Question is not relevant or Can make sure questions are culturally the questions questions for decision at of interest that reflect real helpful appropriate and respect local of interest hand world decision-making Question may be too governance structures Hold consultations with theoretical or unrealistic, Can make sure questions are mindful of sectors and experts to providing limited value social norms refine questions Communicate which types of questions may and may not be appropriate to ask in the local context Designing the Determining and Elucidate understanding of Model may not fit or Can help identify and connect to model mapping out the the system capture the needs of the people in the country who can help components of the Help identify connections and decision-maker provide access to needed information system and their relationships Model may not capture all and expertise on processes and connections and relevant parts of the system mechanisms Communicate which details relationships or mechanisms Can help identify and describe key matter and why Determining the type Processes may not components that may have otherwise and structure of the accurately reflect reality gone unnoticed by those not familiar model with the local context Design may not be sufficiently transparent Implementing Representing the Help provide and find data for May not use correct or Can help identify and provide data the model relationships, processes populating, calibrating and appropriate data sources sources to populate, calibrate and and actions in the validating models Model data sources may validate the model that otherwise may system with equations or Identify those with relevant not reflect situation or be challenging to identify and access algorithms expertise to aid in validation circumstances Can help identify those with relevant Populating, calibrating expertise for validation that may be hard and validating the model to identify otherwise continues Strengthening pandemic preparedness and response through integrated modelling Table 2.  continued How policy-makers Dangers of not having Special considerations for low- and Step Description can be involved policy-makers involved middle-income countries Selecting the Determining the base Identify relevant and realistic Scenarios and experiments Can inform scenarios to make sure they scenarios and case and experimental scenarios may not be relevant to local are realistic and help those that may not experiments scenarios of interest Help provide appropriate circumstances or context be familiar with the in-country context Changing parameter assumptions for scenarios Sensitivity analyses may not Can help identify variables that may be inputs to explore how Help determine plausible be performed on all relevant important to vary in sensitivity analyses changes result in ranges of values for variables different outcomes sensitivity analyses Parameters may not be varied over full plausible ranges Analysing and Analysing results for Help ascertain if trends and Results may be Can help elucidate patterns and trends interpreting trends and drivers, and patterns make sense misinterpreted in the local context that may otherwise the results determining thresholds Help contextualize and Findings may not be relevant not be understood Determining what the interpret findings Implications of assumptions Can provide context-relevant results mean Identify decision rules to aid and of limitations of results benchmarking and thresholds to use decision-making may not be fully understood when interpreting results Translating the Translating results into Convey ethical, political, Full range of implications of Having a buy-in can help in results into practical solutions that resource or contextual proposed solutions will not understanding proposed solutions actions can be applied to real considerations be apparent without human Can help communicate and disseminate world challenges Successfully implement input findings to others in a country solutions Solutions may not be viable Can help ensure actionable solutions or implemented in the real are correctly implemented world 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 37 Box 12.  Sustained communication and collaboration between local and international modelling institutes, policy-makers and the WHO: the experience of the Philippines’ FASSSTER team With funding support from WHO headquarters and the Regional Office for the Western Pacific, the FASSSTER team worked with the Australian Tuberculosis Modelling Network (AuTuMN) and then with modellers at Imperial College London. The initial phase of the work started in May 2020 and involved understanding and subsequently providing feedback on the epidemiological model and methodology used by AuTuMN. Regular biweekly or weekly online meetings took place to ensure that AuTuMN was up to date with whatever COVID-19-related developments in the country would be relevant to their modelling. The scenarios used in generating projections were formulated based on the advice of the FASSSTER team and Philippines Department of Health Epidemiology Bureau (DOH-EB). In the second half of 2022, AuTuMN provided twice-a-month training sessions to FASSSTER and DOH-EB for capacity-building. These sessions focused on the tools, programmes and methods that AuTuMN had used over the previous two years to generate case projections for the country. In 2022, FASSSTER and Imperial College London started working together to develop a localized integrated model, with regular online meetings throughout the year. Once the integrated model development phase was completed, the collaboration between the DAEDALUS and FASSSTER teams continued throughout 2023. The collaboration involved developing what-if scenarios on lockdowns from the first year of the COVID-19 pandemic in the Philippines that could have optimized specific economic outcomes, notably reducing poverty, using the localized integrated model. Source: FASSSTER Ateneo de Manila University (88) 38 Strengthening pandemic preparedness and response through integrated modelling Existing networks of epidemiologist and macroeconomic modellers may be extended to social scientists beyond economists, such as those working on behavioural sciences Networks can be extended to involve response teams that work on the ground where an epidemic has occurred or is occurring to ensure that modelling efforts and plans incorporate relevant contextual information and inform current and future response operations (see Box 10). This would facilitate collaboration using interdisciplinary data, and develop models for future applications, including epidemic and pandemic preparedness (76). The ecosystem of international partners that was formed to fight the COVID-19 pandemic can be used as the basis for expanding the network, strengthening the links within it, and developing collaborations (21). Rather than a passive directory, this network should be active with frequent engagement (Box 13). Policy-makers and government units from various countries can play key roles in keeping the network alive Policy-makers, or their government units, who network as regularly as possible can help catalyse interactions between policy-making and modelling. Countries may draw policy recommendations from each other, but the modelling behind recommendations may not always be public, so involving representatives from different governments and experts from different countries could help contextualize these recommendations. International organizations, including policy and research centres or consortiums, can play a role in assembling and maintaining such networks, given that they have played such a role in other ways (105). 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 39 Box 13.  Collaboratory, an initiative of the WHO Hub for Pandemic and Epidemic Intelligence The COVID-19 pandemic has further highlighted important gaps and challenges in the way the public health and pandemic and epidemic intelligence communities access, analyse and use information to prepare and respond to pandemics and epidemics. The global pandemic and epidemic intelligence community has lacked a common unified space to address these challenges, retain knowledge and build on best practice. Collaboratory, an initiative of the WHO Hub for Pandemic and Epidemic Intelligence, has a primary goal of establishing a digital environment in which the pandemic and epidemic intelligence community can convene to address critical challenges that affect the way data are accessed, analysed, visualized and communicated for better pandemic and epidemic policy and response decision-making. The Collaboratory vision is a world where an interconnected pandemic and epidemic intelligence community responds collaboratively and rapidly, with improved data, enhanced analysis and actionable insights (Fig. 4). Within this digital environment, public health professionals, academics, epidemiologists, data scientists, developers, modellers and decision-makers will find a fertile ground to brainstorm, cultivate ideas and codevelop innovative solutions to complex public health problems. The Collaboratory aims to be more than just a platform – it will be an interactive knowledge-sharing space where members can jointly analyse data, share code and models and methodologies, learn from peers, tap into expert opinions and amalgamate resources to produce timely, effective and actionable insights. The Collaboratory will support the facilitation and convening of modelling groups to address challenges and bring best practices to the forefront, building a culture and environment that fosters collaborative analysis. continued 40 Strengthening pandemic preparedness and response through integrated modelling Box 13.  continued Fig. 4.  Collaboratory, an interactive knowledge-sharing space for the epidemic and pandemic intelligence community Community Digital environment Technical expertise Ease of access Diverse perspectives Collaborative Sandbox Knowledge sharing analysis Technical integration Connecting people Benchmarking Source: WHO (106) 3.2.5 Increasing integrated modelling literacy and communication across society It is useful to develop a communication strategy to modelling before the emergence of an epidemic-prone pathogen Modelling results can be easily misinterpreted if the appropriate context is not provided in language that is readily digestible by the public. Thus, whenever policy-makers – and those who support them – and modelling groups communicate modelling findings to the public, there should be an associated communications strategy (100). The strategy should be formal and consistent and must account for the complexity, heterogeneity and diversity that exists across the population. Media and other communications experts should be consulted or commissioned to develop communication strategies, including when these strategies should be implemented. The strategy should strike a balance between being transparent, so that it shows how science is driving policy-making, while taking uncertainty into account and not offering vague information that can be readily misinterpreted (see Annex 5). Such communication requires engaging many different disciplines, sectors and stakeholders, including communications experts. 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 41 Participatory mechanisms can improve the quality and legitimacy of policy-making Many participatory mechanisms have been introduced throughout the world in recent years to improve the quality and legitimacy of public decision- making, including in health (12). From a pragmatic point of view, it is important to identify the existing participatory mechanisms that contribute to epidemic-related decision-making, to consider how they can best be leveraged (like the participatory surveillance systems for influenza used to collect information on health seeking and testing behaviour) and to ensure that policy-making deliberations are informed by and inform scientific expertise. A participatory system of governance for answering complex questions will usually consist of a combination of mechanisms to involve different groups for different purposes (12). If the mechanisms are to serve their purpose, they should be institutionalized rather than ad hoc. This makes inclusive, transparent, accountable decision-making a routine feature of governance before, during and after epidemics as part of “building back better” efforts (12). 3.3 Initiative 3: Match the appropriate integrated model with the right question No single model can answer every single possible question that a policy-maker may have. Once a specific question has been identified, the appropriateness of the model for that question can be determined. Under this initiative, four activities are proposed. Each activity may be tailored to each country context depending on the extent to which modelling is used to inform policy- making (see Annex 4 and Table A4.3). 3.3.1 Building base-modelling structures before epidemics and pandemics or during interepidemic or pandemic times It is important to develop and establish base-modelling structures that can be rapidly used and adapted when emergencies arise; this ensures greater preparedness Emergencies may not afford the time and opportunity to build the needed modelling structures and capabilities. Building these base-modelling structures when there is more time available can help build preparedness. For example, a model structure may include interactions and mixing among people depending on their work occupation; the connections between health outcomes, work productivity and broader economic measures; and the relationships between school attendance and future livelihoods. Even though pathogens can differ 42 Strengthening pandemic preparedness and response through integrated modelling substantially from each other, and one epidemic may not be representative of others, some principles and mechanisms are common across all epidemics (107, 108). 3.3.2 Building capabilities to determine how to match the appropriate models with the right policy questions Developing checklists of basic criteria for models can help policy-makers or their units better assess the fitness and purposes of different models for their given question Checklists can show the characteristics and components of a model and what systems the model represents and the level of detail. Qualities include the key relationships and mechanisms the model explores, and its granularity, scales, perspectives and time frames. Other qualities include how time progresses in the model, what the model outputs are, what populations and what aspects of these populations the model represents, and what locations and what aspects of these locations the model represents (109). This checklist can be assembled from reports on epidemiological and economic models. For example, in August 2020, the COVID-19 Multi-Model Comparison Collaboration Policy Group released a report, Guidance on use of modelling for policy responses, that provided some criteria on how to evaluate epidemiological models, which would apply specifically to the epidemiological portions of integrated epidemiological–macroeconomic models (109). 3.3.3 Ensuring integrated modelling can adapt to policy priorities during epidemics and pandemics New models and modelling approaches can complement traditional modelling approaches When the Multi-Model Comparison Collaboration reviewed COVID-19 dynamic epidemiological models, they found that existing “models are not designed to answer all COVID-19-related questions decision-makers may have” (110). They highlighted the importance of distinguishing questions that can be addressed by specific models from those that cannot, and communicating this to decision-makers who may need to resort to other research methods or groups for answers. In many ways, integrated modelling is still in its early stages and while the availability and use of modelling have grown in recent years, there is still abundant room for growth and improvements in the field (18, 111–113). Also, of the models available, some will continue to be useful for specific questions, some will need to be adapted to address other questions and some will become obsolete, meaning that new ones will need to be created. 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 43 3.3.4 Creating procedures and systems for comparative integrated modelling Comparing model structures and results can generate important insights for policy-making As noted in section 2, each model has its own viewpoint, strengths and weaknesses. Each model represents a modellers’ conceptualizations of the situation, which may be similar to or different from others, depending on the circumstances. Each model may be weighted towards certain aspects of the issue and look at the question in its own way. When the results of different models are similar, there can be more confidence in the results. In the case of differing results from different models, there may be a risk of inaction by setting aside models and their results if there is not enough time for modellers to coordinate and investigate the drivers of these differences. Thus, a solid process to organize model comparisons, in terms of architecture, data, key assumptions and results, and to communicate the main reasons for diverging results should be established as part of epidemic preparedness (14, 114). 3.4 Initiative 4: Build local integrated modelling production capacity Although in many countries there are both epidemiology and economic university departments, there are few dedicated modelling training programmes in these disciplines around the world. There are even fewer modelling programmes that truly integrate epidemiological and macroeconomic concepts and processes, because the disciplines of epidemiology and economics have long remained very siloed from each other, often sitting in completely different schools. As a result, individuals who can fully integrate epidemiological and economic representations in models are rare. Such individuals exist despite the existing lack of training programmes and infrastructure. Reliance on such incidental career paths means that many large gaps exist in integrated epidemiological–economic modelling capabilities. Therefore, there is a need to develop a workforce that cannot only develop and use such models but also digest and translate them. At the same time, for greater system resilience, there is a need to build awareness and understanding in integrated modelling for policy-making among policy-makers, government units and sectors and scientific advisory groups. Under this initiative, three activities are proposed. Each activity may be tailored to each country context depending on the extent to which modelling is used to inform policy- making. 44 Strengthening pandemic preparedness and response through integrated modelling 3.4.1 Establishing networks that facilitate cooperation and collaboration between integrated modelling groups Networks of modellers and policy-makers can help conduct translational, educational and training activities Section 3.2.4 proposed creating international, regional and national networks to facilitate communication between modellers and policy-makers, but also communications among modellers so that they can learn from each other (14). In countries where there is modelling capacity, national networks can be established. Where local capacity is not sufficient or where capacity can be further developed, modelling groups can be connected to regional and international networks (Box 14). Such collaborative networks can in turn provide the opportunity for modelling groups to establish periodic integrated epidemiological–economic modelling meetings in which modellers can exchange ideas, provide short-term training opportunities (21), and help develop plans to create the next generation workforces. Such meetings can occur on at least an annual basis and help identify leaders willing to help catalyse the next steps. Technology can help policy-makers and modelling groups around the world communicate and share information Data sources (to populate, calibrate and validate models), model descriptions and specifications, model codes, directories of modellers, lessons learned from various policy-making situations, and procedures and guidance may be shared with the help of digital technologies (Box 13). 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 45 Box 14.  The MIDAS network Initially formed in 2003 with funding from the National Institute of General Medical Sciences, the Models of Infectious Disease Agents Study (MIDAS) network has allowed infectious disease modellers throughout the United States to connect with each other as well as with decision-makers at the federal level. For example, during the 2009 H1N1 influenza pandemic, MIDAS investigators were embedded in the United States Department of Health and Human Services to use mathematical and computational modelling to help with the national response. The MIDAS investigators helped with various decisions, ranging from how to allocate vaccines that were at the time in short supply to how a new intravenous antiviral medication should be used to how a potential third wave of the pandemic could have been prevented. Having modellers in person right on site allowed for close daily iterative interactions with decision-makers at the Administration for Strategic Preparedness and Response, the Biomedical Advanced Research and Development Authority, and other entities within Health and Human Services, as well as with the Department of Homeland Security. The modellers, in turn, could better understand in real time what key questions, pressures and constraints the decision-makers were facing. Meanwhile, the decision-makers were better able to get to know the models and their relative strengths and limitations. This helped match the questions of interest to the most relevant models. The MIDAS network has assisted with the response to nearly every major infectious threat to the United States since its formation, including the threats of avian influenza, the spread of methicillin-resistant Staphylococcus aureus, the Zika outbreak and the COVID-19 pandemic. The National Institute of General Medical Sciences has brought modellers into the network by issuing funding opportunities that all modellers can respond to and compete for in a competitive peer-reviewed process. Sources: Lee et al. (13, 15, 115–117) 46 Strengthening pandemic preparedness and response through integrated modelling 3.4.2 Incorporating integrated modelling and real world decision-making into educational programmes Develop a new cohort of individuals with both interdisciplinary and policy expertise Modelling can work best and be most impactful when policy-makers, their teams or units, and modellers truly understand each other. Thus, increasing the entry points and available career paths will help build, expand and diversify the public service workforce. For example, exposing people to modelling and integrated modelling earlier in their educational paths – that is, before they reach university – may bolster the number of people interested in these areas. Later in people’s career paths, offering continuing education opportunities for working professionals will allow people to learn some aspects of modelling without having to dedicate themselves to full-time graduate programmes. Even if they do not eventually become modellers, earlier education or continuing education exposure can help them become more fluent in modelling so that they can better distinguish among different types of modelling and interpret the results. At the same time, making more training opportunities available at the graduate and postgraduate levels can help bolster the local modelling workforce and increase the trust and utility of integrated models in policy-making (Box 15). Currently, there is a dearth of such training opportunities for integrated modelling (118). 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 47 Box 15.  Key success factors in translating modelled evidence into effective policy-making and policy impact Recognizing that modelled results are only impactful if they are used by policy-makers, R4D (Results for Development) worked with researchers in Burkina Faso, India, Kenya, Nigeria and South Africa to identify how to improve policy-maker access to and use of modelling for decision-making. One key finding was that knowledge brokers, who are the people and organizations that engage intentionally with both policy-makers and modelling groups, have a potentially catalytic role in fostering dialogue, distilling findings into policy recommendations, and ensuring sustainability when there is turnover among policy-makers. Intentional investing in knowledge translation mechanisms was reported to be key, and often used formalized technical working groups, tasks forces and model comparison exercises. Capacity-building initiatives to produce, manage and analyse data were important, as were greater transparency in modelling methods and assumptions and increased funding for capacity-building among modellers and policy-makers. Additional important contributors for improving access and use of modelling by policy-making included: • aligning modelling work with policy priorities; • maintaining long-term, stable relationships with government partners; • sustained government commitment to evidence-based decision-making; • credible modelling practices and research organizations; and • routine communication with policy-makers throughout the modelling process, including interactive and iterative communication processes. It is important to note that there were country-specific differences, and not all ecosystems functioned in the same way. The maturity of the modelling organizations, cultural and political context, role of development partners and strength of academic institutions all played roles. However, despite these differences between countries, the trust and utility of models for decision-making was highest when models were locally developed, context-specific and based on high-quality data. Source: R4D (119) 48 Strengthening pandemic preparedness and response through integrated modelling Integrate real world decision-making into integrated modelling training programmes Training programmes in integrated modelling should reflect a broad span of interests. Not everyone entering the modelling arena will have the same interests. At one end of the spectrum are those who gravitate to the theoretical aspects of modelling. At the other end are those who want to be involved in highly applied modelling. Yet there is a dearth of training programmes that integrate the real world problems that policy-makers are facing. To incorporate real world policy-making issues in training programmes, universities and other teaching institutions should actively seek collaborations with the government to better train the next generation workforce. 3.4.3 Creating new interdisciplinary modelling positions along with appropriate incentives to fill those positions There should be strong incentives to fill interdisciplinary positions, and such positions should have accompanying responsibilities, authority and career paths Creating more training opportunities will not be enough to build a workforce. Those completing the training must have positions available once they complete the training. Moreover, it will not be enough to create such positions. Talented individuals will be less inclined to serve in positions where they see no opportunity for advancement and to affect decision-making. This third and last section of the guide proposes four initiatives and associated activities that can enhance the production and use of integrated modelling in policy-making. Each initiative and associated activities should be tailored and prioritized based on context. Not all initiatives and activities may start completely anew. Thus, surveying the current landscape can help identify what capacity and capabilities already exist that can be connected and leveraged. In turn, this can determine what gaps remain and where investments are needed. Substantial initial investments in time and resources may be needed in settings with limited modelling capacity and current use of modelling in policy-making. It is also important to bear in mind that some activities must be maintained throughout inter- and intra-epidemic times once initiated. Thus, it is useful to identify the types and quantities of additional resources needed under each initiative and potential sources of funding. The prioritization of initiatives may be informed by the available resources and funding prospects and supplemented by return-on-investment analyses. This process should stay flexible as circumstances change. For example, when a novel pathogen emerges, the priority could be to get modelling collaborations up and running as soon as possible if not already established. There might also be cost savings from shared investments across multiple use cases or through 3.  Enhancing the production and use of integrated modelling to strengthen pandemic preparedness and response 49 leveraging current and future initiatives to build integrated modelling capacities for policy-making. Finally, systems to monitor the implementation progress and impacts of initiatives should be established to allow plans to enhance the production and use of integrated modelling to be adapted as implementation progresses and circumstances change. 50 Strengthening pandemic preparedness and response through integrated modelling Concluding remarks Policy-makers work to reduce the impacts of an epidemic or pandemic on many dimensions of importance to society. In doing so, they must make difficult choices from a growing variety of intervention options and amid great uncertainty. At the onset of a new epidemic or pandemic, policy-makers make decisions on a narrow set of response options informed by continuously evolving information. As the situation evolves, they continue to face ongoing uncertainty about the potential emergence of new variants, population immunity dynamics and behavioural shifts, and an expanding array of PHSM and medical countermeasures. In this uncertain environment, policy-making should be informed by evidence on the benefits and costs of alternative response strategies, and the evidence should be explicitly evaluated from the perspective of all stakeholders likely to be affected by these policies. Such evaluation is especially important when the risks posed by the severity of an epidemic are high, because choosing the optimal response policy will maximize benefits at minimum cost. Mathematical modelling can help overcome the inherent difficulties in projecting likely outcomes of alternative response policies in a rapidly changing environment. Even though few policy questions can be dealt with by one discipline alone, disciplines have remained siloed. To address the complexity of epidemics and pandemics and their multidimensional factors, a multiperspective approach is needed. Specifically, integrating epidemiological and macroeconomic models in a shared analytical framework provides a helpful framework within which interactions between health and economic systems are explicitly considered. In addition, integrated modelling can benefit policy-making by providing information on the benefits and costs of alternative response strategies and their distributions within society while accounting for the uncertainties in various policy and behavioural response outcomes. Finally, integrated modelling is a way to explicitly acknowledge inherent policy trade-offs and can thus help make policy-making processes more comprehensive, systematic and transparent. Only through an iterative process initiated before an epidemic or pandemic risk emerges can modelling groups – with the support of policy-makers, experts and advisers – build robust models that can then be used to project what could happen under different scenarios. A collaborative team of policy-makers (or their support teams) and of modelling groups from epidemiological, economic and behavioural sciences should thus be part of a preparedness strategy and be ready to be deployed when a pathogen with epidemic or pandemic potential emerges. Concluding remarks 51 References 1. World Health Organization, Organisation for Economic Co-operation and Development. Sustaining lives and livelihoods: a decision framework for calibrating social and movement measures during the COVID-19 pandemic. Geneva: World Health Organization; 2020 (https://apps.who. int/iris/handle/10665/339598, accessed 20 June 2023). 2. Preparedness and Resilience for Emerging Threats (PRET). Geneva: World Health Organization; 2023 (https://www.who.int/initiatives/preparedness- and-resilience-for-emerging-threats, accessed 20 June 2023). 3. Horton R. Offline: COVID-19 is not a pandemic. Lancet. 2020;396:874. doi:10.1016/S0140-6736(20)32000-6. 4. Diotaiuti P, Valente G, Mancone S, Falese L, Bellizzi F, Anastasi D et al. Perception of risk, self-efficacy and social trust during the diffusion of COVID-19 in Italy. Int J Environ Res Public Health. 2021;18:3427. doi:10.3390/ijerph18073427. 5. Managing epidemics: key facts about major deadly diseases. Geneva: World Health Organization; 2018 (https://apps.who.int/iris/ handle/10665/272442, accessed 20 June 2023). 6. Factors that contributed to undetected spread of the Ebola virus and impeded rapid containment. Geneva: World Health Organization; 2015 (https://www.who.int/news-room/spotlight/one-year-into-the-ebola- epidemic/factors-that-contributed-to-undetected-spread-of-the-ebola- virus-and-impeded-rapid-containment, accessed 20 June 2023). 7. Rehfuess EA, Movsisyan A, Pfadenhauer LM, Burns J, Ludolph R, Michie S et al. Public health and social measures during health emergencies such as the COVID-19 pandemic: an initial framework to conceptualize and classify measures. Influenza Other Respir Viruses. 2023;17:e13110. doi:10.1111/irv.13110. 8. Sachs JD, Karim SSA, Aknin L, Allen J, Brosbøl K, Colombo F et al. The Lancet Commission on lessons for the future from the COVID-19 pandemic. Lancet. 2022;400:1224–80. doi:10.1016/S0140- 6736(22)01585-9. 9. Falk L, Neumann-Böhme S, Sabat I, Schreyögg J. Public perceptions of COVID-19 lockdown policies in Europe: socioeconomic status and trust were factors. Health Affairs. 2023;42:1706–14. doi:10.1377/ hlthaff.2023.00707. 52 Strengthening pandemic preparedness and response through integrated modelling 10. Lim TY, Xu R, Ruktanonchai N, Saucedo O, Childs LM, Jalali MS et al. Why similar policies resulted in different COVID-19 outcomes: how responsiveness and culture influenced mortality rates. Health Affairs. 2023;42:1637–46. doi:10.1377/hlthaff.2023.00713. 11. Walker S, Griffin S, Asaria M, Tsuchiya A, Sculpher M. Striving for a societal perspective: a framework for economic evaluations when costs and effects fall on multiple sectors and decision makers. Appl Health Econ Health Policy. 2019;17:577–90. doi:10.1007/s40258-019-00481- 8. 12. Norheim OF, Abi-Rached JM, Bright LK, Bærøe K, Ferraz OLM, Gloppen S et al. Difficult trade-offs in response to COVID-19: the case for open and inclusive decision making. Nat Med. 2021;27:10–3. doi:10.1038/ s41591-020-01204-6. 13. Lee BY, Haidari LA, Lee MS. Modelling during an emergency: the 2009 H1N1 influenza pandemic. Clin Microbiol Infect. 2013;19:1014–22. doi:10.1111/1469-0691.12284. 14. Jit M, Ainslie K, Althaus C, Caetano C, Colizza V, Paolotti D et al. Reflections on epidemiological modeling to inform policy during the COVID-19 pandemic in Western Europe, 2020–23. Health Affairs. 2023;42:1630–6. doi:10.1377/hlthaff.2023.00688. 15. Lee BY, Wiringa AE. The 2009 H1N1 influenza pandemic: a case study of how modeling can assist all stages of vaccine decision-making. Hum Vaccin. 2011;7:115–9. doi:10.4161/hv.7.1.13740. 16. Bonnet G, Pearson CAB, Torres-Rueda S, Ruiz F, Lines J, Jit M et al. A scoping review & taxonomy of epidemiological-macroeconomic models of COVID-19. Value Health. 2023. doi:10.1016/j.jval.2023.10.008. 17. Duarte A, Walker S, Metry A, Wong R, Panovska-Griffiths J, Sculpher M. Jointly modelling economics and epidemiology to support public policy decisions for the COVID-19 response: a review of UK studies. Pharmacoeconomics. 2021;39:879–87. doi:10.1007/s40273-021- 01045-2. 18. Melosi L. Comments on ‘Epidemics in the New Keynesian model’ by Eichenbaum, Rebelo, and Trabandt. J Econ Dyn Control. 2022;140:104307. doi:10.1016/j.jedc.2022.104307. 19. Shin Y, Vandenbrouche G. Covid-19 economics: introduction. Journal of Economic Dynamics & Control. 2022;140. doi:10.1016/j. jedc.2022.104429 20. Haw DJ, Morgenstern C, Forchini G, Johnson R, Doohan P, Smith PC et al. Data needs for integrated economic-epidemiological models of pandemic mitigation policies. Epidemics. 2022;41:100644. doi:10.1016/j. epidem.2022.100644. References 53 21. 10 proposals to build a safer world together – strengthening the Global Architecture for Health Emergency Preparedness, Response and Resilience: draft for consultation. Geneva: World Health Organization; 2022 (https://www.who.int/publications/m/item/10-proposals-to-build-a- safer-world-together---strengthening-the-global-architecture-for-health- emergency-preparedness--response-andresilience--white-paper-for- consultation--june-2022, accessed 20 June 2023). 22. WHO launches new initiative to improve pandemic preparedness. Geneva: World Health Organization; 2023 (https://www.who.int/news/item/26-04- 2023-who-launches-new-initiative-to-improve-pandemic-preparedness, accessed 20 June 2023). 23. Considerations for implementing and adjusting public health and social measures in the context of COVID-19: interim guidance, 30 March 2023. Geneva: World Health Organization; 2023 (https://iris.who.int/ handle/10665/366669, accessed 24 November 2023). 24. Norris SL, Rehfuess EA, Smith H, Tunçalp Ö, Grimshaw JM, Ford NP et al. Complex health interventions in complex systems: improving the process and methods for evidence-informed health decisions. BMJ Glob Health. 2019;4:e000963. doi:10.1136/bmjgh-2018-000963. 25. Rehfuess EA, Stratil JM, Scheel IB, Portela A, Norris SL, Baltussen R. The WHO-INTEGRATE evidence to decision framework version 1.0: integrating WHO norms and values and a complexity perspective. BMJ Glob Health. 2019;4:e000844. doi:10.1136/bmjgh-2018-000844. 26. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. doi:10.1186/1748-5908-6-42. 27. WHO Council on the Economics of Health For All. Geneva: World Health Organization; 2023 (https://www.who.int/groups/who-council-on-the- economics-of-health-for-all, accessed 20 June 2023). 28. COVID-19 and the social determinants of health and health equity: evidence brief. Geneva: World Health Organization; 2021 (https://apps. who.int/iris/handle/10665/348333, accessed 20 June 2023). 29. World Development Report 2022: finance for an equitable recovery. Washington, DC: World Bank; 2022 (https://dx.doi.org/10.1596/978-1- 4648-1730-4, accessed 20 June 2023). 30. Technical note from the WHO Technical Advisory Group on behavioural insights and science for health. Geneva: World Health Organization; 2021 (https://www.who.int/publications/m/item/technical-note-from-the- who-technical-advisory-group-on-behavioural-insights-and-science-for- health, accessed 20 June 2023). 54 Strengthening pandemic preparedness and response through integrated modelling 31. Camerer C. Behavioral economics: reunifying psychology and economics. Proc Natl Acad Sci. 1999;96:10575–7. doi:10.1073/pnas.96.19.10575. 32. COVID-19 social and behavioural data collection. Geneva: World Health Organization; 2023 (https://www.who.int/initiatives/behavioural-sciences/ covid-sbi-data-collection, accessed 20 June 2023). 33. Social and behavioural insights COVID-19 data collection tool for Africa – case study: pilot implementation in Nigeria and Zambia. Brazzaville: World Health Organization Regional Office for Africa; 2021 (https://www. afro.who.int/publications/social-and-behavioural-insights-covid-19-data- collection-tool-africa-case-study-pilot, accessed 20 June 2023). 34. Ghebreyesus TA. Using behavioural science for better health. Bull World Health Organ. 2021;99:755. doi:10.2471/blt.21.287387. 35. Altieri E, Grove J, Davies OL, Habersaat KB, Okeibunor J, Samhouri D et al. Harnessing the power of behavioural science to improve health. Bull World Health Organ. 2021;99:754–A. doi:10.2471/blt.21.287375. 36. Squires H, Kelly MP, Gilbert N, Sniehotta F, Purshouse RC. The long-term effectiveness and cost-effectiveness of public health interventions; how can we model behavior? A review. Health Econ. 2023;32:2836–54. doi:10.1002/hec.4754. 37. López Gómez A, Dogmanas D, Brunet-Adami N, Bagattini N, Bernardi R. Using behavioural and social sciences to inform public policies during COVID-19, Uruguay. Bull World Health Organ. 2021;99:843–4. doi:10.2471/blt.21.287071. 38. Persad G, Pandya A. A comprehensive COVID-19 response – the need for economic evaluation. NEJM. 2022;386:2449–51. doi:10.1056/ NEJMp2202828. 39. Health Emergency Preparedness and Response Umbrella Program annual report 2021. Washington, DC: World Bank; 2022 (https://thedocs.worldbank.org/en/ doc/62688b8a3f8fc6c92036e9035c69c34e-0200022022/ original/HEPR-2021-Annual-Report-Publ-VB-220425.pdf, accessed 20 June 2023). 40. Strengthening pandemic preparedness planning for respiratory pathogens: policy brief, 27 April 2022. Geneva: World Health Organization; 2022 (https://apps.who.int/iris/handle/10665/353577, accessed 20 June 2023). 41. Strengthening the global architecture for health emergency prevention, preparedness, response and resilience. Geneva: World Health Organization; 2023 (https://cdn.who.int/media/docs/default-source/ emergency-preparedness/who_hepr_wha2023-21051248b.pdf, accessed 27 November 2023). References 55 42. Defining collaborative surveillance: a core concept for strengthening the global architecture for health emergency preparedness, response, and resilience (HEPR). Geneva: World Health Organization; 2023 (https://iris. who.int/handle/10665/367927, accessed 27 November 2023). 43. Epidemic analysis for response decision-making : systematic organization of multi-source information to inform response decisions. Manila: World Health Organization. Regional Office for the Western Pacific; 2020 (https://iris.who.int/handle/10665/333046, accessed 27 November 2023). 44. Carter SE, Gobat N, Pfaffmann Zambruni J, Bedford J, van Kleef E, Jombart T et al. What questions we should be asking about COVID-19 in humanitarian settings: perspectives from the Social Sciences Analysis Cell in the Democratic Republic of the Congo. BMJ Glob Health. 2020;5. doi:10.1136/bmjgh-2020-003607. 45. Carter SE, Ahuka-Mundeke S, Zambruni JP, Colorado CN, Kleef Ev, Lissouba P et al. How to improve outbreak response: a case study of integrated outbreak analytics from Ebola in Eastern Democratic Republic of the Congo. BMJ Global Health. 2021;6:e006736. doi:10.1136/ bmjgh-2021-006736. 46. Evidence, policy, impact: WHO guide for evidence-informed decision-making. Geneva: World Health Organization; 2021 (https://apps. who.int/iris/handle/10665/350994, accessed 20 June 2023). 47. Morgan O. How decision makers can use quantitative approaches to guide outbreak responses. Philos Trans R Soc Lond B Biol Sci. 2019;374:20180365. doi:10.1098/rstb.2018.0365. 48. Friedman L, Friedman H, Pollack S. The role of modeling in scientific disciplines: a taxonomy. Rev Bus. 2008;29:61–7. 49. Kivits J, Ricci L, Minary L. Interdisciplinary research in public health: the ‘why’ and the ‘how’. J Epidemiol Community Health. 2019;73:1061–2. doi:10.1136/jech-2019-212511. 50. Kivits J, Fournier C, Mino J, Frattini M, Winance M, Lefève C et al. Jalons pour une recherche interdisciplinaire en santé et en sciences humaines et sociales : apports d’un séminaire de recherche. Santé Publique. 2013;25:579–86. doi:10.3917/spub.135.0579. 51. Aprigliano V, Borin A, Conteduca F, Emiliozzi S, Flaccadoro M, Marchetti S et al. Forecasting Italian GDP growth with epidemiological data. Occasional Paper No. 664. In: Questioni di Economia e Finanza. Rome: Bank of Italy; 2021 (https://www.bancaditalia.it/pubblicazioni/qef/2021- 0664, accessed 20 June 2023). 56 Strengthening pandemic preparedness and response through integrated modelling 52. Conteduca FP, Borin A. A new dataset for local and national COVID-19–related restrictions in Italy. Ital Econ J. 2022;8:435–70. doi:10.1007/s40797-022-00197-0. 53. Marchetti S, Borin A, Conteduca FP, Ilardi G, Guzzetta G, Poletti P et al. An epidemic model for SARS-CoV-2 with self-adaptive containment measures. PLoS One. 2022;17:e0272009. doi:10.1371/journal. pone.0272009. 54. Koronautvalget. Evaluering av pandemihåndteringen. Rapport fra Koronautvalget (in Norwegian). Oslo: Departementenes sikkerhets og serviceorganisasjon; 2023 (https://www.regjeringen.no/no/dokumenter/ nou-2023-16/id2982388/, accessed 28 November 2023). 55. Samfunnsøkonomiske vurderinger av smitteverntiltak (COVID-19) (in Norwegian). Oslo: Helsedirektoratet (Norwegian Directorate of Health); 2021 (https://www.helsedirektoratet.no/rapporter/samfunnsokonomisk- vurdering-av-smitteverntiltak-covid-19, accessed 28 November 2023). 56. Goenka A, Liu L. Smart lockdowns to control COVID-19. Nat Comput Sci. 2022;2:217–8. doi:10.1038/s43588-022-00239-8. 57. Pianella M, Doohan P, Parchani K, Patouillard E, Hauck K. Integrated epidemiological–economic models for pandemic mitigation policies: a scoping review. Report to WHO. Geneva: World Health Organization; 2023. 58. Doohan P, Hauck K. What is the return on investment of pandemic preparedness? London: Center for Global Development; 2022 (https:// www.cgdev.org/event/what-return-investment-pandemic-preparedness, accessed 20 June 2023). 59. World Health Organization, World Bank, International Monetary Fund, European Investment Bank. Report on economic vulnerabilities and risks to pandemics and potential policy measures. G20; 2023. 60. Measuring the effectiveness and impact of public health and social measures. Geneva: World Health Organization; 2023 (https://www.who. int/activities/measuring-the-effectiveness-and-impact-of-public-health- and-social-measures, accessed 20 June 2023). 61. Global COVID-19 vaccination – strategic vision for 2022. Geneva: World Health Organization; 2021 (https://cdn.who.int/media/docs/default- source/immunization/sage/covid/global-covid-19-vaccination-strategic- vision-for-2022_sage-yellow-book.pdf, accessed 20 June 2023). 62. Strategy to achieve global COVID-19 vaccination by mid-2022. Geneva: World Health Organization; 2021 (https://www.who.int/publications/m/ item/strategy-to-achieve-global-covid-19-vaccination-by-mid-2022, accessed 27 November 2023). References 57 63. Haw DJ, Forchini G, Doohan P, Christen P, Pianella M, Johnson R et al. Optimizing social and economic activity while containing SARS-CoV-2 transmission using DAEDALUS. Nat Comput Sci. 2022;2:223–33. doi:10.1038/s43588-022-00233-0. 64. Doohan P, Haw DJ, Pianella M, Johnson R, Hauck K. Epidemiological and economic projections of mitigation measures for the COVID-19 pandemic in Sri Lanka. 2021. 65. The G20 established a joint Finance-Health Task Force to strengthen pandemic prevention, preparedness and response. Rome: Ministero dell’Economia e delle Finanze; 2021 (https://www.mef.gov.it/en/ inevidenza/The-G20-established-a-joint-Finance-Health-Task-Force- to-strengthen-pandemic-prevention-preparedness-and-response, accessed 27 November 2023). 66. G20 Bali Leaders’ Declaration. Washington, DC: United States Government; 2022 (https://www.whitehouse.gov/briefing-room/ statements-releases/2022/11/16/g20-bali-leaders-declaration, accessed 27 November 2023). 67. Lee BY, Bartsch SM, Ferguson MC. How COVID-19 economic and epidemiologic modelling can help decision making. Report to WHO. 2022. 68. McBryde ES, Meehan MT, Adegboye OA, Adekunle AI, Caldwell JM, Pak A et al. Role of modelling in COVID-19 policy development. Paediatr Respir Rev. 2020;35:57–60. doi:10.1016/j.prrv.2020.06.013. 69. Haw DJ, Biggerstaff M, Prasad P, Walker J, Grenfell B, Arinaminpathy N. Using real-time data to guide decision-making during an influenza pandemic: a modelling analysis. PLoS Comput Biol. 2023;19:e1010893. doi:10.1371/journal.pcbi.1010893. 70. Thompson RN, Brooks-Pollock E. Detection, forecasting and control of infectious disease epidemics: modelling outbreaks in humans, animals and plants. Philos Trans R Soc Lond B Biol Sci. 2019;374:20190038. doi:10.1098/rstb.2019.0038. 71. Bausch DG, Edmunds J. Real-time modeling should be routinely integrated into outbreak response. Am J Trop Med Hyg. 2018;98:1214–5. doi:10.4269/ajtmh.18-0150. 72. Mistry D, Litvinova M, Pastore y Piontti A, Chinazzi M, Fumanelli L, Gomes MFC et al. Inferring high-resolution human mixing patterns for disease modeling. Nat Commun. 2021;12:323. doi:10.1038/s41467-020- 20544-y. 73. South African COVID-19 Modelling Consortium. Tekanelo: a summary of model methods and assumptions. Report to WHO. 2022. 58 Strengthening pandemic preparedness and response through integrated modelling 74. Pan American Health Organization, World Health Organization. Strategy for building resilient health systems and post COVID-19 pandemic recovery to sustain and protect public health gains. Washington, DC: Pan American Health Organization; 2021 (https://www.paho.org/en/ documents/cd5911-strategy-building-resilient-health-systems-and-post- covid-19-pandemic-recovery, accessed 27 November 2023). 75. Weather Program Office. The Weather Act: congressional support for improvements in forecasting. Washington, DC: National Oceanic and Atmospheric Administration; 2019 (https://wpo.noaa.gov/the-weather- act, accessed 20 June 2023). 76. Bedson J, Skrip LA, Pedi D, Abramowitz S, Carter S, Jalloh MF et al. A review and agenda for integrated disease models including social and behavioural factors. Nat Hum Behav. 2021;5:834–6. doi:10.1038/ s41562-021-01136-2. 77. Ortenzi F, Marten R, Valentine NB, Kwamie A, Rasanathan K. Whole of government and whole of society approaches: call for further research to improve population health and health equity. BMJ Global Health. 2022;7:e009972. doi:10.1136/bmjgh-2022-009972. 78. Rojhani A. Glossary: whole-of-government, whole-of-society, health in all policies, and multisectoral. International Center for Development of Science and Technology; 2016 (https://dl.icdst.org/pdfs/files2/ bb0495d49d4a4ec51efccf3e70b45885.pdf, accessed 20 June 2023). 79. Abramowitz SA, Hipgrave DB, Witchard A, Heymann DL. Lessons from the West Africa Ebola epidemic: a systematic review of epidemiological and social and behavioral science research priorities. J Infect Dis. 2018;218:1730–8. doi:10.1093/infdis/jiy387. 80. Kretzschmar ME, Ashby B, Fearon E, Overton CE, Panovska-Griffiths J, Pellis L et al. Challenges for modelling interventions for future pandemics. Epidemics. 2022;38:100546. doi:10.1016/j. epidem.2022.100546. 81. Scientific Advisory Group for Emergencies. London: Government of the United Kingdom; 2022 (https://www.gov.uk/government/organisations/ scientific-advisory-group-for-emergencies, accessed 20 June 2023). 82. Isaacs L. Govt to launch multi-sectoral advisory committee on COVID-19. Johannesburg: Eyewitness News South Africa; 2020 (https://iafrica.com/ government-to-launch-new-covid-19-ministerial-advisory-committee/, accessed 20 June 2023). 83. About us. Health Data Collaborative; 2023 (https://www. healthdatacollaborative.org/about-us, accessed 20 June 2023). References 59 84. Joint statement on data protection and privacy in the COVID-19 response. Geneva: World Health Organization; 2020 (https://www.who.int/news/ item/19-11-2020-joint-statement-on-data-protection-and-privacy-in- the-covid-19-response, accessed 20 June 2023). 85. A growing collection of resources for computational model- based science. CoMSES Net; 2023 (https://www.comses.net, accessed 20 June 2023). 86. Amaratunga D, Fernando N, Haigh R, Jayasinghe N. The COVID-19 outbreak in Sri Lanka: a synoptic analysis focusing on trends, impacts, risks and science-policy interaction processes. Prog Disaster Sci. 2020;8:100133. doi:10.1016/j.pdisas.2020.100133. 87. Sri Lanka: 2021 Article IV consultation – press release; staff report; and statement by the Executive Director for Sri Lanka. Washington, DC: International Monetary Fund; 2022 (https://www.imf.org/en/Publications/ CR/Issues/2022/03/25/Sri-Lanka-2021-Article-IV-Consultation-Press- Release-Staff-Report-and-Statement-by-the-515737, accessed 20 June 2023). 88. FASSSTER team. Report on the close collaboration of the FASSSTER team with Imperial College London on the development of the Philippines- Daedalus epidemiological-economic model. Report to WHO. 2022. 89. What is FASSSTER? Manila, Philippines: Republic of the Philippines, Department of Health; 2023 (DOH: FASSSTER COVID-19 v4.0 (ehealth. ph), accessed 01 November). 90. Social Sciences Analytics Cell. Exploring community dynamics around the plague outbreak in Ituri province, DRC. United Nations Children’s Fund (UNICEF); 2021 (https://www.unicef.org/drcongo/en/node/3121, accessed 27 November 2023). 91. Cellule d’Analyse en Sciences Sociales. Recommandations clés de la Cellule d’Analyse en Sciences Sociales pour les équipes de réponse contre la MVE en Equateur (Juin 2020) – 7. Rapport de synthèse (note d’orientation). ReliefWeb; 2020 (https://reliefweb.int/report/democratic- republic-congo/recommandations-cl-s-de-la-cellule-d-analyse-en- sciences-sociales-5, accessed 27 November 2023). 92. Cellule d’Analyse en Sciences Sociales. The impacts of the COVID-19 outbreak response on women and girls in the Democratic Republic of the Congo. Social Science in Humanitarian Action Platform; 2020 (https:// www.socialscienceinaction.org/resources/the-impacts-of-the-covid-19- outbreak-response-on-women-and-girls-in-the-democratic-republic-of- the-congo, accessed 27 November 2023). 60 Strengthening pandemic preparedness and response through integrated modelling 93. Cellule d’Analyses Intégrées. Integrated analytics for public health emergencies and programmes. ReliefWeb; 2022 (https://reliefweb.int/ report/democratic-republic-congo/integrated-analytics-public-health- emergencies-and-programmes, accessed 27 November 2023). 94. Integrated Analytics Cell. Integrated analyses of the determinants of child nutritional status: Manono and Kabalo health zones, Tanganyika province, DRC – Report no. 1. ReliefWeb; 2022 (https://reliefweb.int/ report/democratic-republic-congo/integrated-analyses-determinants- child-nutritional-status-manono-and-kabalo-health-zones-tanganyika- province-drc-report-ndeg-1, accessed 27 November 2023). 95. Integrated Analytics Cell. Integrated analyses of factors contributing to cholera outbreaks – Kalemie, Nyemba and Moba health zones, Tanganyika Province, DRC. ReliefWeb; 2022 (https://reliefweb.int/report/democratic- republic-congo/integrated-analytics-cell-integrated-analyses-factors- contributing-cholera-outbreaks-kalemie-nyemba-and-moba-health- zones-tanganyika-province-drc, accessed 27 November 2023). 96. Cellule d’Analyses Intégrées. Analyse opérationnelle sur les dynamiques autour de l’épidémie de rougeole à Kinshasa: synthèse des résultats clefs et des recommandations. ReliefWeb; 2022 (https://reliefweb.int/report/ democratic-republic-congo/analyse-op-rationnelle-sur-les-dynamiques- autour-de-l-pid-mie-de, accessed 27 November 2023). 97. Cellule d’Analyses Intégrées. RD Congo: Cellule d’Analyse Intégrée – Rapport annuel 2022. ReliefWeb; 2023 (https://reliefweb.int/report/ democratic-republic-congo/rd-congo-cellule-danalyse-integree-cai- rapport-annuel-2022, accessed 27 November 2023). 98. Community-centred approaches to health emergencies: progress, gaps and research priorities. Geneva: World Health Organization; 2021 (https:// www.who.int/publications/m/item/community-centred-approaches-to- health-emergencies-progress-gaps-and-research-priorities-who-covid- 19-social-science-in-outbreak-response, accessed 27 November 2023). 99. Bedford J. Using social science in response to the 2022 Ebola outbreak in Uganda. Social Science in Humanitarian Action Platform; 2022 (https://www.socialscienceinaction.org/resources/using-social- science-in-response-to-the-2022-ebola-outbreak-in-uganda, accessed 27 November 2023). 100. Citizen engagement and stakeholder consultations during COVID-19. Washington, DC: World Bank; 2020 (https://www.worldbank.org/en/ news/factsheet/2020/12/01/citizen-engagement-and-stakeholder- consultations-during-covid-19, accessed 20 June 2023). References 61 101. United Nations e-government survey 2012: e-government for the people. Chapter 3: Taking a whole-of-government approach. New York: United Nations; 2012 (https://publicadministration.un.org/egovkb/en-us/ Reports/UN-E-Government-Survey-2012, accessed 20 June 2023). 102. The United States’ whole-of-government approach to global health challenges. Paris: Organisation for Economic Co-operation and Development; 2022 (https://www.oecd.org/development-cooperation- learning/practices/the-united-states-whole-of-government-approach-to- global-health-challenges-ed48d383, accessed 20 June 2023). 103. Rubinstein A, Filippini F, Santoro A, Lopez Osornio AL, Bardach AL, Navarro E et al. Lives versus livelihoods: the epidemiological, social, and economic impact of COVID-19 in Latin America and the Caribbean. Health Affairs. 2023;42:1647–56. doi:10.1377/hlthaff.2023.00706. 104. Rubinstein A, Santoro A, Filippini F, Osornio AL, Bardach A, Navarro E et al. Technical report on epidemiological, macro-economic and social impact model to inform policy making on COVID-19 strategies in Latin America and the Caribbean. Final report to the Interamerican Development Bank (IDB) Buenos Aires: Instituto de Efectividad Clínica y Sanitaria; 2023. 105. SPH portal: overview of the strategic partnership for IHR and health security (SPH) portal. Geneva: World Health Organization; 2019 (https:// apps.who.int/iris/handle/10665/330768, accessed 20 June 2023). 106. Call for experts – the Collaboratory Technical Advisory Group. Geneva: World Health Organization; 2013 (https://pandemichub.who.int/news- room/news/31-10-2023-call-for-experts-the-collaboratory-technical- advisory-group, accessed 1 November 2023). 107. Bjørnstad ON, Shea K, Krzywinski M, Altman N. Modeling infectious epidemics. Nat Methods. 2020;17:455–6. doi:10.1038/s41592-020- 0822-z. 108. Bloom D, Kuhn M, Prettner K. Modern infectious diseases: macroeconomic impacts and policy responses. CEPR discussion paper DP15997. London: Centre for Economic Policy Research; 2021 (https:// cepr.org/publications/dp15997, accessed 20 June 2023). 109. Technical Group, COVID-19 Multi-Model Comparison Collaboration. Technical Group report: model fitness-for-purpose assessment report. Decide Health Decision Hub; 2020 (https://decidehealth. world/sites/default/files/2020-09/Technical_Report_Sept1.pdf, accessed 20 June 2023). 110. Teerawattananon Y, Kc S, Chi YL, Dabak S, Kazibwe J, Clapham H et al. Recalibrating the notion of modelling for policymaking during pandemics. Epidemics. 2022;38:100552. doi:10.1016/j.epidem.2022.100552. 62 Strengthening pandemic preparedness and response through integrated modelling 111. Eichenbaum MS, Rebelo S, Trabandt M. Epidemics in the New Keynesian model. J Econ Dyn Control. 2022;140:104334. doi:10.1016/j. jedc.2022.104334. 112. Gottardi P. Comment on ‘Optimal management of an epidemic: lockdown, vaccine and value of life’. J Econ Dyn Control. 2022;140:104314. doi:10.1016/j.jedc.2022.104314. 113. Garriga C, Manuelli R, Sanghi S. Optimal management of an epidemic: lockdown, vaccine and value of life. J Econ Dyn Control. 2022;140:104351. doi:10.1016/j.jedc.2022.104351. 114. Penny MA, Verity R, Bever CA, Sauboin C, Galactionova K, Flasche S et al. Public health impact and cost-effectiveness of the RTS,S/AS01 malaria vaccine: a systematic comparison of predictions from four mathematical models. Lancet. 2016;387:367–75. doi:10.1016/S0140- 6736(15)00725-4. 115. Lee BY, Brown ST, Bailey RR, Zimmerman RK, Potter MA, McGlone SM et al. The benefits to all of ensuring equal and timely access to influenza vaccines in poor communities. Health Aff (Millwood). 2011;30:1141–50. doi:10.1377/hlthaff.2010.0778. 116. Lee BY, Tai JH, Bailey RR, McGlone SM, Wiringa AE, Zimmer SM et al. Economic model for emergency use authorization of intravenous peramivir. Am J Manag Care. 2011;17:e1–9. 117. Lee BY, Brown ST, Cooley P, Grefenstette JJ, Zimmerman RK, Zimmer SM et al. Vaccination deep into a pandemic wave: potential mechanisms for a “third wave” and the impact of vaccination. Am J Prev Med. 2010;39:e21–9. doi:10.1016/j.amepre.2010.07.014. 118. One-day course: integrated macroeconomic–epidemiology modelling. London: City University of London; 2021 (https://www.city.ac.uk/news- and-events/events/2021/09/integrated-macroeconomic-epidemiology- modelling-one-day-course, accessed 20 June 2023). 119. Translation of modeled evidence for decision-making: final global report. Washington, DC: Results for Development; 2023 (https:// r4d.org/wp-content/uploads/Results-for-Development-Translating- Modeled-Evidence-for-Decision-Making-TMED-Final-Global-Report- April-2023-5-04-2023.pdf, accessed 6 December 2023). References 63 Annex 1  Methods and declarations of interest The development of this guide was informed by contributions from members of a technical working group, four case studies, a series of literature reviews on related topics and a public webinar organized by the WHO Department of Health Financing and Economics in May 2022. The technical working group was composed of 10 external individuals. All members participated in their individual capacities and not as representatives of their countries, governments or organizations. All members submitted to WHO a declaration of interest, disclosing potential conflicts of interest that might affect, or might reasonably be perceived to affect, their objectivity and independence in relation to the subject matter of the guide. WHO reviewed each of those and found that the interests declared were not directly related to the topics covered by the guide. All the declarations were made known and available to all technical working group members. Four case studies informed institutional-level illustrations on the production and use of integrated modelling to inform policy-making during the COVID-19 pandemic across four countries: Argentina, the Philippines, South Africa and Sri Lanka. Each study was developed in the context of technical and financial support provided by WHO to each institution that had expressed an interest in exploring the feasibility and added value of integrated modelling to inform policy decisions in their country and that demonstrated a working relationship with policy-makers during the COVID-19 pandemic. Contracted suppliers were invited to answer a semi-structured questionnaire on their experiences during the development and use of integrated models in their country. The research proposal and associated questionnaire were submitted and reviewed by the WHO Ethics Review Committee. Answers to these questions were used to develop institutional- or country-level illustrations for the guide. A series of literature reviews on topics related to the subject matter were commissioned, including: • a systematic review of studies that were published between 1 January 2020 and 31 August 2022 and that used an integrated epidemiological–macroeconomic model to measure the benefits and costs of alternative policies in response to the COVID-19 pandemic and identified the optimal policies that maximized health outcomes and minimized economic costs under different constraints, such as keeping hospital admissions under a given threshold and keeping schools opened; 64 Strengthening pandemic preparedness and response through integrated modelling • a rapid review of studies that were published between 1 January 2020 and 15 February 2022 and that used an epidemiological model combined with an economic model accounting for individual behaviour regarding the adoption of COVID-19 control measures; • a rapid review of key insights and lessons learned during the pandemic in the field of behavioural economics, the interdisciplinary field that combines economics and psychology; and • a rapid review of key considerations that should be made when determining how a mathematical model and its results may or may not fit different types of pandemic-related policy questions and key elements needed to close the gaps between the decisions made by policy-makers and the models developed by modelling groups. A public webinar was organized by the WHO Department of Health Financing and Economics in May 2022. The objectives of the webinar were to give a brief review of the unique features of integrated epidemiological–macroeconomic models; to reflect how these models have been used in the response to the COVID-19 pandemic; to discuss shortcomings and areas where improvements are needed; and to discuss future applications of these types of models to support pandemic preparedness and response. A discussion session provided opportunities to webinar participants from the general public to provide feedback, questions and suggestions of how to move the subject matter forward. Annex 1  Methods and declarations of interest 65 Annex 2  COVID-19 integrated modelling Figure A2.1 provides a simplified representation of how COVID-19 integrated models analysed the health and economic impacts of PHSM and medical countermeasures. Fig. A2.1.  Simplified representation of COVID-19 integrated modelling PHSM and medical countermeasures A B C Epidemiology Economy D PHSM: public health and social measures Source: based on Pianella et al. (1) Arrow A represents the beneficial effect on epidemiological outcomes of an intervention. This effect can be modelled in various ways; for example, by reducing the number of in-person contacts (for example, through isolation and service closures) or by making in-person contacts safer (for example, through physical distancing and wearing a face mask). Arrow B captures the direct financial cost of a measure (such as the cost of deploying personal protective equipment) or the indirect economic losses of a measure (such as production or consumption foregone due to service modifications or closures). The costs may be incurred on the supply side – possibly through a change in the availability of workers (for example, through isolation of infected individuals who would have worked otherwise) – or on the 66 Strengthening pandemic preparedness and response through integrated modelling demand side – possibly through a change in consumption opportunities for all individuals regardless of their infection status (which may be caused by service closures) – or both. A policy may be changed in response to the evolving epidemiological or economic situations (for example, imposing or phasing out service modifications or closures in response to changes in hospital admission capacity), as illustrated by the bidirectional A and B arrows. Arrows C and D represent the integration channels between epidemiology and economy. Arrow C represents the effect of changes in the epidemiology on the economy; for example, the impacts of infections or deaths on the productivity of workers and the impacts of infections, deaths or changes in behavioural patterns due to infection risk perceptions on the consumption of goods and services. Arrow D represents the effect of changes in economic activity on the epidemiology. In integrated models, this is typically represented by the spread of infections. For example, it may capture changes in transmission in the workplace because of changes in the supply of workers and in in-person contact rates between workers (because of stay-at-home orders and other factors) or because of changes in the probability of transmission of in-person contact at the workplace (for example, because of environmental modifications and use of face masks). Various avenues of transmission may be modelled, including among workers, among consumers and between workers and consumers, as well as their knock-on effects on transmission in transport and community settings (1). There should be representation of epidemiological and economic systems and mechanisms in COVID-19 integrated models In integrated models, the epidemiological component can adopt either an agent-based (or network) structure, in which individuals or networks of agents are represented, or an aggregated approach, in which agents are homogenized into population groups. On the economic side, there are several ways to represent the economy of a particular location. Some macroeconomic models are relatively aggregated and focus on how production factors such as the workforce, the physical capital2 and technological progress3 determine 2 Physical capital refers to the tangible assets or resources that are used in the production of goods and services. They include buildings, machinery, equipment, vehicles and tools that are essential for businesses to operate and produce goods and services. 3 Technological progress refers to advancements and improvements in technology that lead to increased efficiency, productivity and economic growth. They include the development of new ideas, methods, tools and processes that make tasks easier, faster and more effective, and can lead to the creation of new products and services. Annex 2  COVID-19 integrated modelling 67 the performance of an economy.4 Other types of integrated models use descriptions of the macroeconomy that incorporate more sectoral detail and focus on the relative contributions of different sectors to the total economic output – but these models tend to be annual or static models and do not trace how the economy evolves from month to month or quarter to quarter. A final set of integrated models work with a macroeconomy based on the behavioural choices of various economic agents – such as how households and firms take decisions based on their preferences and resource constraints and, in the aggregate, determine the supply and demand of labour, goods and services in an economy (1). References 1. Pianella M, Doohan P, Parchani K, Patouillard E, Hauck K. Integrated epidemiological–economic models for pandemic mitigation policies: a scoping review. Report to WHO. Geneva: World Health Organization; 2023. 4 The performance of an economy refers to how well it is functioning and is typically assessed using various indicators that provide insights into different aspects of economic activity such as the gross domestic product and the unemployment rate. 68 Strengthening pandemic preparedness and response through integrated modelling Annex 3  Key data sources used in COVID-19 integrated models Integrated models start with similar data to epidemiological and macroeconomic models Context-specific epidemiological parameters important for predicting health outcomes may include the age of infected cases and deaths over time, the infection-to-fatality ratio (which affects the number of predicted deaths and their distribution across population groups with different underlying profiles), and the transmission and in-person contact rates (which affect the spread of the infectious disease). On the economic side, data needs depend on the type of model, but typically involve GDP, labour force statistics (including labour supply, employment and workforce composition, including skills, training and hours worked), consumption, savings, capital stocks and so on. In models that represent transactions between economic entities (including sectors, industries or agents like households and firms), national account input–output tables or social accounting matrices can be used. For integration, localized economic activity and data about human contact causing transmission are needed at the same level of disaggregation at various points in time (1) The social, demographic and economic characteristics of a population are important in determining the contact patterns that drive infection spread (2). Many methodologies have been used to study human-mixing patterns, including surveys (3–6), contact diaries (7–13), wearable sensors (14, 15), analysis of time-use data (16), development of synthetic populations (17–19) and mixed approaches, such as integrating diary-based contact data with time-use data (20, 21) or combining contact data with modelling techniques (2, 20, 22, 23). Each methodology has its own limitations and assumptions because contact patterns among individuals vary according to the geographical scale, the disease under consideration, and the detailed socioeconomic and demographic characteristics of the population (2). Common approaches for integrating COVID-19 epidemiological and macroeconomic models included using contact rates between different populations in different settings (including age and workplace), workers’ productivity and behavioural patterns due to the perceived risk of infection in the community. Annex 3  Key data sources used in COVID-19 integrated models 69 First, data on the numbers and types of in-person contacts, disaggregated by age, socioeconomic status, employment status, economic sector and type of workplace were ideally required but their availability was extremely limited (1). Alternative data from contact surveys, people’s mobility trends and transaction patterns to parameterize the interface between epidemiology and economics were used, each with their advantages and disadvantages (1). Such data included, for example, contact data estimates by age group, occupation, size of workplace and activity (for example, during shopping and while travelling) estimated from demographic health surveys and other surveys and sources (22); survey data on physical proximity and frequent interactions between workers and between workers and consumers (24); mobility data describing the number of visits and visit durations for trips to retail, recreation and workplaces (2, 25, 26); air travel data (26); and other mobility data such as those collected from commercial sources (including foot traffic data from anonymized GPS location information from panels of smartphones) or transaction data from banks and credit card companies. Second, integrated models that considered changes in population behaviours due to COVID-19 and associated control measures generally made assumptions about how classes of individuals may behave differently depending on their age, employment status and infection status, with individuals of the same group generally assumed to behave the same way (27). Third, integrated models that incorporated individual behaviours and choices considered the trade-offs that individuals face between living their normal lives (enjoying the benefits of doing so) and the risk of getting infected (price to be paid). Data needs for the calibration of individual human behaviours depend on which individual preferences are represented and on how individuals weigh up the benefits and costs of a particular course of action to maximize their preferences in the model (1). References 1. Haw DJ, Morgenstern C, Forchini G, Johnson R, Doohan P, Smith PC et al. Data needs for integrated economic-epidemiological models of pandemic mitigation policies. Epidemics. 2022;41:100644. doi:10.1016/j. epidem.2022.100644. 2. Mistry D, Litvinova M, Pastore y Piontti A, Chinazzi M, Fumanelli L, Gomes MFC et al. Inferring high-resolution human mixing patterns for disease modeling. Nat Commun. 2021;12:323. doi:10.1038/s41467- 020-20544-y. 70 Strengthening pandemic preparedness and response through integrated modelling 3. Fenton KA, Korovessis C, Johnson AM, McCadden A, McManus S, Wellings K et al. Sexual behaviour in Britain: reported sexually transmitted infections and prevalent genital Chlamydia trachomatis infection. Lancet. 2001;358:1851–4. doi:10.1016/S0140- 6736(01)06886-6. 4. Dodd PJ, Looker C, Plumb ID, Bond V, Schaap A, Shanaube K et al. Age- and sex-specific social contact patterns and incidence of Mycobacterium tuberculosis infection. Am J Epidemiol. 2016;183:156–66. doi:10.1093/aje/kwv160. 5. Zhang J, Litvinova M, Liang Y, Wang Y, Wang W, Zhao S et al. Changes in contact patterns shape the dynamics of the COVID-19 outbreak in China. Science. 2020;368:1481–6. doi:10.1126/science.abb8001. 6. Jarvis CI, Van Zandvoort K, Gimma A, Prem K, Klepac P, Rubin GJ et al. Quantifying the impact of physical distance measures on the transmission of COVID-19 in the UK. BMC Med. 2020;18:124. doi:10.1186/s12916- 020-01597-8. 7. Wallinga J, Teunis P, Kretzschmar M. Using data on social contacts to estimate age-specific transmission parameters for respiratory-spread infectious agents. Am J Epidemiol. 2006;164:936–44. doi:10.1093/aje/ kwj317. 8. Mossong J, Hens N, Jit M, Beutels P, Auranen K, Mikolajczyk R et al. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Med. 2008;5:e74. doi:10.1371/journal.pmed.0050074. 9. Hens N, Goeyvaerts N, Aerts M, Shkedy Z, Van Damme P, Beutels P. Mining social mixing patterns for infectious disease models based on a two-day population survey in Belgium. BMC Infect Dis. 2009;9:5. doi:10.1186/1471-2334-9-5. 10. Horby P, Pham QT, Hens N, Nguyen TT, Le QM, Dang DT et al. Social contact patterns in Vietnam and implications for the control of infectious diseases. PLoS One. 2011;6:e16965. doi:10.1371/journal. pone.0016965. 11. Read JM, Lessler J, Riley S, Wang S, Tan LJ, Kwok KO et al. Social mixing patterns in rural and urban areas of southern China. Proc Biol Sci. 2014;281:20140268. doi:10.1098/rspb.2014.0268. 12. Ajelli M, Litvinova M. Estimating contact patterns relevant to the spread of infectious diseases in Russia. J Theor Biol. 2017;419:1–7. doi:10.1016/j.jtbi.2017.01.041. 13. Melegaro A, Del Fava E, Poletti P, Merler S, Nyamukapa C, Williams J et al. Social contact structures and time use patterns in the Manicaland province of Zimbabwe. PLoS One. 2017;12:e0170459. doi:10.1371/ journal.pone.0170459. Annex 3  Key data sources used in COVID-19 integrated models 71 14. Cattuto C, Van den Broeck W, Barrat A, Colizza V, Pinton JF, Vespignani A. Dynamics of person-to-person interactions from distributed RFID sensor networks. PLoS One. 2010;5:e11596. doi:10.1371/journal. pone.0011596. 15. Kiti MC, Tizzoni M, Kinyanjui TM, Koech DC, Munywoki PK, Meriac M et al. Quantifying social contacts in a household setting of rural Kenya using wearable proximity sensors. EPJ Data Sci. 2016;5:21. doi:10.1140/epjds/ s13688-016-0084-2. 16. Zagheni E, Billari FC, Manfredi P, Melegaro A, Mossong J, Edmunds WJ. Using time-use data to parameterize models for the spread of close- contact infectious diseases. Am J Epidemiol. 2008;168:1082–90. doi:10.1093/aje/kwn220. 17. Fumanelli L, Ajelli M, Manfredi P, Vespignani A, Merler S. Inferring the structure of social contacts from demographic data in the analysis of infectious diseases spread. PLoS Comput Biol. 2012;8:e1002673. doi:10.1371/journal.pcbi.1002673. 18. Grefenstette JJ, Brown ST, Rosenfeld R, DePasse J, Stone NT, Cooley PC et al. FRED (a Framework for Reconstructing Epidemic Dynamics): an open-source software system for modeling infectious diseases and control strategies using census-based populations. BMC Public Health. 2013;13:940. doi:10.1186/1471-2458-13-940. 19. Gallagher S, Richardson LF, Ventura SL, Eddy WF. SPEW: Synthetic Populations and Ecosystems of the World. J Comput Graph Stat. 2018;27:773–84. doi:10.1080/10618600.2018.1442342. 20. Iozzi F, Trusiano F, Chinazzi M, Billari FC, Zagheni E, Merler S et al. Little Italy: an agent-based approach to the estimation of contact patterns – fitting predicted matrices to serological data. PLoS Comput Biol. 2010;6:e1001021. doi:10.1371/journal.pcbi.1001021. 21. De Cao E, Zagheni E, Manfredi P, Melegaro A. The relative importance of frequency of contacts and duration of exposure for the spread of directly transmitted infections. Biostat. 2014;15:470–83. doi:10.1093/ biostatistics/kxu008. 22. Prem K, Cook AR, Jit M. Projecting social contact matrices in 152 countries using contact surveys and demographic data. PLoS Comput Biol. 2017;13:e1005697. doi:10.1371/journal.pcbi.1005697. 23. Litvinova M, Liu QH, Kulikov ES, Ajelli M. Reactive school closure weakens the network of social interactions and reduces the spread of influenza. Proc Natl Acad Sci USA. 2019;116:13174–81. doi:10.1073/ pnas.1821298116. 72 Strengthening pandemic preparedness and response through integrated modelling 24. O*NET OnLine. Physical proximity. Washington, DC: United States Department of Labor; 2023 (https://www.onetonline.org/find/descriptor/ result/4.C.2.a.3?a=1, accessed 28 November 2023). 25. See how your community moved differently due to COVID-19. Mountain View, CA: Google; 2023 (https://www.google.com/covid19/mobility, accessed 20 June 2023). 26. Alessandretti L. What human mobility data tell us about COVID-19 spread. Nat Rev Phys. 2022;4:12–3. doi:10.1038/s42254-021-00407-1. 27. Pianella M, Doohan P, Parchani K, Patouillard E, Hauck K. Integrated epidemiological–economic models for pandemic mitigation policies: a scoping review. Report to WHO. Geneva: World Health Organization; 2023. Annex 3  Key data sources used in COVID-19 integrated models 73 Annex 4  A framework to contextualize the four initiatives proposed to enhance the production and use of integrated modelling A framework representing a spectrum of country characteristics along two dimensions is used to translate and adapt each of the four initiatives proposed in section 3 (Fig. A4.1) to different contexts. One dimension represents the degree to which modelling may currently be used to assist policy-making for epidemics in a country (vertical axis), and the other relates to the current availability of integrated modelling capacity in the same country (horizonal axis). Country type A (“advancing”) refers to countries where some modelling is used to inform policies before, during and/or after some or all epidemics, and some integrated modelling capacity exists, although its use to inform policy-making may be limited. Country type B refers to countries where there is capacity in integrated modelling but where the use of modelling in policy-making is relatively limited (B1 type – “increasing use”) or countries where modelling is used to inform policy-making but where there is more limited capacity in local integrated modelling (B2 type – “developing capacity”). Country type C (“emerging”) refers to countries where the use of modelling for decision-making is developing and where local integrated modelling capacity is yet to be established. The four initiatives proposed in section 3 may help countries progress in one or both dimensions of Fig. A4.1. A country may progress through the different quadrants depending on where they are on the spectrum of each of the two dimensions. The current availability of any type of modelling capacity may range from limited (far-left of the horizontal axis) to the current availability of integrated modelling capacity (far-right of the horizontal axis) through a spectrum of modelling capacities in single and potentially siloed disciplines. Similarly, the current use of modelling for policy-making may vary across countries and in a country across epidemics and across time (for example, before, during and after an epidemic; vertical axis). 74 Strengthening pandemic preparedness and response through integrated modelling Because there is a continuum of country characteristics along two dimensions, the short-term goals of type C countries may be oriented towards moving closer or into type B; that is, either developing modelling capacity (towards or into B2) or increasing modelling use during policy-making (towards or into B1), or both. Similarly, the short-term goals of type B countries may be to progress towards or into type A. As for country type A, their short-term goals may include further improving their integrated modelling capacity and growing the use of integrated modelling for policy-making. Countries of type A may also assist countries of types B or C in improving in the direction(s) of their choice. Table A4.1.  Summary of short-term activities by country type to formally incorporate integrated modelling into policy-making (Initiative 1) Dimension: Increasing the use of modelling for policy-making Activity: Institutionalizing integrated modelling in policy-making (see section 3.1.1) In B1 and A countries, policy-makers or those who support them can develop procedures detailing collaborations with modelling groups and establish or extend interactions with integrated modelling groups in their country. In C and B2 countries, policy-makers or those who support them can develop procedures detailing how to establish interactions and collaborations with modelling groups from other countries with that capacity. Activity: Assigning clear modelling-related responsibilities (see section 3.1.2) In B1 and A countries, policy-makers or those who support them can, when seeking support, be matched with modelling groups within the country. In C and B2 countries, policy-makers or those who support them can, when seeking support, be matched with modelling groups in other countries with that capacity. Activity: Establishing procedures on the production of adequate integrated models (see section 3.1.3) B1 and A countries can share experiences with each other and facilitate collaborations with C and B1 countries. C and B2 countries can establish collaboration with countries further along at using modelling and integrated modelling in policy-making. Activity: Creating an ecosystem of key data and data sources for policy-making (see section 3.1.4) B1 and A countries can build or strengthen their data ecosystems and expand open-source model, code and data repositories to facilitate collaboration and access for C and B2 countries. See section 3.3 for a how-to guide on integrated modelling, including data needs. C and B2 countries can map out the data sources that are relevant to integrated modelling for epidemics and are currently available within their countries and identify where missing data could be drawn from. Notes: In type A countries, there is capacity in integrated modelling and modelling is commonly used in policy-making for epidemics, but integrated modelling may not be widely used yet. In type B1 countries, there is capacity in integrated modelling and the use of modelling in policy-making is limited or increasing. In type B2 countries, there is limited or developing capacity in integrated modelling and modelling is used to inform policy-making. In type C countries, the use of modelling for decision-making is developing and local integrated modelling capacity is yet to be established or is emerging. Annex 4  A framework to contextualize the four initiatives proposed to enhance the production and use of integrated modelling75 Fig. A4.1.  A framework of country types along two dimensions: the degree to which modelling is currently used to inform policy-making and the current availability of modelling and integrated modelling Greater use of modelling to inform policy-making TYPE B2 TYPE A Developing capacity Advancing • Modelling used to inform • Modelling used to inform decision-making decision-making • Existing modelling capacity • Existing integrated is mostly discipline-specific modelling capacity • Limited or no integrated Note: While integrated modelling capacity exists it may not yet be modelling capacity mainstreamed in decision-making during epidemics and pandemics Limited local Single-discipline Existing local modelling to siloed integrated capacity single-discipline modelling local modelling capacity TYPE C TYPE B1 Emerging Increasing use • Modelling not commonly • Modelling not commonly used to inform decisions used to inform decisions • Limited modelling capacity • Existing integrated • No integrated modelling modelling capacity capacity Limited use of modelling to inform decision-making 76 Strengthening pandemic preparedness and response through integrated modelling Table A4.2.  Summary of short-term activities by country type to establish and maintain active communications (Initiative 2) Dimension: Increasing modelling use to inform Dimension: Increasing policy-making local modelling capacity Activity: Creating formal communication channels for before, during and after epidemics (see section 3.2.1) In B1 and A countries, policy-makers and those working with them can establish or strengthen formal channels with modelling groups to form a community of practice for networking, capacity-building and collaboration. In C and B2 countries, policy-makers and those working with them can establish communication channels with modelling groups from other countries who have that capacity. Activity: Communicating policy questions and how integrated modelling can help (see section 3.2.2) In C and B1 countries, policy-makers and those working NA with them can work with other countries or organizations that have experience in modelling for policy-making to formulate relevant policy questions and scenarios. Activity: Facilitating interactions at all stages of all new model development (see section 3.2.3) In all countries, modelling groups should communicate actively and regularly with government units responsible for modelling activities before, during and after epidemics. Activity: Creating active networks of policy-makers and modelling groups (see section 3.2.4) In C and B2 countries, policy-makers and those who NA support them can establish procedures on how to interact with integrated modelling groups they work with from other countries, or collaborate with countries and organizations to establish procedures on how to engage and interact with integrated modelling groups from other countries. A, B1 and B2 countries can establish procedures on how to interact with integrated modelling groups within their countries. All countries can establish or join in-country or regional or international networks Activity: Increasing integrated modelling literacy and communication across society (see section 3.2.5) All countries can map out what approaches and systems to improve communication and increase literacy could look like in their own contexts, and identify current gaps, opportunities and obstacles. Countries with established approaches and systems can share their experiences and processes with other countries. NA = not applicable Notes: In type A countries, there is capacity in integrated modelling and modelling is commonly used in policy-making for epidemics, but integrated modelling may not be widely used yet. In type B1 Annex 4  A framework to contextualize the four initiatives proposed to enhance the production and use of integrated modelling77 countries, there is capacity in integrated modelling and the use of modelling in policy-making is limited or increasing. In type B2 countries, there is limited or developing capacity in integrated modelling and modelling is used to inform policy-making. In type C countries, the use of modelling for decision-making is developing and local integrated modelling capacity is yet to be established or is emerging. Table A4.3.  Summary of short-term activities by country type to match the appropriate integrated model with the right policy question (Initiative 3) Dimension: Increasing modelling use to Dimension: Increasing local inform policy-making modelling capacity Activity: Building base-modelling structures before epidemics and pandemics or during inter-epidemic or pandemic times (see section 3.3.1) All countries can identify potential policy questions and identify what modelling structures and associated capabilities are needed and where gaps exist. C and B2 countries can identify countries or organizations to help them establish plans to have access to modelling structures and capabilities, depending on policy questions. B1 and A countries can share policy questions and associated modelling structures (e.g. repositories) with other countries and can identify and address gaps in existing modelling structures and capabilities. Activity: Building capabilities to determine how to match the appropriate integrated models with the right policy questions (see section 3.3.2) In B2 and A countries, a checklist of In B1 and A countries, modelling teams basic criteria for an integrated model can develop how-to guides about the to be useful for policy-making could integration of epidemiological and be developed and shared with other macroeconomic models (modelling countries. techniques, data sources etc.). Activity: Ensuring integrated modelling can adapt to policy priorities during epidemics and pandemics (see section 3.3.3) In C and B2 countries, government units B1 and A countries can refine their can identify countries or organizations integrated model, share their model that do have such capacity or that can structure, help build infrastructure that help them establish connections to have facilitates such sharing, and participate access to integrated models. in capacity-building or technical support initiatives to adapt, calibrate and validate existing models for their own context. Activity: Creating procedures and systems for comparative integrated modelling (see section 3.3.4) In B1 and A countries, government NA units and modelling groups can create procedures and systems for comparative modelling. NA = not applicable Notes: In type A countries, there is capacity in integrated modelling and modelling is commonly used in policy-making for epidemics, but integrated modelling may not be widely used yet. In type B1 countries, there is capacity in integrated modelling and the use of modelling in policy-making is 78 Strengthening pandemic preparedness and response through integrated modelling limited or increasing. In type B2 countries, there is limited or developing capacity in integrated modelling and modelling is used to inform policy-making. In type C countries, the use of modelling for decision-making is developing and local integrated modelling capacity is yet to be established or is emerging. Table A4.4.  Summary of short-term activities by country type to build local integrated modelling production capacity (Initiative 4) Dimension: Increasing local modelling capacity Activity: Establishing networks that facilitate cooperation and collaboration between integrated modelling groups (see section 3.4.1) In B1 and A countries, modelling groups can further expand networks to include more in-country and outside-country integrated modelling groups to gain broader and more diverse insights. In C and B2 countries, academic institutions can establish plans on how to build or join such networks. Activity: Incorporating integrated modelling and real world decision-making into educational programmes (see section 3.4.2) In C and B2 countries, modellers and those supporting policy-makers with some modelling expertise can connect with educational programmes in other countries that are able to incorporate integrated modelling. In B1 and A countries, integrated modelling groups can organize learning courses for modellers or those interested in learning about it across all countries. They may also offer studentships to participants who may not have the financial capacity to attend such courses. Activity: Creating new interdisciplinary modelling positions along with appropriate incentives to fill those positions (see section 3.4.3) In B1 and A countries, modelling groups with positions on integrated modelling can share relevant documentation with other countries; for example, through the networks (see section 3.2). For those countries that may not have the resources, a short-term goal can be to write out the job descriptions for such positions, the plans to establish such positions, and the plans to procure the resources to support such positions. Notes: In type A countries, there is capacity in integrated modelling and modelling is commonly used in policy-making for epidemics, but integrated modelling may not be widely used yet. In type B1 countries, there is capacity in integrated modelling and the use of modelling in policy-making is limited or increasing. In type B2 countries, there is limited or developing capacity in integrated modelling and modelling is used to inform policy-making. In type C countries, the use of modelling for decision-making is developing and local integrated modelling capacity is yet to be established or is emerging. Annex 4  A framework to contextualize the four initiatives proposed to enhance the production and use of integrated modelling79 Annex 5 Ten key principles to help effective communication about modelling Table A5.1.  Ten key principles to help effective communication about modelling Key principle Strategy to implement more effective communication about models Use common When communicating about the model and its results, avoid using language as much discipline-specific jargon. Such jargon can inhibit communication as possible and lead to misunderstanding because a technical term can mean different things in different fields. Moreover, people can rely on jargon when they do not truly comprehend something. Understand the To make a model useful for decision support, the modeller must perspectives of understand and account for the perspective of the decision-maker the policy-makers and what incentives and constraints he or she has, and generate and the timing of results in a timely, often urgent, manner. their decisions Make use Data visualizations such as maps, graphs, infographics and of effective other figures can make model structures and results easier to visualizations comprehend and patterns, trends, relationships, thresholds and outliers easier to identify. Visualizations help present complexity in a more easily digestible manner. New visualization approaches may be needed for more effective communications depending on the model and context. Clearly provide It is important to clearly state all key strengths, assumptions and the background, limitations up front. Avoid making conclusions that go beyond what strengths, the model can show. Give indications of how strong the results may assumptions and be. Stay away from making definitive statements. Instead, properly limitations behind communicate the uncertainty and variability of any results and each set of results specify whether findings are statistically significant. Modellers including potential must make clear the implications of each of the limitations and uncertainty and assumptions. variability continued 80 Strengthening pandemic preparedness and response through integrated modelling Table A5.1.  continued Key principle Strategy to implement more effective communication about models Clearly provide No policies or interventions are without their risks, uncertainties and explain the and potential negative consequences. Selecting which to use trade-offs for is about balancing the trade-offs between the positives and all policies and negatives of each individually and in various combinations. One of interventions the major values of integrated epidemiological–macroeconomic models is showing these relative trade-offs, including the ones between benefits and costs. Modellers should clearly communicate these trade-offs and show how they may change with changing assumptions and circumstances. This includes using measures of these trade-offs that are relevant to decision-makers and the population, and clearly defining, explaining and socializing these measures. This also includes clearly communicating how different interventions/products/policies are represented, as this may impact the trade-offs and results. Tailor your One size fits all does not apply to communicating a model’s design communications and results. Instead, tailor communications to suit the audience and approaches setting. Consider the amount of time available, the breadth of the and methods audience, and the communications platform. appropriately to the audiences and settings Provide Documentation should provide adequate details of the model documentation of structure, inputs and validation. Written documentation is all claims preferable to verbal documentation. It should be written in understandable language, preferably undergo some type of peer-review, and be published by a reputable source. It should allow others to recreate the scenarios, and apply them to the specific set of model results. Anticipate how Be proactive and anticipate how model results may be results and misinterpreted or misused, rather than being reactive after findings may be misinterpretation or misuse. This includes considering who might misinterpreted or use the results and for what purposes. Misinterpretation or misuse miscommunicated may be accidental or deliberate. and be proactive about preventing this Continued active As described in section 2, ideally, modelling should be an iterative engagement process. Modelling interactions work better as an ongoing dialogue and iteration is between modellers and decision-makers. This helps “socialize” the preferable model and its results with decision-makers and the public. continued Annex 5  Ten key principles to help effective communication about modelling 81 Table A5.1.  continued Key principle Strategy to implement more effective communication about models Communicate in There is tremendous value in having many models address a given a collaborative question. Thus, there must be ways to encourage the use of multiple rather than a models. This includes openly communicating to decision-makers competitive the need for multiple models, clarifying that decisions will be made manner from multiple models, and iterating the value and advantages that this brings. The emphasis should be on cooperation rather than competition. Such cooperation can entail identifying and finding approaches and mechanisms for modellers to communicate with each other. 82 Strengthening pandemic preparedness and response through integrated modelling Department of Health Financing and Economics World Health Organization 20, Avenue Appia 1211 Geneva 27 Switzerland Email: whochoice@who.int Website: https://www.who.int Directorate for Employment, Labour and Social Affairs Organisation for Economic Co-operation and Development 2, rue André Pascal 75016 Paris France Email: health.contact@oecd.org Website: https://www.oecd.org/health DOI: https://doi.org/10.1787/5f046115-en International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 United States of America Website: www.worldbank.org