June 2022 Technical Paper COUNTRY-LEVEL SEASONAL THREAT PROFILES: Operationalizing Seasonal Forecasts into Decision-Relevant Risk Metrics © 2022 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522- 2625; e-mail: pubrights@worldbank.org. Acknowledgements This note was prepared by the International Research Institute for Climate and Society (IRI), Earth Institute at Columbia University and World Bank’s Global Crisis Risk Platform (GCRP). The note was written by Agathe Bucherie (Staff Associate Researcher) and Andrew Kruczkiewicz (Senior Researcher) at Columbia University and Lindsey Jones (Senior Operations Officer) and Bianca Adam (Senior Operations Officer) at the World Bank. The team wishes to thank Luc Bonnafous (Urban, Disaster Risk, Resilience, and Land) for invaluable comments and suggestions on earlier drafts. Thanks also to Indira Konjhodzic for inputs and guidance as well as Arno Boersma and Susie Youngyun Lee (all GCRP) for support with layout and editing. Funding for this research came from the GCRP’s Multidimensional Crisis Risk Assessment and Monitoring Approaches program as an input to the GCRP Compound Risk Monitor. Contacts: Lindsey Jones (ljones12@worldbank.org), Andrew Kruczkiewicz (andrewk@iri.columbia.edu) 2 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Executive Summary Seasonal forecasts play an important role in alerting development and humanitarian actors to the potential for hydrometeorological threats to impact on their operations. When precipitation and temperature is significantly above or below average over a season, individual and compound hazards can be triggered leading to considerable implications for organizations like the World Bank and other development partners. The appropriate use of seasonal forecasts can therefore strengthen anticipatory action and promote crisis preparedness (Troccoli et al. 2008, Tall et al. 2012). This includes opportunities to feed seasonal risk information into activities related to a range of sectors including disaster risk management, disaster risk financing, social protection, anticipatory action, climate change adaptation as well as infrastructure design and maintenance. Despite these advantages, seasonal forecasts are rarely used in support of humanitarian and development activities. One reason for this shortfall is that existing forecast information is rarely tailored to the operational environments of national and regional decision makers. In particular, while seasonal forecasts can provide detailed outlooks of the geographic distribution of upcoming threats – often displayed as high resolution (~25km) grid cells – their use in addressing most development and humanitarian needs is limited without careful consideration of the overlaps between seasonal climate and wider socio-economic, political and demographic drivers of risk. Translating raw seasonal climate forecasts into country-level threat profiles also presents notable challenges, both methodologically and operationally. This technical note details methodological steps to analyze seasonal precipitation anomaly forecasts, and integrate outcomes into emerging risk information at country levels. Three prototypes are detailed, each with increasing levels of complexity and input data. Prototype 1.0 presents a country level threat profile based only on precipitation anomalies from IRI seasonal forecast output, based on precipitation forecasts anomalies, or deviations from ‘climate normal’ conditions, over the next three months. Prototype 1.1 builds off 1.0, with integration of persistence in dry conditions by combining observed precipitation anomalies over recent months with forecast information. Finally, Prototype 1.2 includes criteria related to both population exposure and land use to enhance Prototype 1.1. 3 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Workflow: A prototype for country-level seasonal risk profiles For each prototype we present a justification for the methodology, including a rationale for both the input data used and definition of thresholds. Acknowledging the influence of method selection and threshold definition has on outputs, our methodology systematically assesses sensitivity of these critical variables by noting how Prototype outputs change over time, using 12 consecutive forecasts from April 2020 to March 2021. We also assess spatial distribution of Prototype 1.2 outcomes, analyzing the averaged proportion of flagged countries per specific region, and per income status. As part of the Global Crisis Risk Platform’s work on compound risk, we present a prototype classification system for showcasing country-level seasonal risk information that is tailored to national decision- making environments. We outline the methodological steps taken to combine seasonal precipitation anomaly forecasts with information on exposure and sensitivity to seasonal hazards. Finally, we showcase an approach to communicating high-level seasonal risk information at country levels using a simple flagging system with the intention to be presented to decision makers. This work allows for progress to be made towards developing standards for governance structures by which seasonal forecasts can be incorporated into global, regional and national scale ex-ante decision making for periods of months and seasons. 4 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Table of Contents Introduction 6 Prototype 1.0 – Country level threat profile based on seasonal forecast precipitation anomalies 8 1- Seasonal forecasts for precipitation 8 2- Defining country borders 9 3- Defining critical thresholds for probability of above/below normal 9 4- Defining country proportion thresholds 10 5- Prototype 1.0 results and challenges 11 Prototype 1.1 – Enhancement with dry conditions persistence data 14 1- Calculation and meaning of the persistence of dry/wet conditions 14 2- Integrating persistence within the prototype 15 3- Prototype 1.1 results and challenges 17 Prototype 1.2 – Integration of the exposure dimension 19 1- World population density 19 2- Cropland and pasture coverage 20 3- A filter for low exposure areas 20 4- Prototype 1.2 results and challenges 21 Results and Sensitivity Analysis 24 1- Sensitivity analysis on critical probability thresholds. 24 2- Sensitivity analysis on critical thresholds of country proportion 25 3- Results of analysis per country classification of interest 28 Discussion and recommendation for future work 32 1- Additional statistical analyses of protype results, over longer historical ranges 32 2- Integrating uncertainty related to seasonal forecast skill 33 3- Validation assessment with historical data 34 4- Towards a risk scale flagging system 35 Conclusion 37 References 39 5 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Introduction Delivery of timely and actionable risk information is key to supporting anticipatory action and crisis preparedness as part of development and humanitarian operations. Information related to weather and climate are especially relevant given the considerable potential for adverse impacts on economies and societies. Seasonal extremes related to precipitation and temperature can trigger individual and compound hazards with considerable implications for organizations like the World Bank and other development partners. Seasonal forecasts are of particular interest given their ability to inform decisions, annually or semi- annually recurrent, around allocation of resources for future periods of varying length (Tadesse et al. 2016). The appropriate use of seasonal forecasts can therefore strengthen anticipatory action and promote crisis preparedness (Troccoli et al. 2008, Tall et al. 2012). This includes opportunities to feed seasonal risk information into activities related to a range of sectors including disaster risk management, disaster risk financing, social protection, anticipatory action, climate change adaptation as well as infrastructure design and maintenance. Seasonal timescales are also unique in bridging the decision support needs of both humanitarian and development sectors—with outlooks ranging from 2 to 6 months. Despite these opportunities, seasonal forecasts are rarely used in support of humanitarian and development activities. One reason for this shortfall is that existing forecast information is rarely tailored to the operational environments of national and regional decision makers. In particular, while seasonal forecasts can provide detailed outlooks of the geographic distribution of upcoming threats – often displayed as high resolution (~25km) grid cells – their use in addressing most development and humanitarian needs is limited without careful consideration of the overlaps between seasonal climate and wider socio-economic, political and demographic drivers of risk. Translating raw seasonal climate forecasts into country-level threat profiles also presents notable challenges, both methodologically and operationally. In seeking to address this shortfall, this technical note details a prototype for classifying country-level seasonal risk information that is tailored to national decision-making environments. It outlines the methodological steps taken to combine seasonal forecasts of precipitation anomalies with exposure data. Finally, an approach is showcased to communicate high-level seasonal risk information at country level using a flagging system that can be communicated with decision makers. In acknowledgment of the various use cases of seasonal forecasts within the development and humanitarian sector, we do not outline a methodology here that implies a single ‘best practice’ approach. Rather, the only element that can be considered guidance for ‘best practice’ is a prerequisite reflection on key elements related to decisions and actions to be potentially influenced by integration of the seasonal forecasts such as target period, spatial scale and tolerance to uncertainty (Taylor et al. 2015, Kruczkiewicz et al. 2021). Presenting options by which users can select from, rather than prescribing a single approach, is important irrespective of the timescale of climate data of interest, but is of particular value for seasonal forecasts given the implicit trade-offs related to skill and uncertainty, spatial granularity and temporal range (Cash et al. 2006, Kumar 2010). While there are indeed far more than three use cases, 6 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles the intention of this work is to highlight examples that could be further iterated upon, rather than prescribe a set number of approaches that must be selected from. To demonstrate opportunities for using seasonal forecasts to inform development and humanitarian decision making, three different prototypes are detailed, including the rationale, assumptions, results and challenges. Prototype 1.0 uses seasonal forecasts for precipitation anomalies to produce country level threat profiles. Prototype 1.1 is an enhancement from 1.0, with integration of persistence in wet/dry conditions over recent months. Finally, Prototype 1.2 builds off 1.1, including exposure of population and land use data. For each prototype we present the input data used, the rationale for the specific methodology and defined thresholds. We report on prototype outputs being i) the total number of countries flagged as emerging threat; ii) the number of countries flagged as emergent threat for each dry and wet indicator; iii) the proportion of flagged countries per country size group. Acknowledging the influence of method selection and threshold definition has on outputs, our methodology systematically assesses sensitivity of these critical variables by noting how Prototype outputs change over time, using 12 consecutive forecasts from April 2020 to March 2021. Finally, we assess spatial distribution of Prototype 1.2 outcomes, analyzing the averaged proportion of flagged countries per specific region, and per income status. 7 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Prototype 1.0 – Country level threat profile based on seasonal forecast precipitation anomalies The term Seasonal Forecast is indicative of a forecast describing the conditions of a noted variable over a set of months in the future. the next full 3-month target period (Mason et al. 2015). It is common for seasonal forecasts to be presented over 3-month target periods, with precipitation being one of the most likely variables. However, seasonal forecasts are also developed for a variety of target periods, including on 2,3- and 4-month lead times, including, but not limited to a 3-month target period. Prototype 1.0 uses the IRI Multi-Model Probability Forecast for Precipitation to flag countries based on emerging risks driven by conditions likely to deviate from normal. 1- Seasonal forecasts for precipitation Each month, IRI issues a probabilistic forecast for precipitation anomalies the next 3 months, produced at 1-degree latitude-longitude resolution (Mason et al. 1999, Barnston et al. 2003, Kirtman et al., 2014). The product is based on a re-calibration of model output from the NOAA’s North American Multi-Model Ensemble Project (NMME). Probability forecasts are produced from each individual NMME ensemble mean precipitation model using extended logistic regression (Wilks, 2009), and averaged together with equal weight to create a multi-model forecast probability (Vigaud et al., 2017). The climatological base period used is 1982-2010, using hindcast ensemble-means and observed tercile-category occurrences of precipitation from the CPC-CMAP-URD dataset. Figure 1 shows an example of such a forecast, issued in January 2021, for the following February-March- April (FMA) period. It indicates for each pixel a probability of above, below or normal precipitation conditions, however it is important to note that the color of each pixel only indicates the probability for the dominant category. For example, if the pixel of interest is dark blue (such as in central and northern areas of The Philippines in Fig. 1) or green (such as in southern Vietnam) it can be gleaned that the most likely category is above normal. However, we cannot infer the specific value of the other categories (normal and below normal) for that same pixel. That said, if there is a 70% probability for above average conditions, there could be a 15 and 5 % probability of normal and below average, respectively, or there could be a 10 and 10 % probability, respectively. This map, and most if not all representations of seasonal forecast output, do not indicate a more granular breakdown of the non-dominant categories in a single map in attempt to facilitate simplified communication of risk of the dominant category Historical precipitation tercile seasonal forecasts are downloadable on the IRI Data Library platform in various formats, including in Geotiff, for each forecast issued monthly since February 2017 (Blumenthal et al. 2014). Link to data: https://iridl.ldeo.columbia.edu/maproom/Global/Forecasts/NMME_Seasonal_Forecasts/Precipitation_ELR.html 8 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 1. IRI Seasonal Forecast for Precipitation Anomalies for February-March-April 2021 (Issued January 2021) 2- Defining country borders The reference for world country boundary divisions is the TM WORLD BORDERS 0.3 shapefile, provided by Bjorn Sandvik, thematicmapping.org. This was used to denote the spatial units for all further raster zonal statistics. As an example, we extracted the total number of raster “pixels” of the forecast product present within each country boundary. To do so we used a rasterizing method including all pixels 1 touching country boundaries, ultimately leading to countries having an even pixel size with a minimum of 1. Including all cells along the country boundaries may introduce biased results in some cases, but overall, better represents the regional risk link to forecast anomalies above a specific country. 3- Defining critical thresholds for probability of above/below normal To use the IRI seasonal precipitation forecast product as an input for assessing risk at the country level, probability thresholds must be defined by which we will assign the number of pixels exceeding a “critical” signal. A key consideration in doing so is to identify the potential threats linked to the probability of the country to experience some abnormal dry or wet condition in an upcoming period of months in the future. 1 https://pythonhosted.org/rasterstats/manual.html#zonal-statistics 9 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles As we acknowledge the importance of and challenges in communicating risk and forecast uncertainty, we employ a strategy to develop a replicable methodology with outputs that are able to influence development sector decision making at the programmatic level. The current prototype is using only a single critical probability threshold (being the inverse for dry and wet indicators), however further investigation should be done to explore the use of different thresholds per season or climate regions, and/or to indicate risk of other and multiple hazards occurring either at the same time or in close succession The choice of a specific threshold was defined based on the analysis of different values on a set of historical forecast data (see chapter on Sensitivity, page 24). We concluded that a probability threshold of 50% was the most appropriate value for this use case, as it is perceived to capture sufficient signal, with an appropriate spatial distribution for a global analysis focusing on the country scale. We note that further analyses can be conducted to assess the degree to which this threshold may shift based on a specific decision-making context. 4- Defining country proportion thresholds Once the probability threshold is defined, the proportion of pixels below/above a threshold that must be achieved in order to appropriately flag a country as being at risk, both for the dry and the wet indicators independently, must be defined. While the use of a single proportion threshold for all countries seems the most straightforward from each a methodological, communication and policy context, we ultimately conclude that using a method scaled based on country size is likely to be more appropriate for the WBG and more broadly the use case for development banks making programmatic funding allocation decisions at the national level. Using a single country proportion threshold is not recommended due to the wide country size variation globally, gradients in climate classifications and differences in capacity to address risk (Jones and Preston 2011). A single country proportion will induce a strong bias toward small countries being flagged more often than relatively large and medium sized countries. In order to assign different thresholds depending on the country size, we categorized the countries into 5 classes based on their size, using a quantile classification method. We distributed the critical proportion threshold per class of country size from 100 % for small countries, such as Guinea-Bissau and Dominica (all pixels need to show a precipitation signal in the same direction for the country to be flagged) to only 10 % for large countries, such as the USA and Australia (see table 1). This method has the advantage of eliminating the non-normal skewed country size distribution, which would be more challenging to handle and communicate with the use of a pixel size versus country proportion threshold equation. 10 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles TABLE 1. Country size class definition and associated critical pixel proportion thresholds per class Class Quantiles Country pixel size Critical Country proportion threshold (%) 1 < Q20 1-3 100 2 Q20-Q40 4-9 80 3 Q40-Q60 10 - 24 60 4 Q60-Q80 25- 75 20 5 > Q80 ≥76 10 5- Prototype 1.0 results and challenges One example of the result of Prototype 1.0 is presented in Table 2 and Figure 2 using the seasonal precipitation forecast issues in January 2021 for the FMA period. The critical probability threshold of the precipitation to be above/below normal is fixed to 50%, and the critical country proportions required for the country to be flagged are class dependent as described in Table 1. This resulted in 30 countries being flagged with an acceptable distribution across country size classes, less biased toward small countries when using a single country proportion threshold (see Sensitivity Chapter, section 2). TABLE 2. Results for running Prototype 1.0 for the precipitation forecast issued January 2021 for FMA. This shows the number of flagged countries related to the wet and dry anomaly indicators, individually and combined. Wet anomaly Dry anomaly wet + dry anomaly Critical precipitation anomaly probability threshold >50 < - 50 < - 50 or > 50 Number of Countries 19 11 30 1 Country size ≤ 3 Pixels 3 3 6 2 Country size 4-9 Pixels 3 0 3 3 Country size 10-24 Pixels 3 2 5 4 Country size 25-75 Pixels 5 2 7 5 Country size ≥ 76 Pixels 5 4 9 Notes: Critical spatial country proportion thresholds 1: 100% , 2: 80% , 3: 60% , 4: 20% , 5:10% 11 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 2. Resulting number of flagged countries, using Prototype 1.0 applied to the precipitation forecast issued January 2021 for FMA. The critical precipitation probability threshold is set at 50%, with the critical country proportions dependent on country size classes. The methodology of Prototype 1.0 has been tested on several historical issued forecasts to better understand the variability of the results. It has been observed that over the last six months of issued seasonal forecasts for precipitation (from the forecast issued in September 2020 for October-November- December to the forecast issued in March 2021 for April-May-June), the number of final flagged countries would vary from 29 to 37, suggesting a fairly small variability during that period. However, one month lead-time forecasts issued from April to August 2020 results in a lower number of total flagged countries, with a substantially higher level of variability, ranging from 4 to 18. This could be the result of different spatial distribution of precipitation on a global scale during those months, as indicated by a higher number of small island countries in the Pacific being flagged. Interestingly, results consistently show a lower number of countries flagged as a result of forecast below average anomalies compared to above average. These observations suggest that more analysis should be conducted on the seasonal, year to year and decadal variability of the results based on a more extensive historical dataset and projections of future impacts. Further, while this method remains intentionally simplified, a more advanced approach could be developed to extract only clusters of adjacent pixels with critical signals. Alternatively, certain large countries could be disaggregated and analyzed at higher administrative resolution, to avoid large countries to be flagged as a result of sparse signals. 12 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 3. Number of wet, dry, and total flagged countries for forecast issued for 12 consecutive months (2020- 2021) using the Prototype 1.0 methodology. 13 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Prototype 1.1 – Enhancement with dry conditions persistence data While Prototype 1.0 considered seasonal precipitation forecasts, as a proxy to extract emergent risk related to anormal dry and wet conditions, in Prototype 1.1 observations of precipitation anomalies in recent months are integrated within the forecast. Adding these observations allows us to extract information on the degree to which conditions may likely continue to be drier or wetter in an area of interest. The resulting persistence in wet and dry conditions is considered a risk factor related to the development of weather-related disasters, especially for drought and more complex hazard contexts involving multiple and compound risks. 1- Calculation and meaning of the persistence of dry/wet conditions To identify where and when conditions currently are and will continue to be anomalously wet and dry, the IRI developed a suite of derived products and tools titled, Forecasts in Context. These interactive mapping tools allow for computing the intersection of 1. where and to what extent locations experienced the past 3 months of anomalously wet and dry conditions AND where and to what extent locations will continue to see the same conditions persist over the next 3 months. The observation data for precipitation anomalies for the most recent 3 month period are calculated based on NOAA Climate prediction center monthly CAMS-OPI data, gridded at 2.5° lat/lon resolution, and compared to 30 years climatology for the same 3-month period (NCEP). Observed precipitation anomalies are then ranked from 0 to 1 (the most dry to the most wet values, respectively); values below a threshold of 0.333 are considered dry (below-normal), and above 0.667 are considered wet (above-normal). The observation data are then coupled with the dominant tercile probabilities for seasonal (3-month) precipitation for the first lead time of the IRI forecast, issued every month at 2.5° lat/lon resolution, and the same that is used in Prototype 1.0. To produce the persistence map presented in Figure 4, the forecast probability has been grouped in two classes, the "Enhanced probability" class corresponds to forecast probability ranging from 40-50%, while the "Greatly Enhanced" class corresponds to a forecast probability of 50% and higher. As a result, Figure 4 is indicating where precipitation forecast for the next 3-month season shows an enhanced likelihood of above-normal (below-normal) precipitation AND where precipitation received in the 3-month season before the forecast was issued was also above normal (below normal), i.e. the forecast has the same tendency, in terms of precipitation, as was observed in the most recent season. Below is the link to download data from the IRI Data Library. http://iridl.ldeo.columbia.edu/maproom/IFRC/FIC/pic3mo_same.html?taxa=iridl%3ADataset_Search&it emClass=iridl%3Adataset&sem=iridl%3Aforecast&sem=iridl%3APrecipitation&sem=iridl%3AProbability& sem=iridl%3AGlobal&sem=iridl%3AIRI 14 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 4. IRI Forecasts in Context tool. This Indicates the probability of persistence in dry (wet) conditions for the next 3 months over areas that have experienced dry (wet) conditions in the previous 3 months. The example here is Issued January 2021 for the period FMA. Produced by IRI with NOAA data, and freely available online. 2- Integrating persistence within the prototype Not all below-normal precipitation periods result in drought conditions. However, continuous abnormally dry conditions, especially when associated with high evaporation rates and occurring in areas with little or no irrigation infrastructure, pose a substantially increased risk for developing drought conditions potentially leading to more severe consequences on food security and water scarcity (Moreland, 1993). Indeed, disasters triggered by precipitation amounts lower than normal (such as droughts) have a slower onset than disasters triggered by precipitation excess (such as floods), which justified a prioritization to include the persistence of dry conditions in prototype 1.1 as an indicator for increased risk for drought development. Considering the continuity in wet conditions, even though it could be suggested that an area presenting abnormal wet conditions for consecutive months is at increased flood risk, there is no research proving this hypothesis globally. We therefore did not include this dimension for the prototype 1.1, as the intention here is to be more operational than research oriented. In addition, prototype 1.0 revealed that fewer countries were flagged due to anomalous dry conditions than wet conditions, suggesting that prototype 1.1 could act to calibrate this analysis. 15 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles In prototype 1.1, we include the persistence of dry conditions into the indicators established for critical levels of dryness. In prototype 1.0, only pixels with a probability higher than 50% for drier than normal conditions based on the seasonal forecast for precipitation over the next 3 months were selected. In 1.1, in addition to the method in 1.0, we propose to include pixels with a lower probability [40-50%] of drier than normal that intersect with pixels showing past observed dry conditions over the most recent 3-month period. To do so, the IRI persistence of dry conditions methodology is used. The yellow category of Figure 4 is added within the new prototype 1.1, summarized in Figure 5. In Figure 5, the red and blue colors indicate signals where risk of seasonal precipitation is forecast to be more likely below and above normal (respectively, in the coming months), with probability at or above 50% for those outcomes to occur. These are the resulting signals from prototype 1.0 applied to the forecast issued in January 2021 for the FMA period. The Orange color indicates signal where both observations of precipitation over the previous three-month period have been below normal and the seasonal forecast for precipitation indicates a 40-50% probability of below normal. In prototype 1.1, both red and orange signals are defined as the ‘dry indicator’. For an example of the differences in 1.1 compared to 1.0, in January 2021 dry signal was shown in western and southern Kazakhstan (Fig. 5), potentially indicative of an emerging threat related to drier than normal conditions in the area, and therefore leading to Kazakhstan to be flagged. Kazakhstan was not flagged as a country of concern in prototype 1.0. FIGURE 5. Global distribution of critical signals defined for Prototype 1.1, based on both observed most recent 3- month precipitation was below normal and seasonal forecast for precipitation for the next 3 months below (above normal). This map is produced based on the seasonal forecast issued January 2021 for the period FMA, and most recent past 3-month (OND) precipitation observations 16 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles 3- Prototype 1.1 results and challenges The output data from prototype 1.1, using the seasonal precipitation forecast of January 2021 for the FMA period combined with the precipitation observations of the most recent season, October-November- December (OND) 2021, are presented in the map of Figure 6 and Table 3. Compared to prototype 1.0 results (Figure 2 and Table 2) which uses only seasonal forecast information, prototype 1.1 results in the flagging of 9 additional countries presenting potential threats linked to drier than normal signals, including Kazakhstan as an example. The analysis of the results of prototype 1.1 ran over a 12 month period (Figure 7) reveals an even greater number of countries flagged as being at risk for impacts related to above and below average precipitation anomalies, compared to prototype 1.0. FIGURE 6. Output from Prototype 1.1 developed in January 2021, indicating flagged countries, based on precipitation forecast (FMA) and persistence in precipitation conditions (from OND). While preliminary analysis notes the lack of a significant relationship between seasonal forecasts indicating above average precipitation, and persistence of those conditions, with ‘positive extreme’ hydrometeorological events such as floods, we note that the lack of historical flood data that is disaggregated by flood type may be a significant factor. Further work should be done to assess the importance of persistence of wet conditions on various types of floods, as well as on compound risks that involve floods and non-flood hazards and events that involve multiple types of floods. This work should consider the non-uniform spatial and temporal relationships between where extreme precipitation occurs and where flooding occurs. For example, riverine flooding in a location can be driven by prolonged intense rainfall that occurs far away from the location. More specifically, the river flood may occur multiple pixels away from where the rainfall which caused the flood occurs. This distance should be considered in both observed rainfall as well as in. 17 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles TABLE 3. Results of Prototype 1 showing the number of flagged countries related to the wet and dry anomaly indicators individually and combined, applied to the seasonal forecast for precipitation issued January 2021 for FMA. Wet anomaly Dry anomaly wet + dry anomaly Critical precipitation anomaly probability < -50 >50 wet or dry threshold [-40 to -50] + past dry Number of Countries 19 20 39 1 Country size ≤ 3 Pixels 3 3 6 2 Country size 4-9 Pixels 3 0 3 3 Country size 10-24 Pixels 3 2 5 4 Country size 25-75 Pixels 5 5 10 5 Country size ≥ 76 Pixels 5 10 15 Note: Critical spatial country proportion thresholds 1: 100% , 2: 80% , 3: 60% , 4: 20% , 5:10% FIGURE 7. Number of wet, dry, and total flagged countries for 12 issued forecast months (2020-2021) using the prototype 1.1 methodology. 18 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Prototype 1.2 – Integration of the exposure dimension The exposure of people and assets to climate related hazards is an important part of the risk assessment process. While prototype 1.0 and 1.1 considered the potential threatened areas related to the hazard dimension only, for prototype 1.2 we integrate socioeconomic factors. Given the intent to develop a globally applicable process, two variables that are considered to be generalizable, population density as well as cropland and pasture coverage, are used to represent exposure. 1- World population density We used the CIESIN Gpw 2020 population density data as a static reference layer to represent population exposure (CIESIN, 2018). The population density gridded data, available at 30 arc-second resolution, are resampled to fit the 1-degree resolution gridded IRI seasonal precipitation forecast dataset. As a result, each "pixel" for the IRI seasonal forecast probabilities is now associated with a population density in inhabitants per square kilometer. FIGURE 8. CIESIN Gpw 2020 global population density data resampled at the 1-degree resolution of IRI seasonal forecast products. Link to data: https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-density-rev11 19 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles 2- Cropland and pasture coverage We used the Earthstat Cropland and Pasture Area coverage, available for the year 2000 at 5 min (10km) resolution, to characterize the static agriculture land exposure globally (Ramankutty et al., 2008). As with the population density data, the cropland and pasture coverage data are resampled to equal the 1-degree resolution of IRI seasonal forecasts. The resulting map, showing the spatial proportion of combined pasture and crop production (from 0 to 1) of each "pixel”, is presented in Figure 9. FIGURE 9. Earthstat cropland and pasture coverage proportion resampled at the 1-degree resolution of IRI seasonal forecasts. Link to data: http://www.earthstat.org/cropland-pasture-area-2000/ 3- A filter for low exposure areas To acknowledge that not all areas of the globe are equally exposed to threats, we designed a method to filter out both low population areas and/or areas without considerable agricultural economic activity. In doing so, we are aiming to prevent very low exposure areas to be highlighted in the emerging threat flagging system. To do so, low exposure areas are defined and excluded from the Prototype 1.2 analysis. We defined low exposure areas as any pixels having both a population density of less than 25 inhabitants per square km and an agricultural activity (pasture or cropland) fraction lower than 35%. These thresholds are defined based on a sensitivity analysis to capture most of the uninhabited and non-farmed lands of 20 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles the globe. Following this definition, the mapped low exposure areas are represented in grey in Figure 10, and account for about 46% of Earth’s surface (excluding the Greenland and the Antarctic continent). FIGURE 10. Low exposure areas shown in grey, corresponding to less than 25 inhabitants per square km and an agricultural (pasture or crops) fraction lower than 35%. (Antarctica and Greenland are masked). 4- Prototype 1.2 results and challenges The number of relevant pixels per country, now filtered out from the low exposure areas, differs from the previous prototypes. As a consequence, the five classes of country size, defined by equal numbers of countries per class, results in different outcomes. In addition, the critical spatial country proportion thresholds have been adapted accordingly in order to achieve a homogeneous distribution of flagged countries per size class. The results from the sensitivity analysis conducted on prototype 1.2 using different thresholds of critical proportion per country size class are detailed in the next section. The best- case scenario and output for Prototype 1.2 is presented in Table 4. Sensitivity analysis reveals that varying the thresholds that define low exposure areas does not lead to significant differences in outcomes in terms of countries flagged In addition, the final Prototype 1.2 output data are presented in the composite maps of Figure 11 (A, B, C and D). These represent the seasonal forecast for precipitation, the critical signal extracted, low exposure areas, as well as the countries flagged based on the example derived from the seasonal forecast issued 21 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles January 2021 for the FMA period. Further, we acknowledge that the definition of low exposure could be improved by integrating other types of indicators of socioeconomic activity, such as mining locations, dam sites, airports and/or other critical infrastructure layers. In addition, the total number of pixels that make up each country, which is used as the denominator in calculating the proportion of critical signals per country, is reduced by the number of pixels defined as ‘low exposure’ per country. TABLE 4. Output of Prototype 1.2 for the forecast issued January 2021 for the FMA period showing the number of flagged countries related to the wet and dry anomaly indicators both individually and combined. Wet anomaly Dry anomaly wet + dry anomaly Critical precipitation anomaly probability threshold >50 < - 50 < - 50 or > 50 Number of Countries 21 15 36 1 Country size ≤ 2 Pixels 5 3 8 2 Country size 3-6 Pixels 5 0 5 3 Country size 7-18 Pixels 5 2 7 4 Country size 19-48 Pixels 2 4 6 5 Country size ≥ 49 Pixels 4 6 10 Note: Critical spatial country proportion thresholds 1: 100% , 2: 66.6% , 3: 50% , 4: 33.3% , 5:33.3% 22 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 11. Composite maps showing the intermediary steps, from precipitation forecast to flagged country results using the Prototype 1.2 method, based on data available in January 2021 for seasonal forecasts for the FMA period and observations from OND. A) IRI Multi-Model Probability Forecast for Precipitation for February-March-April 2021, Issued January 2021. B) Critical signals extracted from both seasonal forecasts for precipitation and persistence in dry conditions, C) Low exposure areas. D) Resulting map flagging countries with emerging threats. A) B) C) D) 23 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Results and Sensitivity Analysis This section presents results across each prototype for both analyses conducted to explore sensitivity of outputs based on changes in critical thresholds and an analysis of outputs of each over the period of 12 consecutive months of issued forecasts. While a wide range of sensitivities have been performed on thresholds defining critical observed precipitation anomaly, or low exposure areas, it is observed that the variables having the most influence on the outcomes are 1) the critical thresholds defined for probabilities within the seasonal forecasts of precipitation; and 2) the critical thresholds defined for country spatial proportion which indicate if that country is to be flagged. Finally, we explore these results by analyzing the average proportion of countries flagged per specific region of the world within the period April 2020 to March 2021. 1- Sensitivity analysis on critical probability thresholds. Sensitivities on critical probability thresholds of precipitation forecast have been conducted over a 12 month period of consecutively issued forecasts. The sensitivities were conducted to test the impact of using three critical probability thresholds - 40%, 50% and 60% - on the number of countries flagged for emerging threat. The sensitivity analysis was performed initially on Prototype 1.0. The three sensitivity results presented in Figures 12, 13 and 14 reveal that the use of a probability threshold for above/below normal precipitation of 50% provides the most ideal range of resulting flagged countries (from 4 to 37) given the intention of the current research application. While using a threshold of 60% results in too few, a threshold of 40% would lead to months with more than a hundred of countries flagged, leading to subsequent challenges in selecting which to prioritize and deprioritize FIGURE 12. Number of total, wet and dry flagged countries for 12 issued forecast months (2020-2021) using a probability threshold for above/below normal precipitation of 40%. 24 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 13. Number of total, wet and dry flagged countries for 12 issued forecast months (2020-2021) using a probability threshold for above/below normal precipitation of 50%. FIGURE 14. Number of total, wet and dry flagged countries for 12 issued forecast months (2020-2021) using a probability threshold for above/below normal precipitation of 60%. 2- Sensitivity analysis on critical thresholds of country proportion Even small changes in the thresholds defined as the critical proportion of critical pixels to total pixels will lead to significant variation in the output of the Prototypes. Here we show the influence of using different country proportion thresholds and methods on the number of flagged countries (relative to the total, wet and dry anomaly) for the same series of 12 consecutively issued seasonal forecasts for precipitation. In addition, we analyzed the sensitivity of the critical country proportion thresholds on the distribution of 25 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles flagged countries number per class of country size. The classes are based on the 5 equal categories of country size defined in Table 4, from small (class 1) to large (class 5) countries. The sensitivity results are presented below for each of the 12 consecutive monthly precipitation forecasts, illustrating 4 cases using a fixed proportion threshold for all classes at 50% (case A) as well as different proportion thresholds per class (cases B, C and D). The 4 cases produce the same range of total flagged countries, however results differ in terms of proportions of flagged countries per country size class. FIGURE 15. Sensitivity analysis results for the choice of different country proportion thresholds and methods on the total number of flagged countries (relative to the total, wet and dry anomaly), as well as on the average proportion of countries flagged in each size class. Case A uses a fixed proportion threshold for all classes at 50%, while cases B, C and D use different proportion thresholds per class. Results are presented for the same series of 12 consecutive monthly precipitation forecasts. A) Fixed proportion threshold for all classes at 50 % B) Different proportion threshold per class - 1: 100% , 2: 80% , 3: 60% , 4: 40% , 5:20% 26 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles C) Different proportion threshold per class -1: 100% , 2: 60% , 3: 50% , 4: 40% , 5:40% D) Different proportion threshold per class -1: 100% , 2: 66.6% , 3: 50% , 4: 33.3% , 5:33.3% 27 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 16. Results showing the shifts in averaged proportion of flagged countries per country class, generated for the 12 consecutive months and based on the case disaggregation noted above in figure 15. N indicates the average number of flagged countries for each case A, B,C and D. A -Fixed proportion threshold for all classes at 50 % B -Different proportion threshold per class - 1: 100% , 2: 80% , 3: 60% , 4: 40% , 5: 20% C -Different proportion threshold per class - 1: 100% , 2: 60% , 3: 50% , 4: 40% , 5: 40% D -Different proportion threshold per class - 1:100% , 2: 66.6% , 3: 50% , 4: 33.3% , 5: 33.3% 3- Results of analysis per country classification of interest In this section, we explore the outcome of Prototype 1.2 by analyzing the averaged proportion of flagged countries per specific region of the world, applied to the 12 consecutively issued seasonal forecasts for the period April 2020 to March 2021. Hereafter we use the three following country classifications presented in Figure 17: 28 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles • 23 subregions of the worlds as defined by the UN Geoscheme2 • 8 regions of the world as defined by World Bank • 4 income categories as defined by World Bank Results of the averaged proportion of countries flagged per subregion, region and income country classification over the 12-month period are presented in Figure 18. It is noteworthy to highlight relatively lower values of proportion of flagged countries in Sub-Saharan Africa and Europe during the period April 2020 to March 2021. However, it should be noted that the forecasts are issued during a La Niña period, which will lead to some degree of bias in describing the results during this limited temporal range. In addition to potential biases related to La Niña conditions. The lower proportion of flagged countries in Sub-Saharan Africa is also related to the known lower levels of seasonal forecast skill to predict precipitation in this region during the dry season, which results in little or no forecast signal in most of the Sub-Saharan region. This suggests that, as it is not possible to predict conditions drier than zero, there might always be a bias towards a lower number of countries flagged as emergent threats in these, and other, extremely dry regions. As a result, as most of the countries within the low-income class are Sub- Saharan countries, we observe a lower proportion of flagged countries in the low-income class compared to the other classes. 2 UNSD M49 classification Methodology. https://unstats.un.org/unsd/methodology/m49/ 29 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 17. World subregions, regions and income class country classifications used for results analysis. 30 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 18. Averaged proportion of countries flagged per subregion, region and income country classification over 12 successive months, applying Prototype 1.2 method. Labelled numbers indicate the total number of countries in each category. 31 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Discussion and recommendation for future work Defining the timescales of both the decision-making context and the available climate information is a critical step to understand if and to what extent they are an appropriate match. That said, assessing ‘appropriateness’ of timescales is not necessarily the same as matching timescales. For example, Seasonal forecasts can inform decision making occurring on a seasonal cycle/timeline (such as decisions related to agriculture activities such as those that may be captured within a cropping calendar and actions related to pre-positioning of disaster preparedness items), but they can also inform shorter and longer term decisions, such as weekly and monthly decisions related to activities regarding inspection of infrastructure and in planning field visits for agriculture extension officers. However, it is also important to highlight that some ‘mis-matched’ timescales may not be appropriate to align. For example, longer term climate change information (such as projections for sea level rise in 2100) should not be used to inform shorter term decision making over weeks and months, with doing so potentially poses risk for unintended consequences. This is especially true for projections that 1. Will likely change over time (in terms of initialization date of the forecast) in the magnitude, potentially sign, for the same target period (2100, for example) and 2. Have fluctuations in magnitude and sign from the current period through to the target period (for example, temperature will likely rise by 2C in 2100, but will cool by -.1C during a decade in the near future, but before 2100). While the work presented here presents options for understanding how and to what extent seasonal forecasts can benefit the development sector, we also highlight various challenges associated with ‘off the shelf’ selection of forecast models and hydro meteorological indices. More exploration could be done in the future to testing the potential value of integrating other drought indices (SPI or sc-PDSI) in prototype 1.1 to look at potential drought persistence, as skill of indices is likely to vary across regions and climatic zones (Delgado et al. 2018, Sutanto et al. 2020). Even with the various approaches presented here, we candidly note both the shortcomings and opportunities of first assessing appropriateness of seasonal forecast use and second, integration of seasonal forecast data into decision making, program development and monitoring and evaluation within the development sector. To enhance this work, we suggest the following steps to be taken. 1- Additional statistical analyses of protype results, over longer historical ranges Here we performed a sensitivity analysis only for 12 consecutive months, however assessments should be conducted over a longer period to allow for new and perhaps different insights. With this preliminary analysis, we do have an estimation of the range of number of flagged countries to expect, as well as potential seasonal and spatial trends observed in various outcomes across the Prototypes. However, we highlight the importance of performing more in depth and robust sensitivity analysis of these results, analysing the statistics of results over a period of at least several years, and across various timescales and geographic areas. Such analyses are key to understanding how our outcome can vary over periods of years and decades, allowing for the opportunity to explore how decadal variability (changes in climate patterns that persist for periods of years to decades) and interannual variability (year to year changes in climate) can influence predictability and outputs of our methods. 32 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles For an example of the latter, ENSO phases, such as El Niño and La Niña, should be controlled for as predictability within seasonal climate forecasts will change when they are present (Balmaseda et al. 1995). Fortunately, El Niño and La Niña generally lead to greater seasonal forecast skill, however the degree to which skill increases will vary from place to place, and further it is unclear if and to what extent any level of increased skill will propagate within some or all Prototypes. Calculating results at this level of granularity may also increase the confidence in our results, and give insights on the most appropriate ways to use such derived products. In addition to potential interannual and decadal shifts in skill for the seasonal forecast, and thus the Prototype methods, it is also important to consider seasonal fluctuations. The Spring Predictability Barrier is a term used to describe the relatively lower level of skill present in seasonal forecasts issued in approximately March-June for target periods of approximately April-August. One of the primary reasons for the barrier is the weaker coupling in oceanic-atmospheric systems, due to lower sea surface temperature gradients in the eastern equatorial Pacific Ocean. The (boreal) spring is also a period of transition of ENSO, whereby a shift away from/towards an El Niño and La Niña is more likely to occur. Further work can include exploration of the influence of the spring predictability barrier on Prototype output, which can lead to a better understanding of where and when some Prototypes should be prioritized over others, and if we should not use some of the Prototypes when forecasts are issued in particular months of the year. 2- Integrating uncertainty related to seasonal forecast skill Care must be taken with using seasonal forecast products for decision making, in order to ensure both the benefits and constraints presented within the scope of a seasonal climate forecast are understood. Indeed, seasonal precipitation forecast skill is not homogeneously distributed in time and space. For instance, seasonal forecasts for precipitation have generally higher skill in Southeast Asia than in East Asia. In addition, in Southeast Asia the seasonal forecast skill is higher for the Nov-Dec-Jan season than for the May-Jun-Jul season. Confidence related to the seasonal forecast results based on seasonal forecast skill should therefore be considered for integration into the future Prototypes. IRI precipitation forecast outputs are compared to observed precipitation (NOAA/Climate Prediction Center CAMS/OPI satellite data) to generate skill scores based on the period from 1997 to present (https://iri.columbia.edu/our-expertise/climate/forecasts/verification/). For instance, the ROC (relative operating characteristics) score indicates the degree of correct probabilistic discrimination between different categories (for instance "above normal" or "below normal" precipitation categories). Forecast skill results can be generated spatially, for all or individual seasons, at a 2.5-degree resolution. Figure 19 illustrates an example of a ROC skill score map for above-normal precipitation forecast, averaged over the period of a year. Uncertainty associated with seasonal climate forecasts that should be better communicated. That said, developing multi-disciplinary partnerships such as those that include climate scientists and policy makers, as well as translators and boundary organizations, increases the chance of effective use of forecasts to drive effective and sustainable seasonal-forecast informed decision making. 33 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles FIGURE 19. ROC skill scores for above-normal precipitation forecast, over all seasons at the 5-month lead time. Gray indicates no (or negative) skill, Purple indicates highest skill. 3- Validation assessment with historical data A country threat profile based on seasonal forecasts is not meant to be a tool to predict the occurrence of specific disaster events such as a flood or drought, and using it as such could introduce unnecessary, and unmanageable, levels of uncertainty. Therefore, the validation of our outputs with historical disaster data is key. However, assessing where and when Prototype 1.2 outputs correspond with historical disaster impacts would be valuable to understand, and better communicate, the limitations and uncertainty within these methods. For instance, as a first step, we suggest collecting EMDAT historical monthly drought data, complemented with data from WB historical operational data, for all countries and regions of interest. This should be done for at least the period used in this report (April 2020 to March 2021). The occurrence of drought events per 3 months could be generated for all countries, and correlated with the Prototype outputs in the form of flagged countries. Sensitivity analyses could then be recalculated referencing the historical data, which could potentially aid in the adjustment of critical thresholds noted in chapter 5. Further, as one overarching goal of this use case for seasonal forecasts is to understand where and to what extent the Prototypes should be prioritized for different months of the year, evaluating the hits and misses using historical data will allow for increased understanding of how it could have been used in the past to prioritize allocation of resources before impact from a threat was realized. 34 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles 4- Towards a risk scale flagging system Our current approach uses a simple binary flagging system for the identification of emerging threats, implying some countries (those flagged) are at or above a level of heightened risk compared to others. While this supports the considerations on the elements to integrate in the design of country threat prototypes, we believe that moving toward a risk scale flagging system would be beneficial for decision making within the development community, including the WB, resulting in outcomes easier to communicate with disaster risk management communities (Lin et al., 2017). The appropriateness of risk scales has been studied, however there is not a clear answer that works for every decision-making context. For the use case addressed here, there are various options for risk scales. Given the spatial scale of the decision making and the necessity to not ignore the existence of uncertainty (even if the exact value is unknown), we suggest the exploration of using a risk scale that is either qualitative or semi-quantitative. For the former, the categories can be designed as low, medium and high. Doing so would allow for a comparative analysis across spatial units of interest, with results being presented in a risk matrix format (one of the most common types of semi-quantitative risk representation). From a disaster risk management perspective, consequence and likelihood/uncertainty are two of the more common dimensions that are used to design a semi-quantitative risk matrix that could inform prioritization for ex- ante disaster risk management (Oldfield, 2012). 5- Recommendations for future policy-oriented research. As the urgency to take action to mitigate disaster risk grows, increasingly there are calls for using climate data that is open and available. However, while climate data availability and accessibility rapidly increases, similar scales of growth in integration of the data within structured approaches for decision making have lagged behind. This paper presents challenges and opportunities to integrate available and accessible climate data into a specific use case, highlighting necessary, and frequently ignored or deprioritized, elements of uncertainty, lead time and spatial scale. While this exercise has highlighted opportunities for future policy consideration, the implications vary. The following set of recommendations is intended not only to inform opportunities to advance the science around seasonal forecast development and applications of the outputs, but rather it is based on the needs of the development and humanitarian community to each; identify forecast datasets that could potentially be useful, understand that these prognostic datasets are inherently uncertain yet still can and should inform policy on timescales both including and longer than, seasonal; and lastly that there are multiple knock-on benefits for strengthening policy in these areas - some that far surpass the socio economic arguments for improved, more targeted allocations of funds, and speak to the importance of using climate data, including climate forecast data, to ensure the lives and livelihoods of the most vulnerable populations are addressed as well as the industries that support them both directly and indirectly. 35 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles • Assess compound event risk that is both present in our current climate and how they are likely to evolve as the climate changes. Methods should be developed that are sufficiently generalizable to be reproduced in other areas that potentially have a different suite of hazard type risk. • Build capacity for integrating seasonal climate information within national level decision making related to prioritization and deprioritization of resources, including but not limited to within the Ministry of Finance • Identify options for prioritization and allocation of resources, particularly those that are relatively high impact at lower levels of investment. • Promote a process whereby ministries of finance/central banks could monitor shifts in risk, for other countries, and allow for structures to explore indirect socioeconomic impacts on their internal activities and external interests/investments. This is needed as many developing countries are not able to lead the design of a trans-border system to assess indirect financial impact on their development and security. • Allow for development programs and activities to contribute to adaptation strategies and sustainable development goals. • Promote a ‘data-driven’ approach to de-prioritize resource allocation, potentially leading to more clarity in why certain areas each received pre-disaster support over others, why this support occurred at different quantities, and why it occurred at specific points in time. • Define where within the enabling environments for seasonal forecast use social justice implications must be included. This should include direct and indirect impacts and focus on how to decrease the disproportionate impacts faced by traditionally underserved and underrepresented communities in support of progress towards environmental justice. • Facilitate cross-border cooperation and crisis management, allowing for innovative approaches to address cross and trans-border risks on a regional scale. • Advocate for a review and potential revision of regulations around transparency of how and to what extent climate data is integrated into decision making. Doing so would allow for this work to exist within a framework, so future iterations can evolve in a systematic way, as climate changes and new risks emerge. The value of this data for consumers would also increase as increased transparency will foster trust and lead to a higher likelihood for matched expectations of what the seasonal forecasts can and cannot be used for. 36 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Conclusion Prior to this activity, there were significant challenges around using seasonal forecasts in a structured approach, related to the development of methodology, policy, and financial mechanism. With this research, we contribute to address some of the technological challenges associated with seasonal forecasts. These advancements are centered around geospatial, climate and decision-making aspects of using seasonal forecasts. Here we outline methods to understand if and to what extent national level decision making can be informed by seasonal forecasts. Particularly, we provide guidance on the best methodology to translate precipitation forecast anomalies into a list of countries with emergent threats. The sensitivity analysis results provide confidence in the choice of methods used, as well as how to define critical thresholds to obtain an adequate number of flagged countries and distribution in time and space. While additional work will allow for an enhancement of this work, our approach shows promising results, in particular moving towards standards in spatial scale considerations and definition of critical thresholds. In the context of global analysis focusing on country scale, we found that using a single probability threshold of 50% below/above normal precipitation to be the best indicator of ‘critical signal’ status. However, the proportion of critical signals needed to flag a country is the most sensitive variable to define and should vary with country size. Another notable result is the tendency for the number of flagged countries associated with drier than normal conditions to be lower than the number of flagged countries associated with wet signals, likely related to limits of the precipitation data, bonded to zero but not towards high extreme values. We therefore highlight in this note the need to integrate dry condition persistence when identifying critical signals. In terms of seasonal fluctuations in the value of our approach, while only evaluated over a small temporal range of 1 year, more intense critical signals did occur from October to June, resulting in a higher number of flagged countries during this period. This can be at least partially explained by differences in spatial and temporal distribution of precipitation anomalies, and the uneven distribution of countries globally. Finally, the spatial analysis of our results confirms that, due to the larger number of small countries exposed to extreme weather in the Pacific, country threat profile tools will result in a higher proportion of flagged countries in East Asia and Pacific compared to other regions. On the contrary, seasonal precipitation forecast skill in most of the Sub-Saharan region during the dry season is relatively lower than other seasons, resulting in a bias toward less countries flagged. As a result, and as most of the countries within the low- income class are Sub-Saharan countries, we observe a lower proportion of flagged countries in the low income class compared to the other classes. These results suggest that integrating information on seasonal precipitation forecast skill, spatial coherence and variance across regions of interest is needed to understand if and to what extent prioritization, and de-prioritization, is justified based on this precipitation data alone. Our current approach is driven by a binary flagging system for the identification of emerging threats, to support prioritization at the national level. To move toward an operational risk scale flagging real-time system, beneficial for decision making within the World Bank, we recommend 3 pathways for future activities: 1. Quality and skill assessments, 2. Mapping of client’s services and mandates to identify how Prototype outputs could be integrated, and 3. Tool design and implementation within operational decision making standard operating procedures. While some elements of these 3 pathways can be 37 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles conducted in parallel, before working on the design and implementation of a real-time system, we recommend doing a proper evaluation of the final Prototype 2.1 using a validation assessment with historical data, as described in chapter 5 above. Assessing where and when, and to what extent, our results correlate with historical disaster impacts would be valuable to understand and better communicate the limitations and uncertainty around the use of our product, in order to ensure that decision maker perceptions are understood, and expectations are matched. This work allows for progress to be made towards developing standards for governance structures by which seasonal forecasts can be incorporated into global, regional and national scale ex-ante decision making for periods of months and seasons. Given the ability to apply this method to a variety of climatic hazards, the potential application for compound events exists, however more work must be done to understand the ability to do so. Also, while various datasets can be used, the processes to develop, review and revise seasonal forecast-driven early action should be standardized. Doing so will allow for new data on each of the climate, socio-economic and financial elements to sustainably improve the processes over time. The policy implications of this work indicate further investment is warranted, however next steps should be informed directly by a set of stakeholders, including central banks, to allow for concepts related to scale, cost-benefit and fiduciary responsibility to be central. These are critical themes in order for any forecast-based decision-making process to be sustainable beyond a short-term pilot project, and to ensure the direct and indirect benefits of such systems are properly assessed. 38 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles References Balmaseda, M.A., Davey, M.K. and Anderson, D.L., 1995. Decadal and seasonal dependence of ENSO prediction skill. Journal of Climate, 8(11), pp.2705-2715. Barnston, A.G., Mason, S.J., Goddard, L., DeWitt, D.G. and Zebiak, S.E.(2003). Multimodel ensembling in seasonal climate forecasting at IRI. Bulletin of the American Meteorological Society, 84(12), pp.1783-1796. Blumenthal, M.B., Bell, M., del Corral, J., Cousin, R. and Khomyakov, I., 2014. IRI Data Library: enhancing accessibility of climate knowledge. Earth Perspectives, 1(1), pp.1-12. Bruno Soares M., Daly, M., and Dessai, S. (2018). Assessing the value of seasonal climate forecasts for decision-making, Wiley Interdiscip. Rev. Clim. Chang., 9, 1–19, https://doi.org/10.1002/wcc.523 Cash, D.W., Borck, J.C. and Patt, A.G., 2006. Countering the loading-dock approach to linking science and decision making: comparative analysis of El Niño/Southern Oscillation (ENSO) forecasting systems. Science, technology, & human values, 31(4), pp.465-494. CIESIN (2018). Gridded Population of the World, Version 4 (GPWv4): Population Density, Revision 11. NASA Socioeconomic Data and Applications Center (SEDAC). https://doi.org/10.7927/H49C6VHW Delgado, J.M., Voss, S., Bürger, G., Vormoor, K., Murawski, A., Rodrigues Pereira, J.M., Martins, E., Vasconcelos Júnior, F. and Francke, T., 2018. Seasonal drought prediction for semiarid northeastern Brazil: verification of six hydro-meteorological forecast products. Hydrology and Earth System Sciences, 22(9), pp.5041-5056. Jones, R.N. and Preston, B.L., 2011. Adaptation and risk management. Wiley Interdisciplinary Reviews: Climate Change, 2(2), pp.296-308. Jones, L., Harvey, B., Cochrane, L. et al. Designing the next generation of climate adaptation research for development (2018). Reg Environ Change 18, 297–304. https://doi.org/10.1007/s10113-017-1254-x Kirtman, B. P., Min, D., Infanti, J. M., Kinter, J. L., Paolino, D. A., Zhang, Q., Van Den Dool, H., Saha, S., Mendez, M. P., Becker, E., Peng, P., Tripp, P., Huang, J., Dewitt, D. G., Tippett, M. K., Barnston, A. G., Li, S., Rosati, A., Schubert, S. D., Wood, E. F. (2014). The North American multimodel ensemble: Phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction. Bulletin of the American Meteorological Society, 95(4), 585–601. https://doi.org/10.1175/BAMS-D-12-00050.1 Kruczkiewicz, A., Sayeed, S., Hansen, J., Furlow, J., Rose, A. and Dinh, D. (2018). Review of Climate Services Governance Structures: Case Studies from Mali, Jamaica, and India. CCAFS Working Paper no. 236. Wageningen, Netherlands: CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS). Kruczkiewicz, A., Braun, M., McClain, S., Greatrex, H., Padilla, L., Hoffman-Hernandez, L., Siahaan, K., Nielsen, M., Llamanzares, B. and Flamig, Z., 2021. Flood Risk and Monitoring Data for Preparedness and Response: From Availability to Use. Global Drought and Flood: Observation, Modeling, and Prediction, pp.289-306. Kumar, A., 2010. On the assessment of the value of the seasonal forecast information. Meteorological Applications, 17(4), pp.385- 392. Kusunose, Y. and Mahmood, R. (2016): Imperfect forecasts and decision making in agriculture, Agric. Syst., 146, 103 –110, https://doi.org/https://doi.org/10.1016/j.agsy.2016.04.006 Lexin Lin, Claudia Rivera, Marcus Abrahamsson & Henrik Tehler (2017). Communicating risk in disaster risk management systems – experimental evidence of the perceived usefulness of risk descriptions, Journal of Risk Research, 20:12, 1534- 1553, DOI: 10.1080/13669877.2016.1179212 Mason, S.J., Goddard, L., Graham, N.E., Yulaeva, E., Sun, L. and Arkin, P.A. (1999). The IRI seasonal climate prediction system and the 1997/98 El Niño event. Bulletin of the American Meteorological Society, 80(9), pp.1853-1874. 39 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles Mason, S.J., Kruckiewicz, A., Ceccato, P. and Crawford, A. (2015). Accessing and using climate data and information in fragile, data-poor states. Moreland, J. A. (1993). Drought. In USGS Open-File Report. https://doi.org/10.3133/ofr93642 Oldfield, S. G (2012). Impacts of Climate Change on Urban Infrastructure & the Built Environment . Toolbox. An Introduction to Risk Assessment, n.d. Ramankutty, N., Evan, A. T., Monfreda, C., & Foley, J. A. (2008). Farming the planet: 1. Geographic distribution of global agricultural lands in the year 2000. Global Biogeochemical Cycles, 22(1), 1–19. https://doi.org/10.1029/2007GB002952 Sutanto, S.J., Weert, M.V.D., Blauhut, V. and Van Lanen, H.A., 2020. Skill of large-scale seasonal drought impact forecasts. Natural Hazards and Earth System Sciences, 20(6), pp.1595-1608. Tall, A., Mason, S. J., van Aalst, M., Suarez, P., Ait-Chellouche, Y., Diallo, A. A., and Braman, L. (2012). Using Seasonal Climate Forecasts to Guide Disaster Management: The Red Cross Experience during the 2008 West Africa Floods, 2012, 986016, https://doi.org/10.1155/2012/986016. Tadesse, T., Haigh, T., Wall, N., Shiferaw, A., Zaitchik, B., Beyene, S., Berhan, G., and Petr, J. , (2016). Linking Seasonal Predictions to Decision-Making and Disaster Management in the Greater Horn of Africa, 97, ES89 –ES92, https://doi.org/10.1175/BAMS-D-15-00269.1 Taylor, A.L., Dessai, S. and De Bruin, W.B., 2015. Communicating uncertainty in seasonal and interannual climate forecasts in Europe. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences , 373(2055), p.20140454. Troccoli, A., Harrison, M., Anderson, D.L. and Mason, S.J. eds. (2008). Seasonal climate: forecasting and managing risk (Vol. 82). Springer Science & Business Media. Vigaud, N., Robertson, A. W., & Tippett, M. K. (2017). Multimodel ensembling of subseasonal precipitation forecasts over North America. Monthly Weather Review, 145(10), 3913–3928. https://doi.org/10.1175/MWR-D-17-0092.1 Wilks, D. S. (2009). Extending logistic regression to provide full-probability-distribution MOS forecasts. Meteorological Applications, 16(3), 361–368. https://doi.org/https://doi.org/10.1002/met.134 40 Country-Level Seasonal Threat Profiles: Operationalizing Seasonal Forecasts into Decision-Relevant Threat Profiles