Publication: Assessing Forecast Uncertainty: An Information Bayesian Approach
Loading...
Published
2017-08
ISSN
Date
2017-08-24
Author(s)
Editor(s)
Abstract
Regardless of the field, forecasts are widely used and yet assessments of the embedded uncertainty—the magnitude of the downside and upside risks of the prediction itself—are often missing. Particularly in policy-making and investment, accounting for these risks around baseline predictions is of outstanding importance for making better and more informed decisions. This paper introduces a procedure to assess risks associated with a random phenomenon. The methodology assigns probability distributions to baseline-projections of an economic or social random variable—for example gross domestic product growth, inflation, population growth, poverty headcount, and so forth—combining ex-post and ex-ante market information. The generated asymmetric density forecasts use information derived from surveys on expectations and implied statistics of predictive models. The methodology also decomposes the variance and skewness of the predictive distribution accounting for the shares of selected risk factors. The procedure relies on a Bayesian information-theoretical approach, which allows the inclusion of judgment and forecaster expertise. For reliability purposes and transparency, the paper also evaluates the constructed density forecasts assigning a score. The continuous ranked probability score is used to assess the prediction accuracy of elicited density forecasts. The selected score incentivizes the forecaster to provide its true and best predictive distribution. An empirical application to forecast world gross domestic product growth is used to test the Bayesian entropy methodology. Predictive variance and skewness of world gross domestic product growth are associated with ex-ante information of four risk factors: term spreads, absolute deviations of headline inflation targets, energy prices, and the Standard and Poor's 500 index prices. The Bayesian entropy technique is benchmarked with naïve-generated density forecasts that utilize information from historical forecast errors. The results show that the Bayesian density forecasts outperform the naïve-generated benchmark predictions, illustrating the value added of the introduced methodology.
Link to Data Set
Citation
“Mendez-Ramos, Fabian. 2017. Assessing Forecast Uncertainty: An Information Bayesian Approach. Policy Research Working Paper;No. 8165. © World Bank. http://hdl.handle.net/10986/27972 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Variance and Skewness in Density Predictions(World Bank, Washington, DC, 2017-08)The paper introduces a Bayesian cross-entropy forecast (BCEF) procedure to assess the variance and skewness in density forecasting. The methodology decomposes the variance and skewness of the predictive distribution accounting for the shares of selected risk factors. The method assigns probability distributions to baseline-projections of an economic or social random variable—for example, gross domestic product growth, inflation, population growth, poverty headcount, among others—combining ex-post and ex-ante market information. The generated asymmetric density forecasts use information derived from surveys on expectations and implied statistics of predictive models. The BCEF procedure is applied to produce world GDP growth forecasts for three-year horizons using information spanning the period of October 2005–August 2015. The scores indicate that the BCEF density forecasts are more accurate and reliable than some naïve—symmetric and normal distributed confidence interval—predictions, illustrating the value-added of the introduced methodology.Publication Uncertainty in Ex-Ante Poverty and Income Distribution(World Bank, Washington, DC, 2019-05)This paper studies future poverty, inequality, and shared prosperity outcomes using a panel data set with 150 countries over 1980-2014. The findings suggest that global extreme poverty will decrease in absolute and relative terms in the period 2015-2030. However, absolute poverty is likely to increase by 2030 in resource-output oriented countries and economies with low rates of output per capita growth. Countries with high growth rates of output are expected to achieve poverty levels below 3 percent by 2030. Global and country aggregations show a decrease in income inequality by 2030; though, significant downside risks could increase wealth inequality in high- and low-output growth economies by 2030. Substantial uncertainty, as measured by the variability of the simulated outcomes, exists on shared prosperity gaps across the studied country typologies.Publication Estimating Parameters and Structural Change in CGE Models Using a Bayesian Cross-Entropy Estimation Approach(World Bank Group, Washington, DC, 2015-01)This paper uses a three-step Bayesian cross-entropy estimation approach in an environment of noisy and scarce data to estimate behavioral parameters for a computable general equilibrium model. The estimation also measures how labor-augmenting productivity and other structural parameters in the model may have shifted over time to contribute to the generation of historically observed changes in the economic arrangement. In this approach, the parameters in a computable general equilibrium model are treated as fixed but unobserved, represented as prior mean values with prior error mass functions. Estimation of the parameters involves using an information-theoretic Bayesian approach to exploit additional information in the form of new data from a series of social accounting matrices, which are assumed were measured with error. The estimation procedure is "efficient" in the sense that it uses all available information and makes no assumptions about unavailable information. As illustration, the methodology is applied to estimate the parameters of a computable general equilibrium model using alternative data sets for the Republic of Korea and Sub-Saharan Africa.Publication Estimation of Normal Mixtures in a Nested Error Model with an Application to Small Area Estimation of Poverty and Inequality(World Bank Group, Washington, DC, 2014-07)This paper proposes a method for estimating distribution functions that are associated with the nested errors in linear mixed models. The estimator incorporates Empirical Bayes prediction while making minimal assumptions about the shape of the error distributions. The application presented in this paper is the small area estimation of poverty and inequality, although this denotes by no means the only application. Monte-Carlo simulations show that estimates of poverty and inequality can be severely biased when the non-normality of the errors is ignored. The bias can be as high as 2 to 3 percent on a poverty rate of 20 to 30 percent. Most of this bias is resolved when using the proposed estimator. The approach is applicable to both survey-to-census and survey-to-survey prediction.Publication Instrumental Variables Regressions with Honestly Uncertain Exclusion Restrictions(World Bank, Washington, DC, 2008-05)The validity of instrumental variables (IV) regression models depends crucially on fundamentally untestable exclusion restrictions. Typically exclusion restrictions are assumed to hold exactly in the relevant population, yet in many empirical applications there are reasonable prior grounds to doubt their literal truth. In this paper I show how to incorporate prior uncertainty about the validity of the exclusion restriction into linear IV models, and explore the consequences for inference. In particular I provide a mapping from prior uncertainty about the exclusion restriction into increased uncertainty about parameters of interest. Moderate prior uncertainty about exclusion restrictions can lead to a substantial loss of precision in estimates of structural parameters. This loss of precision is relatively more important in situations where IV estimates appear to be more precise, for example in larger samples or with stronger instruments. The author illustrates these points using several prominent recent empirical papers that use linear IV models.
Users also downloaded
Showing related downloaded files
Publication Statistical Performance Indicators and Index(World Bank, Washington, DC, 2021-03)The World Bank’s Statistical Capacity Index has been widely employed to measure country statistical capacity since its inception two decades ago. This paper builds on the existing advantages of the Statistical Capacity Index, conceptually and empirically, to offer new statistical performance indicators and the Statistical Performance Index, which can better measure a country’s statistical performance. The new index has clearer conceptual motivations, employs a stronger mathematical foundation, and significantly expands the number of indicators and countries covered. The paper further provides empirical evidence that illustrates the strong correlation of the new index with other commonly used development indicators of human capital, governance, poverty, and inequality. The framework can accommodate future directions to improve the index as the global data landscape evolvesPublication Evolution of the World Bank’s Thinking on Governance(World Bank, Washington, DC, 2016-01)The preparation work for the 2017 World Development Report on Governance and the Law in 2016 will coincide with the 25th anniversary of the World Bank’s decision to broaden its ongoing work on public sector management to embrace key issues of governance in the Bank’s borrowing countries. This paper provides a broad overview of the major Bank reports on governance that went through a review process at a sufficiently high level in the institution that they can reasonably be described as reflecting the Bank’s considered views at the time on the subject. The objective is to review the evolution of the Bank’s thinking on governance and assess the relevance and effectiveness of the work and its implications for the forthcoming World Development Report. Section two of this paper begins with a brief account of how the Bank came to focus on issues of governance, reviewing the major upheaval of governance in many of the Bank’s borrowing countries in the 1980s, the legal constraints the Articles of Agreement impose on the Bank’s work on governance, and a brief overview of the Bank’s initial policy statement on governance issued to the Bank’s Board in June 1991. Section three reviews major Bank work on governance as reflected in successive World Development Reports and examines the Bank’s analysis of the issue of corruption, reviewing how the Bank’s thinking on this symptom of poor governance has evolved. Section four steps back to assess what the Bank got right and some of the issues it missed or failed to address adequately. Section five draws attention to the dramatic changes experienced by the developing world in these past 25 years, and points to the need to better understand the implications of these changes for the governance context facing developing countries.Publication Governance in Sub-Saharan Africa in the 21st Century(Washington, DC: World Bank, 2024-03-04)What can be learned from the governance trajectory of African countries since the beginning of the 21st century What is the quality of governance on the African continent and how does it shape development The first decade of the millennium saw promising growth and poverty reduction in much of the continent. Yet, Sub-Saharan Africa has also been the stage of a stream of governance reform failures and policy reversals, and many countries continue to suffer from the consequences of poor governance. This paper explores the dynamics of governance reform on the continent over the past two decades and points to four key trends. First, effective state institutions, capable of maintaining peace, fostering growth, and delivering services, have developed unevenly. Second, progress has been made on enhancing the inclusiveness and accountability of institutions, but it remains constrained by the weakness of checks and balances and the persistence of patterns of centralized and exclusive power arrangements. Third, civic capacity has risen considerably, but the inability of institutions to respond to social expectations and political mobilization threatens to turn liberal civic engagement into distrust, populism, and radicalization. Fourth, the combination of these three trends contributes to the rise of political instability, which constitutes a major threat for the continent.Publication Data Transparency and Growth in Developing Economies during and after the Global Financial Crisis(World Bank, Washington, DC, 2020-12)The study explores the effects of data transparency on economic growth for developing economies over a unique time period—at the onset of the 2007–2009 global financial crisis and thereafter. Data transparency is defined as the timely production of credible statistics as measured by the Statistical Capacity Indicator. The paper finds that data transparency has a positive effect on real gross domestic product per capita during a period of considerable uncertainty. The estimates indicate an elasticity of the magnitude of 0.03 percent per year, which is much larger than the elasticity of trade openness and schooling in the estimation sample. The empirics employ a variety of econometric estimators, including dynamic panel and cross-sectional instrumental variables estimators, with the latter approach yielding a higher estimated elasticity. The findings are robust to the inclusion of several factors in addition to political institutions and exogenous commodity-price and external debt-financing shocks.Publication Economic Governance Improvements and Sovereign Financing Costs in Developing Countries(World Bank, Washington, DC, 2021-05)Low- and middle-income country governments are increasingly tapping the global debt capital markets. This is increasing the amount of finance available for development, but at a considerably higher cost than traditional external borrowing on concessional terms. Using a novel methodology based on estimating sovereign credit ratings using the Moody’s scorecard, and examining the associations between these ratings and the World Bank’s Country Policy and Institutional Assessment scores, this paper examines how making improvements in the quality of economic policies and institutions can help lower governments’ financing costs. This method aims to overcome the small-sample problem due to the number of rated developing country sovereigns still being relatively limited (although growing). Better economic governance Country Policy and Institutional Assessment scores are associated with better estimated ratings and materially lower financing costs; on average, improvements that are sufficient to increase the Country Policy and Institutional Assessment economic governance indicator score by one point are associated with interest costs that are lower by about 40 basis points, even setting aside the direct impact on ratings of better governance indicators. There are many reasons why improving governance is a good thing. Among them is the potential payoff to the public purse — savings of $40 million or more on a standard $1 billion, 10-year bond.