Publication: Machine Learning Imputation of High Frequency Price Surveys in Papua New Guinea
Loading...
Published
2023-09-28
ISSN
Date
2023-09-28
Author(s)
Editor(s)
Abstract
Capabilities to track fast-moving economic developments re-main limited in many regions of the developing world. This complicates prioritizing policies aimed at supporting vulnerable populations. To gain insight into the evolution of fluid events in a data scarce context, this paper explores the ability of recent machine-learning advances to produce continuous data in near-real-time by imputing multiple entries in ongoing surveys. The paper attempts to track inflation in fresh produce prices at the local market level in Papua New Guinea, relying only on incomplete and intermittent survey data. This application is made challenging by high intra-month price volatility, low cross-market price correlations, and weak price trends. The modeling approach uses chained equations to produce an ensemble prediction for multiple price quotes simultaneously. The paper runs cross-validation of the prediction strategy under different designs in terms of markets, foods, and time periods covered. The results show that when the survey is well-designed, imputations can achieve accuracy that is attractive when compared to costly–and logistically often infeasible–direct measurement. The methods have wider applicability and could help to fill crucial data gaps in data scarce regions such as the Pacific Islands, especially in conjunction with specifically designed continuous surveys.
Link to Data Set
Citation
“Andrée, Bo Pieter Johannes; Pape, Utz Johann; Andree, Bo, Pieter Johannes. 2023. Machine Learning Imputation of High Frequency Price Surveys in Papua New Guinea. Policy Research Working Papers; 10559. © World Bank. http://hdl.handle.net/10986/40410 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Machine Learning Guided Outlook of Global Food Insecurity Consistent with Macroeconomic Forecasts(World Bank, Washington, DC, 2022-10)Motivated by the deterioration in global food security conditions, this paper develops a parsimonious machine learning model to derive a multi-year outlook of global severe food insecurity from macro-economic projections. The objective is to provide forecasts that are internally consistent with wider economic assessments, allowing both food security policies and economic development policies to be informed by a cohesive set of expectations. The model is validated on holdout data that explicitly test the ability to forecast new data from history and extrapolate beyond observed intervals. It is then applied to the World Economic Outlook database of April 2022 to project the severely food insecure population across all 144 World Bank lending countries. The analysis estimates that the global severely food insecure population may remain above 1 billion through 2027 unless large-scale interventions are made. The paper also explores counterfactual scenarios, first to investigate additional risks in a downside economic scenario, and second, to investigate whether restoring macroeconomic targets is sufficient to revert food insecurity back to pre-pandemic levels. The paper concludes that the proposed model provides a robust and low-cost approach to maintain reliable long-term projections and produce scenario analyses that can be revised systematically and interpreted within the context of available economic outlooks.Publication Estimating Food Price Inflation from Partial Surveys(World Bank, Washington, DC, 2021-12)The traditional consumer price index is often produced at an aggregate level, using data from few, highly urbanized, areas. As such, it poorly describes price trends in rural or poverty-stricken areas, where large populations may reside in fragile situations. Traditional price data collection also follows a deliberate sampling and measurement process that is not well suited for monitoring during crisis situations, when price stability may deteriorate rapidly. To gain real-time insights beyond what can be formally measured by traditional methods, this paper develops a machine-learning approach for imputation of ongoing subnational price surveys. The aim is to monitor inflation at the market level, relying only on incomplete and intermittent survey data. The capabilities are highlighted using World Food Programme surveys in 25 fragile and conflict-affected countries where real-time monthly food price data are not publicly available from official sources. The results are made available as a data set that covers more than 1200 markets and 43 food types. The local statistics provide a new granular view on important inflation events, including the World Food Price Crisis of 2007–08 and the surge in global inflation following the 2020 pandemic. The paper finds that imputations often achieve accuracy similar to direct measurement of prices. The estimates may provide new opportunities to investigate local price dynamics in markets where prices are sensitive to localized shocks and traditional data are not available.Publication Climate Shocks and Their Effects on Food Security, Prices, and Agricultural Wages in Afghanistan(Washington, DC: World Bank, 2024-12-17)This study examines the effects of climate and weather shocks on Afghanistan's agricultural economy, with an emphasis on food security, prices, and wages. By utilizing a dynamical model and a unique data set that includes monthly global and local food prices, agricultural wages, unofficial exchange rates, and local climate data, the research provides econometric estimates of the impacts of droughts and floods. The findings reveal that both flooding and drought significantly increase food insecurity, directly and indirectly. Directly, these climatic shocks are linked to heightened risks of food insecurity in the following months, even when controlling for price and wage fluctuations. Indirectly, droughts and floods drive up food prices and depress agricultural wages, further exacerbating food insecurity. The study suggests that enhancing climate resilience in the agriculture sector could mitigate these risks, stabilize local food prices and wages, and strengthen food security and the broader agricultural economy. The results also show that price data effectively capture food security shocks from various non-economic sources, and can serve as a versatile monitoring tool in situations where detailed data on food security are unavailable.Publication Are International Food Price Spikes the Source of Egypt's High Inflation?(World Bank, Washington, DC, 2012-08)This paper examines whether domestic inflation spikes in Egypt during 2001-2011 were primarily the result of external food price shocks. To estimate the pass-through of international food price inflation to domestic price inflation, two different methodologies are used: a two-step regression model estimates the pass-through in the long run, and a vector autoregression model provides the short-run estimates. The empirical evidence confirms that pass-through is high in the short term, but not in the long run. More precisely, the results show that (i) long-run pass-through to domestic food inflation is relatively low, lying between 13 and 16 percent, while the long-term spill-over from domestic food inflation to core inflation is moderate, lying around 60 percent; (ii) in the short term, pass-through is relatively high, estimated around 29 percent after 6 months and around two-thirds after a year, but the spill-over effect to core inflation is limited; (iii) international food price shocks explain only a small portion of domestic inflation shocks in both the short and long terms; and (iv) international price inflation has asymmetric effects on domestic prices.Publication 2023 Food security monitoring in Papua New Guinea - Insights from high frequency phone surveys(Washington, DC: World Bank, 2024-05-06)The objective of the Pacific Observatory is to improve welfare for the poor and vulnerable in Papua New Guinea and the Pacific Island Countries through expanding socio-economic information for better data-driven policymaking.
Users also downloaded
Showing related downloaded files
Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.Publication Business Ready 2024(Washington, DC: World Bank, 2024-10-03)Business Ready (B-READY) is a new World Bank Group corporate flagship report that evaluates the business and investment climate worldwide. It replaces and improves upon the Doing Business project. B-READY provides a comprehensive data set and description of the factors that strengthen the private sector, not only by advancing the interests of individual firms but also by elevating the interests of workers, consumers, potential new enterprises, and the natural environment. This 2024 report introduces a new analytical framework that benchmarks economies based on three pillars: Regulatory Framework, Public Services, and Operational Efficiency. The analysis centers on 10 topics essential for private sector development that correspond to various stages of the life cycle of a firm. The report also offers insights into three cross-cutting themes that are relevant for modern economies: digital adoption, environmental sustainability, and gender. B-READY draws on a robust data collection process that includes specially tailored expert questionnaires and firm-level surveys. The 2024 report, which covers 50 economies, serves as the first in a series that will expand in geographical coverage and refine its methodology over time, supporting reform advocacy, policy guidance, and further analysis and research.Publication Using Immunization Coverage Rates for Monitoring Health Sector Performance : Measurement and Interpretation Issues(World Bank, Washington, DC, 2000-08)Immunization against childhood diseases such as diphtheria, pertussis, tetanus, polio and measles is one of the most important means of preventing childhood morbidity and mortality. Despite the low cost of basic childhood immunizations, nearly 3 million children still die each year from vaccine-preventable diseases. Achieving and maintaining high levels of immunization coverage must therefore be a priority for all health systems. In order to monitor progress in achieving this objective, immunization coverage data can serve as an indicator of a health system's capacity to deliver essential services to the most vulnerable members of a population. This note discusses the use of trends in immunization coverage data, and argues that immunization is a health output with a strong impact on child morbidity, child mortality and permanent disability. This note discusses measurement and interpretation issues for coverage data collected through surveys and administrative records.Publication The Container Port Performance Index 2023(Washington, DC: World Bank, 2024-07-18)The Container Port Performance Index (CPPI) measures the time container ships spend in port, making it an important point of reference for stakeholders in the global economy. These stakeholders include port authorities and operators, national governments, supranational organizations, development agencies, and other public and private players in trade and logistics. The index highlights where vessel time in container ports could be improved. Streamlining these processes would benefit all parties involved, including shipping lines, national governments, and consumers. This fourth edition of the CPPI relies on data from 405 container ports with at least 24 container ship port calls in the calendar year 2023. As in earlier editions of the CPPI, the ranking employs two different methodological approaches: an administrative (technical) approach and a statistical approach (using matrix factorization). Combining these two approaches ensures that the overall ranking of container ports reflects actual port performance as closely as possible while also being statistically robust. The CPPI methodology assesses the sequential steps of a container ship port call. ‘Total port hours’ refers to the total time elapsed from the moment a ship arrives at the port until the vessel leaves the berth after completing its cargo operations. The CPPI uses time as an indicator because time is very important to shipping lines, ports, and the entire logistics chain. However, time, as captured by the CPPI, is not the only way to measure port efficiency, so it does not tell the entire story of a port’s performance. Factors that can influence the time vessels spend in ports can be location-specific and under the port’s control (endogenous) or external and beyond the control of the port (exogenous). The CPPI measures time spent in container ports, strictly based on quantitative data only, which do not reveal the underlying factors or root causes of extended port times. A detailed port-specific diagnostic would be required to assess the contribution of underlying factors to the time a vessel spends in port. A very low ranking or a significant change in ranking may warrant special attention, for which the World Bank generally recommends a detailed diagnostic.Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.