Publication: Comparative Analysis of AI-Predicted and Crowdsourced Food Prices in an Economically Volatile Region
Loading...
Published
2024-04-23
ISSN
Date
2024-04-23
Editor(s)
Abstract
High-frequency monitoring of food commodity prices is important for assessing and responding to shocks, especially in fragile contexts where timely and targeted interventions for food security are critical. However, national price surveys are typically limited in temporal and spatial granularity. It is cost prohibitive to implement traditional data collection at frequent timescales to unravel spatiotemporal price evolution across market segments and at subnational geographic levels. Recent advancements in data innovation offer promising solutions to address the paucity of commodity price data and guide market intelligence for diverse development stakeholders. The use of artificial intelligence to estimate missing price data and a parallel effort to crowdsource commodity price data are both unlocking cost-effective opportunities to generate actionable price data. Yet, little is known about how the data from these alternative methods relate to independent ground truth data. To evaluate if these data strategies can meet the long-standing demand for real-time intelligence on food affordability, this paper analyzes open-source daily crowdsourced data (104,931 datapoints) from a recently published data set in Nature Journal, relative to complementary ground truth sample. The paper subsequently compares these data to open-source monthly artificial intelligence–generated price data for identical commodities over a 36-month period in northern Nigeria, from 2019 to 2022. The results show that all the data sources share a high degree of comparability, with variation across commodity and market segments. Overall, the findings provide important support for leveraging these new and innovative data approaches to enable data-driven decision-making in near real time.
Link to Data Set
Citation
“Adewopo, Julius; Andrée, Bo Pieter Johannes; Peter, Helen; Solano-Hermosilla, Gloria; Micale, Fabio. 2024. Comparative Analysis of AI-Predicted and Crowdsourced Food Prices in an Economically Volatile Region. Policy Research Working Paper; 10758. © World Bank. http://hdl.handle.net/10986/41467 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication A Data-Driven Approach for Early Detection of Food Insecurity in Yemen's Humanitarian Crisis(Washington, DC: World Bank, 2024-05-09)The Republic of Yemen is enduring the world's most severe protracted humanitarian crisis, compounded by conflict, economic collapse, and natural disasters. Current food insecurity assessments rely on expert evaluation of evidence with limited temporal frequency and foresight. This paper introduces a data-driven methodology for the early detection and diagnosis of food security emergencies. The approach optimizes for simplicity and transparency, and pairs quantitative indicators with data-driven optimal thresholds to generate early warnings of impending food security emergencies. Historical validation demonstrates that warnings can be reliably issued before sharp deterioration in food security occurs, using only a few critical indicators that capture inflation, conflict, and agricultural productivity shocks. These indicators signal deterioration most accurately at five months of lead time. The paper concludes that simple data-driven approaches show a strong capability to generate reliable food security warnings in Yemen, highlighting their potential to complement existing assessments and enhance lead time for effective intervention.Publication Altered Destinies: The Long-Term Effects of Rising Prices and Food Insecurity in the Middle East and North Africa(Washington, DC : World Bank, 2023-04-06)Growth is forecasted to slow down for the Middle East and North Africa region. The war in Ukraine in 2022 exacerbated inflationary pressures as the world recovered from the COVID 19 pandemic induced recession. The response by central banks to raise rates to curb inflation is slowing economic activity, while rising food prices are making it difficult for families to put meals on the table. Inflation, when it stems from food prices, hits the poor harder than the rich, thus compounding food insecurity in MENA that had been rising over decades. The immediate effects of food insecurity can be a devastating loss of life, but even temporary increases in food prices can cause long-term irreversible damages, especially to children. The rise in food prices due to the war in Ukraine may have altered the destinies of hundreds of thousands of children in the region, setting them on paths to limited prosperity. Food insecurity imposes challenges to a region where the state of child nutrition and health were inadequate before the shocks from the COVID-19 pandemic. The report discusses policy options and highlights the need for data to guide effective decision making.Publication Estimating Food Price Inflation from Partial Surveys(World Bank, Washington, DC, 2021-12)The traditional consumer price index is often produced at an aggregate level, using data from few, highly urbanized, areas. As such, it poorly describes price trends in rural or poverty-stricken areas, where large populations may reside in fragile situations. Traditional price data collection also follows a deliberate sampling and measurement process that is not well suited for monitoring during crisis situations, when price stability may deteriorate rapidly. To gain real-time insights beyond what can be formally measured by traditional methods, this paper develops a machine-learning approach for imputation of ongoing subnational price surveys. The aim is to monitor inflation at the market level, relying only on incomplete and intermittent survey data. The capabilities are highlighted using World Food Programme surveys in 25 fragile and conflict-affected countries where real-time monthly food price data are not publicly available from official sources. The results are made available as a data set that covers more than 1200 markets and 43 food types. The local statistics provide a new granular view on important inflation events, including the World Food Price Crisis of 2007–08 and the surge in global inflation following the 2020 pandemic. The paper finds that imputations often achieve accuracy similar to direct measurement of prices. The estimates may provide new opportunities to investigate local price dynamics in markets where prices are sensitive to localized shocks and traditional data are not available.Publication Predicting Food Crises(World Bank, Washington, DC, 2020-09)Globally, more than 130 million people are estimated to be in food crisis. These humanitarian disasters are associated with severe impacts on livelihoods that can reverse years of development gains. The existing outlooks of crisis-affected populations rely on expert assessment of evidence and are limited in their temporal frequency and ability to look beyond several months. This paper presents a statistical forecasting approach to predict the outbreak of food crises with sufficient lead time for preventive action. Different use cases are explored related to possible alternative targeting policies and the levels at which finance is typically unlocked. The results indicate that, particularly at longer forecasting horizons, the statistical predictions compare favorably to expert-based outlooks. The paper concludes that statistical models demonstrate good ability to detect future outbreaks of food crises and that using statistical forecasting approaches may help increase lead time for action.Publication Climate Shocks and Their Effects on Food Security, Prices, and Agricultural Wages in Afghanistan(Washington, DC: World Bank, 2024-12-17)This study examines the effects of climate and weather shocks on Afghanistan's agricultural economy, with an emphasis on food security, prices, and wages. By utilizing a dynamical model and a unique data set that includes monthly global and local food prices, agricultural wages, unofficial exchange rates, and local climate data, the research provides econometric estimates of the impacts of droughts and floods. The findings reveal that both flooding and drought significantly increase food insecurity, directly and indirectly. Directly, these climatic shocks are linked to heightened risks of food insecurity in the following months, even when controlling for price and wage fluctuations. Indirectly, droughts and floods drive up food prices and depress agricultural wages, further exacerbating food insecurity. The study suggests that enhancing climate resilience in the agriculture sector could mitigate these risks, stabilize local food prices and wages, and strengthen food security and the broader agricultural economy. The results also show that price data effectively capture food security shocks from various non-economic sources, and can serve as a versatile monitoring tool in situations where detailed data on food security are unavailable.
Users also downloaded
Showing related downloaded files
Publication Economic Governance Improvements and Sovereign Financing Costs in Developing Countries(World Bank, Washington, DC, 2021-05)Low- and middle-income country governments are increasingly tapping the global debt capital markets. This is increasing the amount of finance available for development, but at a considerably higher cost than traditional external borrowing on concessional terms. Using a novel methodology based on estimating sovereign credit ratings using the Moody’s scorecard, and examining the associations between these ratings and the World Bank’s Country Policy and Institutional Assessment scores, this paper examines how making improvements in the quality of economic policies and institutions can help lower governments’ financing costs. This method aims to overcome the small-sample problem due to the number of rated developing country sovereigns still being relatively limited (although growing). Better economic governance Country Policy and Institutional Assessment scores are associated with better estimated ratings and materially lower financing costs; on average, improvements that are sufficient to increase the Country Policy and Institutional Assessment economic governance indicator score by one point are associated with interest costs that are lower by about 40 basis points, even setting aside the direct impact on ratings of better governance indicators. There are many reasons why improving governance is a good thing. Among them is the potential payoff to the public purse — savings of $40 million or more on a standard $1 billion, 10-year bond.Publication Evolution of the World Bank’s Thinking on Governance(World Bank, Washington, DC, 2016-01)The preparation work for the 2017 World Development Report on Governance and the Law in 2016 will coincide with the 25th anniversary of the World Bank’s decision to broaden its ongoing work on public sector management to embrace key issues of governance in the Bank’s borrowing countries. This paper provides a broad overview of the major Bank reports on governance that went through a review process at a sufficiently high level in the institution that they can reasonably be described as reflecting the Bank’s considered views at the time on the subject. The objective is to review the evolution of the Bank’s thinking on governance and assess the relevance and effectiveness of the work and its implications for the forthcoming World Development Report. Section two of this paper begins with a brief account of how the Bank came to focus on issues of governance, reviewing the major upheaval of governance in many of the Bank’s borrowing countries in the 1980s, the legal constraints the Articles of Agreement impose on the Bank’s work on governance, and a brief overview of the Bank’s initial policy statement on governance issued to the Bank’s Board in June 1991. Section three reviews major Bank work on governance as reflected in successive World Development Reports and examines the Bank’s analysis of the issue of corruption, reviewing how the Bank’s thinking on this symptom of poor governance has evolved. Section four steps back to assess what the Bank got right and some of the issues it missed or failed to address adequately. Section five draws attention to the dramatic changes experienced by the developing world in these past 25 years, and points to the need to better understand the implications of these changes for the governance context facing developing countries.Publication Statistical Performance Indicators and Index(World Bank, Washington, DC, 2021-03)The World Bank’s Statistical Capacity Index has been widely employed to measure country statistical capacity since its inception two decades ago. This paper builds on the existing advantages of the Statistical Capacity Index, conceptually and empirically, to offer new statistical performance indicators and the Statistical Performance Index, which can better measure a country’s statistical performance. The new index has clearer conceptual motivations, employs a stronger mathematical foundation, and significantly expands the number of indicators and countries covered. The paper further provides empirical evidence that illustrates the strong correlation of the new index with other commonly used development indicators of human capital, governance, poverty, and inequality. The framework can accommodate future directions to improve the index as the global data landscape evolvesPublication Governance in Sub-Saharan Africa in the 21st Century(Washington, DC: World Bank, 2024-03-04)What can be learned from the governance trajectory of African countries since the beginning of the 21st century What is the quality of governance on the African continent and how does it shape development The first decade of the millennium saw promising growth and poverty reduction in much of the continent. Yet, Sub-Saharan Africa has also been the stage of a stream of governance reform failures and policy reversals, and many countries continue to suffer from the consequences of poor governance. This paper explores the dynamics of governance reform on the continent over the past two decades and points to four key trends. First, effective state institutions, capable of maintaining peace, fostering growth, and delivering services, have developed unevenly. Second, progress has been made on enhancing the inclusiveness and accountability of institutions, but it remains constrained by the weakness of checks and balances and the persistence of patterns of centralized and exclusive power arrangements. Third, civic capacity has risen considerably, but the inability of institutions to respond to social expectations and political mobilization threatens to turn liberal civic engagement into distrust, populism, and radicalization. Fourth, the combination of these three trends contributes to the rise of political instability, which constitutes a major threat for the continent.Publication Data Transparency and Growth in Developing Economies during and after the Global Financial Crisis(World Bank, Washington, DC, 2020-12)The study explores the effects of data transparency on economic growth for developing economies over a unique time period—at the onset of the 2007–2009 global financial crisis and thereafter. Data transparency is defined as the timely production of credible statistics as measured by the Statistical Capacity Indicator. The paper finds that data transparency has a positive effect on real gross domestic product per capita during a period of considerable uncertainty. The estimates indicate an elasticity of the magnitude of 0.03 percent per year, which is much larger than the elasticity of trade openness and schooling in the estimation sample. The empirics employ a variety of econometric estimators, including dynamic panel and cross-sectional instrumental variables estimators, with the latter approach yielding a higher estimated elasticity. The findings are robust to the inclusion of several factors in addition to political institutions and exogenous commodity-price and external debt-financing shocks.