Publication: Monitoring Global Aid Flows: A Novel Approach Using Large Language Models
Loading...
Published
2025-11-04
ISSN
Date
2025-11-05
Editor(s)
Abstract
Effective monitoring of development aid is the foundation for assessing the alignment of flows with their intended development objectives. Existing reporting systems, such as the Organisation for Economic Co-operation and Development’s Creditor Reporting System, provide standardized classification of aid activities but have limitations when it comes to capturing new areas like climate change, digitalization, and other cross-cutting themes. This paper proposes a bottom-up, unsupervised machine learning framework that leverages textual descriptions of aid projects to generate highly granular activity clusters. Using the 2021 Creditor Reporting System data set of nearly 400,000 records, the model produces 841 clusters, which are then grouped into 80 subsectors. These clusters reveal 36 emerging aid areas not tracked in the current Creditor Reporting System taxonomy, allow unpacking of “multi-sectoral” and “sector not specified” classifications, and enable estimation of flows to new themes, including World Bank Global Challenge Programs, International Development Association–20 Special Themes, and Cross-Cutting Issues. Validation against both Creditor Reporting System benchmarks and International Development Association commitment data demonstrates robustness. This approach illustrates how machine learning and the new advances in large language models can enhance the monitoring of global aid flows and inform future improvements in aid classification and reporting. It offers a useful tool that can support more responsive and evidence-based decision-making, helping to better align resources with evolving development priorities.
Link to Data Set
Citation
“Luo, Xubei; Rajasekaran, Arvind Balaji; Scruggs, Andrew Conner. 2025. Monitoring Global Aid Flows: A Novel Approach Using Large Language Models. Policy Research Working Paper; 11248. © World Bank. http://hdl.handle.net/10986/43937 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Rigging the Scores: Corruption through Scoring Rule Manipulation in Public Procurement Auctions(Washington, DC: World Bank, 2025-12-02)Public procurement is highly susceptible to corruption, especially in developing countries. Although open auctions are widely adopted to curb it, this paper finds that corruption remains prevalent even within this procurement format. Procurement officers can collaborate with firms to manipulate scoring rules, ensuring predetermined winners, while corrupt firms submit noncompetitive bids to meet minimum bidder requirements. Using extensive data from Chinese public procurement auctions, the paper introduces model-driven statistical tools to detect such corruption, identifying a corruption rate of 65 percent. A procurement expert audit survey confirms the tools’ reliability, with a 91 percent probability that experts recognize suspicious scoring rules when flagged. Firm-level analysis reveals that local, state-owned, and less productive firms are favored in corrupt auctions. Lastly, the paper explores policy implications. Analysis of the national anti-corruption campaign since 2012 suggests that general investigations may be insufficient to address deeply ingrained corrupt practices. Using counterfactuals based on an estimated structural model, the paper shows that implementing anonymous call-for-tender evaluations could improve social welfare by 10 percent by eliminating suspicious rules and encouraging broader participation.Publication Labor Demand in the Age of Generative AI: Early Evidence from the U.S. Job Posting Data(Washington, DC: World Bank, 2025-11-18)This paper examines the causal impact of generative artificial intelligence on U.S. labor demand using online job posting data. Exploiting ChatGPT’s release in November 2022 as an exogenous shock, the paper applies difference-in-differences and event study designs to estimate the job displacement effects of generative artificial intelligence. The identification strategy compares labor demand for occupations with high versus low artificial intelligence substitution vulnerability following ChatGPT’s launch, conditioning on similar generative artificial intelligence exposure levels to isolate substitution effects from complementary uses. The analysis uses 285 million job postings collected by Lightcast from the first quarter of 2018 to the second quarter of 2025Q2. The findings show that the number of postings for occupations with above-median artificial intelligence substitution scores fell by an average of 12 percent relative to those with below-median scores. The effect increased from 6 percent in the first year after the launch to 18 percent by the third year. Losses were particularly acute for entry-level positions that require neither advanced degrees (18 percent) nor extensive experience (20 percent), as well as those in administrative support (40 percent) and professional services (30 percent). Although generative artificial intelligence generates new occupations and enhances productivity, which may increase labor demand, early evidence suggests that some occupations may be less likely to be complemented by generative artificial intelligence than others.Publication Investment Policy Reforms and Foreign Direct Investment Inflows(Washington, DC: World Bank, 2025-12-01)Foreign direct investment has the potential to introduce much-needed capital and expertise in emerging and developing economies. To attract foreign direct investment, many countries have eased restrictions on foreign ownership in various sectors, reformed their institutions, and set up investment promotion agencies. Until the mid-2010s, Ethiopia remained one of the few countries that resisted this trend, with several stringent restrictions in place on foreign direct investment entry and operations in the country. This study employs a synthetic control method to examine patterns in foreign capital inflows following a series of investment policy reforms that were substantively introduced in the mid-2010s (circa 2015). The study offers evidence that investment policy reforms contributed to a significant foreign direct investment inflow in Ethiopia, compared to what would have occurred in the absence of these policies. An alternative strategy that conservatively specifies the donor country pool using an AI-assisted deep search technique changes the donor pool weighting matrix of the synthetic control method, but the estimated policy effects largely remain robust to this specification. The findings highlight the importance of targeted reforms in promoting foreign direct investment inflow in developing countries.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Estimating the Gravity Model When Zero Trade Flows are Frequent and Economically Determined(World Bank, Washington, DC, 2015-06)This paper evaluates the performance of alternative estimators of the gravity equation when zero trade flows result from economically-based data-generating processes with heteroscedastic residuals and potentially-omitted variables. In a standard Monte Carlo analysis, the paper finds that this combination can create seriously biased estimates in gravity models with frequencies of zero frequently observed in real-world data, and that Poisson Pseudo-Maximum-Likelihood models can be important in solving this problem. Standard threshold–Tobit estimators perform well in a Tobit-based data-generating process only if the analysis deals with the heteroscedasticity problem. When the data are generated by a Heckman sample selection model, the Zero-Inflated Poisson model appears to have the lowest bias. When the data are generated by a Helpman, Melitz, and Rubinstein-type model with heterogeneous firms, a Zero-Inflated Poisson estimator including firm numbers appears to provide the best results. Testing on real-world data for total trade throws up additional puzzles with truncated Poisson Pseudo-Maximum-Likelihood and Poisson Pseudo-Maximum-Likelihood estimators being very similar, and Zero-Inflated Poisson and truncated Poisson Pseudo-Maximum-Likelihood identical. Repeating the Monte Carlo analysis taking into account the high frequency of very small predicted trade flows in real-world data reconciles these findings and leads to specific recommendations for estimators.Publication Using Large Language Models for Qualitative Analysis can Introduce Serious Bias(World Bank Washington, DC, 2023-11-08)Large Language Models (LLMs) are quickly becoming ubiquitous, but the implications for social science research are not yet well understood. This paper asks whether LLMs can help us analyse large-N qualitative data from open-ended interviews, with an application to transcripts of interviews with displaced Rohingya people in Cox’s Bazaar, Bangladesh. The analysis finds that a great deal of caution is needed in using LLMs to annotate text as there is a risk of introducing biases that can lead to misleading inferences. Here this refers to bias in the technical sense, that the errors that LLMs make in annotating interview transcripts are not random with respect to the characteristics of the interview subjects. Training simpler supervised models on high-quality human annotations with flexible coding leads to less measurement error and bias than LLM annotations. Therefore, given that some high quality annotations are necessary in order to asses whether an LLM introduces bias, this paper argues that it is probably preferable to train a bespoke model on these annotations than it is to use an LLM for annotation.Publication Empirical Econometric Evaluation of Alternative Methods of Dealing with Missing Values in Investment Climate Surveys(2010-06-01)Investment climate Surveys are valuable instruments that improve our understanding of the economic, social, political, and institutional factors determining economic growth, particularly in emerging and transition economies. However, at the same time, they have to overcome some difficult issues related to the quality of the information provided; measurement errors, outlier observations, and missing data that are frequently found in these datasets. This paper discusses the applicability of recent procedures to deal with missing observations in investment climate surveys. In particular, it presents a simple replacement mechanism -- for application in models with a large number of explanatory variables -- which in turn is a proxy of two methods: multiple imputations and an export-import algorithm. The performance of this method in the context of total factor productivity estimation in extended production functions is evaluated using investment climate surveys from four countries: India, South Africa, Tanzania, and Turkey. It is shown that the method is very robust and performs reasonably well even under different assumptions on the nature of the mechanism generating missing data.Publication Large Country-Lot Quality Assurance Sampling : A New Method for Rapid Monitoring and Evaluation of Health, Nutrition and Population Programs at Sub-National Levels(World Bank, Washington, DC, 2008-05)Sampling theory facilitates development of economical, effective and rapid measurement of a population. While national policy maker value survey results measuring indicators representative of a large area (a country, state or province), measurement in smaller areas produces information useful for managers at the local level. It is often not possible to disaggregate a national survey to obtain local information if that was not the intent of the original survey design. Cluster sampling is typically used for national or large area surveys because sampling in clusters lowers the cost of a survey. Lot Quality Assurance Sampling (LQAS) is used to measure results at a local level, since it requires small random samples and produces results useful to local managers. However, current LQAS methodology requires all local areas (strata) be included in the survey in order to be aggregated to produce point estimates for the nation or state. In large countries it is not feasible to sample all strata for logistical and financial reasons. This paper resolves this problem by presenting Large Country (LC)-LQAS, a method with two concurrent objectives: 1) provide local managers with accurate local information to enable data driven decisions, and 2) provide central policy makers with the aggregate information they require. These are achieved by integrating cluster sampling with LQAS methodologies. Two examples of the implementation of LC-LQAS are provided, in an HIV/AIDS program in Kenya and a Malaria Booster Project in Nigeria. Classifications of local health units into performance categories and aggregate estimates of coverage, with associated confidence intervals, are provided for select indicators in order to demonstrate its use, analysis, and costs. This paper is written as a manual to support the use of LC-LQAS by others.Publication Criss-Crossing Globalization : Uphill Flows of Skill-Intensive Goods and Foreign Direct Investment(2009-09-01)This paper documents an unusual and possibly significant phenomenon: the export of skills, embodied in goods, services or capital from poorer to richer countries. The authors first present a set of stylized facts. Then, using a measure that combines the sophistication of a country s exports with the average income level of destination countries, they show that the performance of a number of developing countries - notably China, Mexico and South Africa - matches that of much more advanced countries - such as Japan, Spain and the United States. The authors create a new combined dataset on foreign direct investment (covering greenfield investment as well as mergers and acquisitions). The analysis shows that flows of foreign direct investment to developed countries from developing countries - like Brazil, India, Malaysia and South Africa - as a share of their GDP, are as large as flows from developed countries - like Japan, Korea and the United States. The authors suggest that it is not just the composition of exports but their destination that matters. In both cross-sectional and panel regressions, with a range of controls, a measure of uphill flows of sophisticated goods is significantly associated with better growth performance. These results suggest the need for a deeper analysis of whether the benefits of development might derive not from deifying comparative advantage but from defying it.
Users also downloaded
Showing related downloaded files
Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication Poverty, Prosperity, and Planet Report 2024(Washington, DC: World Bank, 2024-10-15)The Poverty, Prosperity, and Planet Report 2024 is the latest edition of the series formerly known as Poverty and Shared Prosperity. The report emphasizes that reducing poverty and increasing shared prosperity must be achieved in ways that do not come at unacceptably high costs to the environment. The current “polycrisis”—where the multiple crises of slow economic growth, increased fragility, climate risks, and heightened uncertainty have come together at the same time—makes national development strategies and international cooperation difficult. Offering the first post-Coronavirus (COVID)-19 pandemic assessment of global progress on this interlinked agenda, the report finds that global poverty reduction has resumed but at a pace slower than before the COVID-19 crisis. Nearly 700 million people worldwide live in extreme poverty with less than US$2.15 per person per day. Progress has essentially plateaued amid lower economic growth and the impacts of COVID-19 and other crises. Today, extreme poverty is concentrated mostly in Sub-Saharan Africa and fragile settings. At a higher standard more typical of upper-middle-income countries—US$6.85 per person per day—almost one-half of the world is living in poverty. The report also provides evidence that the number of countries that have high levels of income inequality has declined considerably during the past two decades, but the pace of improvements in shared prosperity has slowed, and that inequality remains high in Latin America and the Caribbean and Sub-Saharan Africa. Worldwide, people’s incomes today would need to increase fivefold on average to reach a minimum prosperity threshold of US$25 per person per day. Where there has been progress in poverty reduction and shared prosperity, there is evidence of an increasing ability of countries to manage natural hazards, but climate risks are significantly higher in the poorest settings. Nearly one in five people globally is at risk of experiencing welfare losses due to an extreme weather event from which they will struggle to recover. The interconnected issues of climate change and poverty call for a united and inclusive effort from the global community. Development cooperation stakeholders—from governments, nongovernmental organizations, and the private sector to communities and citizens acting locally in every corner of the globe—hold pivotal roles in promoting fair and sustainable transitions. By emphasizing strategies that yield multiple benefits and diligently monitoring and addressing trade-offs, we can strive toward a future that is prosperous, equitable, and resilient.Publication Business Ready 2024(Washington, DC: World Bank, 2024-10-03)Business Ready (B-READY) is a new World Bank Group corporate flagship report that evaluates the business and investment climate worldwide. It replaces and improves upon the Doing Business project. B-READY provides a comprehensive data set and description of the factors that strengthen the private sector, not only by advancing the interests of individual firms but also by elevating the interests of workers, consumers, potential new enterprises, and the natural environment. This 2024 report introduces a new analytical framework that benchmarks economies based on three pillars: Regulatory Framework, Public Services, and Operational Efficiency. The analysis centers on 10 topics essential for private sector development that correspond to various stages of the life cycle of a firm. The report also offers insights into three cross-cutting themes that are relevant for modern economies: digital adoption, environmental sustainability, and gender. B-READY draws on a robust data collection process that includes specially tailored expert questionnaires and firm-level surveys. The 2024 report, which covers 50 economies, serves as the first in a series that will expand in geographical coverage and refine its methodology over time, supporting reform advocacy, policy guidance, and further analysis and research.Publication State of Social Protection Report 2025(Washington, DC: World Bank, 2025-04-07)Social protection goes well beyond cash transfers; it includes policies and programs that bridge skill, financial, and information gaps, aiding people in securing better jobs. The three pillars of social protection—social assistance, social insurance, and labor market programs—support households and workers in handling crises, escaping poverty, facing transitions, and seizing employment opportunities. But despite a substantial expansion over the past decade, 2 billion people remain uncovered or inadequately covered across low- and middle-income countries. Drawing from administrative and household survey data from the World Bank’s Atlas of Social Protection Indicators of Resilience and Equity (ASPIRE), the "State of Social Protection Report 2025: The 2-Billion-Person Challenge" documents advances and challenges to strengthening social protection and labor systems across low- and middle-income countries, analyzing the evolution of expenditure, coverage, and adequacy of support. This report details four policy action areas governments can embrace to maximize the benefits of adequate social protection for all: extending social protection to those in need; strengthening the adequacy of social protection support; building shock-proof social protection systems; and optimizing social protection financing. The report discusses how the path of reforms will depend on country context, capacity, and fiscal space. The rising frequency of shocks and crises calls for major investments in the adaptability and preparedness of social protection and labor systems. Amid a world in transition, social protection is more important and necessary than ever.Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.