Publication: Monitoring Global Aid Flows: A Novel Approach Using Large Language Models
Loading...
Published
2025-11-04
ISSN
Date
2025-11-05
Editor(s)
Abstract
Effective monitoring of development aid is the foundation for assessing the alignment of flows with their intended development objectives. Existing reporting systems, such as the Organisation for Economic Co-operation and Development’s Creditor Reporting System, provide standardized classification of aid activities but have limitations when it comes to capturing new areas like climate change, digitalization, and other cross-cutting themes. This paper proposes a bottom-up, unsupervised machine learning framework that leverages textual descriptions of aid projects to generate highly granular activity clusters. Using the 2021 Creditor Reporting System data set of nearly 400,000 records, the model produces 841 clusters, which are then grouped into 80 subsectors. These clusters reveal 36 emerging aid areas not tracked in the current Creditor Reporting System taxonomy, allow unpacking of “multi-sectoral” and “sector not specified” classifications, and enable estimation of flows to new themes, including World Bank Global Challenge Programs, International Development Association–20 Special Themes, and Cross-Cutting Issues. Validation against both Creditor Reporting System benchmarks and International Development Association commitment data demonstrates robustness. This approach illustrates how machine learning and the new advances in large language models can enhance the monitoring of global aid flows and inform future improvements in aid classification and reporting. It offers a useful tool that can support more responsive and evidence-based decision-making, helping to better align resources with evolving development priorities.
Link to Data Set
Citation
“Luo, Xubei; Rajasekaran, Arvind Balaji; Scruggs, Andrew Conner. 2025. Monitoring Global Aid Flows: A Novel Approach Using Large Language Models. Policy Research Working Paper; 11248. © World Bank. http://hdl.handle.net/10986/43937 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication Direct and Indirect Impacts of Transport Mobility on Access to Jobs: Evidence from South Africa(Washington, DC: World Bank, 2025-11-12)Access to jobs is essential for economic growth. In Africa, unemployment rates are notably high. This paper reexamines the relationship between transport mobility and labor market outcomes, with a particular focus on the direct and indirect effects of transport connectivity. As predicted by theory, wages are influenced by the level of commuting deterrence. Generally, higher earnings are associated with longer commute times and/or higher commuting costs. Local accessibility is also important, especially for individuals with time constraints. Both direct and indirect impacts are found to be significant in South Africa, where job accessibility has been challenging since the end of apartheid. For the direct impact, the wage elasticity associated with commuting costs is significant. Returns on commute are particularly high for women. Local accessibility to socioeconomic facilities, such as shops and health services, is also found to have a significant impact, consistent with the concept of mobility of care. To enhance employment, therefore, it is crucial to connect people not only to job locations but also to various socioeconomic points of interest, such as markets and hospitals, in an integrated manner. This integration will enable individuals to spend more time working and commuting longer distances.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication From Policy to Practice: Lessons from the Implementation of the Refugee Work Rights Policy in Ethiopia(Washington, DC: World Bank, 2025-11-10)This paper examines the early implementation of Ethiopia’s refugee work rights policy, with a focus on the issuance of permits that enable refugees to engage in economic activities. Building on significant legal and institutional advances under the 2019 Refugee Proclamation and subsequent directives, the analysis explores how these reforms are being operationalized in practice. Using a mixed-methods approach, combining document review, administrative data analysis, and semi-structured interviews, the paper identifies both progress and remaining challenges. Permit issuance has increased since the adoption of detailed operational guidance in 2024, reflecting the Government of Ethiopia’s commitment to operationalizing its progressive legal framework and ensuring that refugees can exercise their right to work. However, take-up remains modest, with about 5.2 percent of the working-age population holding a permit. Preliminary evidence suggests that coordination gaps, limited subnational capacity, low awareness among refugees and employers, and disincentives to formalize in a largely informal labor market are contributing to the low take-up. The paper offers policy suggestions, grounded in the Ethiopian context and emerging evidence, to help translate legal commitments into improved labor market outcomes for refugees.Publication It’s Not (Just) the Tariffs: Rethinking Non-Tariff Measures in a Fragmented Global Economy(Washington, DC: World Bank, 2025-10-22)As tariffs have declined, non-tariff measures (NTMs) have become central to trade policy, especially in high-income countries and regulated sectors like food and green technologies. Although NTMs may serve legitimate goals, they could also sort countries and firms into or out of markets based on compliance capacity and differences in product mix. Documenting recent advances in the estimation of ad valorem equivalents (AVEs), this paper uncovers new patterns of use and exposure of NTMs. High-income countries rely more heavily on NTMs relative to tariffs, while low- and middle-income countries face steeper AVEs on their exports. Firm-level evidence shows that NTMs disproportionately affect smaller firms, leading to market exit and concentration. Poorly designed NTMs can harm productivity and welfare, while coordinated, capacity-aware use can deliver inclusive outcomes. Policy design, transparency, and diagnostics must evolve to reflect the growing role—and risks—of NTMs in a fragmented global trade landscape.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Estimating the Gravity Model When Zero Trade Flows are Frequent and Economically Determined(World Bank, Washington, DC, 2015-06)This paper evaluates the performance of alternative estimators of the gravity equation when zero trade flows result from economically-based data-generating processes with heteroscedastic residuals and potentially-omitted variables. In a standard Monte Carlo analysis, the paper finds that this combination can create seriously biased estimates in gravity models with frequencies of zero frequently observed in real-world data, and that Poisson Pseudo-Maximum-Likelihood models can be important in solving this problem. Standard threshold–Tobit estimators perform well in a Tobit-based data-generating process only if the analysis deals with the heteroscedasticity problem. When the data are generated by a Heckman sample selection model, the Zero-Inflated Poisson model appears to have the lowest bias. When the data are generated by a Helpman, Melitz, and Rubinstein-type model with heterogeneous firms, a Zero-Inflated Poisson estimator including firm numbers appears to provide the best results. Testing on real-world data for total trade throws up additional puzzles with truncated Poisson Pseudo-Maximum-Likelihood and Poisson Pseudo-Maximum-Likelihood estimators being very similar, and Zero-Inflated Poisson and truncated Poisson Pseudo-Maximum-Likelihood identical. Repeating the Monte Carlo analysis taking into account the high frequency of very small predicted trade flows in real-world data reconciles these findings and leads to specific recommendations for estimators.Publication Using Large Language Models for Qualitative Analysis can Introduce Serious Bias(World Bank Washington, DC, 2023-11-08)Large Language Models (LLMs) are quickly becoming ubiquitous, but the implications for social science research are not yet well understood. This paper asks whether LLMs can help us analyse large-N qualitative data from open-ended interviews, with an application to transcripts of interviews with displaced Rohingya people in Cox’s Bazaar, Bangladesh. The analysis finds that a great deal of caution is needed in using LLMs to annotate text as there is a risk of introducing biases that can lead to misleading inferences. Here this refers to bias in the technical sense, that the errors that LLMs make in annotating interview transcripts are not random with respect to the characteristics of the interview subjects. Training simpler supervised models on high-quality human annotations with flexible coding leads to less measurement error and bias than LLM annotations. Therefore, given that some high quality annotations are necessary in order to asses whether an LLM introduces bias, this paper argues that it is probably preferable to train a bespoke model on these annotations than it is to use an LLM for annotation.Publication Empirical Econometric Evaluation of Alternative Methods of Dealing with Missing Values in Investment Climate Surveys(2010-06-01)Investment climate Surveys are valuable instruments that improve our understanding of the economic, social, political, and institutional factors determining economic growth, particularly in emerging and transition economies. However, at the same time, they have to overcome some difficult issues related to the quality of the information provided; measurement errors, outlier observations, and missing data that are frequently found in these datasets. This paper discusses the applicability of recent procedures to deal with missing observations in investment climate surveys. In particular, it presents a simple replacement mechanism -- for application in models with a large number of explanatory variables -- which in turn is a proxy of two methods: multiple imputations and an export-import algorithm. The performance of this method in the context of total factor productivity estimation in extended production functions is evaluated using investment climate surveys from four countries: India, South Africa, Tanzania, and Turkey. It is shown that the method is very robust and performs reasonably well even under different assumptions on the nature of the mechanism generating missing data.Publication Large Country-Lot Quality Assurance Sampling : A New Method for Rapid Monitoring and Evaluation of Health, Nutrition and Population Programs at Sub-National Levels(World Bank, Washington, DC, 2008-05)Sampling theory facilitates development of economical, effective and rapid measurement of a population. While national policy maker value survey results measuring indicators representative of a large area (a country, state or province), measurement in smaller areas produces information useful for managers at the local level. It is often not possible to disaggregate a national survey to obtain local information if that was not the intent of the original survey design. Cluster sampling is typically used for national or large area surveys because sampling in clusters lowers the cost of a survey. Lot Quality Assurance Sampling (LQAS) is used to measure results at a local level, since it requires small random samples and produces results useful to local managers. However, current LQAS methodology requires all local areas (strata) be included in the survey in order to be aggregated to produce point estimates for the nation or state. In large countries it is not feasible to sample all strata for logistical and financial reasons. This paper resolves this problem by presenting Large Country (LC)-LQAS, a method with two concurrent objectives: 1) provide local managers with accurate local information to enable data driven decisions, and 2) provide central policy makers with the aggregate information they require. These are achieved by integrating cluster sampling with LQAS methodologies. Two examples of the implementation of LC-LQAS are provided, in an HIV/AIDS program in Kenya and a Malaria Booster Project in Nigeria. Classifications of local health units into performance categories and aggregate estimates of coverage, with associated confidence intervals, are provided for select indicators in order to demonstrate its use, analysis, and costs. This paper is written as a manual to support the use of LC-LQAS by others.Publication Criss-Crossing Globalization : Uphill Flows of Skill-Intensive Goods and Foreign Direct Investment(2009-09-01)This paper documents an unusual and possibly significant phenomenon: the export of skills, embodied in goods, services or capital from poorer to richer countries. The authors first present a set of stylized facts. Then, using a measure that combines the sophistication of a country s exports with the average income level of destination countries, they show that the performance of a number of developing countries - notably China, Mexico and South Africa - matches that of much more advanced countries - such as Japan, Spain and the United States. The authors create a new combined dataset on foreign direct investment (covering greenfield investment as well as mergers and acquisitions). The analysis shows that flows of foreign direct investment to developed countries from developing countries - like Brazil, India, Malaysia and South Africa - as a share of their GDP, are as large as flows from developed countries - like Japan, Korea and the United States. The authors suggest that it is not just the composition of exports but their destination that matters. In both cross-sectional and panel regressions, with a range of controls, a measure of uphill flows of sophisticated goods is significantly associated with better growth performance. These results suggest the need for a deeper analysis of whether the benefits of development might derive not from deifying comparative advantage but from defying it.
Users also downloaded
Showing related downloaded files
Publication Togo Economic Update, August 2025: Boosting Growth and Restoring Fiscal Space in Uncertain Times(Washington, DC: World Bank, 2025-08-28)Togo’s economic trajectory in recent years has been shaped by both resilient performance and emerging vulnerabilities. The 2025 Economic Update underscores the urgency of restoring fiscal space and implementing strategic structural reforms to sustain private sector-led growth and job creation. Through an integrated analysis in two chapters, the report presents a nuanced narrative of the country’s macroeconomic outlook and delineates actionable policy paths to foster inclusive, sustainable development.Publication Jobs in a Changing Climate: Insights from World Bank Group Country Climate and Development Reports Covering 93 Economies(Washington, DC: World Bank, 2025-11-05)The World Bank Group’s Country Climate and Development Reports (CCDRs) provide a crosscutting look at how countries’ development prospects, and the job opportunities they offer to their people, can be threatened by climate impacts and supported by climate policies. Climate change and policies affect jobs through impacts on productivity, energy and material efficiency, and physical, human, and natural capital. They can also transform employment opportunities, especially through complementary measures that help workers and firms adapt to and benefit from new technologies and production practices. Prepared by the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA), CCDRs integrate country perspectives, climate science and economic modeling, private sector information, and policy analysis to assess how countries can successfully grow and develop their economies and create jobs despite increasing climate risks and while achieving their climate objectives and commitments. Each CCDR starts from the country’s development priorities, opportunities, and challenges, and is developed in close consultation with governments, businesses, and civil society, ensuring the recommendations reflect national priorities. By combining evidence on adaptation, resilience, and emissions pathways, CCDRs highlight where climate action can reinforce development and job creation, and where targeted policies are needed to manage risks and smooth labor market transitions. Taken together, these elements can help create local jobs, ensure economic transitions are just and inclusive, and equip workers and firms to navigate the disruptions and opportunities of a changing climate and changing technologies.Publication Uganda Country Climate and Development Report(Washington, DC: World Bank, 2025-09-10)The Country Climate and Development Report (CCDR) for Uganda examines the interplay between climate change and development. It presents how addressing climate change can support achieving the goals in Uganda’s Vision 2040 and Ten-Fold Growth Strategy, and help propel the country to upper-middle-income status. Uganda is the 14th most vulnerable nation to climate change, yet it is 163rd in readiness to address these risks, facing threats such as droughts and floods. The report highlights that, of the poorest households exposed to climate change, 80 percent already experience income loss from climate shocks. GDP could also drop by up to 3.1 percent by 2050 without additional climate action. Climate change poses numerous challenges, including increased variability in crop yields, potential internal climate migration of 12 million people by 2050, notable drop in labor productivity due to heat stress, increased health risks from waterborne diseases and malaria, and exposure of the country’s physical infrastructure to extreme climate events. To combat these challenges, the report recommends transitioning to a low-carbon, climate-resilient growth path by implementing four multisectoral intervention packages. These include boosting resilience through jobs for youth and services for the poor; promoting resilient and productive agriculture and natural resources with lower GHG emissions; developing climate-responsive energy, transport, and digital infrastructure; and fostering planned and climate-positive urbanization. Additionally, the report calls for whole-of-economy measures that strengthen governance of climate action, enhance preparedness for climate hazards including through improved early warning systems, operationalization of the national climate finance strategy, and incentivizing private sector participation. By implementing these intervention packages and whole-of-economy measures, Uganda can lower its risk to climate change and achieve sustainable economic growth.Publication Sahel Irrigation Strategy(Washington, DC: World Bank, 2025-09-05)The Sahel Irrigation Strategy aims to guide participating countries to scale up irrigation expansion and modernization rapidly and responsibly with a more diverse suite of interventions to optimize the use of their natural and financial resources. It provides a comprehensive roadmap to unlock the full potential of sustainable irrigation as a driver of agricultural productivity, food security, and resilience across the Sahel. Its primary objectives include taking stock of the progress achieved over the past decade, identifying emerging trends and challenges, and moving beyond problem identification to actionable solutions rooted in lessons learned. The strategy establishes updated and shared goals for irrigated agriculture tailored to short-term (2035), medium-term (2045), and long-term (2055) horizons. These targets are accompanied by a clear framework for intervention, designed to foster collective commitment and coordinated efforts among the six Sahelian countries.Publication Accelerating Investment: Challenges and Policies(Washington, DC: World Bank, 2025-09-19)Investment is the engine that expands productive capacity, modernizes infrastructure, creates jobs, and drives progress toward development and climate goals. Yet developing economies face an investment shortfall of historic proportions. Even as development needs rise, investment growth has slowed to about half its pace in the 2000s. Public investment alone cannot fill this gap. Private investment must play the leading role in reigniting growth, creating jobs, and supporting the transition to a more sustainable and resilient future. This book presents the World Bank’s most comprehensive assessment of investment in emerging and developing economies. It examines why investment matters, why it has stalled, and what it will take to revive it. The analysis highlights that countries that have successfully triggered investment booms combined sound macroeconomic frameworks with reforms that improved the business climate, strengthened governance, and mobilized private capital. History shows that when investment surges, economies grow faster, job creation accelerates, and poverty declines more rapidly. Reversing today’s slowdown is within reach — through credible fiscal and monetary policies, well-targeted public investment that attracts private capital, and stronger international cooperation to mobilize finance at scale. Reigniting investment is not just a national priority — it is a global imperative.