Publication: Missing Evidence: Tracking Academic Data Use around the World
Loading...
Published
2024-01-25
ISSN
Date
2024-01-25
Author(s)
Editor(s)
Abstract
Data-driven research on a country is key to producing evidence-based public policies. Yet little is known about where data-driven research is lacking and how it could be expanded. This paper proposes a method for tracking academic data use by country of subject, applying natural language processing to open-access research papers. The model’s predictions produce country estimates of the number of articles using data that are highly correlated with a human-coded approach, with a correlation of 0.99. Analyzing more than 1 million academic articles, the paper finds that the number of articles on a country is strongly correlated with its gross domestic product per capita, population, and the quality of its national statistical system. The paper identifies data sources that are strongly associated with data-driven research and finds that availability of subnational data appears to be particularly important. Finally, the paper classifies countries into groups based on whether they could most benefit from increasing their supply of or demand for data. The findings show that the former applies to many low- and lower-middle-income countries, while the latter applies to many upper-middle- and high-income countries.
Link to Data Set
Citation
“Stacy, Brian; Kitzmüller, Lucas; Wang, Xiaoyu; Mahler, Daniel Gerszon; Serajuddin, Umar. 2024. Missing Evidence: Tracking Academic Data Use around the World. Policy Research Working Paper; 10673. © World Bank. http://hdl.handle.net/10986/40965 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication When Is There Enough Data to Create a Global Statistic ?(Washington, DC: World Bank, 2022-05-05)To monitor progress toward global goals such as the Sustainable Development Goals, global statistics are needed. Yet cross-country data sets are rarely truly global, creating a trade-off for producers of global statistics: the lower is the data coverage threshold for disseminating global statistics, the more statistics can be made available, but the lower is the accuracy of these statistics. This paper quantifies the availability-accuracy trade-off by running more than 10 million simulations on the World Development Indicators. It shows that if the fraction of the world’s population for which data are lacking is x, then the global value will on expectation be off by 0.37*x standard deviation, and it could be off by as much as x standard deviations. The paper shows the robustness of this result to various assumptions and provides recommendations on when there is enough data to create global statistics. Although the decision will be context specific, in a baseline scenario, it is suggested not to create global statistics when there are data for less than half of the world’s population.Publication Missing SDG Gender Indicators(Washington, DC: World Bank, 2023-08-15)The Sustainable Development Goal agenda lays out an ambitious set of 231 indicators to track progress. Countries continue to fall short in terms of reporting on the indicators in general, and this is particularly the case for the subset of 50 gender-related indicators, where countries reported on average on 31 percent of these indicators in at least one year from 2016 to 2020. A closer look at this low coverage reveals four salient fundings. First, this is not just a problem of missing data; lack of reporting on existing data is detected to be a problem. For example, of the 32 gender-related indicators that are sex disaggregated, if countries that had a population estimate also had a sex-disaggregated estimate (which is almost always feasible), the Sustainable Development Goal gender coverage rate would be 43 percent instead of 31 percent. Second, better statistical systems are a major part of the solution, as statistical system strength is correlated with higher coverage. Third, poorer countries are doing no worse in reporting on gender-related Sustainable Development Goal indicators than high-income countries, despite weaker statistical systems. Lastly, sizable over (and under) performance in reporting, conditional on statistical strength, suggests that country-level advocacy and focus can yield wins in Sustainable Development Goal gender indicator coverage.Publication Updating Poverty Estimates at Frequent Intervals in the Absence of Consumption Data : Methods and Illustration with Reference to a Middle-Income Country(World Bank Group, Washington, DC, 2014-09)Obtaining consistent estimates on poverty over time as well as monitoring poverty trends on a timely basis is a priority concern for policy makers. However, these objectives are not readily achieved in practice when household consumption data are neither frequently collected, nor constructed using consistent and transparent criteria. This paper develops a formal framework for survey-to-survey poverty imputation in an attempt to overcome these obstacles, and to elevate the discussion of these methods beyond the largely ad-hoc efforts in the existing literature. The framework introduced here imposes few restrictive assumptions, works with simple variance formulas, provides guidance on the selection of control variables for model building, and can be generally applied to imputation either from one survey to another survey with the same design, or to another survey with a different design. Empirical results analyzing the Household Expenditure and Income Survey and the Unemployment and Employment Survey in Jordan are quite encouraging, with imputation-based poverty estimates closely tracking the direct estimates of poverty.Publication Technical Assessment of Open Data Platforms for National Statistical Organisations(World Bank, Washington, DC, 2014-10-18)The term quot;open dataquot; is generally understood to be data that are made available to the public free of charge, without registration or restrictive licenses, for any purpose whatsoever (including commercial purposes), in electronic, machine-readable formats that ensure data are easy to find, download and use. National Statistics Offices (NSOs) have the potential to play a pivotal role in the implementation of open data initiatives. As producers and curators of data, the objective of making high quality data more accessible and usable is consistent with their guiding principles. NSOs indicate, in research conducted in support of this report, that one of the difficulties they encounter is that the technology they use to publish - or electronically distribute - data for public use is not compatible with open formats. They also indicate that common software packages used for open data portals do not accommodate the data formats and metadata they produce. Two key concerns related to data dissemination products are addresses: (1) Can such products designed primarily for NSOs satisfy requirements for an open data initiative?; and (2) Can such products designed primarily for open data satisfy the requirements of NSOs? Furthermore, data reuse, both by data experts and the public at large, is key to creating new opportunities and benefits from government data. The following recommendations are made to improve the overall utility of data publication platforms to NSOs and the open data community: improve technical documentation; ensure public Application Programming Interfaces (APIs) and endpoints are interoperable; presentation of metadata and Uniform Resource Identifiers (URIs) must conform to W3C standards; natural language search and metadata faceting should be standard; structural metadata and hypercube support are core NSO requirements; dashboards and visualisations are necessary for user engagement; and develop data engagement tools for improving data-quality and reuse.Publication Reviewing Assessment Tools for Measuring Country Statistical Capacity(Washington, DC: World Bank, 2024-03-11)Country statistical capacity is increasingly recognized as crucial for development, but no academic study exists that reviews the available assessment tools. This paper offers the first review study that fills this gap, paying particular attention to data and practical measurement challenges. It compares the World Bank’s recently developed Statistical Performance Indicators and Index with other widely used indexes, such as the Open Data Inventory index, the Global Data Barometer index, and other regional and self-assessment tools. The findings show that each index brings advantages in data sources, number of indicators, measurement focus, coverage of countries and time periods, and correlation with common development indexes. The Open Data Inventory index covers the most countries, the Global Data Barometer index collects data through its surveys, and the Statistical Performance Indicators and Index offer a broader framework for assessing statistical systems. The paper offers further thoughts on the potential mechanisms through which these tools can bring positive impacts on economic activities and some political economy concerns, as well as future directions for development.
Users also downloaded
Showing related downloaded files
Publication Kyrgyz Republic Country Climate and Development Report(Washington, DC: World Bank, 2025-11-03)This Country Climate and Development Report (CCDR) on the Kyrgyz Republic aims to support the country’s development goals amid a changing climate. The CCDR considers two policy scenarios up to 2050: the business-as-usual (BAU) and high-growth scenarios. As it quantifies the likely impacts of climate change on the Kyrgyz economy between now and 2050, the report highlights key government actions to best prepare for and adapt to climate impacts (referred to as “with adaptation” measures), with a particular focus on the time horizon up to 2030. The CCDR also outlines a path to net zero emissions by 2050 (referred to as “with mitigation” measures, “decarbonization,” or, simply, “net zero 2050”), highlighting associated development co-benefits.Publication Mongolia Country Climate and Development Report(Washington, DC: World Bank, 2024-10-22)Mongolia’s development prospects are uniquely challenged by both the impacts of climate change and the global shift toward a low-carbon economy. The country’s efforts toward decarbonization pose significant challenges given the structurally high-emission intensity of its economy. While challenging, climate action also presents Mongolia with opportunities to achieve important development benefits. The effects of climate risks and the shift away from coal will have diverse impacts across different regions, communities, and socioeconomic levels. The report assesses the critical interconnections between Mongolia’s development ambitions and climate change action and identifies ways to transition to a more economically diversified, inclusive, and resilient development path. It highlights key climate and transition risks affecting Mongolia’s future development and presents a pathway to enhance climate mitigation and adaptation. The report also makes a case for strengthening policies to enhance resilience to climate change and ensure a just transition, particularly for the most vulnerable. The report is structured as follows: section 1 gives introduction. Section 2 delves into the linkages between development and climate in Mongolia and presents model-based findings on the economic and poverty impacts of climate change under different scenarios. Section 3 covers four in-depth sectoral analyses. The first two mainly focus on adaptation to climate change in the agriculture and water sectors. The third considers prospects for the extraction sector, while the fourth sectoral analysis focuses on decarbonizing power and heat generation. Section 4 shifts the focus to how the government can boost resilience for climate-vulnerable populations. Section 5 outlines options for mobilizing private and public financing and private investments to support the green transition. Section 6 examines the existing institutional and governance structure for climate action and presents recommendations to improve its effectiveness, and section 7 concludes with a framework for prioritizing the policy actions outlined in this report.Publication Guinea-Bissau Country Climate and Development Report(Washington, DC: World Bank, 2024-10-23)Guinea-Bissau is endowed with a wealth of natural resources, with the highest natural capital per capita in West Africa (US3,874 dollars per capita), which could be leveraged for sustainable and resilient growth. However, Guinea-Bissau faces significant development hurdles, such as high poverty rates, political instability, and economic challenges, including an over-reliance on cashew nuts. Rural poverty has increased, and the nation's infrastructure, education, and health care systems are underdeveloped. Climate change poses a severe threat, potentially impacting agriculture, fisheries, and infrastructure. Without adaptation, it could lead to a significant cut in real GDP per capita (minus 7.3 percent by 2050) and increase in poverty (with up to over 200,000 additional poor by 2050, that is, 5 percent of the expected population, in the worst scenario). The country's low greenhouse gas emissions are expected to rise, mainly due to agriculture and land-use changes, with deforestation being a major contributing factor. Although Guinea-Bissau is a low emitter, it has high mitigation ambitions, targeting a 30 percent reduction in greenhouse gas emissions by 2030. The Nationally Determined Contribution outlines significant climate actions, with initiatives focused on forest conservation, sustainable agriculture, and community development. However, the country's political instability, institutional weaknesses, and limited financial resources pose challenges to implementing these climate commitments, which depend heavily on external funding. The financial sector's underdevelopment and vulnerability to external shocks limit its ability to support green investments, though reforms could enhance resilience. Guinea-Bissau must consider its climate financing as development financing and vice-versa, engage the private sector, and integrate climate goals with national development plans to ensure a sustainable future. Concessional climate financing is vital due to the underdeveloped financial sector and the government’s limited borrowing capacity. Addressing Guinea-Bissau's vulnerability to climate change and its structural issues requires a cohesive approach that integrates development and climate strategies. This could involve improving governance, diversifying the economy, protecting natural capital, developing human capital, and investing in sustainable agriculture and infrastructure. The transition to a more sustainable and inclusive development pathway that supports economic growth is possible, but requires focusing on key strategic sectors, enhancing institutional capacity, and creating the conditions to mobilize finance. As a highly vulnerable country, there are myriad needs in the different sectors; however, to be more efficient and effective, Guinea-Bissau should prioritize actions in a few sectors, especially actions on biodiversity, agriculture, and social protection. Low carbon development, especially in energy and forestry sectors, could provide cost-efficient solutions and attract climate finance, including from the private sector, which will support the overall development agenda.Publication Jobs in a Changing Climate: Insights from World Bank Group Country Climate and Development Reports Covering 93 Economies(Washington, DC: World Bank, 2025-11-05)The World Bank Group’s Country Climate and Development Reports (CCDRs) provide a crosscutting look at how countries’ development prospects, and the job opportunities they offer to their people, can be threatened by climate impacts and supported by climate policies. Climate change and policies affect jobs through impacts on productivity, energy and material efficiency, and physical, human, and natural capital. They can also transform employment opportunities, especially through complementary measures that help workers and firms adapt to and benefit from new technologies and production practices. Prepared by the World Bank, the International Finance Corporation (IFC), and the Multilateral Investment Guarantee Agency (MIGA), CCDRs integrate country perspectives, climate science and economic modeling, private sector information, and policy analysis to assess how countries can successfully grow and develop their economies and create jobs despite increasing climate risks and while achieving their climate objectives and commitments. Each CCDR starts from the country’s development priorities, opportunities, and challenges, and is developed in close consultation with governments, businesses, and civil society, ensuring the recommendations reflect national priorities. By combining evidence on adaptation, resilience, and emissions pathways, CCDRs highlight where climate action can reinforce development and job creation, and where targeted policies are needed to manage risks and smooth labor market transitions. Taken together, these elements can help create local jobs, ensure economic transitions are just and inclusive, and equip workers and firms to navigate the disruptions and opportunities of a changing climate and changing technologies.Publication Comoros Country Climate and Development Report(Washington, DC: World Bank, 2025-06-18)The Union of the Comoros (The Comoros) has significant vulnerability to climate change-related risks but has considerable opportunities to strengthen preparedness and resilience against these challenges. According to the Notre Dame Global Adaptation Index, the Comoros is the 29th-most vulnerable country to climate change and the 163rd most ready to adapt (out of 191). The Comoros archipelago is exposed to many natural hazards that adversely affect the country’s natural capital, people, and physical infrastructure. In 2014, the economic cost of climate-related disasters was estimated at 5.7 million dollars annually, equivalent to 9.2 percent of Gross Domestic Product (GDP). Between 2018 and 2023, as many as 11 tropical depressions or cyclones impacted the country, with Cyclone Kenneth causing the greatest damage, equivalent to 14 percent of GDP, resulting in total economic growth falling from 3.6 percent in 2018 to 1.9 percent in 2019. More than 345,000 people (40 percent of the population) were affected by the cyclone, with 185,000 people experiencing severe impacts and 12,000 people displaced. However, there is an opportunity for the country to grow more robust and shock-responsive, and to establish pre-positioned funding mechanisms to enhance future crisis response efforts. For the Comoros, adaptation and climate-resilient development are the key climate change focus areas, with the country projected to face 836 million dollars 2050 in additional costs due to climate-related impacts. Current plans to adapt to the impacts of climate change in the Comoros include efforts to improve water management, strengthen coastal protection, and develop climate-smart agriculture practices. Given the country’s reliance on its natural resource base for economic growth and mobility, protection of these resources from climate change will be essential for promoting resilient growth and development. In addition to growing the adaptive capacity of the country’s natural resource sectors, strategic economic diversification will be important to help minimize future climate impacts, and development activities will need to be undertaken in such a way as to attract low-carbon co-benefits. The Union of the Comoros is committed to addressing climate change through its Nationally Determined Contribution (NDC) and national priorities. The country’s NDC (which was revised in 2021 for a ten-year horizon) sets ambitious targets, with a goal of reducing greenhouse gas emissions by 23 percent by 2030. The country also plans to significantly increase the share of renewable energy in its energy portfolio, reaching 33 MW by 2030. This will not only promote low-carbon development but also reduce the country’s dependency on imported oil and coal, which currently make up 95 percent of the energy mix. Additionally, the Comoros has declared its intention to increase CO2 removals by 47 percent by 2030, compared to BAU.