Publication: Using Survey-to-Survey Imputation to Fill Poverty Data Gaps at a Low Cost: Evidence from a Randomized Survey Experiment
Loading...
Published
2024-03-26
ISSN
Date
2024-03-27
Author(s)
Editor(s)
Abstract
Survey data on household consumption are often unavailable or incomparable over time in many low- and middle-income countries. Based on a unique randomized survey experiment implemented in Tanzania, this study offers new and rigorous evidence demonstrating that survey-to-survey imputation can fill consumption data gaps and provide low-cost and reliable poverty estimates. Basic imputation models featuring utility expenditures, together with a modest set of predictors on demographics, employment, household assets, and housing, yield accurate predictions. Imputation accuracy is robust to varying the survey questionnaire length, the choice of base surveys for estimating the imputation model, different poverty lines, and alternative (quarterly or monthly) Consumer Price Index deflators. The proposed approach to imputation also performs better than multiple imputation and a range of machine learning techniques. In the case of a target survey with modified (shortened or aggregated) food or non-food consumption modules, imputation models including food or non-food consumption as predictors do well only if the distributions of the predictors are standardized vis-à-vis the base survey. For the best-performing models to reach acceptable levels of accuracy, the minimum required sample size should be 1,000 for both the base and target surveys. The discussion expands on the implications of the findings for the design of future surveys.
Link to Data Set
Citation
“Dang, Hai-Anh; Kilic, Talip; Hlasny, Vladimir; Abanokova, Kseniya; Carletto, Calogero; Abanokova, Ksenia. 2024. Using Survey-to-Survey Imputation to Fill Poverty Data Gaps at a Low Cost: Evidence from a Randomized Survey Experiment. Policy Research Working Paper; 10738. © World Bank. http://hdl.handle.net/10986/41291 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Gender Gaps in the Performance of Small Firms: Evidence from Urban Peru(Washington, DC: World Bank, 2025-09-23)This paper estimates the gender gap in the performance of firms in Peru using representative data on both formal and informal firms. On average, informal female-led firms have lower sales, labor productivity, and profits compared to their male-led counterparts, with differences more pronounced when controlling for observable determinants of firm performance. However, gender gaps are only significant at the bottom of the performance distribution of informal firms, and these gaps disappear at the top of the distribution of informal firms and for formal firms. Possible explanations for the performance gaps at the bottom of the distribution include the higher likelihood of small, female-led firms being home-based, which is linked to lower profits, and their concentration in less profitable sectors. The paper provides suggestive evidence that household responsibilities play a key role in explaining the gender gap in firm performance among informal firms. Therefore, policies that promote access to care services or foster a more equal distribution of household activities may reduce gender productivity gaps and allow for a more efficient allocation of resources.Publication The Exposure of Workers to Artificial Intelligence in Low- and Middle-Income Countries(Washington, DC: World Bank, 2025-02-05)Research on the labor market implications of artificial intelligence has focused principally on high-income countries. This paper analyzes this issue using microdata from a large set of low- and middle-income countries, applying a measure of potential artificial intelligence occupational exposure to a harmonized set of labor force surveys for 25 countries, covering a population of 3.5 billion people. The approach advances work by using harmonized microdata at the level of individual workers, which allows for a multivariate analysis of factors associated with exposure. Additionally, unlike earlier papers, the paper uses highly detailed (4 digit) occupation codes, which provide a more reliable mapping of artificial intelligence exposure to occupation. Results within countries, show that artificial intelligence exposure is higher for women, urban workers, and those with higher education. Exposure decreases by country income level, with high exposure for just 12 percent of workers in low-income countries and 15 percent of workers in lower-middle-income countries. Furthermore, lack of access to electricity limits effective exposure in low-income countries. These results suggest that for developing countries, and in particular low-income countries, the labor market impacts of artificial intelligence will be more limited than in high-income countries. While greater exposure to artificial intelligence indicates larger potential for future changes in certain occupations, it does not equate to job loss, as it could result in augmentation of worker productivity, automation of some tasks, or both.Publication Geopolitical Risks and Trade(Washington, DC: World Bank, 2025-09-23)This paper studies the impact of geopolitical risks on international trade, using the Geopolitical Risk (GPR) index of Caldara and Iacoviello (2022) and an empirical gravity model. The impact of spikes in geopolitical risk on trade is negative, strong, and heterogeneous across sectors. The findings show that increases in geopolitical risk reduce trade by about 30 to 40 percent. These effects are equivalent to an increase of global tariffs of up to 14 percent. Services trade is most vulnerable to geopolitical risks, followed by agriculture, and the impact on manufacturing trade is moderate. These negative effects are partially mitigated by cultural and geographic proximity, as well as by the presence of trade agreements.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Imputing Poverty Indicators without Consumption Data(Washington, DC: World Bank, 2024-08-19)Accurate poverty measurement relies on household consumption data, but such data are often inadequate, outdated, or display inconsistencies over time in poorer countries. To address these data challenges, this paper employs survey-to-survey imputation to produce estimates for several poverty indicators, including headcount poverty, extreme poverty, poverty gap, near-poverty rates, as well as mean consumption levels and the entire consumption distribution. Analysis of 22 multi-topic household surveys conducted over the past decade in Bangladesh, Ethiopia, Malawi, Nigeria, Tanzania, and Viet Nam yields encouraging results. Adding household utility expenditures or food expenditures to basic imputation models with household-level demographic, employment, and asset variables could improve the probability of imputation accuracy by 0.1 to 0.4. Adding predictors from geospatial data could further increase imputation accuracy. The analysis also shows that a larger time interval between surveys is associated with a lower probability of predicting some poverty indicators, and that a better imputation model goodness-of-fit (R2) does not necessarily help. The results offer cost-saving inputs for future survey design.Publication Poverty Imputation in Contexts without Consumption Data(World Bank, Washington, DC, 2021-11)A key challenge with poverty measurement is that household consumption data are often unavailable or infrequently collected or may be incomparable over time. In a development project setting, it is seldom feasible to collect full consumption data for estimating the poverty impacts. While survey-to-survey imputation is a cost-effective approach to address these gaps, its effective use calls for a combination of both ex-ante design choices and ex-post modeling efforts that are anchored in validated protocols. This paper refines various aspects of existing poverty imputation models using 14 multi-topic household surveys conducted over the past decade in Ethiopia, Malawi, Nigeria, Tanzania, and Vietnam. The analysis reveals that including an additional predictor that captures household utility consumption expenditures—as part of a basic imputation model with household-level demographic and employment variables—provides poverty estimates that are not statistically significantly different from the true poverty rates. In many cases, these estimates even fall within one standard error of the true poverty rates. Adding geospatial variables to the imputation model improves imputation accuracy on a cross-country basis. Bringing in additional community-level predictors (available from survey and census data in Vietnam) related to educational achievement, poverty, and asset wealth can further enhance accuracy. Yet, there is within-country spatial heterogeneity in model performance, with certain models performing well for either urban areas or rural areas only. The paper provides operationally-relevant and cost-saving inputs into the design of future surveys implemented with a poverty imputation objective and suggests directions for future research.Publication Data Gaps, Data Incomparability, and Data Imputation(World Bank, Washington, DC, 2017-12)This paper reviews methods that have been employed to estimate poverty in contexts where household consumption data are unavailable or missing. These contexts range from completely missing and partially missing consumption data in cross-sectional household surveys, to missing panel household data. The paper focuses on methods that aim to compare trends and dynamic patterns of poverty outcomes over time. It presents the various methods under a common framework, with pedagogical discussion on the intuition. Empirical illustrations are provided using several rounds of household survey data from Vietnam. Furthermore, the paper provides a practical guide with detailed instructions on computer programs that can be used to implement the reviewed techniques.Publication Is Climate Change Slowing the Urban Escalator out of Poverty?(World Bank, Washington, DC, 2023-03-30)While urbanization has great potential to facilitate poverty reduction, climate shocks represent a looming threat to such upward mobility. This paper empirically analyzes the effects of climatic risks on the function of urban agglomerations to support poor households to escape from poverty. Combining household surveys with climatic datasets, the panel regression analysis for Chile, Colombia, and Indonesia finds that households in large metropolitan areas are more likely to escape from poverty, indicating better access to economic opportunities in those areas. However, the climate shocks offset such benefits of urban agglomerations, as extreme rainfalls and high flood risks significantly reduce the chance of upward mobility. The findings underscore the need to enhance resilience among the urban poor to allow them to fully utilize the benefits of urban agglomerations.Publication The Important Role of Equivalence Scales(World Bank, Washington, DC, 2020-06)Hardly any literature exists on the relationship between equivalence scales and poverty dynamics for transitional countries. This paper offers a new study on the impacts of equivalence scale adjustments on poverty dynamics in the Russian Federation, using equivalence scales constructed from subjective wealth and more than 20 waves of household panel survey data from the Russia Longitudinal Monitoring Survey. The analysis suggests that the equivalence scale elasticity is sensitive to household demographic composition. The adjustments for the equivalence of scales result in lower estimates of poverty lines. The study decomposes poverty into chronic and transient components and finds that chronic poverty is positively related to the adult scale parameter. However, chronic poverty is less sensitive to the child scale factor compared with the adult scale factor. Interestingly, the direction of income mobility might change depending on the specific scale parameters that are employed. The results are robust to different measures of chronic poverty, income expectations, reference groups, functional forms, and various other specifications.
Users also downloaded
Showing related downloaded files
Publication The Container Port Performance Index 2023(Washington, DC: World Bank, 2024-07-18)The Container Port Performance Index (CPPI) measures the time container ships spend in port, making it an important point of reference for stakeholders in the global economy. These stakeholders include port authorities and operators, national governments, supranational organizations, development agencies, and other public and private players in trade and logistics. The index highlights where vessel time in container ports could be improved. Streamlining these processes would benefit all parties involved, including shipping lines, national governments, and consumers. This fourth edition of the CPPI relies on data from 405 container ports with at least 24 container ship port calls in the calendar year 2023. As in earlier editions of the CPPI, the ranking employs two different methodological approaches: an administrative (technical) approach and a statistical approach (using matrix factorization). Combining these two approaches ensures that the overall ranking of container ports reflects actual port performance as closely as possible while also being statistically robust. The CPPI methodology assesses the sequential steps of a container ship port call. ‘Total port hours’ refers to the total time elapsed from the moment a ship arrives at the port until the vessel leaves the berth after completing its cargo operations. The CPPI uses time as an indicator because time is very important to shipping lines, ports, and the entire logistics chain. However, time, as captured by the CPPI, is not the only way to measure port efficiency, so it does not tell the entire story of a port’s performance. Factors that can influence the time vessels spend in ports can be location-specific and under the port’s control (endogenous) or external and beyond the control of the port (exogenous). The CPPI measures time spent in container ports, strictly based on quantitative data only, which do not reveal the underlying factors or root causes of extended port times. A detailed port-specific diagnostic would be required to assess the contribution of underlying factors to the time a vessel spends in port. A very low ranking or a significant change in ranking may warrant special attention, for which the World Bank generally recommends a detailed diagnostic.Publication Digital Progress and Trends Report 2023(Washington, DC: World Bank, 2024-03-05)Digitalization is the transformational opportunity of our time. The digital sector has become a powerhouse of innovation, economic growth, and job creation. Value added in the IT services sector grew at 8 percent annually during 2000–22, nearly twice as fast as the global economy. Employment growth in IT services reached 7 percent annually, six times higher than total employment growth. The diffusion and adoption of digital technologies are just as critical as their invention. Digital uptake has accelerated since the COVID-19 pandemic, with 1.5 billion new internet users added from 2018 to 2022. The share of firms investing in digital solutions around the world has more than doubled from 2020 to 2022. Low-income countries, vulnerable populations, and small firms, however, have been falling behind, while transformative digital innovations such as artificial intelligence (AI) have been accelerating in higher-income countries. Although more than 90 percent of the population in high-income countries was online in 2022, only one in four people in low-income countries used the internet, and the speed of their connection was typically only a small fraction of that in wealthier countries. As businesses in technologically advanced countries integrate generative AI into their products and services, less than half of the businesses in many low- and middle-income countries have an internet connection. The growing digital divide is exacerbating the poverty and productivity gaps between richer and poorer economies. The Digital Progress and Trends Report series will track global digitalization progress and highlight policy trends, debates, and implications for low- and middle-income countries. The series adds to the global efforts to study the progress and trends of digitalization in two main ways: · By compiling, curating, and analyzing data from diverse sources to present a comprehensive picture of digitalization in low- and middle-income countries, including in-depth analyses on understudied topics. · By developing insights on policy opportunities, challenges, and debates and reflecting the perspectives of various stakeholders and the World Bank’s operational experiences. This report, the first in the series, aims to inform evidence-based policy making and motivate action among internal and external audiences and stakeholders. The report will bring global attention to high-performing countries that have valuable experience to share as well as to areas where efforts will need to be redoubled.Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication Business Ready 2024(Washington, DC: World Bank, 2024-10-03)Business Ready (B-READY) is a new World Bank Group corporate flagship report that evaluates the business and investment climate worldwide. It replaces and improves upon the Doing Business project. B-READY provides a comprehensive data set and description of the factors that strengthen the private sector, not only by advancing the interests of individual firms but also by elevating the interests of workers, consumers, potential new enterprises, and the natural environment. This 2024 report introduces a new analytical framework that benchmarks economies based on three pillars: Regulatory Framework, Public Services, and Operational Efficiency. The analysis centers on 10 topics essential for private sector development that correspond to various stages of the life cycle of a firm. The report also offers insights into three cross-cutting themes that are relevant for modern economies: digital adoption, environmental sustainability, and gender. B-READY draws on a robust data collection process that includes specially tailored expert questionnaires and firm-level surveys. The 2024 report, which covers 50 economies, serves as the first in a series that will expand in geographical coverage and refine its methodology over time, supporting reform advocacy, policy guidance, and further analysis and research.