Publication: How Survey-to-Survey Imputation Can Fail
Loading...
Published
2014-07
ISSN
Date
2014-08-15
Author(s)
Editor(s)
Abstract
This paper proposes diagnostics to assess the accuracy of survey-to-survey imputation methods and applies them to examine why imputing from the Household Income and Expenditure Survey into the Labor Force Survey fails to accurately project poverty trends in Sri Lanka between 2006 and 2009. Survey-to-survey imputation methods rely on two key assumptions: (i) that the questions in the two surveys are asked in a consistent way and (ii) that common variables of the two surveys explain a large share of the intertemporal change in household expenditure and poverty. In addition, differences in sampling design can lead validation tests to underestimate the accuracy of survey-to-survey predictions. In Sri Lanka, the causes of failure differ across sectors. In the urban sector, the primary culprit is differences between the two surveys in the design of the questionnaire. In the rural and estate sectors, the set of common variables in the prediction model does not adequately capture changes in poverty. The paper concludes that in Sri Lanka, survey-to-survey imputation between the Household Income and Expenditure Survey and the Labor Force Survey cannot produce accurate poverty estimates unless the Labor Force Survey adds additional questions on assets and is redesigned to use a questionnaire that is compatible with the Household Income and Expenditure Survey. Alternatively, a new welfare-tracking survey that satisfies these conditions could be established.
Link to Data Set
Citation
“Newhouse, D.; Shivakumaran, S.; Takamatsu, S.; Yoshida, N.. 2014. How Survey-to-Survey Imputation Can Fail. Policy Research Working Paper;No. 6961. © http://hdl.handle.net/10986/19364 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Updating Poverty Estimates at Frequent Intervals in the Absence of Consumption Data : Methods and Illustration with Reference to a Middle-Income Country(World Bank Group, Washington, DC, 2014-09)Obtaining consistent estimates on poverty over time as well as monitoring poverty trends on a timely basis is a priority concern for policy makers. However, these objectives are not readily achieved in practice when household consumption data are neither frequently collected, nor constructed using consistent and transparent criteria. This paper develops a formal framework for survey-to-survey poverty imputation in an attempt to overcome these obstacles, and to elevate the discussion of these methods beyond the largely ad-hoc efforts in the existing literature. The framework introduced here imposes few restrictive assumptions, works with simple variance formulas, provides guidance on the selection of control variables for model building, and can be generally applied to imputation either from one survey to another survey with the same design, or to another survey with a different design. Empirical results analyzing the Household Expenditure and Income Survey and the Unemployment and Employment Survey in Jordan are quite encouraging, with imputation-based poverty estimates closely tracking the direct estimates of poverty.Publication Measuring Poverty Dynamics with Synthetic Panels Based on Cross-Sections(World Bank, Washington, DC, 2013-06)Panel data conventionally underpin the analysis of poverty mobility over time. However, such data are not readily available for most developing countries. Far more common are the “snap-shots” of welfare captured by cross-section surveys. This paper proposes a method to construct synthetic panel data from cross sections which can provide point estimates of poverty mobility. In contrast to traditional pseudo-panel methods that require multiple rounds of cross-sectional data to study poverty at the cohort level, the proposed method can be applied to settings with as few as two survey rounds and also permits investigation at the more disaggregated household level. The procedure is implemented using cross-section survey data from several countries, spanning different income levels and geographical regions. Estimates fall within the 95 percent confidence interval— or even one standard error in many cases—of those based on actual panel data. The method is not only restricted to studying poverty mobility but can also accommodate investigation of other welfare outcome dynamics.Publication Is Random Forest a Superior Methodology for Predicting Poverty?(World Bank, Washington, DC, 2016-03)Random forest is in many fields of research a common method for data driven predictions. Within economics and prediction of poverty, random forest is rarely used. Comparing out-of-sample predictions in surveys for same year in six countries shows that random forest is often more accurate than current common practice (multiple imputations with variables selected by stepwise and Lasso), suggesting that this method could contribute to better poverty predictions. However, none of the methods consistently provides accurate predictions of poverty over time, highlighting that technical model fitting by any method within a single year is not always, by itself, sufficient for accurate predictions of poverty over time.Publication Using Repeated Cross-Sections to Explore Movements in and out of Poverty(2011-01-01)Movements in and out of poverty are of core interest to both policymakers and economists. Yet the panel data needed to analyze such movements are rare. In this paper, the authors build on the methodology used to construct poverty maps to show how repeated cross-sections of household survey data can allow inferences to be made about movements in and out of poverty. They illustrate that the method permits the estimation of bounds on mobility, and provide non-parametric and parametric approaches to obtaining these bounds. They test how well the method works on data sets for Vietnam and Indonesia where we are able to compare our method to true panel estimates. The results are sufficiently encouraging to offer the prospect of some limited, basic, insights into mobility and poverty duration in settings where historically it was judged that the data necessary for such analysis were unavailable.Publication Poverty and Economic Growth in Egypt, 1995-2000(World Bank, Washington, DC, 2003-06)After a decade of slow economic growth Egypt's rate of growth recovered in the late 1990s, averaging more than five percent a year. But the effect of this growth on poverty patterns has not been systematically examined using consistent, comparable household datasets. In this paper, the authors use the rich set of unit-level data from the most recent Egyptian household surveys (1995-96 and 1999-2000) to assess changes in poverty and inequality between 1995 and 2000. Their analysis is based on household-specific poverty lines that account for the differences in regional prices, as well as differences in the consumption preferences and size and age composition of poor households. The results show that average household expenditures rose in the second half of the 1990s and the poverty rate fell from 20 percent to less than 17 percent. But, in addition to the ongoing divide in the urban-rural standard of living, a new geographical/regional divide emerged in the late 1990s. Poverty was found predominantly among less-educated individuals, particularly those working in agriculture and construction, and among seasonal and occasional workers. These groups could suffer the most from the slowing economic growth evident after 1999-2000.
Users also downloaded
Showing related downloaded files
Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Ukraine Country Environmental Analysis(World Bank, Washington, DC, 2016-01)The objective of the Country Environmental Analysis (CEA) is to assess the adequacy and performance of the policy, legal, and institutional framework for environmental management in Ukraine, in light of the decentralization process of environmental governance and wider reform objectives, and to provide recommendations to government to address the key gaps identified. Ukraine is the second largest country in Europe and has a population of 43 million, the majority of whom live in urban areas. It is a lower middle income country, with the services, industry and agriculture sectors being main contributors to the country’s Gross Domestic Product (GDP). Ukraine faces a number of environmental challenges, as identified in its National Environmental Strategy 2020 (NES). Key among these are: air pollution; quality of water resources and land degradation; solid waste management; biodiversity loss; human health issues associated with environmental risk factors; in addition to climate change. The scope of Ukrainian environmental legislation is quite broad and comprehensive (more than 300 legal acts) and covers most areas of environmental protection and natural resources management. However, the environmental legislation faces a number of weaknesses:The environmental legislation is largely declaratory in nature and does not have all the essential enforcement mechanisms for the implementation of legal acts and international agreements; Many of the acts are not coordinated with each other; and Legislation undergoes limited analysis of its impact—for example, no in-depth analysis such as Regulatory Impact Analysis is conducted for proposed pieces of legislation.Publication Thailand Monthly Economic Monitor, October 2025(Washington, DC: World Bank, 2025-10-22)Fiscal conditions remained stable, with a modest widening of the deficit to 3.1 percent of GDP. New stimulus measures are expected to support short-term demand without breaching the public debt ceiling. Inflation stayed negative, reflecting lower energy and food prices amid subdued domestic demand. The central bank kept the policy rate unchanged, citing limited policy space. Thailand’s growth momentum has slowed further as manufacturing activity and services weakened as projected. Tourism remained subdued, largely due to fewer Chinese visitors. Goods exports also slowed as earlier front-loaded orders faded, particularly in agriculture and industrial goods. The Thai baht depreciated in early October as the US dollar appreciated and the current account turned negative.Publication Regional Poverty and Inequality Update: Latin America and the Caribbean, October 2025(Washington, DC: World Bank, 2025-10-23)This brief summarizes recent facts related to poverty and inequality in Latin America and the Caribbean (LAC) using the latest wave of harmonized household surveys from the Socio-Economic Database for LAC (SEDLAC). This brief was produced by the Poverty Global Practice in the LAC Region of the World Bank.Publication Rural Employment in Africa(Washington, DC: World Bank, 2022-02-23)Africa’s rural population continues to expand rapidly and labor productivity in agriculture and many rural off farm activities remains low. This paper uses the lens of a dual economy and the associated patterns of agricultural, rural, and structural transformation to review the evolution of Africa’s rural employment and its inclusiveness. Many African countries still find themselves in an early stage of the agricultural and rural transformation. Given smaller sectoral productivity gaps than commonly assumed, greater size effects and larger spillovers, investment in agriculture and the rural off-farm economy remains warranted to broker the transition to more and more productive rural employment. The key policy questions thus become how best to invest in the agri-food system (on and increasingly also off the farm) and how best to generate demand for nonagricultural goods and services which rural households can competitively produce. Informing these choices continues to present a major research agenda, with digitization, the imperative of greening and intra-African liberalization raising many unarticulated and undocumented opportunities and challenges.