Publication: New Algorithm to Estimate Inequality Measures in Cross-Survey Imputation: An Attempt to Correct the Underestimation of Extreme Values
Loading...
Published
2022-04
ISSN
Date
2022-04-22
Author(s)
Editor(s)
Abstract
This paper contributes to the debate on ways to improve the calculation of inequality measures in developing countries experiencing severe budget constraints. Linear regression-based survey-to-survey imputation techniques are most frequently discussed in the literature. These are effective at estimating predictions of poverty indicators but are much less accurate with inequality indicators. To demonstrate this limited accuracy, the first part of the paper discusses several simulations using Moroccan Household Budget Surveys and Labor Force Surveys. The paper proposes a method for overcoming these limitations based on an algorithm that minimizes the sum of the squared difference between a certain number of direct estimates of an index and its empirical version obtained from the predicted values. Indeed, when comparing the estimated results with those directly estimated from the original sample, the bias is negligible. Furthermore, the inequality indices for the years for which there are only model estimates, rather than direct information on expenditures, seem to be consistent with Moroccan economic trends.
Link to Data Set
Citation
“Betti, Gianni; Molini, Vasco; Mori, Lorenzo. 2022. New Algorithm to Estimate Inequality Measures in Cross-Survey Imputation: An Attempt to Correct the Underestimation of Extreme Values. Policy Research Working Paper;10013. © World Bank. http://hdl.handle.net/10986/37329 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Using Poverty Maps to Improve the Design of Household Surveys(World Bank, Washington, DC, 2021-05)This paper proposes a new method for improving the design effect of household surveys based on a two-stage design in which the first stage clusters, or primary selection units, are stratified along administrative boundaries. Improvement of the design effect can result in more precise survey estimates (smaller standard errors and confidence intervals) or reduction of the necessary sample size, that is, a reduction in the budget needed for a survey. The proposed method is based on the availability of a previously conducted poverty mapping, that is, spatial descriptions of the distribution of poverty, which are finely disaggregated in small geographic units, such as cities, municipalities, districts, or other administrative partitions of a country that are linked to primary selection units. Such information is then used to select primary selection units with systematic sampling by introducing further implicit stratification in the survey design, to maximize the improvement of the design effect. The proposed methodology has been implemented for the new 2021 Household Budget Survey in Tunisia, conducted under a cooperation project funded by the World Bank. The underlying poverty mapping is based on the 2015 Household Budget Survey and the 2014 Population and Housing Census.Publication Appraising Cross-National Income Inequality Databases(World Bank, Washington, DC, 2015-11)In response to a growing interest in comparing inequality levels and trends across countries, several cross-national inequality databases are now available. These databases differ considerably in purpose, coverage, data sources, inclusion and exclusion criteria, and quality of documentation. A special issue of the Journal of Economic Inequality, which this paper introduces, is devoted to an assessment of the merits and shortcomings of eight such databases. Five of these sets are microdata-based: CEPALSTAT, Income Distribution Database, Luxembourg Income Study, PovcalNet, and Socio-Economic Database for Latin America and the Caribbean. Two are based on secondary sources: All the Ginis and the World Income Inequality Database; and one is generated entirely through multiple-imputation methods: the Standardized World Income Inequality Database. Although there is much agreement across these databases, there is also a nontrivial share of country/year cells for which substantial discrepancies exist. In some cases, different databases would lead users to radically different conclusions about inequality dynamics in certain countries and periods. The methodological differences that lead to these discrepancies often appear to be driven by a fundamental trade-off between a wish for broader coverage on the one hand, and for greater comparability on the other hand. These differences across databases place considerable responsibility on both producers and users: on the former, to better document and explain their assumptions and procedures, and on the latter, to understand the data they are using, rather than merely taking them as true because available.Publication Estimating Poverty in the Absence of Consumption Data : The Case of Liberia(World Bank Group, Washington, DC, 2014-09)In much of the developing world, the demand for high frequency quality household data for poverty monitoring and program design far outstrips the capacity of the statistics bureau to provide such data. In these environments, all available data sources must be leveraged. Most surveys, however, do not collect the detailed consumption data necessary to construct aggregates and poverty lines to measure poverty directly. This paper benefits from a shared listing exercise for two large-scale national household surveys conducted in Liberia in 2007 to explore alternative methodologies to estimate poverty indirectly. The first is an asset-based model that is commonly used in Demographic and Health Surveys. The second is a survey-to-survey imputation that makes use of small area estimation techniques. In addition to a standard base model, separate models are estimated for urban and rural areas and an expanded model that includes climatic variables. Special attention is paid to the inclusion of cell phones, with implications for other assets whose cost and availability may be changing rapidly. The results demonstrate substantial limitations with asset-based indexes, but also leave questions as to the accuracy and stability of imputation models.Publication Imputed Welfare Estimates in Regression Analysis(World Bank, Washington, D.C., 2004-04)The authors discuss the use of imputed data in regression analysis, in particular the use of highly disaggregated welfare indicators (from so-called "poverty maps"). They show that such indicators can be used both as explanatory variables on the right-hand side and as the phenomenon to explain on the left-hand side. The authors try out practical ways of adjusting standard errors of the regression coefficients to reflect the error introduced by using imputed, rather than actual, welfare indicators. These are illustrated by regression experiments based on data from Ecuador. For regressions with imputed variables on the left-hand side, the authors argue that essentially the same aggregate relationships would be found with either actual or imputed variables. They address the methodological question of how to interpret aggregate relationships found in such regressions.Publication Poverty in Mozambique : New Evidence from Recent Household Surveys(World Bank, Washington, DC, 2012-10)This paper has three primary objectives: (i) to investigate potential problems regarding Mozambique's most recent nationally representative household survey on poverty dynamics; (ii) to assess the robustness and reliability of official poverty statistics; and (iii) to provide alternative estimates of poverty and welfare indicators in light of the methodological and analytical issues raised in areas (i) and (ii). It is determined that at least two significant weaknesses affect the official poverty-rate estimates: measurement errors in consumption data and flaws in the methodology used to calculate poverty lines (the cost-of-basic-needs approach based on provincial food bundles with entropy correction). A number of observations appear to be affected by substantial measurement errors, which severely distort the official poverty statistics. The paper provides methods to correct the consumption distribution by recalculating poverty lines based on a single national food basket -- as opposed to the current estimates, which are based on province-specific food baskets. The revised poverty statistics differ considerably from the official estimates of poverty across provinces and are far more consistent with other poverty indicators. In addition, poverty appears to be highly concentrated in certain areas, with dramatically higher rates found in Central and Northern Mozambique, as well as in rural areas overall, compared with relatively low rates in Southern Mozambique and in the country's urban centers. These findings substantially contradict the government's official poverty figures, which appear to systematically overestimate poverty rates in Mozambique's Southern provinces and urban areas while simultaneously underestimating the prevalence of poverty in the country's Central and Northern regions and in rural areas nationwide.
Users also downloaded
Showing related downloaded files
Publication Panama Poverty and Equity Assessment 2024(Washington, DC: World Bank, 2025-02-12)Panama has been one of the fastest-growing countries in the region, with rapid economic expansion accompanied by significant poverty reduction. Driven by public and private investment as well as labor accumulation, the Panamanian economy grew by an annual average of 5.7 percent between 1990 and 2023, much higher than the regional average of 2.5 percent. This growth contributed to a significant reduction in poverty. Using the poverty line of US$6.85 per day per capita (2017 PPP), the share of Panamanians affected by poverty improved from one in two in 1989 to only one in ten lived in 2023. Nevertheless, Panama remains one of the most unequal countries in the world. While poverty in urban areas was 4.8 percent in 2023, poverty in indigenous regions (comarcas) reached 76 percent—15 times higher. Limited progress in reducing inequality, as measured by the Gini coefficient, contrasts with Panama’s achievements in other areas. Globally, Panama ranked 11th in inequality in 2000, with a Gini coefficient of 53.8. Two decades later, it ranked 8th, with a Gini coefficient of 50.9 as of 2022. This report examines Panama’s achievements and challenges in reducing poverty and inequality to inform policy options. With a special focus on the 2008–2023 period the report documents progress in poverty and equity in recent decades, highlighting access to basic services, expansion of quality jobs, improvement of human capital, and promotion of household resilience as critical policy priorities.Publication Philippines Country Climate and Development Report(World Bank, Washington, DC, 2022-11)Climate change poses major risks for development in the Philippines. Climate shocks, whether in the form of extreme weather events or slow-onset trends, will hamper economic activities, damage infrastructure, and induce deep social disruptions. Adaptation to the risks of climate change, including both extreme events and slow-onset problems, is thus critical for the Philippines. Policy inaction would impose substantial economic and human costs, especially for the poor. Adaptation cannot eliminate the costs of climate change, but it can substantially reduce them. Many adaptation responses also contribute to mitigation; conversely, many mitigation measures generate local co-benefits, such as reduced air pollution. Although the Philippines is a relatively low emitter of greenhouse gas (GHG), it can contribute to global mitigation efforts through an energy transition, including a shift away from coal. The investment costs of such adaptation measures and an energy transition are substantial but not out of reach. The Philippines Country Climate and Development Report (CCDR) comprehensively analyzes how climate change will affect the country's ability to meet its development goals and pursue green, resilient, and inclusive development. The CCDR helps identify opportunities for climate action by both the public and private sectors and prioritizes the most urgent development challenges impacted by climate change in the Philippines.Publication Poverty and Equity Assessment for El Salvador 2024(Washington, DC: World Bank, 2024-12-12)This report proposes an agenda for building on gains to re-accelerate poverty reduction among Salvadorans. The last World Bank Poverty Assessment for El Salvador, from 2015, proposed two key policy recommendations: (a) effective pro-poor spending and (b) reduction of crime and violence through better access to jobs and education. Nine years later, the authorities have managed to achieve a substantial reduction in crime and violence and have indicated an intent to build on such progress to establish a path toward an El Salvador where shared prosperity is achievable. In this report, we propose a three pillar structure to address poverty and inequality reduction: jobs, services, and social protection, with a cross-cutting set of primary conditions that articulates this structure.Publication Poverty and Shared Prosperity 2022(Washington, DC : World Bank, 2022)Poverty and Shared Prosperity 2022: Correcting Course provides the first comprehensive analysis of the pandemic’s toll on poverty in developing countries. It identifies how governments can optimize fiscal policy to help correct course. Fiscal policies offset the impact of COVID-19 on poverty in many high-income countries, but those policies offset barely one quarter of the pandemic’s impact in low-income countries and lower-middle-income countries. Improving support to households as crises continue will require reorienting protective spending away from generally regressive and inefficient subsidies and toward a direct transfer support system—a first key priority. Reorienting fiscal spending toward supporting growth is a second key priority identified by the report. Some of the highest-value public spending often pays out decades later. Amid crises, it is difficult to protect such investments, but it is essential to do so. Finally, it is not enough just to spend wisely - when additional revenue does need to be mobilized, it must be done in a way that minimizes reductions in poor people’s incomes. The report highlights how exploring underused forms of progressive taxation and increasing the efficiency of tax collection can help in this regard. Poverty and Shared Prosperity is a biennial series that reports on global trends in poverty and shared prosperity. Each report also explores a central challenge to poverty reduction and boosting shared prosperity, assessing what works well and what does not in different settings. By bringing together the latest evidence, this corporate flagship report provides a foundation for informed advocacy around ending extreme poverty and improving the lives of the poorest in every country in the world. For more information, please visit worldbank.org/poverty-and-shared-prosperity.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.