Publication: Correcting Sampling and Nonresponse Bias in Phone Survey Poverty Estimation Using Reweighting and Poverty Projection Models
Loading...
Date
2024-01-03
ISSN
Published
2024-01-03
Author(s)
Editor(s)
Abstract
To monitor the evolution of household living conditions during the COVID-19 pandemic, the World Bank conducted COVID-19 High-Frequency Phone Surveys in around 80 countries. Phone surveys are cheap and easy to implement, but they have some major limitations, such as the absence of poverty data, sampling bias due to incomplete telephone coverage in many developing countries, and frequent nonresponses to phone interviews. To overcome these limitations, the World Bank conducted pilots in 20 countries where the Survey of Wellbeing via Instant and Frequent Tracking, a rapid poverty monitoring tool, was adopted to estimate poverty rates based on 10 to 15 simple questions collected via phone interviews, and where sampling weights were adjusted to correct the sampling and nonresponse bias. This paper examines whether reweighting procedures and the Survey of Wellbeing via Instant and Frequent Tracking methodology can eliminate the bias in poverty estimation based on the COVID-19 High-Frequency Phone Surveys. Experiments using artificial phone survey samples show that (i) reweighting procedures cannot fully eliminate bias in poverty estimates, as previous research has demonstrated, but (ii) when combined with Survey of Wellbeing via Instant and Frequent Tracking poverty projections, they effectively eliminate bias in poverty estimates and other statistics.
Link to Data Set
Citation
“Zhang, Kexin; Takamatsu, Shinya; Yoshida, Nobuo. 2024. Correcting Sampling and Nonresponse Bias in Phone Survey Poverty Estimation Using Reweighting and Poverty Projection Models. Policy Research Working Papers; 10656. © World Bank. http://hdl.handle.net/10986/40829 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Asymmetric Bank Distress Amplifier of Recessions(Washington, DC: World Bank, 2025-07-11)One defining feature of financial crises, evident in U.S. and international data, is asymmetric bank distress—concentrated losses on a subset of banks. This paper proposes a model in which shocks to borrowers’ productivity dispersion lead to asymmetric bank losses. The framework exhibits a “bank distress amplifier,” exacerbating economic downturns by causing costly bank failures and raising uncertainty about the solvency of banks, thereby pushing banks to deleverage. Quantitative analysis shows that the bank distress amplifier doubles investment decline and increases the spread by 2.5 times during the Great Recession compared to a standard financial accelerator model. The mechanism helps explain how a seemingly small shock can sometimes trigger a large crisis.Publication From Tailwinds to Headwinds(Washington, DC: World Bank, 2025-07-10)The first quarter of the twenty-first century has been transformative for emerging market and developing economies (EMDEs). These economies now account for about 45 percent of global GDP, up from about 25 percent in 2000, a trend driven by robust collective growth in the three largest EMDEs—China, India, and Brazil (the EM3). Collectively, EMDEs have contributed about 60 percent of annual global growth since 2000, on average, double the share during the 1990s. Their ascendance was powered by swift global trade and financial integration, especially during the first decade of the century. Interdependence among these economies has also increased markedly. Today, nearly half of goods exports from EMDEs go to other EMDEs, compared to one-quarter in 2000. As cross-border linkages have strengthened, business cycles among EMDEs and between EMDEs and advanced economies have become more synchronized, and a distinct EMDE business cycle has emerged. Cross-border business cycle spillovers from the EM3 to other EMDEs are sizable, at about half of the magnitude of spillovers from the largest advanced economies (the United States, the euro area, and Japan). Yet EMDEs confront a host of headwinds at the turn of the second quarter of the century. Progress implementing structural reforms in many of these economies has stalled. Globally, protectionist measures and geopolitical fragmentation have risen sharply. High debt burdens, demographic shifts, and the rising costs of climate change weigh on economic prospects. A successful policy approach to accelerate growth and development should focus on boosting investment and productivity, navigating a difficult external environment, and enhancing macroeconomic stability.Publication Intergenerational Income Mobility around the World(Washington, DC: World Bank, 2025-07-09)This paper introduces a new global database with estimates of intergenerational income mobility for 87 countries, covering 84 percent of the world’s population. This marks a notable expansion of the cross-country evidence base on income mobility, particularly among low- and middle-income countries. The estimates indicate that the negative association between income mobility and inequality (known as the Great Gatsby Curve) continues to hold across this wider range of countries. The database also reveals a positive association between income mobility and national income per capita, suggesting that countries achieve higher levels of intergenerational mobility as they grow richer.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Prospects of Estimating Poverty with Phone Surveys(World Bank, Washington, DC, 2017-10)Telephone surveys enable us to collect data in a cost-effective and timely manner, but may not be conducive for collecting detailed consumption or income data for measuring poverty due to the required length of the interview and complexity of the questions. Combining telephone surveys with a survey-to-survey imputation technique may be a solution, as this technique can produce reliable poverty estimates from only 10 to 20 simple questions. However, this approach may lead to biased results if the interview mode, that is, face-to-face versus telephone interviews, affects how households respond to questions. By conducting the first survey experiment to examine potential differences in poverty estimates between interview modes, this study finds that the reporting patterns changed very little between the two interview modes, and the bias in poverty estimates due to interview mode is statistically insignificant. These findings suggest that poverty monitoring via telephone surveys is promising, but additional experiments in other country contexts are encouraged.Publication Reducing Bias in Phone Survey Samples(World Bank, Washington, DC, 2021-05)Several developing countries are currently implementing phone surveys in response to immediate data needs to monitor the socioeconomic impact of COVID-19. However, phone surveys are often subject to coverage and non-response bias that can compromise the representativeness of the sample and the external validity of the estimates obtained from the survey. Using data from high-frequency phone surveys in Ethiopia, Malawi, Nigeria, and Uganda, this study investigates the magnitude and source of biases present in these four surveys and explores the effectiveness of techniques applied to reduce bias. Varying levels of coverage and non-response bias are found in all four countries. The successfully contacted samples in these four countries were biased toward wealthier households with higher living standards. Left unaddressed, this bias would result in biased estimates from the interviewed sample that do not fully reflect the situation of poorer households in the country. However, phone survey biases can be substantially reduced by applying survey weight adjustments using information from the representative survey from which the sample is drawn. Applying these methods to the four surveys resulted in a substantial reduction in bias, although the bias was not fully eradicated. This highlights one of the potential advantages of drawing phone survey samples from existing face-to-face, representative surveys over random digit dialing or using lists from telecom providers where such adjustment methods can be more limited.Publication Rapid Consumption Method and Poverty and Inequality Estimation in Somalia Revisited(Washington, DC: World Bank, 2022-03)This paper presents updated poverty and inequality estimates from the Somalia High Frequency Survey. This survey used the Rapid Consumption Method to collect consumption data quickly in an environment of high insecurity. Its poverty estimation, therefore, requires imputation of skipped consumption modules. Previous poverty estimates did not properly impute consumption, resulting in the imputation of negative total consumption values for some households. This paper uses the Two-Part Multiple Imputation method to address this issue. The assessment of module-level prediction performance demonstrates that the Two-Part Multiple Imputation handles this issue effectively. In addition, this paper adopts the newly updated 2011 purchasing power parities to convert the High Frequency Survey consumption data for global poverty measurement purposes. Lastly, this paper provides new inequality measures to address issues with the previous exercise. The paper finds that new poverty rates are slightly lower than those using the previous method while inequality is higher with the new method.Publication Rapid Consumption Method and Poverty and Inequality Estimation in South Sudan Revisited(World Bank, Washington, DC, 2021-10)This paper presents updated poverty and inequality estimates from the South Sudan High Frequency Survey (HFS) consumption data. The HFS uses the Rapid Consumption Methodology (RCM), which skips part of consumption module, to save interview time due to the volatile security situation. The previous methodology adopted the Multivariate Normal Regression (MI-MVN) method to impute the skipped consumption data, but it produced improper consumption data like negative total consumption values for some households. Instead, the new methodology uses the Two-Part multiple imputation (MI) method, and improved the reliability of imputed consumption data, although there is still room for improvement. In addition, the new methodology adopts the latest consumer price index (CPI) and purchasing power parities (PPPs). Lastly, this paper updates the inequality estimates, which the previous method overestimated. As a result of all the above adjustments, South Sudan's national poverty headcount rate in 2016-17 is 76.4 percent, which is 5.6 percentage points lower than the previous estimate of 82 percent. Inequality, as measured by the national Gini coefficient, is 44.1 percent, around 3 percentage points higher than the previous estimate of 41.0 percent.Publication How Survey-to-Survey Imputation Can Fail(World Bank Group, Washington, DC, 2014-07)This paper proposes diagnostics to assess the accuracy of survey-to-survey imputation methods and applies them to examine why imputing from the Household Income and Expenditure Survey into the Labor Force Survey fails to accurately project poverty trends in Sri Lanka between 2006 and 2009. Survey-to-survey imputation methods rely on two key assumptions: (i) that the questions in the two surveys are asked in a consistent way and (ii) that common variables of the two surveys explain a large share of the intertemporal change in household expenditure and poverty. In addition, differences in sampling design can lead validation tests to underestimate the accuracy of survey-to-survey predictions. In Sri Lanka, the causes of failure differ across sectors. In the urban sector, the primary culprit is differences between the two surveys in the design of the questionnaire. In the rural and estate sectors, the set of common variables in the prediction model does not adequately capture changes in poverty. The paper concludes that in Sri Lanka, survey-to-survey imputation between the Household Income and Expenditure Survey and the Labor Force Survey cannot produce accurate poverty estimates unless the Labor Force Survey adds additional questions on assets and is redesigned to use a questionnaire that is compatible with the Household Income and Expenditure Survey. Alternatively, a new welfare-tracking survey that satisfies these conditions could be established.
Users also downloaded
Showing related downloaded files
Publication Global Economic Prospects, January 2024(Washington, DC: World Bank, 2024-01-09)Note: Chart 1.2.B has been updated on January 18, 2024. Chart 2.2.3 B has been updated on January 14, 2024. Global growth is expected to slow further this year, reflecting the lagged and ongoing effects of tight monetary policy to rein in inflation, restrictive credit conditions, and anemic global trade and investment. Downside risks include an escalation of the recent conflict in the Middle East, financial stress, persistent inflation, weaker-than-expected activity in China, trade fragmentation, and climate-related disasters. Against this backdrop, policy makers face enormous challenges. In emerging market and developing economies (EMDEs), commodity exporters face the enduring challenges posed by fiscal policy procyclicality and volatility, which highlight the need for robust fiscal frameworks. Across EMDEs, previous episodes of investment growth acceleration underscore the critical importance of macroeconomic and structural policies and an enabling institutional environment in bolstering investment and long-term growth. At the global level, cooperation needs to be strengthened to provide debt relief, facilitate trade integration, tackle climate change, and alleviate food insecurity.Publication Digital Progress and Trends Report 2023(Washington, DC: World Bank, 2024-03-05)Digitalization is the transformational opportunity of our time. The digital sector has become a powerhouse of innovation, economic growth, and job creation. Value added in the IT services sector grew at 8 percent annually during 2000–22, nearly twice as fast as the global economy. Employment growth in IT services reached 7 percent annually, six times higher than total employment growth. The diffusion and adoption of digital technologies are just as critical as their invention. Digital uptake has accelerated since the COVID-19 pandemic, with 1.5 billion new internet users added from 2018 to 2022. The share of firms investing in digital solutions around the world has more than doubled from 2020 to 2022. Low-income countries, vulnerable populations, and small firms, however, have been falling behind, while transformative digital innovations such as artificial intelligence (AI) have been accelerating in higher-income countries. Although more than 90 percent of the population in high-income countries was online in 2022, only one in four people in low-income countries used the internet, and the speed of their connection was typically only a small fraction of that in wealthier countries. As businesses in technologically advanced countries integrate generative AI into their products and services, less than half of the businesses in many low- and middle-income countries have an internet connection. The growing digital divide is exacerbating the poverty and productivity gaps between richer and poorer economies. The Digital Progress and Trends Report series will track global digitalization progress and highlight policy trends, debates, and implications for low- and middle-income countries. The series adds to the global efforts to study the progress and trends of digitalization in two main ways: · By compiling, curating, and analyzing data from diverse sources to present a comprehensive picture of digitalization in low- and middle-income countries, including in-depth analyses on understudied topics. · By developing insights on policy opportunities, challenges, and debates and reflecting the perspectives of various stakeholders and the World Bank’s operational experiences. This report, the first in the series, aims to inform evidence-based policy making and motivate action among internal and external audiences and stakeholders. The report will bring global attention to high-performing countries that have valuable experience to share as well as to areas where efforts will need to be redoubled.Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication The Container Port Performance Index 2023(Washington, DC: World Bank, 2024-07-18)The Container Port Performance Index (CPPI) measures the time container ships spend in port, making it an important point of reference for stakeholders in the global economy. These stakeholders include port authorities and operators, national governments, supranational organizations, development agencies, and other public and private players in trade and logistics. The index highlights where vessel time in container ports could be improved. Streamlining these processes would benefit all parties involved, including shipping lines, national governments, and consumers. This fourth edition of the CPPI relies on data from 405 container ports with at least 24 container ship port calls in the calendar year 2023. As in earlier editions of the CPPI, the ranking employs two different methodological approaches: an administrative (technical) approach and a statistical approach (using matrix factorization). Combining these two approaches ensures that the overall ranking of container ports reflects actual port performance as closely as possible while also being statistically robust. The CPPI methodology assesses the sequential steps of a container ship port call. ‘Total port hours’ refers to the total time elapsed from the moment a ship arrives at the port until the vessel leaves the berth after completing its cargo operations. The CPPI uses time as an indicator because time is very important to shipping lines, ports, and the entire logistics chain. However, time, as captured by the CPPI, is not the only way to measure port efficiency, so it does not tell the entire story of a port’s performance. Factors that can influence the time vessels spend in ports can be location-specific and under the port’s control (endogenous) or external and beyond the control of the port (exogenous). The CPPI measures time spent in container ports, strictly based on quantitative data only, which do not reveal the underlying factors or root causes of extended port times. A detailed port-specific diagnostic would be required to assess the contribution of underlying factors to the time a vessel spends in port. A very low ranking or a significant change in ranking may warrant special attention, for which the World Bank generally recommends a detailed diagnostic.Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.