Publication: Representativeness of Individual-Level Data in COVID-19 Phone Surveys: Findings from Sub-Saharan Africa
Loading...
Published
2021-05-13
ISSN
Date
2021-05-20
Author(s)
Editor(s)
Abstract
The COVID-19 pandemic has created urgent demand for timely data, leading to a surge in mobile phone surveys for tracking the impacts of and responses to the pandemic. This paper assesses, and attempts to mitigate, selection biases in individual-level analyses based on phone survey data. The research uses data from (i) national phone surveys that have been implemented in Ethiopia, Malawi, Nigeria, and Uganda during the pandemic, and (ii) the pre-COVID-19 national face-to-face surveys that served as the sampling frames for the phone surveys. The availability of pre-COVID-19 face-to-face survey data permits comparisons of phone survey respondents with the general adult population. Phone survey respondents are more likely to be household heads or their spouses and non-farm enterprise owners, and on average, are older and better educated vis-à-vis the general adult population. To improve the representativeness of individual-level phone survey data, the household-level phone survey sampling weights are calibrated based on propensity score adjustments that are derived from a model of an individual’s likelihood of being interviewed as a function of individual- and household-level attributes. Reweighting improves the representativeness of the estimates for the phone survey respondents, moving them closer to those of the general adult population. This holds for women and men and a range of demographic, education, and labor market outcomes. However, reweighting increases the variance of the estimates and fails to overcome selection biases. Obtaining reliable data on men and women through phone surveys requires random selection of adult interviewees within sampled households.
Link to Data Set
Citation
“Brubaker, Joshua; Kilic, Talip; Wollburg, Philip. 2021. Representativeness of Individual-Level Data in COVID-19 Phone Surveys: Findings from Sub-Saharan Africa. Policy Research Working Paper;No. 9660. © World Bank. http://hdl.handle.net/10986/35609 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Labor Demand in the Age of Generative AI: Early Evidence from the U.S. Job Posting Data(Washington, DC: World Bank, 2025-11-18)This paper examines the causal impact of generative artificial intelligence on U.S. labor demand using online job posting data. Exploiting ChatGPT’s release in November 2022 as an exogenous shock, the paper applies difference-in-differences and event study designs to estimate the job displacement effects of generative artificial intelligence. The identification strategy compares labor demand for occupations with high versus low artificial intelligence substitution vulnerability following ChatGPT’s launch, conditioning on similar generative artificial intelligence exposure levels to isolate substitution effects from complementary uses. The analysis uses 285 million job postings collected by Lightcast from the first quarter of 2018 to the second quarter of 2025Q2. The findings show that the number of postings for occupations with above-median artificial intelligence substitution scores fell by an average of 12 percent relative to those with below-median scores. The effect increased from 6 percent in the first year after the launch to 18 percent by the third year. Losses were particularly acute for entry-level positions that require neither advanced degrees (18 percent) nor extensive experience (20 percent), as well as those in administrative support (40 percent) and professional services (30 percent). Although generative artificial intelligence generates new occupations and enhances productivity, which may increase labor demand, early evidence suggests that some occupations may be less likely to be complemented by generative artificial intelligence than others.Publication The Lasting Effects of Working while in School(Washington, DC: World Bank, 2025-08-18)This paper provides the first experimental evidence on the long-term effects of work-study programs, leveraging a randomized lottery design from a national program in Uruguay. Participation leads to a persistent 11 percent increase in formal labor earnings, observable seven years after the program. Effects are stronger for youth who participate during pivotal educational transitions and are larger for vulnerable youth and men, while remaining positive for women and non-vulnerable youth. The program is highly cost-effective, with average impacts exceeding those of job training programs and comparable to early childhood investments.Publication It’s Not (Just) the Tariffs: Rethinking Non-Tariff Measures in a Fragmented Global Economy(Washington, DC: World Bank, 2025-10-22)As tariffs have declined, non-tariff measures (NTMs) have become central to trade policy, especially in high-income countries and regulated sectors like food and green technologies. Although NTMs may serve legitimate goals, they could also sort countries and firms into or out of markets based on compliance capacity and differences in product mix. Documenting recent advances in the estimation of ad valorem equivalents (AVEs), this paper uncovers new patterns of use and exposure of NTMs. High-income countries rely more heavily on NTMs relative to tariffs, while low- and middle-income countries face steeper AVEs on their exports. Firm-level evidence shows that NTMs disproportionately affect smaller firms, leading to market exit and concentration. Poorly designed NTMs can harm productivity and welfare, while coordinated, capacity-aware use can deliver inclusive outcomes. Policy design, transparency, and diagnostics must evolve to reflect the growing role—and risks—of NTMs in a fragmented global trade landscape.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Reducing Bias in Phone Survey Samples(World Bank, Washington, DC, 2021-05)Several developing countries are currently implementing phone surveys in response to immediate data needs to monitor the socioeconomic impact of COVID-19. However, phone surveys are often subject to coverage and non-response bias that can compromise the representativeness of the sample and the external validity of the estimates obtained from the survey. Using data from high-frequency phone surveys in Ethiopia, Malawi, Nigeria, and Uganda, this study investigates the magnitude and source of biases present in these four surveys and explores the effectiveness of techniques applied to reduce bias. Varying levels of coverage and non-response bias are found in all four countries. The successfully contacted samples in these four countries were biased toward wealthier households with higher living standards. Left unaddressed, this bias would result in biased estimates from the interviewed sample that do not fully reflect the situation of poorer households in the country. However, phone survey biases can be substantially reduced by applying survey weight adjustments using information from the representative survey from which the sample is drawn. Applying these methods to the four surveys resulted in a substantial reduction in bias, although the bias was not fully eradicated. This highlights one of the potential advantages of drawing phone survey samples from existing face-to-face, representative surveys over random digit dialing or using lists from telecom providers where such adjustment methods can be more limited.Publication COVID-19 High Frequency Phone Surveys in Latin America(World Bank, Washington, DC, 2021-09)The Coronavirus disease 2019 (COVID-19) high-frequency phone survey (HFPS) 2020 was conducted in 13 Latin American countries: Argentina, Bolivia, Chile, Colombia, Costa Rica, Dominican Republic, Ecuador, El Salvador, Guatemala, Honduras, Mexico, Paraguay, and Peru. It followed a panel sample over three waves of data collection in 12 countries and over four waves in Ecuador. All waves spanned from May to August 2020 and each wave’s collection period lasted about ten days on average. The survey was administered to one adult per household. Each respondent was presented with both individual and household-level questions. All national samples were based on a dual frame of cell and landline phones, and selected as a one-stage probability sample, with geographic stratification of landline numbers. The samples were generated through a random digit dialing (RDD) process covering all cell and landline telephone numbers active at the time of the sample selection. Survey estimates represent households with a landline or at least one cell phone and individuals of 18 years of age or above who have an active cell phone number or a landline at home.Publication Challenges and Opportunities of Mobile Phone-Based Data Collection : Evidence from South Sudan(World Bank, Washington, DC, 2013-01)The proliferation of mobile phones in developing countries has generated a wave of interest in collecting high-frequency socioeconomic surveys using this technology. This paper considers lessons from one such survey effort in a difficult environment -- the South Sudan Experimental Phone Survey, which gathered data on living conditions, access to services, and citizen attitudes via monthly interviews by phones provided to respondents. Non-response, particularly in later rounds of the survey, was a substantial problem, largely due to erratic functioning of the mobile network. However, selection due to non-response does not appear to have markedly affected survey results. Response rates were much higher for respondents who owned their own phones. Both compensation provided to respondents in the form of airtime and the type of phone (solar-charged or traditional) were varied experimentally. The type of phone was uncorrelated with response rates and, contrary to expectation, attrition was slightly higher for those receiving the higher level of compensation. The South Sudan Experimental Phone Survey experience suggests that mobile phones can be a viable means of data collection for some purposes, that calling people on their own phones is preferred to handing out phones, and that careful attention should be given to the potential for selective non-response.Publication Economic Sentiments and Expectations in Sub-Saharan Africa in a Time of Multiple Shocks(Washington, DC: World Bank, 2023-11-21)Against the background of high inflation, climate shocks, and concerns about rising food insecurity, this study documents the state of economic sentiments and expectations of households in five African countries—Burkina Faso, Ethiopia, Malawi, Nigeria, and Uganda—that are home to 36 percent of the Sub-Saharan African population. Leveraging nationally representative phone survey data, 57 percent of households across the five countries report that their financial situation and their country’s economic situation have worsened significantly in the past 12 months. While expectations for the future are more positive, there are marked differences across countries that suggest uneven recovery prospects and nonnegligible uncertainty about the future. Households overwhelmingly report prices to have increased considerably over the past 12 months and expect prices to increase faster, or at the same rate, over the next 12 months. Close to 54 percent of households—home to 206 million individuals—further expect that climate shocks will have adverse impacts on their finances in the next year. Economic sentiments are closely related to livelihood outcomes such as food insecurity, lack of access to staple foods, income loss, and unemployment, and sentiments about the household financial situation, country economic situation, price increases, and climate shocks are also interdependent. Households whose financial situation has worsened in the past year are consistently more pessimistic about their financial future. Food insecure households, in particular, are not only more likely to report a worsening financial situation in the recent past and pessimism about the future, but also more likely to expect to be adversely impacted by climate shocks.Publication Linking Representative Household Models with Household Surveys for Poverty Analysis: A Comparison of Alternative Methodologies(World Bank, Washington, D.C., 2004-06)The authors compare three approaches to linking representative-household macro models with micro household income data in terms of their implications for measuring the poverty and distributional effects of policy shocks. These approaches are a simple micro-accounting method, an extension of that method to account for changes in employment structure, and the Beta distribution approach. Even though in the authors simulation exercises the three methods do not lead to fundamentally different results in absolute terms, they show that potential differences in the measurement of distributional and poverty effects of policy shocks can be very large.
Users also downloaded
Showing related downloaded files
Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.Publication The Container Port Performance Index 2020 to 2024: Trends and Lessons Learned(Washington, DC: World Bank, 2025-09-22)The Container Port Performance Index (CPPI) provides a global benchmark of how container ports perform in handling vessel calls. Developed jointly by the World Bank and S&P Global Market Intelligence, it measures the time ships spend in port and relates this to the number of containers moved during that time. This approach makes the CPPI a unique diagnostic tool that can highlight patterns in port operations and shed light on global and regional supply chain dynamics. Now in its fifth edition, the CPPI report covers the period from 2020 to 2024. It builds on a well-established methodology to generate scores for more than 400 container ports worldwide. Over time, the CPPI has become a trusted reference point for policymakers, industry stakeholders, and researchers who seek to understand how ports adapt to shocks, recover from disruptions, and identify opportunities for investments, reform and modernization. A major innovation in this edition is the introduction of multi-year trend analysis. Rather than presenting annual snapshots, the report now tracks how CPPI scores have changed across five years. This longitudinal perspective reveals shifts in port performance, showing where scores have risen, fallen, or remained stable. By linking these movements to external factors, the CPPI offers insights into how global and regional supply chains evolve under pressure. The results clearly mirror the crises that have shaken global trade. During the COVID-19 pandemic, CPPI scores in different regions declined sharply as congestion, equipment shortages, and delays overwhelmed many ports. By 2023, global averages rebounded in parallel with easing freight markets and reduced congestion. Yet 2024 brought new challenges: the Red Sea crisis disrupted major trade lanes, while climate-related constraints at the Panama Canal added further stress. These shocks were reflected in lower global and several regional average scores, underscoring the vulnerability of maritime transport to geopolitical and environmental events. The CPPI is not about comparing one port against another, but about understanding changes in performance over time. Ports that improved their scores often did so by reducing time at anchor, optimizing berth operations, investing in digital tools, and strengthening coordination across logistics partners. The evidence confirms that improvements are possible across ports of all sizes, and that rising scores are linked to deliberate actions to minimize time in port relative to containers moved. By consolidating five years of results, this edition transforms the CPPI into a long-term reference point. It shows how global crises have affected shipping, how different regions have adapted, and what lessons can be drawn for future resilience. The World Bank and S&P Global Market Intelligence remain committed to maintaining the CPPI as a global public good, providing transparency, comparability, and practical insights to support more reliable and sustainable maritime supply chains.Publication Digital Progress and Trends Report 2023(Washington, DC: World Bank, 2024-03-05)Digitalization is the transformational opportunity of our time. The digital sector has become a powerhouse of innovation, economic growth, and job creation. Value added in the IT services sector grew at 8 percent annually during 2000–22, nearly twice as fast as the global economy. Employment growth in IT services reached 7 percent annually, six times higher than total employment growth. The diffusion and adoption of digital technologies are just as critical as their invention. Digital uptake has accelerated since the COVID-19 pandemic, with 1.5 billion new internet users added from 2018 to 2022. The share of firms investing in digital solutions around the world has more than doubled from 2020 to 2022. Low-income countries, vulnerable populations, and small firms, however, have been falling behind, while transformative digital innovations such as artificial intelligence (AI) have been accelerating in higher-income countries. Although more than 90 percent of the population in high-income countries was online in 2022, only one in four people in low-income countries used the internet, and the speed of their connection was typically only a small fraction of that in wealthier countries. As businesses in technologically advanced countries integrate generative AI into their products and services, less than half of the businesses in many low- and middle-income countries have an internet connection. The growing digital divide is exacerbating the poverty and productivity gaps between richer and poorer economies. The Digital Progress and Trends Report series will track global digitalization progress and highlight policy trends, debates, and implications for low- and middle-income countries. The series adds to the global efforts to study the progress and trends of digitalization in two main ways: · By compiling, curating, and analyzing data from diverse sources to present a comprehensive picture of digitalization in low- and middle-income countries, including in-depth analyses on understudied topics. · By developing insights on policy opportunities, challenges, and debates and reflecting the perspectives of various stakeholders and the World Bank’s operational experiences. This report, the first in the series, aims to inform evidence-based policy making and motivate action among internal and external audiences and stakeholders. The report will bring global attention to high-performing countries that have valuable experience to share as well as to areas where efforts will need to be redoubled.Publication The Container Port Performance Index 2023(Washington, DC: World Bank, 2024-07-18)The Container Port Performance Index (CPPI) measures the time container ships spend in port, making it an important point of reference for stakeholders in the global economy. These stakeholders include port authorities and operators, national governments, supranational organizations, development agencies, and other public and private players in trade and logistics. The index highlights where vessel time in container ports could be improved. Streamlining these processes would benefit all parties involved, including shipping lines, national governments, and consumers. This fourth edition of the CPPI relies on data from 405 container ports with at least 24 container ship port calls in the calendar year 2023. As in earlier editions of the CPPI, the ranking employs two different methodological approaches: an administrative (technical) approach and a statistical approach (using matrix factorization). Combining these two approaches ensures that the overall ranking of container ports reflects actual port performance as closely as possible while also being statistically robust. The CPPI methodology assesses the sequential steps of a container ship port call. ‘Total port hours’ refers to the total time elapsed from the moment a ship arrives at the port until the vessel leaves the berth after completing its cargo operations. The CPPI uses time as an indicator because time is very important to shipping lines, ports, and the entire logistics chain. However, time, as captured by the CPPI, is not the only way to measure port efficiency, so it does not tell the entire story of a port’s performance. Factors that can influence the time vessels spend in ports can be location-specific and under the port’s control (endogenous) or external and beyond the control of the port (exogenous). The CPPI measures time spent in container ports, strictly based on quantitative data only, which do not reveal the underlying factors or root causes of extended port times. A detailed port-specific diagnostic would be required to assess the contribution of underlying factors to the time a vessel spends in port. A very low ranking or a significant change in ranking may warrant special attention, for which the World Bank generally recommends a detailed diagnostic.