Publication: How Well Did Real-Time Indicators Track Household Welfare Changes in Developing Countries during the COVID-19 Crisis?
Loading...
Published
2024-09-23
ISSN
Date
2024-09-23
Editor(s)
Abstract
This paper investigates the extent to which real-time indicators derived from internet search, cell phones, and satellites predict changes in household socioeconomic indicators across approximately 300 administrative level-1 regions in 20 countries during the COVID-19 crisis. Measures of changes in socioeconomic status in each region are taken from high-frequency phone surveys. When using the first wave of data, fielded between April and August 2020, models selected using the least absolute shrinkage and selection operator explain 37 percent of the cross-regional variation in the share of households reporting declines in total income and 34 percent of the share of respondents reporting work stoppages since the onset of the crisis. Real-time indicators explain a lower amount of the within-region variation in income losses and current employment over time, with an R2 of 15 percent for current employment and 22 to 26 percent for the prevalence of income declines. When limiting the sample to urban regions, real-time indicators are far more effective at explaining within-region variation in income losses and current employment, with R2 values of approximately 0.54 and 0.38, respectively. Income gains, self-reported food insecurity, social distancing behavior, and child school engagement are more difficult to predict, with R2 values ranging from 0.06 to 0.17. Google search terms related to food, money, jobs, and religion were the most powerful predictors of work stoppage and income declines in the first survey wave, while those related to food, exercise, and religion better tracked changes in income declines and employment over time. Google mobility measures are also strong predictors of changes in employment and the prevalence of specific types of income declines. In general, satellite data on vegetation, pollution, and nighttime lights are far less predictive. Google mobility and search data, and to a lesser extent vegetation and pollution data, can provide a meaningful signal of regional economic distress and recovery, particularly during the early phases of a major crisis such as COVID-19.
Link to Data Set
Citation
“Newhouse, David; Swindle, Rachel; Wang, Shun; Merfeld, Joshua D.; Pape, Utz; Tafere , Kibrom; Weber, Michael. 2024. How Well Did Real-Time Indicators Track Household Welfare Changes in Developing Countries during the COVID-19 Crisis?. Policy Research Working Paper; 10916. © World Bank. http://hdl.handle.net/10986/42188 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Labor Demand in the Age of Generative AI: Early Evidence from the U.S. Job Posting Data(Washington, DC: World Bank, 2025-11-18)This paper examines the causal impact of generative artificial intelligence on U.S. labor demand using online job posting data. Exploiting ChatGPT’s release in November 2022 as an exogenous shock, the paper applies difference-in-differences and event study designs to estimate the job displacement effects of generative artificial intelligence. The identification strategy compares labor demand for occupations with high versus low artificial intelligence substitution vulnerability following ChatGPT’s launch, conditioning on similar generative artificial intelligence exposure levels to isolate substitution effects from complementary uses. The analysis uses 285 million job postings collected by Lightcast from the first quarter of 2018 to the second quarter of 2025Q2. The findings show that the number of postings for occupations with above-median artificial intelligence substitution scores fell by an average of 12 percent relative to those with below-median scores. The effect increased from 6 percent in the first year after the launch to 18 percent by the third year. Losses were particularly acute for entry-level positions that require neither advanced degrees (18 percent) nor extensive experience (20 percent), as well as those in administrative support (40 percent) and professional services (30 percent). Although generative artificial intelligence generates new occupations and enhances productivity, which may increase labor demand, early evidence suggests that some occupations may be less likely to be complemented by generative artificial intelligence than others.Publication The Lasting Effects of Working while in School(Washington, DC: World Bank, 2025-08-18)This paper provides the first experimental evidence on the long-term effects of work-study programs, leveraging a randomized lottery design from a national program in Uruguay. Participation leads to a persistent 11 percent increase in formal labor earnings, observable seven years after the program. Effects are stronger for youth who participate during pivotal educational transitions and are larger for vulnerable youth and men, while remaining positive for women and non-vulnerable youth. The program is highly cost-effective, with average impacts exceeding those of job training programs and comparable to early childhood investments.Publication It’s Not (Just) the Tariffs: Rethinking Non-Tariff Measures in a Fragmented Global Economy(Washington, DC: World Bank, 2025-10-22)As tariffs have declined, non-tariff measures (NTMs) have become central to trade policy, especially in high-income countries and regulated sectors like food and green technologies. Although NTMs may serve legitimate goals, they could also sort countries and firms into or out of markets based on compliance capacity and differences in product mix. Documenting recent advances in the estimation of ad valorem equivalents (AVEs), this paper uncovers new patterns of use and exposure of NTMs. High-income countries rely more heavily on NTMs relative to tariffs, while low- and middle-income countries face steeper AVEs on their exports. Firm-level evidence shows that NTMs disproportionately affect smaller firms, leading to market exit and concentration. Poorly designed NTMs can harm productivity and welfare, while coordinated, capacity-aware use can deliver inclusive outcomes. Policy design, transparency, and diagnostics must evolve to reflect the growing role—and risks—of NTMs in a fragmented global trade landscape.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication How Well Can Real-Time Indicators Track the Economic Impacts of a Crisis Like COVID-19?(World Bank, Washington, DC, 2022-06)This paper presents evidence on the extent to which a set of real-time indicators tracked changes in gross domestic product across 142 countries in 2020. The real-time indicators include Google mobility, Google search trends, food price information, nitrogen dioxide, and nighttime lights. Google mobility and staple food prices both declined sharply in March and April, followed by a rapid recovery that returned to baseline levels by July and August. Mobility and staple food prices fell less in low-income countries. Nitrogen dioxide levels show a similar pattern, with a steep fall and rapid recovery in high-income and upper-middle-income countries but not in low-income and lower-middle-income countries. In April and May, Google search terms reflecting economic distress and religiosity spiked in some regions but not others. Data on nighttime lights show no clear drop in March outside East Asia. Linear models selected using the Least Absolute Shrinkage and Selection Operator explain about a third of the variation in annual gross domestic product growth rates across 72 countries. In a smaller subset of higher income countries, real-time indicators explain about 40 percent of the variation in quarterly gross domestic product growth. Overall, mobility and food price data, as well as pollution data in more developed countries, appeared to be best at capturing the widespread economic disruption experienced during the summer of 2020. The results indicate that these real-time indicators can track a substantial percentage of both annual and quarterly changes in gross domestic product.Publication The Welfare Implications of COVID-19 for Fragile and Conflict-Affected Areas(World Bank, Washington, DC, 2022-06)Understanding the impacts of the COVID-19 pandemic on households’ welfare in areas at the admin-1 level subject to fragility, conflict, and violence is important to inform programs and policies in this context. Harmonized data from high-frequency phone surveys indicate that, at the onset of the pandemic, a higher fraction of households in areas affected by fragility, conflict, and violence reported income declines and a higher fraction of respondents reported that they had stopped working since the beginning of the crisis. Households in areas affected by fragility, conflict, and violence were far less likely to report receiving government assistance than those in other areas. These findings suggest that the initial effects of the pandemic exacerbated preexisting economic gaps between areas affected by fragility, conflict, and violence and other areas, indicating that an even larger effort will be necessary in areas affected by fragility, conflict, and violence to recover from COVID-19, with implications for funding needs and policy as well as program design.Publication Near-Real-Time Welfare and Livelihood Impacts of an Active Civil War(World Bank, Washington, DC, 2022-04)Ethiopia is currently embroiled in a large-scale civil war that has continued for more than a year. Using unique High-Frequency Phone Survey data, which spans several months before and after the outbreak of the war, this paper provides fresh evidence on the ex durante impacts of the conflict on the food security and livelihood activities of affected households. The analysis uses difference-in-differences estimation to compare trends in the outcomes of interest across affected and unaffected regions (households) and before and after the outbreak of the civil war. The findings show that seven months into the conflict, the outbreak of the civil war increased the probability of moderate to severe food insecurity by 38 percentage points. Using the Armed Conflict Location and Event Data on households’ exposure to violent conflict, the analysis shows that exposure to one additional battle leads to a 1 percentage point increase in the probability of moderate to severe food insecurity. The conflict has reduced households’ access to food through supply chain disruptions while also curtailing non-farm livelihood activities. Non-farm and wage-related activities have been the most affected by the conflict, while farming activities have been relatively more resilient. Similarly, economic activities in urban areas have been much more affected than those in rural areas. These substantial impact estimates, which are likely to be underestimates of the true average effects on the population, constitute novel evidence of the near-real-time impacts of an ongoing civil conflict, providing direct evidence of how violent conflict disrupts the functioning of market supply chains and livelihoods activities. The paper highlights the potential of phone surveys to monitor active and large-scale conflicts, especially in contexts where conventional data sources are not immediately available.Publication Insuring Well-Being?(World Bank, Washington, DC, 2017-11)This paper estimates the causal effects of index insurance coverage on subjective well-being among livestock herders in southern Ethiopia. The randomization of incentives to purchase index-based livestock insurance and three rounds of panel data are exploited to separately identify ex ante welfare gains from insurance that reduces risk exposure and ex post buyer's remorse effects that may arise after the resolution of uncertainty. Insurance coverage currently in force generates subjective well-being gains that are significantly higher than the buyer's remorse effect of an insurance that lapsed without paying out. Given the temporal correlation in insurance purchase propensity, failure to control for potential buyer's remorse effects can bias downward estimates of welfare gains from current insurance coverage.Publication Does Climbing the Jobs Ladder Promote Poverty Reduction?(Washington, DC: World Bank, 2024-07-24)This paper explores trends in and the potential determinants of the types of jobs held by workers, and their relationship with poverty reduction, in an unbalanced panel of 89 countries over the past 30 years. Jobs are classified into five categories according to formality, occupation or level of skills required, and wage work versus self-employment. Net shifts into "upper tier" or skilled informal wage jobs, defined as professionals, managers, technicians, or clerks, from "lower tier" or lower skilled informal jobs were strongly associated with poverty reduction at the $1.90 and $3.20 lines. In contrast, net shifts into formal wage jobs from lower tier informal jobs were associated with modest poverty reductions at the $5.50 poverty line. The share of workers in informal upper tier jobs represents less than 2 percent of the workforce and has increased little over the past 30 years in low- and middle-income countries. The findings show that increases in upper tier informal wage jobs are associated with shifts of the workforce from microenterprises to small firms in lower- and upper-middle-income countries, but they are not discernibly associated with higher educational attainment or urbanization. In contrast, increases in the share of formal wage jobs are strongly associated with increases in the share of workers with post-secondary education, driven by high-income countries. The results suggest that upper tier informal wage jobs and the skills they require play a potentially important role in poverty reduction but are not automatically generated by increased educational attainment, urbanization, or firm size.
Users also downloaded
Showing related downloaded files
Publication World Development Report 2024(Washington, DC: World Bank, 2024-08-01)Middle-income countries are in a race against time. Many of them have done well since the 1990s to escape low-income levels and eradicate extreme poverty, leading to the perception that the last three decades have been great for development. But the ambition of the more than 100 economies with incomes per capita between US$1,100 and US$14,000 is to reach high-income status within the next generation. When assessed against this goal, their record is discouraging. Since the 1970s, income per capita in the median middle-income country has stagnated at less than a tenth of the US level. With aging populations, growing protectionism, and escalating pressures to speed up the energy transition, today’s middle-income economies face ever more daunting odds. To become advanced economies despite the growing headwinds, they will have to make miracles. Drawing on the development experience and advances in economic analysis since the 1950s, World Development Report 2024 identifies pathways for developing economies to avoid the “middle-income trap.” It points to the need for not one but two transitions for those at the middle-income level: the first from investment to infusion and the second from infusion to innovation. Governments in lower-middle-income countries must drop the habit of repeating the same investment-driven strategies and work instead to infuse modern technologies and successful business processes from around the world into their economies. This requires reshaping large swaths of those economies into globally competitive suppliers of goods and services. Upper-middle-income countries that have mastered infusion can accelerate the shift to innovation—not just borrowing ideas from the global frontiers of technology but also beginning to push the frontiers outward. This requires restructuring enterprise, work, and energy use once again, with an even greater emphasis on economic freedom, social mobility, and political contestability. Neither transition is automatic. The handful of economies that made speedy transitions from middle- to high-income status have encouraged enterprise by disciplining powerful incumbents, developed talent by rewarding merit, and capitalized on crises to alter policies and institutions that no longer suit the purposes they were once designed to serve. Today’s middle-income countries will have to do the same.Publication World Development Report 2023: Migrants, Refugees, and Societies(Washington, DC : World Bank, 2023-04-25)Migration is a development challenge. About 184 million people—2.3 percent of the world’s population—live outside of their country of nationality. Almost half of them are in low- and middle-income countries. But what lies ahead? As the world struggles to cope with global economic imbalances, diverging demographic trends, and climate change, migration will become a necessity in the decades to come for countries at all levels of income. If managed well, migration can be a force for prosperity and can help achieve the United Nations’ Sustainable Development Goals. World Development Report 2023 proposes an innovative approach to maximize the development impacts of cross-border movements on both destination and origin countries and on migrants and refugees themselves. The framework it offers, drawn from labor economics and international law, rests on a “Match and Motive Matrix” that focuses on two factors: how closely migrants’ skills and attributes match the needs of destination countries and what motives underlie their movements. This approach enables policy makers to distinguish between different types of movements and to design migration policies for each. International cooperation will be critical to the effective management of migration.Publication World Development Report 2018(Washington, DC: World Bank, 2018)Every year, the World Bank's World Development Report takes on a topic of central importance to global development. The 2018 Report, Learning to Realize Education's Promise, is the first ever devoted entirely to education. Now is an excellent time for it: education has long been critical for human welfare, but is even more so in a time of rapid economic change. The Report explores four main themes. First, education's promise: Education is a powerful instrument for eradicating poverty and promoting shared prosperity, but fulfilling its potential requires better policies - both within and outside the education system. Second, the learning crisis: Despite gains in education access, recent learning assessments show that many young people around the world, especially from poor families, are leaving school unequipped with even the most foundational skills they need for life. At the same time, internationally comparable learning assessments show that skills in many middle-income countries lag far behind what those countries aspire to. Third, promising interventions to improve learning: Research from areas such as brain science, pedagogical innovations, or school management have identified interventions that promote learning by ensuring that learners are prepared, that teachers are skilled as well as motivated, and that other inputs support the teacher-learner relationship. Fourth, learning at scale: Achieving learning throughout an education system will require more than just scaling up effective interventions. Change requires overcoming technical and political barriers by deploying salient metrics for mobilizing actors and tracking progress, building coalitions for learning, and being adaptive when implementing programs.Publication Western Balkans 6 Country Climate and Development Report(Washington, DC: World Bank Group, 2024-07-16)This Regional Western Balkans Countries Climate and Development Report (CCDR) stands out in several ways. In a region that often lacks cohesive regional alliances, this report emphasizes how the challenges faced across countries are often common and interconnected, and, importantly, that climate action requires coordination on multiple fronts. Simultaneously, it illustrates the differences across countries, places, and people that require targeted strategies and interventions. This report demonstrates how shocks and stressors re intensifying and how investments in adaptation could bring significant benefits in the form of avoided losses, accelerated economic potential, and amplified social and economic spillovers. Given the region’s high emission and energy intensity and the limitations of its current fossil fuel-based development model, the report articulates a path to greener and more resilient growth, a path that is more consistent with the aspiration of accession to the EU. The report finds that the net zero transition can be undertaken without compromising the economic potential of the Western Balkans and that it could lead to higher growth than under the Reference Scenario (RS) with appropriate structural reforms.Publication World Development Report 2004(World Bank, 2003)Too often, services fail poor people in access, in quality, and in affordability. But the fact that there are striking examples where basic services such as water, sanitation, health, education, and electricity do work for poor people means that governments and citizens can do a better job of providing them. Learning from success and understanding the sources of failure, this year’s World Development Report, argues that services can be improved by putting poor people at the center of service provision. How? By enabling the poor to monitor and discipline service providers, by amplifying their voice in policymaking, and by strengthening the incentives for providers to serve the poor. Freedom from illness and freedom from illiteracy are two of the most important ways poor people can escape from poverty. To achieve these goals, economic growth and financial resources are of course necessary, but they are not enough. The World Development Report provides a practical framework for making the services that contribute to human development work for poor people. With this framework, citizens, governments, and donors can take action and accelerate progress toward the common objective of poverty reduction, as specified in the Millennium Development Goals.