Publication: Improving Estimates of Mean Welfare and Uncertainty in Developing Countries
Loading...
Date
2023-03
ISSN
Published
2023-03
Author(s)
Editor(s)
Abstract
Reliable estimates of economic welfare for small areas are valuable inputs into the design and evaluation of development policies. This paper compares the accuracy of point estimates and confidence intervals for small area estimates of wealth and poverty derived from four different prediction methods: linear mixed models, Cubist regression, extreme gradient boosting, and boosted regression forests. The evaluation draws samples from unit-level household census data from four developing countries, combines them with publicly and globally available geospatial indicators to generate small area estimates, and evaluates these estimates against aggregates calculated using the full census. Predictions of wealth are evaluated in four countries and poverty in one. All three machine learning methods outperform the traditional linear mixed model, with extreme gradient boosting and boosted regression forests generally outperforming the other alternatives. The proposed residual bootstrap procedure reliably estimates confidence intervals for the machine learning estimators, with estimated coverage rates across simulations falling between 94 and 97 percent. These results demonstrate that predictions obtained using tree-based gradient boosting with a random effect block bootstrap generate more accurate point and uncertainty estimates than prevailing methods for generating small area welfare estimates.
Link to Data Set
Citation
“Merfeld, Joshua D.; Newhouse, David. 2023. Improving Estimates of Mean Welfare and Uncertainty in Developing Countries. Policy Research Working Papers; 10348. © World Bank. http://hdl.handle.net/10986/39530 License: CC BY-NC 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Small Area Estimation of Monetary Poverty in Mexico Using Satellite Imagery and Machine Learning(World Bank, Washington, DC, 2022-09)Estimates of poverty are an important input into policy formulation in developing countries. The accurate measurement of poverty rates is therefore a first-order problem for development policy. This paper shows that combining satellite imagery with household surveys can improve the precision and accuracy of estimated poverty rates in Mexican municipalities, a level at which the survey is not considered representative. It also shows that a household-level model outperforms other common small area estimation methods. However, poverty estimates in 2015 derived from geospatial data remain less accurate than 2010 estimates derived from household census data. These results indicate that the incorporation of household survey data and widely available satellite imagery can improve on existing poverty estimates in developing countries when census data are old or when patterns of poverty are changing rapidly, even for small subgroups.Publication Small Area Estimation of Poverty and Wealth Using Geospatial Data(World Bank, Washington, DC, 2023-07-18)This paper offers a nontechnical review of selected applications that combine survey and geospatial data to generate small area estimates of wealth or poverty. Publicly available data from satellites and phones predicts poverty and wealth accurately across space, when evaluated against census data, and their use in model-based estimates improve the accuracy and efficiency of direct survey estimates. Although the evidence is scant, models based on interpretable features appear to predict at least as well as estimates derived from Convolutional Neural Networks. Estimates for sampled areas are significantly more accurate than those for non-sampled areas due to informative sampling. In general, estimates benefit from using geospatial data at the most disaggregated level possible. Tree-based machine learning methods appear to generate more accurate estimates than linear mixed models. Small area estimates using geospatial data can improve the design of social assistance programs, particularly when the existing targeting system is poorly designed.Publication Micro-Level Estimation of Welfare(World Bank, Washington, DC, 2002-10)The authors construct and derive the properties of estimators of welfare that take advantage of the detailed information about living standards available in small household surveys and the comprehensive coverage of a census or large sample. By combining the strengths of each, the estimators can be used at a remarkably disaggregated level. They have a clear interpretation, are mutually comparable, and can be assessed for reliability using standard statistical theory. Using data from Ecuador, the authors obtain estimates of welfare measures, some of which are quite reliable for populations as small as 15,000 households--a "town." They provide simple illustrations of their use. Such estimates open up the possibility of testing, at a more convincing intra-country level, the many recent models relating welfare distributions to growth and a variety of socioeconomic and political outcomes.Publication Combining Survey and Geospatial Data Can Significantly Improve Gender-Disaggregated Estimates of Labor Market Outcomes(World Bank, Washington, DC, 2022-06)Better understanding the geography of women’s labor market outcomes within countries is important to inform targeted efforts to increase women’s economic empowerment. This paper assesses the extent to which a method that combines simulated survey data from urban areas in Mexico with broadly available geospatial indicators from Google Earth Engine and OpenStreetMap can significantly improve estimates of labor force participation and unemployment rates. Incorporating geospatial information substantially increases the accuracy of male and female labor force participation and unemployment rates at the state level, reducing mean absolute deviation by 50 to 62 percent for labor force participation and 25 to 52 percent for unemployment. Small area estimation using a nested error conditional random effect model also greatly improves municipal estimates of labor force participation, as the mean absolute error falls by approximately half, while the mean squared error falls by almost 75 percent when holding coverage rates constant. In contrast, the results for municipal unemployment rate estimates are not reliable because values of unemployment rates are low and therefore poorly suited for linear models. The municipal results hold in repeated simulations of alternative samples. Models utilizing Basic Geo-Statistical Area (AGEB)–level auxiliary information generate more accurate predictions than area-level models specified using the same auxiliary data. Overall, integrating survey data and publicly available geospatial indicators is feasible and can greatly improve state-level estimates of male and female labor force participation and unemployment rates, as well as municipal estimates of male and female labor force participation.Publication Imputed Welfare Estimates in Regression Analysis(World Bank, Washington, D.C., 2004-04)The authors discuss the use of imputed data in regression analysis, in particular the use of highly disaggregated welfare indicators (from so-called "poverty maps"). They show that such indicators can be used both as explanatory variables on the right-hand side and as the phenomenon to explain on the left-hand side. The authors try out practical ways of adjusting standard errors of the regression coefficients to reflect the error introduced by using imputed, rather than actual, welfare indicators. These are illustrated by regression experiments based on data from Ecuador. For regressions with imputed variables on the left-hand side, the authors argue that essentially the same aggregate relationships would be found with either actual or imputed variables. They address the methodological question of how to interpret aggregate relationships found in such regressions.
Users also downloaded
Showing related downloaded files
Publication The Journey Ahead(Washington, DC: World Bank, 2024-10-31)The Journey Ahead: Supporting Successful Migration in Europe and Central Asia provides an in-depth analysis of international migration in Europe and Central Asia (ECA) and the implications for policy making. By identifying challenges and opportunities associated with migration in the region, it aims to inform a more nuanced, evidencebased debate on the costs and benefits of cross-border mobility. Using data-driven insights and new analysis, the report shows that migration has been an engine of prosperity and has helped address some of ECA’s demographic and socioeconomic disparities. Yet, migration’s full economic potential remains untapped. The report identifies multiple barriers keeping migration from achieving its full potential. Crucially, it argues that policies in both origin and destination countries can help maximize the development impacts of migration and effectively manage the economic, social, and political costs. Drawing from a wide range of literature, country experiences, and novel analysis, The Journey Ahead presents actionable policy options to enhance the benefits of migration for destination and origin countries and migrants themselves. Some measures can be taken unilaterally by countries, whereas others require close bilateral or regional coordination. The recommendations are tailored to different types of migration— forced displacement as well as high-skilled and low-skilled economic migration—and from the perspectives of both sending and receiving countries. This report serves as a comprehensive resource for governments, development partners, and other stakeholders throughout Europe and Central Asia, where the richness and diversity of migration experiences provide valuable insights for policy makers in other regions of the world.Publication Remarks at the United Nations Biodiversity Conference(World Bank, Washington, DC, 2021-10-12)World Bank Group President David Malpass discussed biodiversity and climate change being closely interlinked, with terrestrial and marine ecosystems serving as critically important carbon sinks. At the same time climate change acts as a direct driver of biodiversity and ecosystem services loss. The World Bank has financed biodiversity conservation around the world, including over 116 million hectares of Marine and Coastal Protected Areas, 10 million hectares of Terrestrial Protected Areas, and over 300 protected habitats, biological buffer zones and reserves. The COVID pandemic, biodiversity loss, climate change are all reminders of how connected we are. The recovery from this pandemic is an opportunity to put in place more effective policies, institutions, and resources to address biodiversity loss.Publication South Asia Development Update, April 2024: Jobs for Resilience(Washington, DC: World Bank, 2024-04-02)South Asia is expected to continue to be the fastest-growing emerging market and developing economy (EMDE) region over the next two years. This is largely thanks to robust growth in India, but growth is also expected to pick up in most other South Asian economies. However, growth in the near-term is more reliant on the public sector than elsewhere, whereas private investment, in particular, continues to be weak. Efforts to rein in elevated debt, borrowing costs, and fiscal deficits may eventually weigh on growth and limit governments' ability to respond to increasingly frequent climate shocks. Yet, the provision of public goods is among the most effective strategies for climate adaptation. This is especially the case for households and farms, which tend to rely on shifting their efforts to non-agricultural jobs. These strategies are less effective forms of climate adaptation, in part because opportunities to move out of agriculture are limited by the region’s below-average employment ratios in the non-agricultural sector and for women. Because employment growth is falling short of working-age population growth, the region fails to fully capitalize on its demographic dividend. Vibrant, competitive firms are key to unlocking the demographic dividend, robust private investment, and workers’ ability to move out of agriculture. A range of policies could spur firm growth, including improved business climates and institutions, the removal of financial sector restrictions, and greater openness to trade and capital flows.Publication Economic Recovery(World Bank, Washington, DC, 2021-04-06)World Bank Group President David Malpass spoke about the world facing major challenges, including COVID, climate change, rising poverty and inequality and growing fragility and violence in many countries. He highlighted vaccines, working closely with Gavi, WHO, and UNICEF, the World Bank has conducted over one hundred capacity assessments, many even more before vaccines were available. The World Bank Group worked to achieve a debt service suspension initiative and increased transparency in debt contracts at developing countries. The World Bank Group is finalizing a new climate change action plan, which includes a big step up in financing, building on their record climate financing over the past two years. He noted big challenges to bring all together to achieve GRID: green, resilient, and inclusive development. Janet Yellen, U.S. Secretary of the Treasury, mentioned focusing on vulnerable people during the pandemic. Kristalina Georgieva, Managing Director of the International Monetary Fund, focused on giving everyone a fair shot during a sustainable recovery. All three commented on the importance of tackling climate change.Publication Media and Messages for Nutrition and Health(World Bank, Washington, DC, 2020-06)The Lao People’s Democratic Republic (Lao PDR) has experienced rapid and significant economic growth over the past decade. However, poor nutritional outcomes remain a concern. Rates of childhood undernutrition are particularly high in remote, rural, and upland areas. Media have the potential to play an important role in shaping health and nutrition–related behaviors and practices as well as in promoting sociocultural and economic development that might contribute to improved nutritional outcomes. This report presents the results of a media audit (MA) that was conducted to inform the development and production of mass media advocacy and communication strategies and materials with a focus on maternal and child health and nutrition that would reach the most people from the poorest communities in northern Lao PDR. Making more people aware of useful information, essential services and products and influencing them to use these effectively is the ultimate goal of mass media campaigns, and the MA measures the potential effectiveness of media efforts to reach this goal. The effectiveness of communication channels to deliver health and nutrition messages to target beneficiaries to ensure maximum reach and uptake can be viewed in terms of preferences, satisfaction, and trust. Overall, the four most accessed media channels for receiving information among communities in the study areas were village announcements, mobile phones, television, and out-of-home (OOH) media. Of the accessed media channels, the top three most preferred channels were village announcements (40 percent), television (26 percent), and mobile phones (19 percent). In terms of trust, village announcements were the most trusted source of information (64 percent), followed by mobile phones (14 percent) and television (11 percent). Hence of all the media channels, village announcements are the most preferred, have the most satisfied users, and are the most trusted source of information in study communities from four provinces in Lao PDR with some of the highest burden of childhood undernutrition.