Publication: Pull Your Small Area Estimates Up by the Bootstraps
Loading...
Files in English
104 downloads
Date
2021-05-08
ISSN
0094-9655
Published
2021-05-08
Author(s)
Editor(s)
Abstract
This paper presents a methodological update to the World Bank's toolkit for small area estimation. The paper reviews the computational procedures of the current methods used by the institution: the traditional ELL approach and the Empirical Best (EB) addition introduced to imitate the original EB procedure of Molina and Rao [Small area estimation of poverty indicators. Canadian J Stat. 2010;38(3):369–385], including heteroskedasticity and survey weights, but using a different bootstrap approach, here referred to as clustered bootstrap. Simulation experiments provide empirical evidence of the shortcomings of the clustered bootstrap approach, which yields biased and noisier point estimates. The document presents an update to the World Bank’s EB implementation by considering the original EB procedures for point and noise estimation, extended for complex designs and heteroscedasticity. Simulation experiments illustrate that the revised methods yield considerably less biased and more efficient estimators than those obtained from the clustered bootstrap approach.
Link to Data Set
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Citations
- Cited 8 times in Scopus (view citations)
Collections
Related items
Showing items related by metadata.
Publication Pull Your Small Area Estimates Up by the Bootstraps(World Bank, Washington, DC, 2020-05)After almost two decades of poverty maps produced by the World Bank and multiple advances in the literature, this paper presents a methodological update to the World Bank's toolkit for small area estimation. The paper reviews the computational procedures of the current methods used by the World Bank: the traditional approach by Elbers, Lanjouw and Lanjouw (2003) and the Empirical Best/Bayes (EB) addition introduced by Van der Weide (2014). The addition extends the EB procedure of Molina and Rao (2010) by considering heteroscedasticity and includes survey weights, but uses a different bootstrap approach, here referred to as clustered bootstrap. Simulation experiments comparing these methods to the original EB approach of Molina and Rao (2010) provide empirical evidence of the shortcomings of the clustered bootstrap approach, which yields biased point estimates. The main contributions of this paper are then two: 1) to adapt the original Monte Carlo simulation procedure of Molina and Rao (2010) for the approximation of the extended EB estimators that include heteroscedasticity and survey weights as in Van der Weide (2014); and 2) to adapt the parametric bootstrap approach for mean squared error (MSE) estimation considered by Molina and Rao (2010), and proposed originally by González-Manteiga et al. (2008), to these extended EB estimators. Simulation experiments illustrate that the revised Monte Carlo simulation method yields estimators that are considerably less biased and more efficient in terms of MSE than those obtained from the clustered bootstrap approach, and that the parametric bootstrap MSE estimators are in line with the true MSEs under realistic scenarios.Publication A Map of the Poor or a Poor Map?(World Bank, Washington, DC, 2021-04)This paper evaluates the performance of different small area estimation methods using model and design-based simulation experiments. Design-based simulation experiments are carried out using the Mexican Intra Censal survey as a census of roughly 3.9 million households from which 500 samples are drawn using a two-stage selection procedure similar to that of Living Standards Measurement Study surveys. Several unit-level methods are considered as well as a method that combines unit and area level information, which has been proposed as an alternative when the available census data is outdated. The findings show the importance of selecting a proper model and data transformation so that the model assumptions hold. A proper data transformation can lead to a considerable improvement in mean squared errors. The results from design-based validation show that all small area estimation methods represent an improvement, in terms of mean squared errors, over direct estimates. However, methods that model unit level welfare using only area level information suffer from considerable bias. Because the magnitude and direction of the bias are unknown ex ante, methods that rely only on aggregated covariates should be used with caution, but they may be an alternative to traditional area level models when these are not applicable.Publication How Good a Map? Putting Small Area Estimation to the Test(World Bank, Washington, DC, 2007-03)The authors examine the performance of small area welfare estimation. The method combines census and survey data to produce spatially disaggregated poverty and inequality estimates. To test the method, they compare predicted welfare indicators for a set of target populations with their true values. They construct target populations using actual data from a census of households in a set of rural Mexican communities. They examine estimates along three criteria: accuracy of confidence intervals, bias, and correlation with true values. The authors find that while point estimates are very stable, the precision of the estimates varies with alternative simulation methods. While the original approach of numerical gradient estimation yields standard errors that seem appropriate, some computationally less-intensive simulation procedures yield confidence intervals that are slightly too narrow. The precision of estimates is shown to diminish markedly if unobserved location effects at the village level are not well captured in underlying consumption models. With well specified models there is only slight evidence of bias, but the authors show that bias increases if underlying models fail to capture latent location effects. Correlations between estimated and true welfare at the local level are highest for mean expenditure and poverty measures and lower for inequality measures.Publication SAE - A Stata Package for Unit Level Small Area Estimation(World Bank, Washington, DC, 2018-10)This paper presents a new family of Stata functions devoted to small area estimation. Small area methods attempt to solve low representativeness of surveys within areas, or the lack of data for specific areas/sub-populations. This is accomplished by incorporating information from outside sources. Such target data sets are becoming increasingly available and can take the form of a traditional population census, but also large scale administrative records from tax administrations, or geospatial information produced using remote sensing. The strength of these target data sets is their granularity on the subpopulations of interest, however, in many cases they lack the ability to collect analytically relevant variables such as welfare or caloric intake. The family of functions introduced follow a modular design to have the flexibility with which these can be expanded in the future. This can be accomplished by the authors and/or other collaborators from the Stata community. Thus far, a major limitation of such analysis in Stata has been the large size of target data sets. The package introduces new mata functions and a plugin used to circumvent memory limitations that inevitably arise when working with big data. From an estimation perspective, the paper starts by implementing a methodology that has been widely used for the production of several poverty maps.Publication Frontiers in Small Area Estimation Research(Washington, DC: World Bank, 2024-06-28)This paper reviews the main methods for small area estimation of welfare indicators. It begins by discussing the importance of small area estimation methods for producing reliable disaggregated estimates. It mentions the baseline papers and describes the contents of the different sections. Basic direct estimators obtained from area-specific survey data are described first, followed by simple indirect methods, which include synthetic procedures that do not account for the area effects and composite estimators obtained as a composition (or weighted average) of a synthetic and a direct estimator. The previous estimators are design-based, meaning that their properties are assessed under the sampling replication mechanism, without assuming any model to be true. The paper then turns to proper model-based estimators that assume an explicit model. These models allow obtaining optimal small area estimators when the assumed model holds. The first type of models, referred to as area-level models, use only aggregated data at the area level to fit the model. However, unit-level survey data were previously used to calculate the direct estimators, which act as response variables in the most common area-level models. The paper then switches to unit-level models, describing first the usual estimators for area means, and then moving to general area indicators. Semi-parametric, non-parametric, and machine learning procedures are described in a separate section, although many of the procedures are applicable only to area means. Based on the previous material, the paper identifies gaps or potential limitations in existing procedures from a practitioner’s perspective, which could potentially be addressed through research over the next three to five years.
Users also downloaded
Showing related downloaded files
Publication World Development Report 2011(World Bank, 2011)The 2011 World development report looks across disciplines and experiences drawn from around the world to offer some ideas and practical recommendations on how to move beyond conflict and fragility and secure development. The key messages are important for all countries-low, middle, and high income-as well as for regional and global institutions: first, institutional legitimacy is the key to stability. When state institutions do not adequately protect citizens, guard against corruption, or provide access to justice; when markets do not provide job opportunities; or when communities have lost social cohesion-the likelihood of violent conflict increases. Second, investing in citizen security, justice, and jobs is essential to reducing violence. But there are major structural gaps in our collective capabilities to support these areas. Third, confronting this challenge effectively means that institutions need to change. International agencies and partners from other countries must adapt procedures so they can respond with agility and speed, a longer-term perspective, and greater staying power. Fourth, need to adopt a layered approach. Some problems can be addressed at the country level, but others need to be addressed at a regional level, such as developing markets that integrate insecure areas and pooling resources for building capacity Fifth, in adopting these approaches, need to be aware that the global landscape is changing. Regional institutions and middle income countries are playing a larger role. This means should pay more attention to south-south and south-north exchanges, and to the recent transition experiences of middle income countries.Publication Remarks to the Annual Meetings 2020 Development Committee(World Bank, Washington, DC, 2020-10-16)David Malpass, President of the World Bank Group, announced that the Board approved a fast track approach to emergency health support programs that now covers 111 countries. Most projects are well advanced, with average disbursement upward of 40 percent. The goal is to take broad, fast action early. The operational framework presented back in June has positioned the Bank to help countries address immediate health threats and social and economic impacts and maintain our focus on long-term development. The Bank is making good progress toward the 15-month target of 160 billion dollars in surge financing. Much of it is for the poorest countries and will take the form of grants or low-rate, long-maturity loans. IFC, through the Global Health Platform, will be providing financing to vaccine manufacturers to foster expanded production of COVID-19 vaccines in both part 1 and 2 countries, providing production is reserved for emerging markets. The Development Committee holds a unique place in the international architecture. It is the only global forum in which the Governments of developed countries and the Governments of developing countries, creditor countries and borrower countries, come together to discuss development and the ‘net transfer of resources to developing countries.’ The current International Financial Architecture system is skewed in favor of the rich and creditor countries. It is important that all voices are heard, so Malpass urged the Ministers of developing countries to use their voice and speak their minds today. Malpass urged consideration of how we can build a new approach to debt restructuring that allows for a fair relationship and balance between creditors and debtors. This will be critical in restoring growth in developing countries; and helping reverse the inequality.Publication Doing Business 2014 : Understanding Regulations for Small and Medium-Size Enterprises(Washington, DC: World Bank Group, 2013-10-28)Eleventh in a series of annual reports comparing business regulation in 185 economies, Doing Business 2014 measures regulations affecting 11 areas of everyday business activity: Starting a business, Dealing with construction permits, Getting electricity, Registering property, Getting credit, Protecting investors, Paying taxes, Trading across borders, Enforcing contracts, Closing a business, Employing workers. The report updates all indicators as of June 1, 2013, ranks economies on their overall “ease of doing business”, and analyzes reforms to business regulation – identifying which economies are strengthening their business environment the most. The Doing Business reports illustrate how reforms in business regulations are being used to analyze economic outcomes for domestic entrepreneurs and for the wider economy. Doing Business is a flagship product by the World Bank and IFC that garners worldwide attention on regulatory barriers to entrepreneurship. More than 60 economies use the Doing Business indicators to shape reform agendas and monitor improvements on the ground. In addition, the Doing Business data has generated over 870 articles in peer-reviewed academic journals since its inception.Publication World Development Report 2006(Washington, DC, 2005)This year’s Word Development Report (WDR), the twenty-eighth, looks at the role of equity in the development process. It defines equity in terms of two basic principles. The first is equal opportunities: that a person’s chances in life should be determined by his or her talents and efforts, rather than by pre-determined circumstances such as race, gender, social or family background. The second principle is the avoidance of extreme deprivation in outcomes, particularly in health, education and consumption levels. This principle thus includes the objective of poverty reduction. The report’s main message is that, in the long run, the pursuit of equity and the pursuit of economic prosperity are complementary. In addition to detailed chapters exploring these and related issues, the Report contains selected data from the World Development Indicators 2005‹an appendix of economic and social data for over 200 countries. This Report offers practical insights for policymakers, executives, scholars, and all those with an interest in economic development.Publication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.