Publication: Do Labor Statistics Depend on How and to Whom the Questions Are Asked? Results from a Survey Experiment in Tanzania
Loading...
Published
2010-01-01
ISSN
Date
2012-03-19
Author(s)
Editor(s)
Abstract
Labor market statistics are critical for assessing and understanding economic development. In practice, widespread variation exists in how labor statistics are measured in household surveys in low-income countries. Little is known whether these differences have an effect on the labor statistics they produce. This paper analyzes these effects by implementing a survey experiment in Tanzania that varied two key dimensions: the level of detail of the questions and the type of respondent. Significant differences are observed across survey designs with respect to different labor statistics. Labor force participation rates, for example, vary by as much as 10 percentage points across the four survey assignments. Using a short labor module without screening questions on employment generates lower female labor force participation and lower rates of wage employment for both men and women. Response by proxy rather than self-report yields lower male labor force participation, lower female working hours, and lower employment in agriculture for men. The differences between proxy and self reporting seem to come from information imperfections within the household, especially with the distance in age between respondent and subject playing an important role, while gender and educational differences seem less important.
Link to Data Set
Citation
“Bardasi, Elena; Beegle, Kathleen; Dillon, Andrew; Serneels, Pieter. 2010. Do Labor Statistics Depend on How and to Whom the Questions Are Asked? Results from a Survey Experiment in Tanzania. Paper is funded by the Knowledge for Change
Program (KCP),Policy Research working paper ; no. WPS 5192. © World Bank. http://hdl.handle.net/10986/3700 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Stanley Fischer Memorial Lecture: Geopolitics and Development—The World Bank ABCDE Conference(Washington, DC: World Bank, 2025-10-01)This paper provides a sweeping historical overview of how geopolitics has shaped economic development and global institutions, culminating in a forward-looking analysis of challenges facing the world in 2025. The paper argues that understanding economic history and the political economy is essential for effective policy making, as geopolitics—encompassing power politics, geography, security, and ecology—fundamentally influences development economics. It traces the dynamic interplay between geopolitics and development across major historical periods: from the European Age of Discovery and the rise of nation-states, through the world wars and the Bretton Woods era, to the Cold War and the age of globalization. It highlights how crises and conflicts have repeatedly transformed global economic systems, spurring new ideas and institutions. Looking ahead, the paper identifies key trends for 2025—fragmented global governance amid ongoing globalization (migration, climate change, and debt), shifting security contexts with territorial disputes and nuclear risks, and the intersection of energy, climate, demographics, and migration with development—as it predicts increased economic nationalism and regionalization in finance and trade. The paper concludes by emphasizing the adaptive role of multilateral institutions like the World Bank and the International Monetary Fund, advocating for a holistic approach to development economics that integrates political and economic institutions, governance, and both macro and micro markets.Publication Gender Gaps in the Performance of Small Firms: Evidence from Urban Peru(Washington, DC: World Bank, 2025-09-23)This paper estimates the gender gap in the performance of firms in Peru using representative data on both formal and informal firms. On average, informal female-led firms have lower sales, labor productivity, and profits compared to their male-led counterparts, with differences more pronounced when controlling for observable determinants of firm performance. However, gender gaps are only significant at the bottom of the performance distribution of informal firms, and these gaps disappear at the top of the distribution of informal firms and for formal firms. Possible explanations for the performance gaps at the bottom of the distribution include the higher likelihood of small, female-led firms being home-based, which is linked to lower profits, and their concentration in less profitable sectors. The paper provides suggestive evidence that household responsibilities play a key role in explaining the gender gap in firm performance among informal firms. Therefore, policies that promote access to care services or foster a more equal distribution of household activities may reduce gender productivity gaps and allow for a more efficient allocation of resources.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Do Labor Statistics Depend on How and to Whom the Questions Are Asked? Results from a Survey Experiment in Tanzania(World Bank, 2011-10-18)Labor market statistics are critical for assessing and understanding economic development. However, widespread variation exists in how labor statistics are collected in household surveys. This paper analyzes the effects of alternative survey design on employment statistics by implementing a randomized survey experiment in Tanzania. Two features of the survey design are assessed – the level of detail of the employment questions and the type of respondent. It turns out that both features have relevant and statistically significant effects on employment statistics. Using a short labor module without screening questions induces many individuals to adopt a broad definition of employment, incorrectly including domestic duties. But after reclassifying those in domestic work as ‘not working’ in order to obtain the correct ILO classification, the short module turns out to generate lower female employment rates, higher working hours for both men and women who are employed, and lower rates of wage employment than the detailed module. Response by proxy rather than self-report has no effect on female labor statistics but yields substantially lower male employment rates, mostly due to underreporting of agricultural activity. The large impacts of proxy responses on male employment rates are attenuated when proxy informants are spouses and individuals with some schooling.Publication Explaining Variation in Child Labor Statistics(2010-09-01)Child labor statistics are critical for assessing the extent and nature of child labor activities in developing countries. In practice, widespread variation exists in how child labor is measured. Questionnaire modules vary across countries and within countries over time along several dimensions, including respondent type and the structure of the questionnaire. Little is known about the effect of these differences on child labor statistics. This paper presents the results from a randomized survey experiment in Tanzania focusing on two survey aspects: different questionnaire design to classify children work and proxy response versus self-reporting. Use of a short module compared with a more detailed questionnaire has a statistically significant effect, especially on child labor force participation rates, and, to a lesser extent, on working hours. Proxy reports do not differ significantly from a child s self-report. Further analysis demonstrates that survey design choices affect the coefficient estimates of some determinants of child labor in a child labor supply equation. The results suggest that low-cost changes to questionnaire design to clarify the concept of work for respondents can improve the data collected.Publication Employment and Shared Growth : Rethinking the Role of Labor Mobility for Development(Washington, DC: World Bank, 2007)This edited volume brings together the papers presented at the conference, "Rethinking the Role of Jobs for Shared Growth," held in Washington, DC, in June 2006. The common theme is that of mobility in the labor market. As growth is related to sectoral shifts in economic activity, the mobility of labor plays a crucial role in ensuring sustainable growth whose benefits are shared amongst all individuals. The papers in this volume focus on selected priority issues at the frontier of research in the microeconomics of labor markets in developing countries, multi-segmented labor markets, the role of informal employment and self-employment, the effect of worker mobility on income, and the impact of firm dynamics on growth and employment. These are important parts of the puzzle and contribute to a better understanding of the role of employment in the economic development of low-income countries.Publication Health Information, Treatment, and Worker Productivity : Experimental Evidence from Malaria Testing and Treatment among Nigerian Sugarcane Cutters(World Bank Group, Washington, DC, 2014-11)Agricultural and other physically demanding sectors are important sources of growth in developing countries but prevalent diseases such as malaria adversely impact the productivity, labor supply, and choice of job tasks among workers by reducing physical capacity. This study identifies the impact of malaria on worker earnings, labor supply, and daily productivity by randomizing the temporal order at which piece-rate workers at a large sugarcane plantation in Nigeria are offered malaria testing and treatment. The results indicate a significant and substantial intent to treat effect of the intervention -- the offer of a workplace-based malaria testing and treatment program increases worker earnings by approximately 10 percent over the weeks following the offer. The study further investigates the effect of health information by contrasting program effects by workers' revealed health status. For workers who test positive for malaria, the treatment of illness increases labor supply, leading to higher earnings. For workers who test negative, and especially for those workers most likely to be surprised by the healthy diagnosis, the health information also leads to increased earnings via increased productivity. Possible mechanisms for this response include selection into higher return tasks within the plantation as a result of changes in the perceived cost of effort. A model of the worker labor decision that allows health expectations partly to determine the supply of effort suggests that, in endemic settings with poor quality health services, inaccurate health perceptions may lead workers to suboptimal labor allocation decisions. The results underline the importance of medical treatment, but also of access to improved information about one's health status, as the absence of either may lead workers to deliver lower effort in lower return jobs.Publication Do Returns to Education Depend on How and Whom You Ask?(Elsevier, 2017-10)Returns to education remain an important parameter of interest in economic analysis. A large literature estimates these returns, often carefully addressing issues such as selection into wage employment and endogeneity in terms of completed schooling. There has been much less exploration of whether the estimates of Mincerian returns depend on how information about wage work is collected. Relying on a survey experiment in Tanzania, this paper finds that estimates of the returns to education vary by questionnaire design, but not by whether the information on employment and wages is self-reported or collected by a proxy respondent. The differences derived from questionnaire type are substantial, varying from higher returns of 5 percentage points among the most well educated men to 16 percentage points among the least well educated women. These differences are at magnitudes similar to the bias in ordinary least squares estimation, which receives considerable attention in the literature. The findings demonstrate that survey design matters in the estimation of returns to schooling and that care is needed in comparing across contexts and over time, particularly if the data are generated through different surveys.
Users also downloaded
Showing related downloaded files
Publication Ten Steps to a Results-Based Monitoring and Evaluation System : A Handbook for Development Practitioners(Washington, DC: World Bank, 2004)An effective state is essential to achieving socio-economic and sustainable development. With the advent of globalization, there are growing pressures on governments and organizations around the world to be more responsive to the demands of internal and external stakeholders for good governance, accountability and transparency, greater development effectiveness, and delivery of tangible results. Governments, parliaments, citizens, the private sector, Non-governmental Organizations (NGOs), civil society, international organizations, and donors are among the stakeholders interested in better performance. As demands for greater accountability and real results have increased, there is an attendant need for enhanced results-based monitoring and evaluation of policies, programs, and projects. This handbook provides a comprehensive ten-step model that will help guide development practitioners through the process of designing and building a results-based monitoring and evaluation system. These steps begin with a 'readiness assessment' and take the practitioner through the design, management, and importantly, the sustainability of such systems. The handbook describes each step in detail, the tasks needed to complete each one, and the tools available to help along the way.Publication World Development Report 2019(Washington, DC: World Bank, 2019)Work is constantly reshaped by technological progress. New ways of production are adopted, markets expand, and societies evolve. But some changes provoke more attention than others, in part due to the vast uncertainty involved in making predictions about the future. The 2019 World Development Report will study how the nature of work is changing as a result of advances in technology today. Technological progress disrupts existing systems. A new social contract is needed to smooth the transition and guard against rising inequality. Significant investments in human capital throughout a person’s lifecycle are vital to this effort. If workers are to stay competitive against machines they need to train or retool existing skills. A social protection system that includes a minimum basic level of protection for workers and citizens can complement new forms of employment. Improved private sector policies to encourage startup activity and competition can help countries compete in the digital age. Governments also need to ensure that firms pay their fair share of taxes, in part to fund this new social contract. The 2019 World Development Report presents an analysis of these issues based upon the available evidence.Publication World Development Report 2014(Washington, DC, 2013-10-06)The past 25 years have witnessed unprecedented changes around the world—many of them for the better. Across the continents, many countries have embarked on a path of international integration, economic reform, technological modernization, and democratic participation. As a result, economies that had been stagnant for decades are growing, people whose families had suffered deprivation for generations are escaping poverty, and hundreds of millions are enjoying the benefits of improved living standards and scientific and cultural sharing across nations. As the world changes, a host of opportunities arise constantly. With them, however, appear old and new risks, from the possibility of job loss and disease to the potential for social unrest and environmental damage. If ignored, these risks can turn into crises that reverse hard-won gains and endanger the social and economic reforms that produced these gains. The World Development Report 2014 (WDR 2014), Risk and Opportunity: Managing Risk for Development, contends that the solution is not to reject change in order to avoid risk but to prepare for the opportunities and risks that change entails. Managing risks responsibly and effectively has the potential to bring about security and a means of progress for people in developing countries and beyond. Although individuals’ own efforts, initiative, and responsibility are essential for managing risk, their success will be limited without a supportive social environment—especially when risks are large or systemic in nature. The WDR 2014 argues that people can successfully confront risks that are beyond their means by sharing their risk management with others. This can be done through naturally occurring social and economic systems that enable people to overcome the obstacles that individuals and groups face, including lack of resources and information, cognitive and behavioral failures, missing markets and public goods, and social externalities and exclusion. These systems—from the household and the community to the state and the international community—have the potential to support people’s risk management in different yet complementary ways. The Report focuses on some of the most pressing questions policy makers are asking. What role should the state take in helping people manage risks? When should this role consist of direct interventions, and when should it consist of providing an enabling environment? How can governments improve their own risk management, and what happens when they fail or lack capacity, as in many fragile and conflict-affected states? Through what mechanisms can risk management be mainstreamed into the development agenda? And how can collective action failures to manage systemic risks be addressed, especially those with irreversible consequences? The WDR 2014 provides policy makers with insights and recommendations to address these difficult questions. It should serve to guide the dialogue, operations, and contributions from key development actors—from civil society and national governments to the donor community and international development organizations.Publication World Development Report 2018(Washington, DC: World Bank, 2018)Every year, the World Bank's World Development Report takes on a topic of central importance to global development. The 2018 Report, Learning to Realize Education's Promise, is the first ever devoted entirely to education. Now is an excellent time for it: education has long been critical for human welfare, but is even more so in a time of rapid economic change. The Report explores four main themes. First, education's promise: Education is a powerful instrument for eradicating poverty and promoting shared prosperity, but fulfilling its potential requires better policies - both within and outside the education system. Second, the learning crisis: Despite gains in education access, recent learning assessments show that many young people around the world, especially from poor families, are leaving school unequipped with even the most foundational skills they need for life. At the same time, internationally comparable learning assessments show that skills in many middle-income countries lag far behind what those countries aspire to. Third, promising interventions to improve learning: Research from areas such as brain science, pedagogical innovations, or school management have identified interventions that promote learning by ensuring that learners are prepared, that teachers are skilled as well as motivated, and that other inputs support the teacher-learner relationship. Fourth, learning at scale: Achieving learning throughout an education system will require more than just scaling up effective interventions. Change requires overcoming technical and political barriers by deploying salient metrics for mobilizing actors and tracking progress, building coalitions for learning, and being adaptive when implementing programs.Publication The Power of Survey Design : A User's Guide for Managing Surveys, Interpreting Results, and Influencing Respondents(Washington, DC: World Bank, 2006)The vast majority of data used for economic research, analysis, and policy design comes from surveys-surveys of households, firms, schools, hospitals, and market participants, and, the accuracy of the estimate will depend on how well the survey is done. This innovative book is both a 'how-to' go about carrying out high-quality surveys, especially in the challenging environment of developing countries, and a 'user's guide' for anyone who uses statistical data. Reading this book will provide data users with a wealth of insight into what kinds of problems, or biases to look for in different data sources, based on the underlying survey approaches that were used to generate the data. In that sense the book is an invaluable 'skeptics guide to data'. Yet, the broad storyline of the book is something that should be absorbed by statistical data users. The book will teach and show how difficult it often is to obtain reliable estimates of important social and economic facts, and, therefore encourages you to approach all estimates with sensible caution.