Publication: Interviewer Effects in Subjective Survey Questions: Evidence from Timor-Leste
Loading...
Files in English
Published
2015-03
ISSN
Date
2015-03-30
Author(s)
Editor(s)
Abstract
There is an inherent tension between traditional norms and survey protocols for quantitative data collected in the developing world. Unexpected interactions between the interviewer and respondent can lead to interviewer effects in the data, particularly in the case of subjective or sensitive questions. This paper makes use of a unique data set available from Timor-Leste containing subjective and objective questions to study these effects. In addition to their age and gender, data were collected from the interviewers regarding their opinions on the subjective questions prior to fieldwork. Fixed effects and mixed effects logit models are used to examine the main effects and interactions between interviewer and respondent characteristics. More objective measures serve as a pseudo control group. The paper finds interviewer effects in the both subjective and objective data, but the magnitude is considerably stronger for subjective questions. The paper also finds that female respondents are more susceptible to influence based on the interviewer's beliefs. Despite methodological shortcomings, the study highlights the need to consider more fully the impact of traditional cultural norms when conducting quantitative surveys in the developing world on topics that are outside the standard objective questions.
Link to Data Set
Citation
“Himelein, Kristen. 2015. Interviewer Effects in Subjective Survey Questions: Evidence from Timor-Leste. Policy Research Working Paper;No. 7208. © World Bank. http://hdl.handle.net/10986/21647 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Error: Could not load results for '/discover/search/objects?query=relation.isSeriesOfPublication%3A26e071dc-b0bf-409c-b982-df2970295c87&f.resourceid=93724322-ca22-56a8-8e72-d703e06c0cbb,notequals'.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Surveying Justice : A Practical Guide to Household Surveys(World Bank, Washington, DC, 2010-01)Though household surveys have long been an established part of development practice and regularly used to gather data on poverty incidence and the range of associated indicators, they have not yet become a common tool of justice reform practitioners. This guide aims to be a practical starting point for integrating justice work and household data collection, targeted both towards justice practitioners interested in survey design, as well as survey researchers interested in incorporating justice questions into their work. It provides guidance on designing a survey, suggested topics and questions, and ideas to facilitate a constructive engagement in discussions around justice in development practice. Household survey data can be beneficial to understanding justice questions as household surveys ordinarily cover a large, randomly selected cross-section of people - including the rich and poor, urban and rural dwellers - capturing a population's most common justice issues. Household survey questions commonly ask respondents about their most frequently experienced justice issues, issues when seeking redress, and knowledge and opinions of the law. Household surveys thus complement data collection techniques more familiar to justice practitioners (such as user surveys or sector assessments) that tend to focus on institutions of the justice sector and hence capture only the views of those who manage to access such institutions and privilege the perspectives of system incumbents. Household surveys have their limitations - not least significant cost, time and complexity implications. In addition, the standardized nature of surveys limits the type of information that can be gleaned and hence household surveys are generally most useful for gaining a picture of the "what" when it comes to justice issues, with complementary research methods often needed to properly understand the "why." Nevertheless, surveys can represent a useful starting point for engagement in a particular context, providing a snap shot of the justice landscape from which more detailed qualitative and quantitative studies can be undertaken.Publication Integrating Quantitative and Qualitative Research in Development Projects(Washington, DC: World Bank, 2000-06)"Much of the early work on poverty was highly quantitative:... It became increasingly clear, however, that while numbers are essential for policy and monitoring purposes, it is also important to understand people's perception of poverty and their mechanisms for coping with poverty and other situations of extreme economic and social stress." Researchers have recognized over the past few years that quantitative analysis of the incidence and trends in poverty, while essential for national economic development planning, must be complemented by qualitative methods that help planners and managers understand the cultural, social, political, and institutional context within which projects are designed and implemented. This report is based on a two-day workshop held in June 1998, where outside research specialists and World Bank staff discussed the importance of integrating these research methods. The participants reviewed experiences in the use of mixed-method approaches in Bank research and project design. This report is a result of those discussions. The report examines the need for integrated research approaches in social and economic development, presents case studies of integrated approaches in practice, and talks about lessons learned. Part I describes the evolution of interest in, and the potential benefits of integrated research, and, Part II presents case studies on how integrated approaches have been used in poverty analysis, education, health, and water supply and sanitation, while Part III discusses lessons learned with respect to the use of integrated approaches, and assesses the benefits that can be achieved.Publication Can Subjective Questions on Economic Welfare Be Trusted? Evidence for Three Developing Countries(World Bank, Washington, DC, 2013-12)While self-assessments of welfare have become popular for measuring poverty and estimating welfare effects, the methods can be deceptive given systematic heterogeneity in respondents' scales. Little is known about this problem. This study uses specially-designed surveys in three countries, Tajikistan, Guatemala, and Tanzania, to study scale heterogeneity. Respondents were asked to score stylized vignettes, as well as their own household. Diverse scales are in evidence, casting considerable doubt on the meaning of widely-used summary measures such as subjective poverty rates. Nonetheless, under the identifying assumptions of the study, only small biases are induced in the coefficients on widely-used regressors for subjective poverty and welfare.Publication Second-Stage Sampling for Conflict Areas(World Bank, Washington, DC, 2016-03)The collection of survey data from war zones or other unstable security situations is vulnerable to error because conflict often limits the implementation options. Although there are elevated risks throughout the process, this paper focuses specifically on challenges to frame construction and sample selection. The paper uses simulations based on data from the Mogadishu High Frequency Survey Pilot to examine the implications of the choice of second-stage selection methodology on bias and variance. Among the other findings, the simulations show the bias introduced by a random walk design leads to the underestimation of the poverty headcount by more than 10 percent. The paper also discusses the experience of the authors in the time required and technical complexity of the associated back-office preparation work and weight calculations for each method. Finally, as the simulations assume perfect implementation of the design, the paper also discusses practicality, including the ease of implementation and options for remote verification, and outlines areas for future research and pilot testing.Publication The Use of Random Geographic Cluster Sampling to Survey Pastoralists(World Bank, Washington, DC, 2013-09)Livestock are an important component of rural livelihoods in developing countries, but data about this source of income and wealth are difficult to collect because of the nomadic and semi-nomadic nature of many pastoralist populations. Most household surveys exclude those without permanent dwellings, leading to undercoverage. This study explores the use of a random geographic cluster sample as an alternative to the household-based sample. In this design, points are randomly selected and all eligible respondents found inside circles drawn around the selected points are interviewed. This approach should eliminate undercoverage of mobile populations. The results of a random geographic cluster sample survey are presented with a total sample size of 784 households to measure livestock ownership in the Afar region of Ethiopia in 2012. The paper explores the data quality of the random geographic cluster sample relative to a recent household survey and discusses the implementation challenges.
Users also downloaded
Showing related downloaded files
Publication World Development Report 2008(Washington, DC, 2007)The world's demand for food is expected to double within the next 50 years, while the natural resources that sustain agriculture will become increasingly scarce, degraded, and vulnerable to the effects of climate change. In many poor countries, agriculture accounts for at least 40 percent of GDP and 80 percent of employment. At the same time, about 70 percent of the world's poor live in rural areas and most depend on agriculture for their livelihoods. World Development Report 2008 seeks to assess where, when, and how agriculture can be an effective instrument for economic development, especially development that favors the poor. It examines several broad questions: How has agriculture changed in developing countries in the past 20 years? What are the important new challenges and opportunities for agriculture? Which new sources of agricultural growth can be captured cost effectively in particular in poor countries with large agricultural sectors as in Africa? How can agricultural growth be made more effective for poverty reduction? How can governments facilitate the transition of large populations out of agriculture, without simply transferring the burden of rural poverty to urban areas? How can the natural resource endowment for agriculture be protected? How can agriculture's negative environmental effects be contained? This year's report marks the 30th year the World Bank has been publishing the World Development Report.Publication Poverty, Prosperity, and Planet Report 2024(Washington, DC: World Bank, 2024-10-15)The Poverty, Prosperity, and Planet Report 2024 is the latest edition of the series formerly known as Poverty and Shared Prosperity. The report emphasizes that reducing poverty and increasing shared prosperity must be achieved in ways that do not come at unacceptably high costs to the environment. The current “polycrisis”—where the multiple crises of slow economic growth, increased fragility, climate risks, and heightened uncertainty have come together at the same time—makes national development strategies and international cooperation difficult. Offering the first post-Coronavirus (COVID)-19 pandemic assessment of global progress on this interlinked agenda, the report finds that global poverty reduction has resumed but at a pace slower than before the COVID-19 crisis. Nearly 700 million people worldwide live in extreme poverty with less than US$2.15 per person per day. Progress has essentially plateaued amid lower economic growth and the impacts of COVID-19 and other crises. Today, extreme poverty is concentrated mostly in Sub-Saharan Africa and fragile settings. At a higher standard more typical of upper-middle-income countries—US$6.85 per person per day—almost one-half of the world is living in poverty. The report also provides evidence that the number of countries that have high levels of income inequality has declined considerably during the past two decades, but the pace of improvements in shared prosperity has slowed, and that inequality remains high in Latin America and the Caribbean and Sub-Saharan Africa. Worldwide, people’s incomes today would need to increase fivefold on average to reach a minimum prosperity threshold of US$25 per person per day. Where there has been progress in poverty reduction and shared prosperity, there is evidence of an increasing ability of countries to manage natural hazards, but climate risks are significantly higher in the poorest settings. Nearly one in five people globally is at risk of experiencing welfare losses due to an extreme weather event from which they will struggle to recover. The interconnected issues of climate change and poverty call for a united and inclusive effort from the global community. Development cooperation stakeholders—from governments, nongovernmental organizations, and the private sector to communities and citizens acting locally in every corner of the globe—hold pivotal roles in promoting fair and sustainable transitions. By emphasizing strategies that yield multiple benefits and diligently monitoring and addressing trade-offs, we can strive toward a future that is prosperous, equitable, and resilient.Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication World Development Report 2017(Washington, DC: World Bank, 2017-01-30)Why are carefully designed, sensible policies too often not adopted or implemented? When they are, why do they often fail to generate development outcomes such as security, growth, and equity? And why do some bad policies endure? This book addresses these fundamental questions, which are at the heart of development. Policy making and policy implementation do not occur in a vacuum. Rather, they take place in complex political and social settings, in which individuals and groups with unequal power interact within changing rules as they pursue conflicting interests. The process of these interactions is what this Report calls governance, and the space in which these interactions take place, the policy arena. The capacity of actors to commit and their willingness to cooperate and coordinate to achieve socially desirable goals are what matter for effectiveness. However, who bargains, who is excluded, and what barriers block entry to the policy arena determine the selection and implementation of policies and, consequently, their impact on development outcomes. Exclusion, capture, and clientelism are manifestations of power asymmetries that lead to failures to achieve security, growth, and equity. The distribution of power in society is partly determined by history. Yet, there is room for positive change. This Report reveals that governance can mitigate, even overcome, power asymmetries to bring about more effective policy interventions that achieve sustainable improvements in security, growth, and equity. This happens by shifting the incentives of those with power, reshaping their preferences in favor of good outcomes, and taking into account the interests of previously excluded participants. These changes can come about through bargains among elites and greater citizen engagement, as well as by international actors supporting rules that strengthen coalitions for reform.Publication Expanding Opportunities: Toward Inclusive Growth(World Bank, Washington, DC, 2023-04-04)South Asia’s outlook is shaped by both good and bad news in the global economy. Lower commodity prices, a strong recovery in the services sector, and reduced disruptions in value chains are aiding South Asia’s recovery but rising interest rates and uncertainty in financial markets are putting downward pressure on the region’s economies. Countries in South Asia, especially those with large external debt, face difficult tradeoffs as they respond to these pressures. Growth prospects have weakened, with large downside risks in most countries given limited fiscal space and depleting foreign reserves. Going forward, broad reform programs, including a sustainable fiscal outlook, are needed to put South Asia on a more robust and inclusive growth path. Inequality of opportunity, which is higher in South Asia than in other regions of the world, is both unfair and inefficient. Reducing inequality of opportunity and increasing economic mobility will help broaden countries’ tax base and boost support from the population for the critical reforms.