Publication: Using Paradata to Assess Respondent Burden and Interviewer Effects in Household Surveys: Evidence from Low- and Middle-Income Countries
Loading...
Date
2023-06-15
ISSN
Published
2023-06-15
Author(s)
Editor(s)
Abstract
Over the past decade, national statistical offices in low- and middle-income countries have increasingly transitioned to computer-assisted personal interviewing and computer-assisted telephone interviewing for the implementation of household surveys. The byproducts of these types of data collection are survey paradata, which can unlock objective, module- and question-specific, actionable insights on respondent burden, survey costs, and interviewer effects. This study does precisely that, using paradata generated by the Survey Solutions computer-assisted personal interviewing platform in recent national household surveys implemented by the national statistical offices in Cambodia, Ethiopia, and Tanzania. Across countries, the average household interview, based on a socioeconomic household questionnaire, ranges from 82 to 120 minutes, while the average interview with an adult household member, based on a multi-topic individual questionnaire, takes between 13 to 25 minutes. Using a multilevel model that is estimated for each household and individual questionnaire module, the paper shows that interviewer effects on module duration are significantly larger than the estimates from high-income contexts. Food consumption, household roster, and non-farm enterprises consistently emerge among the top five household questionnaire modules in terms of total variance in duration, with 5 to 50 percent of the variability being attributable to interviewers. Similarly, labor, health, and land ownership appear among the top five individual questionnaire modules in terms of total variance in duration, with 6 to 50 percent of the variability being attributable to interviewers. These findings, particularly by module, point to where additional interviewer training, fieldwork supervision, and data quality monitoring may be needed in future surveys.
Link to Data Set
Citation
“Hasanbasri, Ardina; Kilic, Talip; Koolwal, Gayatri; Moylan, Heather. 2023. Using Paradata to Assess Respondent Burden and Interviewer Effects in Household Surveys: Evidence from Low- and Middle-Income Countries. Policy Research Working Papers; 10456. © World Bank. http://hdl.handle.net/10986/39878 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication LSMS+ Program(World Bank, Washington, DC, 2021-05-05)Established in 2016, the World Bank living standards measurement study - plus (LSMS+) program works to enhance the availability and quality of intra-household, self-reported, individual-disaggregated survey data collected in low- and middle-income countries on key dimensions of men’s and women’s economic opportunities and welfare. This report presents an overview of the LSMS+ program and provides operational guidance regarding individual-disaggregated data collection in large-scale household surveys, based on the experience with and analysis of the national surveys that have been implemented by the respective national statistical offices (NSOs) in Cambodia, Ethiopia, Malawi, Tanzania over the period 2016-2020, with support from the LSMS+ program.Publication LSMS+ Program in Sub-Saharan Africa(World Bank, Washington, DC, 2021-05-05)Established in 2016, the World Bank living standards measurement study - plus (LSMS+) program works to enhance the availability and quality of intra-household, self-reported, individual-disaggregated survey data collected in low- and middle-income countries on key dimensions of men’s and women’s economic opportunities and welfare. This report presents findings on gender differences in labor market outcomes and ownership of physical and financial assets in Sub-Saharan Africa, based on the national surveys that have been implemented by the respective national statistical offices (NSOs) in Ethiopia, Malawi, and Tanzania over the period 2016-2020, with support from the LSMS+ program.Publication Multidimensionality of Land Ownership among Men and Women in Sub-Saharan Africa(World Bank, Washington, DC, 2022-01)Across Sub-Saharan African countries with customary tenure systems and low levels of documented land ownership, there are limited nationally representative insights on men and women landowners’ rights over land. Variations in institutions and norms governing land ownership further complicate cross-country comparisons. Using machine learning techniques and nationally representative, intrahousehold survey data elicited in private from men and women on their ownership of assets, this paper creates unique profiles of landowners in Ethiopia, Malawi, and Tanzania, anchored in a range of constructs related to self-reported rights and control over land parcels. The analysis reveals a high degree of cross-country consistency in the new insights. Landowners, particularly women, often do not have full rights and decision-making power over land. Multiple correspondence analysis demonstrates that transfer rights (rights to bequeath, sell, rent out, and use as collateral) contribute the most to the variation in the composition of the constructs related to rights and control over land. Hierarchical clustering shows that landowners can effectively be clustered into three categories: (1) owners with mostly exclusive transfer rights, (2) owners with mostly joint transfer rights, and (3) owners with no/limited transfer rights. Owners with transfer rights tend to have all other rights and measures of control. Women are overrepresented in the cluster of landowners with no/limited transfer rights, and in moving from the cluster with mostly joint transfer rights to the one with mostly exclusive transfer rights, the increase in the share of individuals not needing permission to exercise any right is considerably greater among women than men.Publication Individual Wealth and Time Use(World Bank, Washington, DC, 2021-08)A better understanding of how individual wealth and time use are linked—across paid, unpaid, and leisure activities —is important for targeting widespread gender inequalities in time allocation, as well as in accessing economic opportunities. The lack of reliable, individual-level data on asset ownership across different subpopulations, however, has limited discussions of these issues in the literature. Using a unique nationally representative survey from Cambodia, this paper shows that individual wealth, as measured through self-reported ownership of physical and financial assets, is significantly associated with time allocation to different activities. The role of asset ownership in time use is also stronger, particularly among women, vis-à-vis the competing proxies for socioeconomic status. Ownership of financial accounts, motorized vehicles, and mobile phones —all of which can improve access to networks, markets, and services—is associated with less time in unpaid work, and in some cases greater time in paid work, specifically among women in off-farm jobs. There are also distinct gender differences in how men and women shift their time away from leisure and childcare, highlighting the importance of social norms in choices over time use. The analysis highlights the utility of integrated, intra-household, individual-disaggregated data collection on asset ownership, time use, and employment in lower-income contexts.Publication Are You Being Asked? Impacts of Respondent Selection on Measuring Employment(World Bank, Washington, DC, 2020-02)Accurate estimates of men's and women's employment are at the heart of understanding sources of productivity and economic growth and designing well-targeted, gender-sensitive labor policies. How respondent selection in household and labor force surveys affects these estimates is a key question, for which experimental evidence outside high-income settings is limited. Leveraging two concurrent, national surveys in Malawi that differed in their approach to respondent selection, the analysis shows that, compared with the best practice of privately interviewing adults about their employment outcomes, the common "business-as-usual" approach that permits the use of proxy respondents and non-private/group interviews leads to significant underreporting of employment across a range of wage and self-employment activities, with stronger effects for women and for a longer (12-month) recall period. Under the business-as-usual approach, the main factors linked to underreporting include household wealth, proxy reporting, and potential difficulties associated with interpreting/answering questions on household non-farm enterprises.
Users also downloaded
Showing related downloaded files
Publication The Journey Ahead(Washington, DC: World Bank, 2024-10-31)The Journey Ahead: Supporting Successful Migration in Europe and Central Asia provides an in-depth analysis of international migration in Europe and Central Asia (ECA) and the implications for policy making. By identifying challenges and opportunities associated with migration in the region, it aims to inform a more nuanced, evidencebased debate on the costs and benefits of cross-border mobility. Using data-driven insights and new analysis, the report shows that migration has been an engine of prosperity and has helped address some of ECA’s demographic and socioeconomic disparities. Yet, migration’s full economic potential remains untapped. The report identifies multiple barriers keeping migration from achieving its full potential. Crucially, it argues that policies in both origin and destination countries can help maximize the development impacts of migration and effectively manage the economic, social, and political costs. Drawing from a wide range of literature, country experiences, and novel analysis, The Journey Ahead presents actionable policy options to enhance the benefits of migration for destination and origin countries and migrants themselves. Some measures can be taken unilaterally by countries, whereas others require close bilateral or regional coordination. The recommendations are tailored to different types of migration— forced displacement as well as high-skilled and low-skilled economic migration—and from the perspectives of both sending and receiving countries. This report serves as a comprehensive resource for governments, development partners, and other stakeholders throughout Europe and Central Asia, where the richness and diversity of migration experiences provide valuable insights for policy makers in other regions of the world.Publication Economic Recovery(World Bank, Washington, DC, 2021-04-06)World Bank Group President David Malpass spoke about the world facing major challenges, including COVID, climate change, rising poverty and inequality and growing fragility and violence in many countries. He highlighted vaccines, working closely with Gavi, WHO, and UNICEF, the World Bank has conducted over one hundred capacity assessments, many even more before vaccines were available. The World Bank Group worked to achieve a debt service suspension initiative and increased transparency in debt contracts at developing countries. The World Bank Group is finalizing a new climate change action plan, which includes a big step up in financing, building on their record climate financing over the past two years. He noted big challenges to bring all together to achieve GRID: green, resilient, and inclusive development. Janet Yellen, U.S. Secretary of the Treasury, mentioned focusing on vulnerable people during the pandemic. Kristalina Georgieva, Managing Director of the International Monetary Fund, focused on giving everyone a fair shot during a sustainable recovery. All three commented on the importance of tackling climate change.Publication South Asia Development Update, April 2024: Jobs for Resilience(Washington, DC: World Bank, 2024-04-02)South Asia is expected to continue to be the fastest-growing emerging market and developing economy (EMDE) region over the next two years. This is largely thanks to robust growth in India, but growth is also expected to pick up in most other South Asian economies. However, growth in the near-term is more reliant on the public sector than elsewhere, whereas private investment, in particular, continues to be weak. Efforts to rein in elevated debt, borrowing costs, and fiscal deficits may eventually weigh on growth and limit governments' ability to respond to increasingly frequent climate shocks. Yet, the provision of public goods is among the most effective strategies for climate adaptation. This is especially the case for households and farms, which tend to rely on shifting their efforts to non-agricultural jobs. These strategies are less effective forms of climate adaptation, in part because opportunities to move out of agriculture are limited by the region’s below-average employment ratios in the non-agricultural sector and for women. Because employment growth is falling short of working-age population growth, the region fails to fully capitalize on its demographic dividend. Vibrant, competitive firms are key to unlocking the demographic dividend, robust private investment, and workers’ ability to move out of agriculture. A range of policies could spur firm growth, including improved business climates and institutions, the removal of financial sector restrictions, and greater openness to trade and capital flows.Publication Remarks at the United Nations Biodiversity Conference(World Bank, Washington, DC, 2021-10-12)World Bank Group President David Malpass discussed biodiversity and climate change being closely interlinked, with terrestrial and marine ecosystems serving as critically important carbon sinks. At the same time climate change acts as a direct driver of biodiversity and ecosystem services loss. The World Bank has financed biodiversity conservation around the world, including over 116 million hectares of Marine and Coastal Protected Areas, 10 million hectares of Terrestrial Protected Areas, and over 300 protected habitats, biological buffer zones and reserves. The COVID pandemic, biodiversity loss, climate change are all reminders of how connected we are. The recovery from this pandemic is an opportunity to put in place more effective policies, institutions, and resources to address biodiversity loss.Publication Media and Messages for Nutrition and Health(World Bank, Washington, DC, 2020-06)The Lao People’s Democratic Republic (Lao PDR) has experienced rapid and significant economic growth over the past decade. However, poor nutritional outcomes remain a concern. Rates of childhood undernutrition are particularly high in remote, rural, and upland areas. Media have the potential to play an important role in shaping health and nutrition–related behaviors and practices as well as in promoting sociocultural and economic development that might contribute to improved nutritional outcomes. This report presents the results of a media audit (MA) that was conducted to inform the development and production of mass media advocacy and communication strategies and materials with a focus on maternal and child health and nutrition that would reach the most people from the poorest communities in northern Lao PDR. Making more people aware of useful information, essential services and products and influencing them to use these effectively is the ultimate goal of mass media campaigns, and the MA measures the potential effectiveness of media efforts to reach this goal. The effectiveness of communication channels to deliver health and nutrition messages to target beneficiaries to ensure maximum reach and uptake can be viewed in terms of preferences, satisfaction, and trust. Overall, the four most accessed media channels for receiving information among communities in the study areas were village announcements, mobile phones, television, and out-of-home (OOH) media. Of the accessed media channels, the top three most preferred channels were village announcements (40 percent), television (26 percent), and mobile phones (19 percent). In terms of trust, village announcements were the most trusted source of information (64 percent), followed by mobile phones (14 percent) and television (11 percent). Hence of all the media channels, village announcements are the most preferred, have the most satisfied users, and are the most trusted source of information in study communities from four provinces in Lao PDR with some of the highest burden of childhood undernutrition.