Publication: Using Paradata to Assess Respondent Burden and Interviewer Effects in Household Surveys: Evidence from Low- and Middle-Income Countries
Loading...
Date
2023-06-15
ISSN
Published
2023-06-15
Author(s)
Editor(s)
Abstract
Over the past decade, national statistical offices in low- and middle-income countries have increasingly transitioned to computer-assisted personal interviewing and computer-assisted telephone interviewing for the implementation of household surveys. The byproducts of these types of data collection are survey paradata, which can unlock objective, module- and question-specific, actionable insights on respondent burden, survey costs, and interviewer effects. This study does precisely that, using paradata generated by the Survey Solutions computer-assisted personal interviewing platform in recent national household surveys implemented by the national statistical offices in Cambodia, Ethiopia, and Tanzania. Across countries, the average household interview, based on a socioeconomic household questionnaire, ranges from 82 to 120 minutes, while the average interview with an adult household member, based on a multi-topic individual questionnaire, takes between 13 to 25 minutes. Using a multilevel model that is estimated for each household and individual questionnaire module, the paper shows that interviewer effects on module duration are significantly larger than the estimates from high-income contexts. Food consumption, household roster, and non-farm enterprises consistently emerge among the top five household questionnaire modules in terms of total variance in duration, with 5 to 50 percent of the variability being attributable to interviewers. Similarly, labor, health, and land ownership appear among the top five individual questionnaire modules in terms of total variance in duration, with 6 to 50 percent of the variability being attributable to interviewers. These findings, particularly by module, point to where additional interviewer training, fieldwork supervision, and data quality monitoring may be needed in future surveys.
Link to Data Set
Citation
“Hasanbasri, Ardina; Kilic, Talip; Koolwal, Gayatri; Moylan, Heather. 2023. Using Paradata to Assess Respondent Burden and Interviewer Effects in Household Surveys: Evidence from Low- and Middle-Income Countries. Policy Research Working Papers; 10456. © World Bank. http://hdl.handle.net/10986/39878 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication LSMS+ Program(World Bank, Washington, DC, 2021-05-05)Established in 2016, the World Bank living standards measurement study - plus (LSMS+) program works to enhance the availability and quality of intra-household, self-reported, individual-disaggregated survey data collected in low- and middle-income countries on key dimensions of men’s and women’s economic opportunities and welfare. This report presents an overview of the LSMS+ program and provides operational guidance regarding individual-disaggregated data collection in large-scale household surveys, based on the experience with and analysis of the national surveys that have been implemented by the respective national statistical offices (NSOs) in Cambodia, Ethiopia, Malawi, Tanzania over the period 2016-2020, with support from the LSMS+ program.Publication LSMS+ Program in Sub-Saharan Africa(World Bank, Washington, DC, 2021-05-05)Established in 2016, the World Bank living standards measurement study - plus (LSMS+) program works to enhance the availability and quality of intra-household, self-reported, individual-disaggregated survey data collected in low- and middle-income countries on key dimensions of men’s and women’s economic opportunities and welfare. This report presents findings on gender differences in labor market outcomes and ownership of physical and financial assets in Sub-Saharan Africa, based on the national surveys that have been implemented by the respective national statistical offices (NSOs) in Ethiopia, Malawi, and Tanzania over the period 2016-2020, with support from the LSMS+ program.Publication Multidimensionality of Land Ownership among Men and Women in Sub-Saharan Africa(World Bank, Washington, DC, 2022-01)Across Sub-Saharan African countries with customary tenure systems and low levels of documented land ownership, there are limited nationally representative insights on men and women landowners’ rights over land. Variations in institutions and norms governing land ownership further complicate cross-country comparisons. Using machine learning techniques and nationally representative, intrahousehold survey data elicited in private from men and women on their ownership of assets, this paper creates unique profiles of landowners in Ethiopia, Malawi, and Tanzania, anchored in a range of constructs related to self-reported rights and control over land parcels. The analysis reveals a high degree of cross-country consistency in the new insights. Landowners, particularly women, often do not have full rights and decision-making power over land. Multiple correspondence analysis demonstrates that transfer rights (rights to bequeath, sell, rent out, and use as collateral) contribute the most to the variation in the composition of the constructs related to rights and control over land. Hierarchical clustering shows that landowners can effectively be clustered into three categories: (1) owners with mostly exclusive transfer rights, (2) owners with mostly joint transfer rights, and (3) owners with no/limited transfer rights. Owners with transfer rights tend to have all other rights and measures of control. Women are overrepresented in the cluster of landowners with no/limited transfer rights, and in moving from the cluster with mostly joint transfer rights to the one with mostly exclusive transfer rights, the increase in the share of individuals not needing permission to exercise any right is considerably greater among women than men.Publication Individual Wealth and Time Use(World Bank, Washington, DC, 2021-08)A better understanding of how individual wealth and time use are linked—across paid, unpaid, and leisure activities —is important for targeting widespread gender inequalities in time allocation, as well as in accessing economic opportunities. The lack of reliable, individual-level data on asset ownership across different subpopulations, however, has limited discussions of these issues in the literature. Using a unique nationally representative survey from Cambodia, this paper shows that individual wealth, as measured through self-reported ownership of physical and financial assets, is significantly associated with time allocation to different activities. The role of asset ownership in time use is also stronger, particularly among women, vis-à-vis the competing proxies for socioeconomic status. Ownership of financial accounts, motorized vehicles, and mobile phones —all of which can improve access to networks, markets, and services—is associated with less time in unpaid work, and in some cases greater time in paid work, specifically among women in off-farm jobs. There are also distinct gender differences in how men and women shift their time away from leisure and childcare, highlighting the importance of social norms in choices over time use. The analysis highlights the utility of integrated, intra-household, individual-disaggregated data collection on asset ownership, time use, and employment in lower-income contexts.Publication Are You Being Asked? Impacts of Respondent Selection on Measuring Employment(World Bank, Washington, DC, 2020-02)Accurate estimates of men's and women's employment are at the heart of understanding sources of productivity and economic growth and designing well-targeted, gender-sensitive labor policies. How respondent selection in household and labor force surveys affects these estimates is a key question, for which experimental evidence outside high-income settings is limited. Leveraging two concurrent, national surveys in Malawi that differed in their approach to respondent selection, the analysis shows that, compared with the best practice of privately interviewing adults about their employment outcomes, the common "business-as-usual" approach that permits the use of proxy respondents and non-private/group interviews leads to significant underreporting of employment across a range of wage and self-employment activities, with stronger effects for women and for a longer (12-month) recall period. Under the business-as-usual approach, the main factors linked to underreporting include household wealth, proxy reporting, and potential difficulties associated with interpreting/answering questions on household non-farm enterprises.
Users also downloaded
Showing related downloaded files
Publication Economic Recovery(World Bank, Washington, DC, 2021-04-06)World Bank Group President David Malpass spoke about the world facing major challenges, including COVID, climate change, rising poverty and inequality and growing fragility and violence in many countries. He highlighted vaccines, working closely with Gavi, WHO, and UNICEF, the World Bank has conducted over one hundred capacity assessments, many even more before vaccines were available. The World Bank Group worked to achieve a debt service suspension initiative and increased transparency in debt contracts at developing countries. The World Bank Group is finalizing a new climate change action plan, which includes a big step up in financing, building on their record climate financing over the past two years. He noted big challenges to bring all together to achieve GRID: green, resilient, and inclusive development. Janet Yellen, U.S. Secretary of the Treasury, mentioned focusing on vulnerable people during the pandemic. Kristalina Georgieva, Managing Director of the International Monetary Fund, focused on giving everyone a fair shot during a sustainable recovery. All three commented on the importance of tackling climate change.Publication Media and Messages for Nutrition and Health(World Bank, Washington, DC, 2020-06)The Lao People’s Democratic Republic (Lao PDR) has experienced rapid and significant economic growth over the past decade. However, poor nutritional outcomes remain a concern. Rates of childhood undernutrition are particularly high in remote, rural, and upland areas. Media have the potential to play an important role in shaping health and nutrition–related behaviors and practices as well as in promoting sociocultural and economic development that might contribute to improved nutritional outcomes. This report presents the results of a media audit (MA) that was conducted to inform the development and production of mass media advocacy and communication strategies and materials with a focus on maternal and child health and nutrition that would reach the most people from the poorest communities in northern Lao PDR. Making more people aware of useful information, essential services and products and influencing them to use these effectively is the ultimate goal of mass media campaigns, and the MA measures the potential effectiveness of media efforts to reach this goal. The effectiveness of communication channels to deliver health and nutrition messages to target beneficiaries to ensure maximum reach and uptake can be viewed in terms of preferences, satisfaction, and trust. Overall, the four most accessed media channels for receiving information among communities in the study areas were village announcements, mobile phones, television, and out-of-home (OOH) media. Of the accessed media channels, the top three most preferred channels were village announcements (40 percent), television (26 percent), and mobile phones (19 percent). In terms of trust, village announcements were the most trusted source of information (64 percent), followed by mobile phones (14 percent) and television (11 percent). Hence of all the media channels, village announcements are the most preferred, have the most satisfied users, and are the most trusted source of information in study communities from four provinces in Lao PDR with some of the highest burden of childhood undernutrition.Publication Remarks at the United Nations Biodiversity Conference(World Bank, Washington, DC, 2021-10-12)World Bank Group President David Malpass discussed biodiversity and climate change being closely interlinked, with terrestrial and marine ecosystems serving as critically important carbon sinks. At the same time climate change acts as a direct driver of biodiversity and ecosystem services loss. The World Bank has financed biodiversity conservation around the world, including over 116 million hectares of Marine and Coastal Protected Areas, 10 million hectares of Terrestrial Protected Areas, and over 300 protected habitats, biological buffer zones and reserves. The COVID pandemic, biodiversity loss, climate change are all reminders of how connected we are. The recovery from this pandemic is an opportunity to put in place more effective policies, institutions, and resources to address biodiversity loss.Publication Global Regulations, Institutional Development, and Market Authorities Perspective Toolkit (GRIDMAP) - Framework and Methodology(Washington, DC: World Bank, 2024-12-05)GRIDMAP--the Global Regulations, Institutional Development, and Market Authorities Perspective Toolkit--provides emerging markets and developing economies (EMDEs) with a “Minimum Package” of policies to build markets that are trustworthy, safe, and competitive. The “Minimum Package” sets out essential regulatory provisions, institutional arrangements, and implementation and enforcement needed for those markets to thrive. GRIDMAP will provide modules focused on various subjects of market regulation, such as consumer protection and data markets.Publication Education, Social Norms, and the Marriage Penalty(Washington, DC: World Bank, 2024-10-16)A growing literature attributes gender inequality in labor market outcomes in part to the reduction in female labor supply after childbirth, the child penalty. However, if social norms constrain married women’s activities outside the home, then marriage can independently reduce employment, even in the absence childbearing. Given the correlation in timing between childbirth and marriage, conventional estimates of child penalties will conflate these two effects. The paper studies the marriage penalty in South Asia, a context featuring conservative gender norms and low female labor force participation. The study introduces a split-sample, pseudo-panel approach that allows for the separation of marriage and child penalties even in the absence of individual-level panel data. Marriage reduces women’s labor force participation in South Asia by 12 percentage points, whereas the marginal penalty of childbearing is small. Consistent with the central roles of both opportunity costs and social norms, the marriage penalty is smaller among cohorts with higher education and less conservative gender attitudes.