Publication: The Power of Survey Design : A User's Guide for Managing Surveys, Interpreting Results, and Influencing Respondents
Loading...
Other Files
15,492 downloads
Published
2006
ISSN
Date
2012-06-04
Author(s)
Editor(s)
Abstract
The vast majority of data used for economic research, analysis, and policy design comes from surveys-surveys of households, firms, schools, hospitals, and market participants, and, the accuracy of the estimate will depend on how well the survey is done. This innovative book is both a 'how-to' go about carrying out high-quality surveys, especially in the challenging environment of developing countries, and a 'user's guide' for anyone who uses statistical data. Reading this book will provide data users with a wealth of insight into what kinds of problems, or biases to look for in different data sources, based on the underlying survey approaches that were used to generate the data. In that sense the book is an invaluable 'skeptics guide to data'. Yet, the broad storyline of the book is something that should be absorbed by statistical data users. The book will teach and show how difficult it often is to obtain reliable estimates of important social and economic facts, and, therefore encourages you to approach all estimates with sensible caution.
Link to Data Set
Citation
“Iarossi, Giuseppe. 2006. The Power of Survey Design : A User's Guide for Managing Surveys, Interpreting Results, and Influencing Respondents. © World Bank. http://hdl.handle.net/10986/6975 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Does Respondent Reticence Affect the Results of Corruption Surveys? Evidence from the World Bank Enterprise Survey for Nigeria(2010-09-01)A potential concern with survey-based data on corruption is that respondents may not be fully candid in their responses to sensitive questions. If reticent respondents are less likely to admit to involvement in corrupt acts, and if the proportion of reticent respondents varies across groups of interest, comparisons of reported corruption across those groups can be misleading. This paper implements a variant on random response techniques that allows for identification of reticent respondents in the World Bank s Enterprise Survey for Nigeria fielded in 2008 and 2009. The authors find that 13.1 percent of respondents are highly likely to be reticent, and that these reticent respondents admit to sensitive acts at a significantly lower rate than possibly candid respondents when survey questions are worded in a way that implies personal wrongdoing on the part of the respondent.Publication Governance and Anti-corrupton Diagnostics : Guidance and Tools for Implementation, Monitoring, and Assessment in the Field(World Bank, Washington, DC, 2010-09)This guide focuses on one of the more granular approaches to governance improvement, the governance and anti-corruption diagnostic. Governance and anti-corruption diagnostics are used as an initial strategy to identify the nature of governance problems, to target the key sources or institutions associated with these problems, and to establish a baseline and indicators which can be used to make reforms and to measure progress over time. Increasingly, these diagnostics are becoming sectoral in nature, meaning they are customized to assess governance and anti-corruption in a targeted sector. If conducted properly, the diagnostic process informs and catalyzes stakeholders to demand reform. If repeated periodically, these diagnostics can become useful tools to monitor governance and anti-corruption over time. More specifically, governance and anti-corruption diagnostics: 1) Unbundle corruption by type - administrative, capture of the state, bidding, theft of goods and public resources, purchase of licenses and regulations; 2) Identify both weak institutions (which are in need of reform) and strong institutions (which provide examples of good governance); 3) Assess the cost of each type of corruption on different groups of stakeholders; 4) Provide insight into the relationship between corruption, service quality and access, and trust in public institutions; 5) Identify key determinants of good governance; and 6) Serve as a strong foundation for policy recommendations and reform.Publication Measuring Financial Capability : Questionnaires and Implementation Guidance for Low-, and Middle-Income Countries(World Bank, Washington, DC, 2013-06)This manual is designed to provide guidance to institutions, researchers, and survey firms on how to measure financial capability in middle, and low income countries using a new survey instrument that was developed and tested, from start to finish, in middle- and low-income countries. This new survey was developed as part of a larger project financed by the Russia Financial Literacy and Education Trust Fund (RTF) and implemented by the World Bank that encompassed both measurement of financial capability and evaluation of financial literacy programs. The development of the new RTF Financial Capability Survey (FCS) was done in collaboration with a team of external experts and teams from a total of 12 low- and middle-income countries. The full description of how the survey was developed can be found in Kempson, Perotti, and Scott (2013) along with findings from the first wave of surveys that have been done. In this manual we briefly summarize reasons why the FCS might be of interest to a country, provide a detailed outline of the issues related to implementing the survey successfully, and demonstrate how to analyze the resulting data. The first chapter of the manual lays out the reasons why this survey could be of use to policy makers. Chapter two describes the FCS questionnaires and their goals and objectives. Topics related to fieldwork, or the implementation of the survey, are covered in chapter three. Guidance on how to analyze the survey is provided in chapter four. The survey instruments can be found in the appendixes of this manual, and related documents, interviewer and supervisor manuals, can be found on the Trust Fund website www.finlitedu.org.Publication Challenges and Opportunities of Mobile Phone-Based Data Collection : Evidence from South Sudan(World Bank, Washington, DC, 2013-01)The proliferation of mobile phones in developing countries has generated a wave of interest in collecting high-frequency socioeconomic surveys using this technology. This paper considers lessons from one such survey effort in a difficult environment -- the South Sudan Experimental Phone Survey, which gathered data on living conditions, access to services, and citizen attitudes via monthly interviews by phones provided to respondents. Non-response, particularly in later rounds of the survey, was a substantial problem, largely due to erratic functioning of the mobile network. However, selection due to non-response does not appear to have markedly affected survey results. Response rates were much higher for respondents who owned their own phones. Both compensation provided to respondents in the form of airtime and the type of phone (solar-charged or traditional) were varied experimentally. The type of phone was uncorrelated with response rates and, contrary to expectation, attrition was slightly higher for those receiving the higher level of compensation. The South Sudan Experimental Phone Survey experience suggests that mobile phones can be a viable means of data collection for some purposes, that calling people on their own phones is preferred to handing out phones, and that careful attention should be given to the potential for selective non-response.Publication Do Labor Statistics Depend on How and to Whom the Questions Are Asked? Results from a Survey Experiment in Tanzania(2010-01-01)Labor market statistics are critical for assessing and understanding economic development. In practice, widespread variation exists in how labor statistics are measured in household surveys in low-income countries. Little is known whether these differences have an effect on the labor statistics they produce. This paper analyzes these effects by implementing a survey experiment in Tanzania that varied two key dimensions: the level of detail of the questions and the type of respondent. Significant differences are observed across survey designs with respect to different labor statistics. Labor force participation rates, for example, vary by as much as 10 percentage points across the four survey assignments. Using a short labor module without screening questions on employment generates lower female labor force participation and lower rates of wage employment for both men and women. Response by proxy rather than self-report yields lower male labor force participation, lower female working hours, and lower employment in agriculture for men. The differences between proxy and self reporting seem to come from information imperfections within the household, especially with the distance in age between respondent and subject playing an important role, while gender and educational differences seem less important.
Users also downloaded
Showing related downloaded files
Publication Assessment of the Zimbabwe Public Finance Management System for Investment Lending Projects(World Bank, Washington, DC, 2015-06-29)This study was undertaken under the leadership of the Ministry of Finance and Economic Development (MOFED) to assess fiduciary risks in using country financial management (FM) systems in full, or in part, for implementing Donor and Bank-financed investment projects in Zimbabwe and to identify risk mitigation measures required for such use. Fiduciary risk is the risk that Bank funds (or donor funds) will not be used for their intended purposes or that they will be used without due attention to economy and efficiency. In projects using country FM systems, Bank funds are potentially commingled with the country’s own funds; therefore, a fiduciary risk assessment also needs to consider broader country PFM risks that could affect the fiduciary risk. This assessment uses a risk-based approach consistent with the interim guidance note issued by the FM Sector Board in 2009, entitled ‘assessment of fiduciary risks in the use of country FM systems in bank-financed investment projects’; and supplemented by the framework methodology for channeling investment lending projects through country financial management systems and the approach used for regular FM assessments. The risk-based approach provides a ranking of the fiduciary risks to be managed as high, substantial, moderate, or low. The decision to use country systems for a specific project then rests with the project's task team, guided by the country management team, after taking into account this fiduciary risk assessment and other factors such as the nature and complexity of the project and an assessment of implementing entities.Publication Innovations in Tax Compliance(Washington, DC: World Bank, 2022-02-17)Recent decades have seen important progress in strengthening country tax systems. Yet many areas of reform have remained stubbornly resistant to major improvements. Overall, revenue collection still falls short of that needed for effective governance and service delivery. Tax collection is too often riddled with high rates of evasion among large corporations and the rich and by disproportionate, though often hidden, burdens on lower-income groups. As countries around the world deal with the large debt burdens induced by COVID-19, an in-depth look at how to strengthen tax systems is especially timely. Innovations in Tax Compliance: Building Trust, Navigating Politics, and Tailoring Reform takes a fresh look at tax reform. The authors draw on recent research and experience for their new conceptual framework to guide more effective approaches to reform. Building on the achievements of recent decades, they argue for a greater emphasis on the overlapping goals of building trust, navigating political resistance, and tailoring reform to unique local contexts—an emphasis achieved by identifying the most binding constraints on reform. This focus not only can lead to greater compliance, a fairer system, and higher revenues, but also can contribute to building state capacity, sustained political support for further reforms, and a stronger fiscal contract between citizens and governments.Publication China Country Climate and Development Report(World Bank Group, Washington DC, 2022-10)The China Country Climate and Development Report (CCDR) provides analysis and recommendations on integrating the country’s efforts to achieve high-quality development with the pursuit of emission reduction and climate resilience. Without adequate mitigation and adaptation efforts, climate risks will become a growing constraint to China’s long-term growth and prosperity, threatening to reverse development gains. Conversely, if efforts to tackle climate risks lead to a significant decline in growth and rising inequality, they would deprive millions of people of development and likely erode support for the reforms necessary to achieve a lasting economic transformation. Hence, China will need to grow and green its economy at the same time. This report offers policy options to achieve these dual objectives by easing inevitable trade-offs and maximizing potential synergies between China’s development and climate objectives.Publication The Road to Results : Designing and Conducting Effective Development Evaluations(World Bank, 2009-12-01)The analytical, conceptual, and political framework of development is changing dramatically. The new development agenda calls for broader understandings of sectors, countries, development strategies, and policies. It emphasizes learning and continuous feedback at all phases of the development cycle. As the development agenda grows in scope and complexity, development evaluation follows suit. Development evaluator are moving away from traditional implementation and output-focused evaluation models toward results-based evaluation models, as the development community calls for results and embraces the millennium development goals. As the development community shifts its focus away from projects in order to comprehensively address country challenges, development evaluators are seeking methods with which to assess results at the country, sector, theme, policy, and even global levels. As the development community recognizes the importance of not only a comprehensive but also a coordinated approach to developing country challenges and emphasizes partnerships, development evaluators are increasingly engaged in joint evaluations. These joint evaluations, while advantageous in many respects, add to the complexity of development evaluation (OECD 2006). Additionally, development evaluators increasingly face the measurement challenge of determining the performance of an individual development organization in this broader context and of identifying its contribution. This text is intended as a tool for use in building development evaluation capacity. It aims to help development evaluators think about and explore the new evaluation architecture and especially to design and conduct evaluations that focus on results in meeting the challenges of development.Publication World Development Report 2014(Washington, DC, 2013-10-06)The past 25 years have witnessed unprecedented changes around the world—many of them for the better. Across the continents, many countries have embarked on a path of international integration, economic reform, technological modernization, and democratic participation. As a result, economies that had been stagnant for decades are growing, people whose families had suffered deprivation for generations are escaping poverty, and hundreds of millions are enjoying the benefits of improved living standards and scientific and cultural sharing across nations. As the world changes, a host of opportunities arise constantly. With them, however, appear old and new risks, from the possibility of job loss and disease to the potential for social unrest and environmental damage. If ignored, these risks can turn into crises that reverse hard-won gains and endanger the social and economic reforms that produced these gains. The World Development Report 2014 (WDR 2014), Risk and Opportunity: Managing Risk for Development, contends that the solution is not to reject change in order to avoid risk but to prepare for the opportunities and risks that change entails. Managing risks responsibly and effectively has the potential to bring about security and a means of progress for people in developing countries and beyond. Although individuals’ own efforts, initiative, and responsibility are essential for managing risk, their success will be limited without a supportive social environment—especially when risks are large or systemic in nature. The WDR 2014 argues that people can successfully confront risks that are beyond their means by sharing their risk management with others. This can be done through naturally occurring social and economic systems that enable people to overcome the obstacles that individuals and groups face, including lack of resources and information, cognitive and behavioral failures, missing markets and public goods, and social externalities and exclusion. These systems—from the household and the community to the state and the international community—have the potential to support people’s risk management in different yet complementary ways. The Report focuses on some of the most pressing questions policy makers are asking. What role should the state take in helping people manage risks? When should this role consist of direct interventions, and when should it consist of providing an enabling environment? How can governments improve their own risk management, and what happens when they fail or lack capacity, as in many fragile and conflict-affected states? Through what mechanisms can risk management be mainstreamed into the development agenda? And how can collective action failures to manage systemic risks be addressed, especially those with irreversible consequences? The WDR 2014 provides policy makers with insights and recommendations to address these difficult questions. It should serve to guide the dialogue, operations, and contributions from key development actors—from civil society and national governments to the donor community and international development organizations.