Publication: The Mystery of the Vanishing Benefits : An Introduction to Impact Evaluation
Loading...
Published
2001-01
ISSN
Date
2014-03-04
Author(s)
Editor(s)
Abstract
This article provides an introduction to the concepts and methods of impact evaluation. The author provides an intuitive explanation in the context of a concrete application. The article takes the form of a short story about a fictional character's on-the-job training in evaluation. Ms. Speedy analyst is an economist in the Ministry of Finance in the fictional country of Labas. In the process of figuring out how to evaluate a human resource program targeted to the poor, Ms. Analyst learns the strengths and weaknesses of the main methods of ex post impact evaluation.
Link to Data Set
Citation
“Ravallion, Martin. 2001. The Mystery of the Vanishing Benefits : An Introduction to Impact Evaluation. World Bank Economic Review. © World Bank. http://hdl.handle.net/10986/17218 License: CC BY-NC-ND 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Journal
World Bank Economic Review
1564-698X
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
No results found.
Users also downloaded
Showing related downloaded files
Publication Ten Steps to a Results-Based Monitoring and Evaluation System : A Handbook for Development Practitioners(Washington, DC: World Bank, 2004)An effective state is essential to achieving socio-economic and sustainable development. With the advent of globalization, there are growing pressures on governments and organizations around the world to be more responsive to the demands of internal and external stakeholders for good governance, accountability and transparency, greater development effectiveness, and delivery of tangible results. Governments, parliaments, citizens, the private sector, Non-governmental Organizations (NGOs), civil society, international organizations, and donors are among the stakeholders interested in better performance. As demands for greater accountability and real results have increased, there is an attendant need for enhanced results-based monitoring and evaluation of policies, programs, and projects. This handbook provides a comprehensive ten-step model that will help guide development practitioners through the process of designing and building a results-based monitoring and evaluation system. These steps begin with a 'readiness assessment' and take the practitioner through the design, management, and importantly, the sustainability of such systems. The handbook describes each step in detail, the tasks needed to complete each one, and the tools available to help along the way.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Impact Evaluation in Practice, Second Edition(Washington, DC: Inter-American Development Bank and World Bank, 2016-09-13)The second edition of the Impact Evaluation in Practice handbook is a comprehensive and accessible introduction to impact evaluation for policy makers and development practitioners. First published in 2011, it has been used widely across the development and academic communities. The book incorporates real-world examples to present practical guidelines for designing and implementing impact evaluations. Readers will gain an understanding of impact evaluations and the best ways to use them to design evidence-based policies and programs. The updated version covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice, as well as an expanded set of examples and case studies that draw on recent development challenges. It also includes new material on research ethics and partnerships to conduct impact evaluation. The handbook is divided into four sections: Part One discusses what to evaluate and why; Part Two presents the main impact evaluation methods; Part Three addresses how to manage impact evaluations; Part Four reviews impact evaluation sampling and data collection. Case studies illustrate different applications of impact evaluations. The book links to complementary instructional material available online, including an applied case as well as questions and answers. The updated second edition will be a valuable resource for the international development community, universities, and policy makers looking to build better evidence around what works in development.Publication Labor and Welfare Impacts of a Large-Scale Livelihoods Program(World Bank, Washington, DC, 2019-06)Improving the livelihoods of poor households and transitioning more women back to the labor force is a major challenge in South Asia. Self-employment promoted through women's groups has often been cited as a promising intervention towards this end. However, the evidence on the impact of such programs on household income and labor outcomes is limited, especially for government programs like the National Rural Livelihoods Mission in India. This study aims to provide empirical evidence on the welfare impacts of an "intensive approach" adopted under this program. The data for the study come from 4,316 household surveys in 727 villages. The study uses matching methods with the population and socioeconomic census, as well as an instrumental variable approach to construct a retrospective control group. The analysis finds that the program has been able to achieve its primary objective of improving livelihoods by transitioning more women into work. The program has also expanded access to credit, increased the proportion of savings, and reduced interest rates on credit for rural households. This is the first study to estimate the annual income effects of a government-run rural livelihoods program in India, and it shows significant increases in median income across the sample. The results for 30th, 40th, and 75th percentiles are also large and significant. However, the study did not find significant average treatment effects for income. Contrary to previous studies, this study finds weaker impacts on assets, except for livestock.Publication Impact Evaluation in International Development(Washington, DC: World Bank, 2022-03-17)Impact evaluations are studies that attempt to measure the causal impact of a project, program, or policy on one or more outcomes. This book provides a comprehensive exposition of how to conduct impact evaluations. Part I provides an overview of impact evaluations and comprises five chapters which are accessible to readers who have few or none of the technical (statistical and econometric) skills that are needed to conduct impact evaluations. Parts II and III make use of statistical and econometric methods and are at a level similar to a graduate-student course but written to make them accessible to the ambitious reader whose skills are not at that level. Part II presents, in Chapters 6-10, a comprehensive discussion of the use of randomized controlled trials (RCTs) to conduct impact evaluations, including a general discussion of the ethical issues involved in conducting impact evaluations. Part III presents the main nonexperimental methods that are used to implement impact evaluations when RCTs are not feasible or not recommended for other reasons. Chapters 11 and 12 present regression methods, including difference-in-differences estimation. Matching methods are described in Chapter 13, after which regression discontinuity methods are covered in Chapter 14. Instrumental variable methods, including the estimation of local average treatment effects (LATE), are discussed in detail in Chapter 15. Chapters 16 and 17 cover more advanced topics: quantile treatment effects and control function methods, respectively. Part IV then considers more practical issues when conducting impact evaluations, including designing questionnaires (Chapter 18), data collection methods and survey management (Chapters 19 and 20), and disseminating results to policymakers (Chapter 21). Finally, Part V addresses two topics in impact evaluation: qualitative methods for conducting impact evaluations (Chapter 22), and cost-benefit and cost-effectiveness analysis (Chapter 23).