Publication: Handbook on Impact Evaluation :
Quantitative Methods and Practices
Loading...
Date
2010
ISSN
Published
2010
Editor(s)
Abstract
This book reviews quantitative methods and models of impact evaluation. The formal literature on impact evaluation methods and practices is large, with a few useful overviews. Yet there is a need to put the theory into practice in a hands-on fashion for practitioners. This book also details challenges and goals in other realms of evaluation, including monitoring and evaluation (M&E), operational evaluation, and mixed-methods approaches combining quantitative and qualitative analyses. This book is organized as follows. Chapter two reviews the basic issues pertaining to an evaluation of an intervention to reach certain targets and goals. It distinguishes impact evaluation from related concepts such as M&E, operational evaluation, qualitative versus quantitative evaluation, and ex-ante versus ex post impact evaluation. Chapter three focuses on the experimental design of an impact evaluation, discussing its strengths and shortcomings. Various non-experimental methods exist as well, each of which are discussed in turn through chapters four to seven. Chapter four examines matching methods, including the propensity score matching technique. Chapter five deal with double-difference methods in the context of panel data, which relax some of the assumptions on the potential sources of selection bias. Chapter six reviews the instrumental variable method, which further relaxes assumptions on self-selection. Chapter seven examines regression discontinuity and pipeline methods, which exploit the design of the program itself as potential sources of identification of program impacts. Specifically, chapter eight presents a discussion of how distributional impacts of programs can be measured, including new techniques related to quantile regression. Chapter nine discusses structural approaches to program evaluation, including economic models that can lay the groundwork for estimating direct and indirect effects of a program. Finally, chapter ten discusses the strengths and weaknesses of experimental and non-experimental methods and also highlights the usefulness of impact evaluation tools in policy making.
Link to Data Set
Citation
“Khandker, Shahidur R.; Koolwal, Gayatri B.; Samad, Hussain A.. 2010. Handbook on Impact Evaluation :
Quantitative Methods and Practices. © World Bank. http://hdl.handle.net/10986/2693 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Evaluating the Impact of Development Projects on Poverty : A Handbook for Practitioners(Washington, DC: World Bank, 2000-05)Very little is known about the actual impact of projects on the poor. Many are reluctant to carry out impact evaluations because they are deemed expensive, time consuming, and technically complex, and because the findings can be politically sensitive. Yet a rigorous evaluation can be powerful in assessing the appropriateness and effectiveness of programs. Evaluating impact is particularly critical in developing where resources are scarce and every dollar spent should aim to maximize its impact on poverty reduction. This handbook seeks to provide project managers and policy analysts with the tools needed for evaluating project impact. It is aimed at readers with a general knowledge of statistics. Chapter 1 presents an overview of concepts and methods, Chapter 2 discusses key steps and related issues to consider in implementation, Chapter 3 illustrates various analytical techniques through a case study, and Chapter 4 includes a discussion of lessons that have been reviewed for this handbook. The case studies, included in Annex I, were selected from a range of evaluations carried out by the Bank, other donor agencies, research institutions, and private consulting firms. Also included in the annexes are samples of the main components that would be necessary in planning any impact evaluation - sample terms of reference, a budget, impact indicators, a log frame, and a matrix of analysis.Publication Using Mixed Methods in Monitoring and Evaluation : Experiences from International Development(2010-03-01)This paper provides an overview of the various ways in which mixing qualitative and quantitative methods could add value to monitoring and evaluating development projects. In particular it examines how qualitative methods could address some of the limitations of randomized trials and other quantitative impact evaluation methods; it also explores the importance of examining "process" in addition to "impact", distinguishing design from implementation failures, and the value of mixed methods in the real-time monitoring of projects. It concludes by suggesting topics for future research -- including the use of mixed methods in constructing counterfactuals, and in conducting reasonable evaluations within severe time and budget constraints.Publication Planning, Monitoring, and Evaluation : Methods and Tools for Poverty and Inequality Reduction Programs(World Bank, Washington, DC, 2013-01)As the author enter the second decade of the twenty-first century, governments, international organizations, nongovernmental organizations (NGOs), philanthropic organizations, and civil society groups worldwide are actively focusing on evidence-based policy and increased accountability to stakeholders (results agenda).The widespread implementation of the Results Agenda has generated a plethora of books, guides, academic papers, trainings, and case studies, which has enabled an ongoing maturation process in the field. Consequently, specialists are now better equipped to understand what works under which circumstances. Broadly speaking there are two interrelated questions which must be answered when assessing the sustainability of a government results agenda. First, is the institutional design and practice of government conducive to evidence-based policy making? Second, are the overarching monitoring and evaluation (M&E) methods and specific tools used appropriate for garnering the evidence demanded by government? These series of notes aim to make a small contribution to the latter question by summarizing and highlighting a selection of PM&E methods and the tools that governments and international organizations around the world have developed to put these into practice in their own contexts. The central goal of this initiative is to prompt a process of learning, reflection and action by providing practical information to those whose leadership role requires them to understand PM&E methods and their potential for enhancing evidence-based policy making.Publication Integrating Qualitative Methods into Investment Climate Impact Evaluations(World Bank Group, Washington, DC, 2014-12)Incorporating qualitative methods into the evaluation of development programs has become increasingly popular in recent years, both for the distinctive insights such approaches can bring in their own right and because of their capacity to complement the strengths -- and where necessary correct some of the weaknesses -- of quantitative approaches. Some initial work deploying mixed methods has been undertaken in the assessment of investment climate reforms, but considerable room for expansion exists. This paper summarizes some of the key principles and practices underpinning mixed methods evaluations in development, highlight some notable examples of how such work has been conducted (and the particular contributions it has made), and offers some guidelines for those seeking to increase the sophistication and utility of qualitative methods in the evaluation of investment climate reforms.Publication Impact Evaluation in Practice, First Edition(World Bank, 2011)The Impact Evaluation in Practice handbook is a comprehensive and accessible introduction to impact evaluation for policymakers and development practitioners. The book incorporates real-world examples to present practical guidelines for designing and implementing evaluations. Readers will gain an understanding of the uses of impact evaluation and the best ways to use evaluations to design policies and programs that are based on evidence of what works most effectively. The handbook is divided into three sections: Part One discusses what to evaluate and why; Part Two outlines the theoretical underpinnings of impact evaluation; and Part Three examines how to implement an evaluation. Case studies illustrate different methods for carrying out impact evaluations.
Users also downloaded
Showing related downloaded files
Publication Commodity Markets Outlook, October 2024(Washington, DC: World Bank, 2024-10-29)Commodity prices are expected to decrease by 5 percent in 2025 and 2 percent in 2026. The projected declines are led by oil prices but tempered by price increases for natural gas and a stable outlook for metals and agricultural raw materials. The possibility of escalating conflict in the Middle East represents a substantial near-term upside risk to energy prices, with potential knock-on consequences for other commodities. However, over the forecast horizon, longer-term dynamics—including decelerating global oil demand, diversifying oil production, and ample oil supply capacity—suggest sizable downside risks to oil prices, especially if OPEC+ unwinds its latest production cuts. There are also dual risks to industrial commodity demand stemming from economic activity. On the one hand, concerted stimulus in China and above-trend growth in the United States could push commodity prices higher. On the other, weaker-than-anticipated global industrial activity could dampen them. Following several overlapping global shocks in the early 2020s, which drove parallel swings in commodity prices, commodity markets appear to be departing from a period of tight synchronization. A Special Focus analyzes commodity price synchronization over time and considers the relative importance across commodity cycles of a wide range of demand and supply shocks, including global demand shocks and shocks specific to different commodity markets. It concludes that, while supply shocks were the dominant commodity price driver in the early 2000s and around the global financial crisis, post-pandemic price movements have been more substantially shaped by commodity-specific shocks, such as those related to conflicts.Publication Latin America and the Caribbean Economic Review, April 2024 - Competition: The Missing Ingredient for Growth?(Washington, DC: World Bank, 2024-04-10)Latin America and the Caribbean has made slow but consistent progress addressing the imbalances induced by the pandemic in an international environment that is just now showing signs of stabilizing. Despite favorable macroeconomic management, high interest rates and fiscal imbalances remain challenging while growth rates remain lackluster due to long-standing structural issues. Looking forward, an aging workforce and rising violence will increasingly complicate policy. This report focuses particularly on weak competitive forces as a source of low productivity, low growth, and low welfare in LAC. It emphasizes the need for effective competition institutions, pro-competition regulatory frameworks, complementary policies to improve the capabilities of workers and firms, and enhanced innovation systems, to prepare local industries to reach the technological frontier and face global competition. Furthermore, the report underscores the need for reforms to prevent large businesses from exerting undue political influence over policy decisions.Publication Global Economic Prospects, January 2024(Washington, DC: World Bank, 2024-01-09)Note: Chart 1.2.B has been updated on January 18, 2024. Chart 2.2.3 B has been updated on January 14, 2024. Global growth is expected to slow further this year, reflecting the lagged and ongoing effects of tight monetary policy to rein in inflation, restrictive credit conditions, and anemic global trade and investment. Downside risks include an escalation of the recent conflict in the Middle East, financial stress, persistent inflation, weaker-than-expected activity in China, trade fragmentation, and climate-related disasters. Against this backdrop, policy makers face enormous challenges. In emerging market and developing economies (EMDEs), commodity exporters face the enduring challenges posed by fiscal policy procyclicality and volatility, which highlight the need for robust fiscal frameworks. Across EMDEs, previous episodes of investment growth acceleration underscore the critical importance of macroeconomic and structural policies and an enabling institutional environment in bolstering investment and long-term growth. At the global level, cooperation needs to be strengthened to provide debt relief, facilitate trade integration, tackle climate change, and alleviate food insecurity.Publication World Development Report 2023: Migrants, Refugees, and Societies(Washington, DC : World Bank, 2023-04-25)Migration is a development challenge. About 184 million people—2.3 percent of the world’s population—live outside of their country of nationality. Almost half of them are in low- and middle-income countries. But what lies ahead? As the world struggles to cope with global economic imbalances, diverging demographic trends, and climate change, migration will become a necessity in the decades to come for countries at all levels of income. If managed well, migration can be a force for prosperity and can help achieve the United Nations’ Sustainable Development Goals. World Development Report 2023 proposes an innovative approach to maximize the development impacts of cross-border movements on both destination and origin countries and on migrants and refugees themselves. The framework it offers, drawn from labor economics and international law, rests on a “Match and Motive Matrix” that focuses on two factors: how closely migrants’ skills and attributes match the needs of destination countries and what motives underlie their movements. This approach enables policy makers to distinguish between different types of movements and to design migration policies for each. International cooperation will be critical to the effective management of migration.Publication At a Crossroads(World Bank, Washington, DC, 2017-05-02)Higher education (HE) has expanded dramatically in Latin America and the Caribbean (LAC) since 2000. While access became more equitable, quality concerns remain. This volume studies the expansion, as well as HE quality, variety and equity in LAC. It investigates the expansion’s demand and supply drivers, and outlines policy implications.