Publication: Meta-Evaluation of IEG
Evaluations (FY15-19)
Loading...
Date
2022-02-14
ISSN
Published
2022-02-14
Author(s)
Editor(s)
Abstract
IEG’s meta-evaluation serves as an input for the upcoming independent external review of its evaluations. The report focuses on aspects of credibility related to the rationale, focus, use of innovative methods, and various research design attributes as formulated in evaluation reports and their respective approach papers. Drawing on a set of 28 evaluations published from fiscal year 2015 to 2019, the meta-evaluation offers six major conclusions and suggestions based on a systematic review of evaluation scope, reliability, validity (including construct, internal, external, and data analysis validity), consistency, and the integration of innovative methods.
Link to Data Set
Citation
“Independent Evaluation Group. 2022. Meta-Evaluation of IEG
Evaluations (FY15-19). © World Bank. http://hdl.handle.net/10986/37022 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Planning, Monitoring, and Evaluation : Methods and Tools for Poverty and Inequality Reduction Programs(World Bank, Washington, DC, 2013-01)As the author enter the second decade of the twenty-first century, governments, international organizations, nongovernmental organizations (NGOs), philanthropic organizations, and civil society groups worldwide are actively focusing on evidence-based policy and increased accountability to stakeholders (results agenda).The widespread implementation of the Results Agenda has generated a plethora of books, guides, academic papers, trainings, and case studies, which has enabled an ongoing maturation process in the field. Consequently, specialists are now better equipped to understand what works under which circumstances. Broadly speaking there are two interrelated questions which must be answered when assessing the sustainability of a government results agenda. First, is the institutional design and practice of government conducive to evidence-based policy making? Second, are the overarching monitoring and evaluation (M&E) methods and specific tools used appropriate for garnering the evidence demanded by government? These series of notes aim to make a small contribution to the latter question by summarizing and highlighting a selection of PM&E methods and the tools that governments and international organizations around the world have developed to put these into practice in their own contexts. The central goal of this initiative is to prompt a process of learning, reflection and action by providing practical information to those whose leadership role requires them to understand PM&E methods and their potential for enhancing evidence-based policy making.Publication Analyzing the Effects of Policy Reforms on the Poor : An Evaluation of the Effectiveness of World Bank Support to Poverty and Social Impact Analyses(Washington, DC: World Bank, 2010)The current global financial and economic crises are likely to put enormous pressure on governments to respond with immediate measures and to undertake far-reaching reforms in the medium term, requiring a substantial increase in donor support. To protect the poor and enhance benefits to them, key policy reforms will need to be underpinned by systematic analysis of their expected poverty and social impacts. The World Bank's experience to date with the Poverty and Social Impact Analysis (PSIA) approach provides useful lessons for addressing these issues. Overall, implementation of the PSIA approach has had considerable limitations. There have been tensions between the various operational objectives assigned to PSIAs. The tensions concern inconsistencies between informing country and Bank policy decisions in a timely way and building country analytic capacity. PSIAs have had limited ownership by Bank staff and managers and have often not been effectively integrated into country assistance programs. Quality assurance, monitoring, and evaluation of the overall effectiveness of PSIAs have been weak. To improve PSIAs' effectiveness, this evaluation recommends that the Bank take measures to ensure that staff fully understands what the PSIA approach is and when to use it, clarify the operational objectives of each PSIA, and ensure that the approach and timeline adopted are aligned with those objectives. Quality assurance mechanisms should be strengthened to ensure that PSIAs are designed to achieve the intended effects.Publication Using Mixed Methods in Monitoring and Evaluation : Experiences from International Development(2010-03-01)This paper provides an overview of the various ways in which mixing qualitative and quantitative methods could add value to monitoring and evaluating development projects. In particular it examines how qualitative methods could address some of the limitations of randomized trials and other quantitative impact evaluation methods; it also explores the importance of examining "process" in addition to "impact", distinguishing design from implementation failures, and the value of mixed methods in the real-time monitoring of projects. It concludes by suggesting topics for future research -- including the use of mixed methods in constructing counterfactuals, and in conducting reasonable evaluations within severe time and budget constraints.Publication Integrating Qualitative Methods into Investment Climate Impact Evaluations(World Bank Group, Washington, DC, 2014-12)Incorporating qualitative methods into the evaluation of development programs has become increasingly popular in recent years, both for the distinctive insights such approaches can bring in their own right and because of their capacity to complement the strengths -- and where necessary correct some of the weaknesses -- of quantitative approaches. Some initial work deploying mixed methods has been undertaken in the assessment of investment climate reforms, but considerable room for expansion exists. This paper summarizes some of the key principles and practices underpinning mixed methods evaluations in development, highlight some notable examples of how such work has been conducted (and the particular contributions it has made), and offers some guidelines for those seeking to increase the sophistication and utility of qualitative methods in the evaluation of investment climate reforms.Publication Integrating Quantitative and Qualitative Research in Development Projects(Washington, DC: World Bank, 2000-06)"Much of the early work on poverty was highly quantitative:... It became increasingly clear, however, that while numbers are essential for policy and monitoring purposes, it is also important to understand people's perception of poverty and their mechanisms for coping with poverty and other situations of extreme economic and social stress." Researchers have recognized over the past few years that quantitative analysis of the incidence and trends in poverty, while essential for national economic development planning, must be complemented by qualitative methods that help planners and managers understand the cultural, social, political, and institutional context within which projects are designed and implemented. This report is based on a two-day workshop held in June 1998, where outside research specialists and World Bank staff discussed the importance of integrating these research methods. The participants reviewed experiences in the use of mixed-method approaches in Bank research and project design. This report is a result of those discussions. The report examines the need for integrated research approaches in social and economic development, presents case studies of integrated approaches in practice, and talks about lessons learned. Part I describes the evolution of interest in, and the potential benefits of integrated research, and, Part II presents case studies on how integrated approaches have been used in poverty analysis, education, health, and water supply and sanitation, while Part III discusses lessons learned with respect to the use of integrated approaches, and assesses the benefits that can be achieved.
Users also downloaded
Showing related downloaded files
Publication Services Unbound(Washington, DC: World Bank, 2024-12-09)Services are a new force for innovation, trade, and growth in East Asia and Pacific. The dramatic diffusion of digital technologies and partial policy reforms in services--from finance, communication, and transport to retail, health, and education--is transforming these economies. The result is higher productivity and changing jobs in the services sector, as well as in the manufacturing sectors that use these services. A region that has thrived through openness to trade and investment in manufacturing still maintains innovation-inhibiting barriers to entry and competition in key services sectors. 'Services Unbound: Digital Technologies and Policy Reform in East Asia and Pacific' makes the case for deeper domestic reforms and greater international cooperation to unleash a virtuous cycle of increased economic opportunity and enhanced human capacity that would power development in the region.Publication At Your Service?: The Promise of Services-led Growth in Uzbekistan(Washington, DC: World Bank, 2025-01-07)In Uzbekistan, the services sector accounts for more than half of all jobs and has been central to the process of structural transformation over the past three decades. In the past decade, the growth of Uzbekistan’s services exports has lagged behind its manufactures' exports while FDI greenfield announcements to both sectors have been even. The growth of the services sector in the past five years was driven by social services, mostly reflecting increased public spending. This report groups the services sector into four categories based on their skill intensity, the extent of their linkages with other sectors, and their tradability in international markets: low-skilled consumer services, low-skilled enabling services , global innovator services. Of these groups, social services accounted for three-fourths of employment growth in the services sector between 2017–2022. These services also experienced relatively high rates of labor productivity growth, which was largely driven by higher public spending on wages and salaries.Publication Timor-Leste Economic Report(Washington, DC: World Bank, 2025-02-12)Timor-Leste faces a defining moment in its economic journey. As the country seeks to transition from petroleum dependency to a more diverse and resilient economy, the stakes cannot be higher. The Petroleum Fund, a lifeline that has sustained much of the national budget, risks depletion by 2035, based on recent Ministry of Finance estimates, unless there are urgent reforms. Ambitious national goals - such as upcoming ASEAN membership - reflect the country’s aspirations to integrate globally and broaden its growth horizons. Yet, these opportunities are weighed down by significant fiscal challenges. More specifically, an expanding national budget that raises concerns about fiscal sustainability and the efficiency of public spending. The question is clear: how can Timor-Leste spend better to fuel sustainable growth while preserving fiscal stabilityPublication An Investment Framework for Nutrition(Washington, DC: World Bank, 2017-04-12)The report estimates the costs, impacts and financing scenarios to achieve the World Health Assembly global nutrition targets for stunting, anemia in women, exclusive breastfeeding and the scaling up of the treatment of severe wasting among young children. To reach these four targets, the world needs $70 billion over 10 years to invest in high-impact nutrition-specific interventions. This investment would have enormous benefits: 65 million cases of stunting and 265 million cases of anemia in women would be prevented in 2025 as compared with the 2015 baseline. In addition, at least 91 million more children would be treated for severe wasting and 105 million additional babies would be exclusively breastfed during the first six months of life over 10 years. Altogether, achieving these targets would avert at least 3.7 million child deaths. Every dollar invested in this package of interventions would yield between $4 and $35 in economic returns, making investing in early nutrition one of the best value-for-money development actions. Although some of the targets—especially those for reducing stunting in children and anemia in women—are ambitious and will require concerted efforts in financing, scale-up, and sustained commitment, recent experience from several countries suggests that meeting these targets is feasible. These investments in the critical 1000 day window of early childhood are inalienable and portable and will pay lifelong dividends – not only for children directly affected but also for us all in the form of more robust societies – that will drive future economies.Publication Business Ready 2024(Washington, DC: World Bank, 2024-10-03)Business Ready (B-READY) is a new World Bank Group corporate flagship report that evaluates the business and investment climate worldwide. It replaces and improves upon the Doing Business project. B-READY provides a comprehensive data set and description of the factors that strengthen the private sector, not only by advancing the interests of individual firms but also by elevating the interests of workers, consumers, potential new enterprises, and the natural environment. This 2024 report introduces a new analytical framework that benchmarks economies based on three pillars: Regulatory Framework, Public Services, and Operational Efficiency. The analysis centers on 10 topics essential for private sector development that correspond to various stages of the life cycle of a firm. The report also offers insights into three cross-cutting themes that are relevant for modern economies: digital adoption, environmental sustainability, and gender. B-READY draws on a robust data collection process that includes specially tailored expert questionnaires and firm-level surveys. The 2024 report, which covers 50 economies, serves as the first in a series that will expand in geographical coverage and refine its methodology over time, supporting reform advocacy, policy guidance, and further analysis and research.