Publication: Meta-Evaluation of IEG
Evaluations (FY15-19)
Loading...
Published
2022-02-14
ISSN
Date
2022-02-23
Author(s)
Editor(s)
Abstract
IEG’s meta-evaluation serves as an input for the upcoming independent external review of its evaluations. The report focuses on aspects of credibility related to the rationale, focus, use of innovative methods, and various research design attributes as formulated in evaluation reports and their respective approach papers. Drawing on a set of 28 evaluations published from fiscal year 2015 to 2019, the meta-evaluation offers six major conclusions and suggestions based on a systematic review of evaluation scope, reliability, validity (including construct, internal, external, and data analysis validity), consistency, and the integration of innovative methods.
Link to Data Set
Citation
“Independent Evaluation Group. 2022. Meta-Evaluation of IEG
Evaluations (FY15-19). © World Bank. http://hdl.handle.net/10986/37022 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Planning, Monitoring, and Evaluation : Methods and Tools for Poverty and Inequality Reduction Programs(World Bank, Washington, DC, 2013-01)As the author enter the second decade of the twenty-first century, governments, international organizations, nongovernmental organizations (NGOs), philanthropic organizations, and civil society groups worldwide are actively focusing on evidence-based policy and increased accountability to stakeholders (results agenda).The widespread implementation of the Results Agenda has generated a plethora of books, guides, academic papers, trainings, and case studies, which has enabled an ongoing maturation process in the field. Consequently, specialists are now better equipped to understand what works under which circumstances. Broadly speaking there are two interrelated questions which must be answered when assessing the sustainability of a government results agenda. First, is the institutional design and practice of government conducive to evidence-based policy making? Second, are the overarching monitoring and evaluation (M&E) methods and specific tools used appropriate for garnering the evidence demanded by government? These series of notes aim to make a small contribution to the latter question by summarizing and highlighting a selection of PM&E methods and the tools that governments and international organizations around the world have developed to put these into practice in their own contexts. The central goal of this initiative is to prompt a process of learning, reflection and action by providing practical information to those whose leadership role requires them to understand PM&E methods and their potential for enhancing evidence-based policy making.Publication Analyzing the Effects of Policy Reforms on the Poor : An Evaluation of the Effectiveness of World Bank Support to Poverty and Social Impact Analyses(Washington, DC: World Bank, 2010)The current global financial and economic crises are likely to put enormous pressure on governments to respond with immediate measures and to undertake far-reaching reforms in the medium term, requiring a substantial increase in donor support. To protect the poor and enhance benefits to them, key policy reforms will need to be underpinned by systematic analysis of their expected poverty and social impacts. The World Bank's experience to date with the Poverty and Social Impact Analysis (PSIA) approach provides useful lessons for addressing these issues. Overall, implementation of the PSIA approach has had considerable limitations. There have been tensions between the various operational objectives assigned to PSIAs. The tensions concern inconsistencies between informing country and Bank policy decisions in a timely way and building country analytic capacity. PSIAs have had limited ownership by Bank staff and managers and have often not been effectively integrated into country assistance programs. Quality assurance, monitoring, and evaluation of the overall effectiveness of PSIAs have been weak. To improve PSIAs' effectiveness, this evaluation recommends that the Bank take measures to ensure that staff fully understands what the PSIA approach is and when to use it, clarify the operational objectives of each PSIA, and ensure that the approach and timeline adopted are aligned with those objectives. Quality assurance mechanisms should be strengthened to ensure that PSIAs are designed to achieve the intended effects.Publication Using Mixed Methods in Monitoring and Evaluation : Experiences from International Development(2010-03-01)This paper provides an overview of the various ways in which mixing qualitative and quantitative methods could add value to monitoring and evaluating development projects. In particular it examines how qualitative methods could address some of the limitations of randomized trials and other quantitative impact evaluation methods; it also explores the importance of examining "process" in addition to "impact", distinguishing design from implementation failures, and the value of mixed methods in the real-time monitoring of projects. It concludes by suggesting topics for future research -- including the use of mixed methods in constructing counterfactuals, and in conducting reasonable evaluations within severe time and budget constraints.Publication Integrating Qualitative Methods into Investment Climate Impact Evaluations(World Bank Group, Washington, DC, 2014-12)Incorporating qualitative methods into the evaluation of development programs has become increasingly popular in recent years, both for the distinctive insights such approaches can bring in their own right and because of their capacity to complement the strengths -- and where necessary correct some of the weaknesses -- of quantitative approaches. Some initial work deploying mixed methods has been undertaken in the assessment of investment climate reforms, but considerable room for expansion exists. This paper summarizes some of the key principles and practices underpinning mixed methods evaluations in development, highlight some notable examples of how such work has been conducted (and the particular contributions it has made), and offers some guidelines for those seeking to increase the sophistication and utility of qualitative methods in the evaluation of investment climate reforms.Publication Integrating Quantitative and Qualitative Research in Development Projects(Washington, DC: World Bank, 2000-06)"Much of the early work on poverty was highly quantitative:... It became increasingly clear, however, that while numbers are essential for policy and monitoring purposes, it is also important to understand people's perception of poverty and their mechanisms for coping with poverty and other situations of extreme economic and social stress." Researchers have recognized over the past few years that quantitative analysis of the incidence and trends in poverty, while essential for national economic development planning, must be complemented by qualitative methods that help planners and managers understand the cultural, social, political, and institutional context within which projects are designed and implemented. This report is based on a two-day workshop held in June 1998, where outside research specialists and World Bank staff discussed the importance of integrating these research methods. The participants reviewed experiences in the use of mixed-method approaches in Bank research and project design. This report is a result of those discussions. The report examines the need for integrated research approaches in social and economic development, presents case studies of integrated approaches in practice, and talks about lessons learned. Part I describes the evolution of interest in, and the potential benefits of integrated research, and, Part II presents case studies on how integrated approaches have been used in poverty analysis, education, health, and water supply and sanitation, while Part III discusses lessons learned with respect to the use of integrated approaches, and assesses the benefits that can be achieved.
Users also downloaded
Showing related downloaded files
Publication Côte d’Ivoire Country Climate and Development Report(Washington, DC: World Bank, 2023-11-02)Le présent CCDR comporte trois messages principaux: (i) En premier lieu, le maintien du statu quo ne permettra plus de soutenir la croissance économique de la Côte d'Ivoire et ses ambitions de devenir un pays à revenu intermédiaire de la tranche supérieure à l’horizon 2030, tout en réduisant considérablement la pauvreté. Toutes choses étant égales par ailleurs, et dans le cadre d’un scénario de climat sec/plus chaud, le changement climatique devrait réduire le produit intérieur brut (PIB) réel de 13 pour cent d’ici à 2050, ce qui empêcherait 1,63 million de personnes de s’affranchir de la pauvreté. Quoique dispendieuses, les mesures d’adaptation peuvent potentiellement compenser une grande partie de l’impact négatif du climat, notamment sur les populations démunies; (ii) Deuxièmement, des secteurs économiques clés, dont le cacao et l’énergie, courent le risque de connaître des contre-performances si aucune mesure n’est prise maintenant même pour faire face aux impacts climatiques et tirer parti des mutations technologiques ou des changements réglementaires. En outre, les centres urbains, qui sont des pôles économiques, sont exposés aux dommages climatiques subis par les infrastructures et aux pertes considérables de moyens de subsistance subies par les populations démunies vivant dans des communautés à faibles revenus. Des menaces planent également sur les routes, les réseaux numériques et les autres infrastructures qui assurent l’interconnectivité au plan national, garantissant l’efficacité des déplacements et l’accès aux marchés et aux services; (iii) Troisièmement, la Côte d'Ivoire n’est pas actuellement prête à faire face aux conséquences du changement climatique. Sa capacité d’adaptation en est encore à ses balbutiements, ses institutions et sa coordination de l’action en faveur du climat sont fragmentaires, et ses politiques et programmes ne sont pas à la hauteur du défi climatique auquel sont confrontées les populations vulnérables. Entre-temps, la mise en œuvre des stratégies et plans existants reste limitée. Les composantes réglementaires, institutionnelles et climatiques nécessaires à la gestion des impacts climatiques doivent être revues ou mises en place. Certes, la croissance du secteur privé a connu une tendance positive, mais elle n’atteint pas encore son potentiel en termes de portée et d’échelle, si bien qu’elle doit encore se développer pour jouer son rôle essentiel à l’adaptation aux effets du changement climatique et à leur atténuation.Publication Kazakhstan Country Climate and Development Report(World Bank, Washington, DC, 2022-11)The World Bank Group’s Country Climate and Development Reports (CCDRs) are new core diagnostic reports that integrate climate change and development considerations. The CCDR for Kazakhstan identifies ways that Kazakhstan can achieve its development objectives while fostering the transition to a more green, resilient, and inclusive development pathway. It sets out policy reforms and investments needed to build resilience to climate change impacts and reduce greenhouse gas (GHG) emissions while creating a more diversified, competitive and sustainable economy.Publication Argentina Country Climate and Development Report(World Bank, Washington, DC, 2022-11)The Argentina Country Climate and Development Report (CCDR) explores opportunities and identifies trade-offs for aligning Argentina’s growth and poverty reduction policies with its commitments on, and its ability to withstand, climate change. It assesses how the country can: reduce its vulnerability to climate shocks through targeted public and private investments and adequation of social protection. The report also shows how Argentina can seize the benefits of a global decarbonization path to sustain a more robust economic growth through further development of Argentina’s potential for renewable energy, energy efficiency actions, the lithium value chain, as well as climate-smart agriculture (and land use) options. Given Argentina’s context, this CCDR focuses on win-win policies and investments, which have large co-benefits or can contribute to raising the country’s growth while helping to adapt the economy, also considering how human capital actions can accompany a just transition.Publication China Country Climate and Development Report(World Bank Group, Washington DC, 2022-10)The China Country Climate and Development Report (CCDR) provides analysis and recommendations on integrating the country’s efforts to achieve high-quality development with the pursuit of emission reduction and climate resilience. Without adequate mitigation and adaptation efforts, climate risks will become a growing constraint to China’s long-term growth and prosperity, threatening to reverse development gains. Conversely, if efforts to tackle climate risks lead to a significant decline in growth and rising inequality, they would deprive millions of people of development and likely erode support for the reforms necessary to achieve a lasting economic transformation. Hence, China will need to grow and green its economy at the same time. This report offers policy options to achieve these dual objectives by easing inevitable trade-offs and maximizing potential synergies between China’s development and climate objectives.Publication Egypt Country Climate and Development Report(World Bank, Washington, DC, 2022-11-08)This Country Climate and Development Report (CCDR) explores the challenges and opportunities of improving the alignment of Egypt’s development goals with its climate ambition. The CCDR offers a set of policy options and investment opportunities that, if implemented within five years, can deliver short-term benefits in selected sectors while also creating momentum toward important long-term benefits. The options identified in this report provide: Cost-effective adaptation approaches to reduce the negative impacts of climate change; Policy interventions to improve efficiency in the use of natural resources, and complement the creation of fiscal space to finance projects that reduce the vulnerability of people and the economy to climate shocks; Actions that can help avoid carbon lock-in through low-cost policy changes; Interventions to strengthen the country’s competitiveness while reducing negative externalities (such as pollution) and incentivize Egypt’s move towards a low carbon growth path in a manner consistent with its development objectives. Overall, the report identifies opportunities to reduce inefficiencies, manage risk, and strengthen the foundation for increased private sector participation.