Results and Performance of the World Bank Group 2022

2.1 This year’s RAP will focus on the country level. The Bank Group’s outcome orientation agenda emphasizes high-level outcomes, and by focusing on the country level, the 2022 RAP aligns with that agenda. This focus is also a response to the interest of members of the Bank Group’s Board of Executive Directors in reporting on countrylevel performance. In this context, the RAP team will conduct an in-depth analysis of country-level evidence contained in IEG’s Country Program Evaluations (CPEs) and Completion and Learning Report Reviews (CLRRs) through two types of analyses. First, overall country program performance will be assessed by tracking country program ratings over time. Second, the country program will be used as the entry point to examine the extent to which the Bank Group’s support (that is, project portfolio and advisory services and analytics) contributed to the achievement of the objectives of the Country Partnership Framework (CPF) and the intended development outcome. Furthermore, the extent to which there was a line of sight between the development outcome and high-level outcomes will also be examined.


1.1
The Results and Performance of the World Bank Group (RAP) report presents an annual review of evidence from Independent Evaluation Group (IEG) evaluations and validation work on the development effectiveness of the World Bank Group. 1 The 2022 RAP will be the 12th in the annual series.

2.1
This year's RAP will focus on the country level. The Bank Group's outcome orientation agenda emphasizes high-level outcomes, and by focusing on the country level, the 2022 RAP aligns with that agenda. This focus is also a response to the interest of members of the Bank Group's Board of Executive Directors in reporting on countrylevel performance. In this context, the RAP team will conduct an in-depth analysis of country-level evidence contained in IEG's Country Program Evaluations (CPEs) and Completion and Learning Report Reviews (CLRRs) through two types of analyses. First, overall country program performance will be assessed by tracking country program ratings over time. Second, the country program will be used as the entry point to examine the extent to which the Bank Group's support (that is, project portfolio and advisory services and analytics) contributed to the achievement of the objectives of the Country Partnership Framework (CPF) and the intended development outcome. Furthermore, the extent to which there was a line of sight between the development outcome and high-level outcomes will also be examined.

2.2
The traditional update of project ratings trends will be undertaken. For the World Bank, the RAP team will also identify some drivers of performance by examining ratings at the two ends of the ratings spectrum (highly satisfactory or satisfactory and highly unsatisfactory or unsatisfactory) and ratings in the middle range. For International Finance Corporation (IFC) investments and advisory services and Multilateral Investment Guarantee Agency (MIGA) guarantees, the analysis of drivers of performance will examine ratings at the two extreme ends of the rating scale (for example, highly successful and highly unsuccessful for IFC investments and advisory services; and excellent and unsatisfactory for MIGA guarantee projects).

2.3
The main audience for the 2022 RAP will be the Bank Group's Board of Executive Directors. Other important stakeholders include senior management; the Operations Policy and Country Services Vice Presidency; management of the Global Practices (Global Industries for IFC) and Regions; directors and senior managers in IFC and MIGA; country directors and representatives; task teams of operational projects; and funders of trust funds.

3.1
The primary data sources for this year's RAP will be IEG's CPEs and CLRRs. Although all CLRRs covering the 10-year period from fiscal year (FY)13 to FY22 and comprising 152 CLRRs will be covered for the ratings trends analysis, 2 a stratified random sample of these CLRRs will be examined to answer the evaluation questions pertaining to country-level performance. The nine completed CPEs will also be examined for the latter. 3 For the World Bank, the analysis of findings from CPEs and CLRRs will be complemented by country-level insights from a purposive sample of sector and thematic evaluations (including corporate evaluations), Project Performance Assessment Reports (PPARs), and other relevant evaluative material. For IFC and MIGA, the analysis of findings from CPEs and CLRRs will be complemented by country-level insights from IFC Sector Highlights, Learning Engagements, Synthesis Notes.

3.2
At the project level, the project ratings trends will be updated from the previous RAP based on IEG's project ratings database.

3.3
Tables 3.1-3.3 provide a count of available IEG evaluations by year.

3.4
The 152 CLRRs cover 112 countries. The 112 countries are classified according to income level in figure 3.1.

4.1
The evaluation questions are presented in box 4.1.

. Evaluation Questions
Country program performance (data sources): • For the World Bank, Country Program Evaluations (CPEs) and Completion and Learning Report Reviews (CLRRs) will be complemented by country-level insights from a purposive sample of sector and thematic evaluations (including corporate evaluations), Project Performance Assessment Reports (PPARs), and other relevant evaluative material.
• For the International Finance Corporation (IFC) and Multilateral Investment Guarantee Agency (MIGA), CPEs and CLRRs will be complemented by country-level insights from sector highlights, learning engagements, and synthesis notes..
• How have Independent Evaluation Group (IEG) country program ratings changed over time and across types of countries (for example, classified by income level, lending group, or status of countries affected by fragility, conflict, and violence FCS status)? a • To what extent did the World Bank Group's support (that is, project portfolio and advisory services and analytics) contribute to the achievement of the Country Partnership Framework's objectives and intended development outcomes? To what extent was there a line of sight between the development outcome and high-level outcomes?
• What role did World Bank advisory services and analytics play in informing the World Bank's support and the policies and programs of governments?
Project portfolio performance (data sources): • For the World Bank, PPARs and ICRRs.
• For IFC and MIGA, IFC XPSR Evaluation Notes, IFC Project Evaluation Summaries, IFC PCR Evaluation Notes, MIGA PER Validation Notes, and IEG PERs.
• For the Bank Group, how have IEG project ratings changed over time and across types of countries and operations? a For the World Bank, what explains variations in the ratings at the two ends of the ratings spectrum (that is, highly satisfactory or satisfactory versus highly unsatisfactory or unsatisfactory) and also, ratings in the middle range? For IFC and MIGA, what explains variations in the ratings at the two extreme ends of the rating spectrum (highly successful and highly unsuccessful for IFC; and excellent and unsatisfactory for MIGA).
• For IFC investment and MIGA guarantee projects (in addition to the previous question), what was the relationship between development outcomes and work quality and between development outcomes and the role and contribution of IFC additionality or MIGA guarantees? For IFC investments, what was the relationship between development outcome and IFC's investment outcome? What was the relationship between IFC advisory services and investments (in particular, those IFC investment projects that had prior or concurrent advisory services)?
Source: Independent Evaluation Group. Note: a. Includes both outcome and Bank performance ratings.

5.1
At the country level, in addition to examining country performance ratings trends emerging from all completed CLRRs, CLRRs for a stratified random sample of 50 countries (table 5.1) will be examined to answer the evaluation questions pertaining to country-level performance. For the Bank Group, country-level insights will also be gleaned from a purposive sample of sector and thematic evaluations (including corporate evaluations), PPARs, and other relevant evaluative material. For IFC and MIGA, the analysis of findings from CPEs and CLRRs will be complemented by country-level insights from sector highlights, learning engagements, and synthesis notes.

5.2
At the project level, the RAP will update the ratings trends. For the World Bank, the RAP will identify some drivers of performance by examining projects rated at the two ends of the ratings spectrum (that is, highly satisfactory or satisfactory versus highly unsatisfactory or unsatisfactory) and also ratings in the middle range using machine learning (supplemented by NVivo as relevant). The machine learning algorithm will pull out from project documents what it identifies as common characteristics of projects in the satisfactory and unsatisfactory ranges.

5.3
For IFC investments and advisory and MIGA guarantee projects, the RAP will identify some drivers of performance at the two extreme ends of the rating spectrum (for example, highly successful and highly unsuccessful for IFC investments and advisory projects; and excellent and unsatisfactory for MIGA projects). 4 Past RAP reports have conducted regression analysis of drivers of performance across the rating spectrum; the focus on the extreme ends aims to elicit new insights on what worked well and not so well. Owing to the small number of evaluated IFC and MIGA projects with development outcome ratings at the two extreme ends of the rating scale, the RAP will conduct manual document review of the projects.

5.4
The evaluation design matrix is presented in appendix A.

6.1
The 2022 RAP team will engage with Board members and advisers, members of World Bank, IFC, and MIGA management, and staff from the World Bank, IFC, and MIGA. The 2022 RAP will be e-submitted to the Bank Group's Board of Executive Directors by August 31, 2022, for presentation to the Board in September, or on the first available Board date.

7.1
The co-task team leaders for this RAP are Soniya Carvalho and Aurora Medina Siy. Other IEG staff and consultants will also contribute.

7.2
The report will be produced under the overall supervision of Alison Evans (Director-General, Evaluation) and the direct supervision of Oscar Calvo-Gonzalez (director, Human Development and Economic Management) and Galina Sotirova (manager, Corporate and Human Development). During the production of the report, the team will also consult with IEG staff, including ICRR coordinators, CLR coordinators, the Financial and Private Sector Micro Unit, and staff and consultants involved in the validation of self-evaluations.

7.3
The budget for the task is $695,000, including $40,000 for dissemination. Staff costs are estimated at 55 percent of the total task budget, and variable costs, mostly for consultants, at 45 percent. The timeline for the evaluation is included in table 7.1.

7.4
The report will also benefit from the advice of three external reviewers: Tamar Manuelyan Atinc (nonresident senior fellow at The Brookings Institution); Monika Huppi (principal adviser, Office of Evaluation and Oversight, Inter-American Development Bank); and Barbara Scott (deputy director general, External Cooperation Management and Project Development, Planning Institute of Jamaica). The reviewers will advise the team on methods and the interpretation of findings. This Concept Note and draft final report will be subject to internal IEG review and the standard process of Bank Group management comments. African countries (Mauritius, the Seychelles, Cabo Verde, and Djibouti), with full CPEs for OECS and Pacific Island countries at regional level, and case studies for Mauritius and the Seychelles.