Elements That Enhance Institutional Capacity Development in World Bank Projects and Country Partnerships in Sub-Saharan Africa September 2023 click on individual headings for immediate access to the related section World Bank projects and country partnerships in Sub-Saharan Africa routinely integrate and show results on institutional capacity development. Using the Institutional Change Assessment Method helps harness four elements that can enhance both the process and the results of institutional capacity development. Routine integration of institutional capacity development in World Bank proj- ects provides multiple entry points for enhancing processes of institutional change. Interventions with better institutional capacity development results tend to have higher outcome ratings. This implies that analyses from the Institutional Change Assessment Method can be used to adapt country portfolios in a way that improves outcome ratings. Because strengthening the ownership of interest groups is the most important dimension of institutional change in World Bank projects and country programs, its prioritization can help enhance results. Support for commitment, coordination, and cooperation helps improve insti- tutional capacity development processes. Methodological approaches involved coding, content analysis, interpretation, and statistical analysis of text. Drawing from the Independent Evaluation Group’s rich knowledge repository, Evaluation Insight Notes respond to the need for more rapid and focused evaluative evidence. These notes systematically analyze data from a range of evaluations, validations, and other studies to generate insights in a timely manner around important strategic and operational issues. Institutional Capacity Development in the World Bank Institutions shape how countries foster poverty reduction, support sustainable growth, and respond during crises (OECD 2008; Otoo, Agapitova, and Behrens 2009; World Bank 2005a, 2005b, 2017b, 2018b, 2018c, 2022a). However, the World Bank has no overarching framework to guide capacity development; the most-recent cross-sectoral publications are almost 15 years old, and the latest major evaluation—which focused on Africa—is close to 20 years old (Otoo, Agapitova, and Behrens 2009; World Bank 2005b). To help navigate institutional capacity development, this Evaluation Insight Note (EIN) answers the ques- tion, How can the World Bank help address institutional capacity development needs in Sub-Saharan Africa based on the body of work of the Independent Evaluation Group (IEG) from 2008 to 2022? Developing institutional capacity has been identified as an important element for improving the effectiveness of development interventions by the World Bank and within international agreements. What is institutional capacity development? In the World Bank, institutional capacity development means improving the effectiveness of country development by changing the formal and informal rules that structure interactions across multiple organizations (World Bank 2005b, 2018b; World Bank Group 2017). This understanding of institutional capacity development provides a broad approach to identifying opportunities for change, especially relating to social and political factors.1 Although an overall framework has not been defined, the World Bank has often usefully diagnosed and addressed institutional capacity development needs in projects and country programs by applying the Institutional Change Assessment Method (ICAM; Otoo, Agapitova, and Behrens 2009; World Bank 2013a, 2018c, 2018d, 2021a, forthcoming). The three dimensions of the ICAM serve as the underlying framework for this analysis—namely, A narrower focus on the capability of individuals to produce outputs is often the core of capacity assessments (for example, Lokshin 2021); 1  individual capacity is considered in this framework only where broader institutionally relevant results are identified. 2 strengthen ownership of interest groups,2 reform policy instruments, and enhance organizational arrangements (box 1). Previously, all three dimensions have been seen as equally important, whereas in this EIN, we suggest that the ownership of interest groups is the most important dimension. Box 1. Dimensions of Institutional Capacity Development with Examples Strengthen ownership of interest groups: Build the capacity to engage different interest groups (government, civil society, communities, and the private sector) to prioritize a development goal, drawing on participation and information in oversight processes that is transparent, accurate, relevant, verifiable, and timely. » This dimension increases meaningful progress in implementing agreed standards, such as improving transparency and accountability in the extractive industries, by providing timely and reliable information to the public. » Without ownership of interest groups, a government’s public financial management reform program slows down as the legislature’s commitment deteriorates and there is little further progress in address- ing budget and expenditure issues. Reform policy instruments: Increase the capacity to change and implement the administrative rules, laws, regulations, standards, and other formal incentives outside of specific organizations that a soci- ety uses to guide actions to achieve development goals. » Analytical work provides policy options to the sector ministry and ministry of finance in a manner that facilitates dialogue on fiscal space and increases resource allocations. » Without reform of policy instruments, corruption issues between institutions are not addressed, which hinders efficient procurement of goods that meet the necessary standards. Enhance organizational arrangements: Improve the capacity to implement needed processes through the systems, rules of action, communications, business processes, personnel, and other resources that are brought together to achieve shared development goals. » These processes allow organizations and the public to work together more easily. For example, a forestry commission could implement a reporting system with government officials and forestry nongovernmental organizations so that community leaders report illegal foresters. » Without ongoing implementation support, the department of public works was unable to manage and supervise private sector contracts, which contributed to ineffective waste management services. Sources: World Bank 2013a, 2018c, 2018d. Interest groups is preferred to stakeholder. The term stakeholder may imply a hierarchy that is not relevant and “has echoes of colonial 2  practices where settler became stakeholders as they staked their claim to Indigenous lands” (Reed and Rudman 2023, 971). 3 In applying the ICAM framework, this EIN uses IEG evidence and some external litera- ture. We used 29 Project Performance Assessment Reports, 30 validations of country reviews, and 5 Country Program Evaluations (from hereon termed “cases”) to prepare this EIN. These cases cover 10 Global Practices and 32 countries in Sub-Saharan Africa.3 Four Elements That Enhance the Process and Results of Institutional Capacity Development We found that World Bank projects and country partnerships operating in challenging contexts in Sub-Saharan Africa routinely integrate and show results in institutional capacity development. These have been characterized as “wicked” problems (Andrews, Pritchett, and Woolcock 2017; Hopson and Cram 2018). Addressing institutional challenges in these and similar contexts involves multiple organizations and is like running through a labyrinth because of the need for quick decisions, unclear processes, shifting objectives, and trial and error. The cases reviewed for this EIN consistently supported institutional reforms in a variety of challenging contexts with, for example, compromised financial systems, corruption, and civil conflict. Analysis using the ICAM helps harness four elements that can enhance both the process and the results of institutional capacity development: (i) routine integration of institutional capacity development provides multiple entry points for targeting different dimensions of institutional change across projects; (ii) analysis of successes and challenges using the ICAM can help adapt country portfolios to reinforce The Global Practices covered are Agriculture and Food; Environment, Natural Resources, and Blue Economy; Finance, Competitiveness, 3  and Innovation; Governance; Health, Nutrition, and Population; Macroeconomics, Trade, and Investment; Social Protection and Jobs; Transport; Urban, Disaster Risk Management, Resilience, and Land; and Water. The countries covered are Benin, Burkina Faso, Burundi, Cabo Verde, Cameroon, the Central African Republic, Chad, the Democratic Republic of Congo, the Republic of Congo, Côte d’Ivoire, Ethiopia, The Gambia, Ghana, Guinea, Kenya, Liberia, Madagascar, Malawi, Mali, Mauritania, Mozambique, Niger, Nigeria, Rwanda, São Tomé and Príncipe, Senegal, the Seychelles, Sierra Leone, South Africa, Tanzania, Uganda, and Zambia. 4 institutional capacity development processes, with a potentially positive effect on outcome ratings; (iii) strengthening the ownership of interest groups, the most important dimension of institutional capacity, can be prioritized to help enhance results; and (iv) support for commitment, coordination, and cooperation can improve the process of institutional capacity development. What Are the Main Insights from This Synthesis? Routine integration of institutional capacity development in World Bank projects provides multiple entry points for enhancing processes of institutional change. Interventions integrating institutional capacity development are widespread in World Bank project objectives. A review of objectives estimates that 72 percent of projects approved from fiscal year 2009 and closed by fiscal year 2022 targeted a dimension of institutional capacity. The Equitable Growth, Finance, and Institutions Practice Group most frequently targeted institutional capacity development, with 93 out of 101 projects (92 percent) having at least one objective related to one of the dimensions. The three other Practice Groups also frequently targeted institutional capacity development, with Human Development (72 percent of projects) followed by Sustainable Development (61 percent) and Infrastructure (60 percent). Stand-alone capacity development projects, such as those focused on statistical systems, are far less common than projects with integrated objectives; fewer than one in every six projects had a primary focus on institutional capacity development. Evidence from the cases also indicates that even when no objectives focused on institutional capacity, World Bank projects and country programs still undertake interventions in one of the dimensions of capacity. Only one project evaluation conducted by IEG between 2008 and 2022 in Sub-Saharan Africa had no text on interventions that support institutional capacity development. For all other relevant country evaluations and validations, almost 80 percent have at least one positive result in institutional capacity development, and 90 percent highlight at least one challenge in developing institutional capacity. Furthermore, although 47 percent of the projects of 5 the Agriculture and Food Global Practice had a project objective targeting institutional capacity development: in all six project evaluations reviewed, it was consistently a feature of implementation. With this routine integration, there are many entry points to enhance the process and results of projects by targeting change in a specific institutional capacity dimension. nterventions with better institutional capacity development I results tend to have higher outcome ratings. This implies that analyses from the ICAM can be used to adapt country portfolios in a way that improves outcome ratings.4 Of the cases rated as moderately satisfactory and above, 66 percent had high levels of positive reporting on institutional capacity development. In contrast, among cases rated as moderately unsatisfactory and below, only 10 percent had a high percentage of positive reporting.5 Similar findings emerge in the analysis of two data sets coded for the recent evaluation of the World Bank’s COVID-19 health and social response. These two data sets also show higher degrees of positive reporting on institutional capacity development in projects with higher rating levels (World Bank 2022b). This finding also aligns with the 2017 World Development Report and other World Bank reports that emphasize the importance of institutions for development results (World Bank 2017b, 2018b, 2022a; World Bank Group 2017).6 Consequently, adapting implementation based on ICAM analysis of institutional successes and challenges can help improve outcomes. Box 2 provides an example of how using the ICAM helps project teams understand important successes and ongoing challenges. 4  We recognize that outcome ratings are a blunt instrument, and in the absence of a granular and universal measure, we use this as a relevant indicator for results. 5  Based on the percentage of positive accounts on institutional capacity development, low is less than 30.0 percent, medium is 30.0 percent to 69.9 percent, and high is 70.0 percent or more. 6  A robust regression analysis of these institutional capacity development dimensions could be undertaken on a larger sample of projects and control for known factors that influence project quality, such as monitoring and evaluation ratings and task team leader quality. 6 Positive Accounts in Cases of Institutional Capacity Development by Figure 1.  Outcome Rating 100 90 80 70 Share of positive accounts (%) 60 50 40 30 20 10 0 Highly Unsatisfactory Moderately Moderately Satisfactory Highly unsatisfactory unsatisfactory satisfactory satisfactory Outcome rating Source: Independent Evaluation Group. Note: Calculated based on the percent of reporting on successful institutional capacity development results. N = 59 cases with five Country Program Evaluations excluded because there is no single outcome rating. Reporting successes or challenges in institutional capacity can help World Bank operations understand progress toward objectives; a larger share of positive text in the cases correlated to higher outcome ratings. 7  nalysis of Successes and Challenges in Institutional Capacity Box 2. A Development Dimensions The Community-Based Rural Land Development Project in Malawi received a satisfactory outcome rating. Content analysis of the project evaluation identified 80 percent of the text related to institutional capacity development as positive in sentiment. Further review of the project demonstrates how the Institutional Change Assessment Method dimensions can help differentiate short-term positive results and long-term contextual challenges showing areas for adaptation even in a relatively successful project. Strengthening ownership of interest groups: The successful implementation of the project enabled the government to demonstrate commitment to strengthening of the land rights—for example, an increase in counterpart funding of an additional MK 80 million was disbursed by 2010 in lieu of transferring state land. Yet by the project’s completion, gaps in government commitment remained because the government disbursed less financing overall than had been expected at design. Reform of policy instruments: Institutional capacity was developed to enable the implementation of the government’s land policy. The project supported the surveying and registration of land parcels through the provision of office equipment and effective staff training. This led to 32 land registries being computerized and operational when the project closed. In addition, under the project, government agencies streamlined the procedures for verifying deed plans, reducing the time for processing property transfers from nine months to four months. However, the momentum after the project was difficult to maintain because land reform was not the focus of policy reform and was not included in either the Malawi Growth and Development Strategy (2012–16) or the World Bank’s Country Assistance Strategy for fiscal years 2013–16. Enhancing organizational arrangements: The project demonstrated a viable approach to community-driven land reform through the Ministry of Land in Malawi. In total, 15,142 households benefited. The impact evaluation showed that these households owned more farmland than nonbeneficiaries and that they were able to obtain a larger increase in farm output volume. However, the project was unable to fully develop efficient coordination arrangements with the Malawi Social Action Fund because actual investments from this fund were reportedly lower than expected for off-farm infrastructure investments for community assets, such as boreholes, access roads, schools, and clinics. In the case of drinking water supply, the benefiting households themselves often installed infrastructure, using part of the initial settlement grant provided by the project. Source: World Bank 2012. S  trengthening the ownership of interest groups arises as the most important dimension of institutional change in World Bank projects and country programs signifying that its prioritization can help enhance results. 8 The ownership of interest groups demonstrates a strongly significant correlation with outcome ratings. Interest groups are associations of individuals and organizations established to influence government and affect decisions about public policy corre- sponding to needs, wants, or, more generally, forms of power (Calhoun 2002). For the two other dimensions of institutional capacity development, no strongly significant correlations were found; enhancing organizational arrangements was also correlated but not at a conventionally statistically significant level, and no correlation was apparent in the dimension of reform of policy instruments.7 This finding was counter to expecta- tions because World Bank frameworks do not prioritize one dimension of institutional capacity over another (Otoo, Agapitova, and Behrens 2009; World Bank 2013a, 2018c). Developing capacity to work across interest groups to secure support for interventions is also identified as important with a range of other experience in international devel- opment (Andrews, Pritchett, and Woolcock 2017; Kusek, Prestidge, and Hamilton 2013; Smithers 2011). Given the significance of the ownership of interest groups to enhanced results, additional prioritization may help further reinforce targeted changes in institu- tional capacity. Interpretation of the content of cases also consistently identifies interest group owner- ship as an important dimension in enabling or hindering institutional capacity develop- ment (box 3). Text related to the ownership of interest groups is often stated in a way that shapes the direction of reporting in the two other dimensions of the ICAM. In other words, positive statements in the ownership of interest groups are coupled with positive accounts in other dimensions and vice versa. When challenges in ownership arose, the development of public financial management reform was hindered. In the examples provided in this EIN, strengthened ownership of interest groups at both the central government and the community levels supported the development of institutional capacities. The lack of a correlation with positive accounts on policy reform could be due to it being the least reported dimension overall, specifically 7  in country reviews. 9 Effects of Strengthening the Ownership of Interest Groups on Efforts Box 3.  in Policy and Organizational Institutional Capacity Development Underlying successes in strengthening the ownership of interest groups contributed to improved institutional capacity development (italicized for emphasis). In Malawi, the framework of the land administration system was strengthened, including capacity at different levels of government, updating the applicable legal framework, and the participation of beneficiary groups. Out of the 666 beneficiary groups, 641 had received group land titles before closing. Impact evaluations found that farm income increased more for beneficiaries than nonbeneficiaries by a sizeable magnitude and in a statistically significant way. In Sierra Leone’s National Social Action Project, the World Bank provided technical assistance on the highly participatory processes. The cash-for-work program was prepared and implemented remarkably quickly as befitting an emergency project, and although it had a number of problems, these were identified during its two-year implementation period, and adjustments were made for a follow-on youth employment project. The government championed the project and effectively backstopped the National Commission for Social Action and its coordination with other government agencies. Inadequate attention to developing capacity for the ownership of interest groups led to challenges that hindered efforts. In Ghana, responsibilities in the agriculture sector are fragmented across different directorates and agencies. In two development policy operations, the World Bank’s dialogue and engagement were largely through counterparts in the Ministry of Food and Agriculture Policy Planning and Monitoring Directorate, whereas other key stakeholders had a limited understanding of the budget support instru- ment. Insufficient efforts were put into engaging and capacity building of other directorates to reduce negative responses; public finance management reforms did not perform as expected, and measures to adequately compensate at the sectoral level were not taken. Many reforms in the agriculture sector are beyond the Ministry of Food and Agriculture’s jurisdiction, and some of the cross-jurisdictional reforms may be best incorporated into national-level budget support. In Sierra Leone’s Integrated Public Financial Management Reform Project, activities to support oversight by nonstate actors could not achieve their expected outcomes because the government did not provide the conditions for the related staff to work independently and be held accountable. Academia, media, and the private sector were not engaged, and efforts only narrowly focused on civil society. The Citizens’ Budget was underfunded, undermining its meaningful dissemination. Sources: World Bank 2013c, 2013d, 2017a, 2018f. 10 At the country level, analysis of the ownership of interest groups helps understand how policy reform or organizational change initiatives can become unclogged. In Mali, the World Bank’s emphasis on strengthening the role of government in steering donors toward prioritization of sector issue helped contribute to achieving results in education (World Bank 2015). Analysis also helps highlight where a focus on a nar- rower set of government champions meant that opportunities were missed to adapt implementation or build ownership through analytical products (box 4).  igeria Country Validation’s Assessment of Strengthening the Box 4. N Ownership of Interest Groups The review of the Nigerian Country Partnership Strategy Completion Report by the Independent Evaluation Group (IEG) rated progress as moderately satisfactory. The overall IEG assessment concluded that modest steps were taken to strengthen the institutions and systems for better governance, with the ownership of interest groups being a key factor in enabling and constraining progress. Success from Ownership IEG’s validation states that the objectives of the Country Partnership Strategy aligned with the federal government’s own priorities with all the core development issues fully reflected. Results in the theme of governance indicate the benefit of this ownership. For example, the effectiveness of Nigeria’s anticorruption agency improved more than expected; the new civil service performance management system was widely adopted at the federal level; public financial management systems, especially budget planning, were strengthened; and some progress was made in building statistical capacity at both the federal and the state levels. Challenges in Ownership IEG’s validation also notes that ownership often did not extend outside of the federal government. Evidence identified a lack of ownership in the national legislature as shown through delays in processing the legal requirements of some projects and consequent delays in effectiveness. These challenges extended to state-level reforms, where the World Bank reportedly struggled with how to work with 36 state governments with a long-term view of engagement. Outside of the government, the World Bank was described as not sufficiently inclusive of a range of interests. In their analytical work, for example, the limited involvement of academia was seen to curtail deeper ongoing engagement. In addition, the country partnership was identified as being notably thin on the demand-side engagement. The World Bank was reported to be attempting to engage more systematically with nongovernmental organizations and the media with some positive results; however, given the size of the country, the IEG assessment suggests that this should be a central feature of the program. Source: World Bank 2014. 11  upport that enhances commitment, coordination, and S cooperation helps improve the process of institutional capacity development in all three ICAM dimensions. Interpretation of the text of reports found that support for commitment, coordination, and cooperation was an important, consistent theme across all three dimensions of institutional capacity development (box 5). These three areas are also identified as the drivers of devel- opment effectiveness in the 2017 World Development Report (World Bank Group 2017).  ommitment, Coordination, and Cooperation across Institutional Box 5. C Capacity Dimensions with Examples of Enabling Actions Commitment of leaders: Support to enable community, civil society, private sector, and political leaders to consistently and frequently engage within policy and organizational reform processes. » Engagement of leaders in a country through dialogue, participatory approaches, and resource mobili- zation to drive action on important development constraints. Coordination of actions: Support decision-making processes that engage different organizations to work toward shared goals. » Establishment of operational and strategic forums of coordination, planning, implementation, and pro- gram monitoring and evaluation. Cooperation in pursuing a goal: Support interest groups to act when change is favorable and poten- tially to contribute to policy processes even when change is considered unfavorable. » Convene government and civil society to take a step-by-step approach toward better governance to improve public perceptions of the government commitment to addressing corruption. Source: Independent Evaluation Group. The evidence shows that leaders’ commitment for institutional capacity development is improved by working with different interest groups. Institutional development processes are hindered when a single leader, often from the government, is relied on. For example, a multisector micro, small, and medium enterprise project in Guinea did not identify risks to government commitment nor did it identify a private sector leader to champion dialogue; both factors hindered the further adoption of business 12 environment reforms (World Bank 2021b). To overcome this kind of challenge, leaders needed to be mobilized from different constituencies representing a diverse array of interests. The validation of Benin’s country program found that weak government commitment and governance processes undermined the policy reform program (World Bank 2018a). During project and country design processes, attention needs to be paid to existing coordination arrangements; otherwise, the processes can undermine successfully established institutional capacity. Existing cross-functional organizational structures, when included in designs, were often very important for supporting capacity devel- opment. In Senegal, the national executive bureau successfully provided day-to-day project management for a nutrition intervention, effectively handling the community subcontracts, supporting capacity strengthening for nongovernmental organizations in financial management procedures, and clarifying institutional arrangements with sector ministries (World Bank 2016). In contrast, in Ghana, the design of the country program missed an opportunity to address challenges in donor coordination, which contributed to weakened support in developing access to land and secure property rights (World Bank 2013b). In addition, in Sierra Leone, external partners provided parallel technical support to the Ministry of Finance, which increased the risk of fragmentation and incon- sistencies, an issue that could be resolved through partners coordinating with a shared framework (World Bank 2018f). Coordination during design helps develop shared action across partners, test for overly complicated assumptions, and identify valuable existing coordination mechanisms. In some examples, developing leadership to support participation of marginalized groups strengthened institutional capacity development efforts, such as the National Social Action Project in Sierra Leone that ensured the participation of women and people who were not literate. 13 Support for cooperation is needed to help maintain dialogue among those who have different views on interventions. Interventions that had cooperation challenges often did not define agreements for a shared understanding of oversight mechanisms, implementation processes, regulations, communications, and standards of transparency. Interventions that promoted enhanced cooperation used quality analytical underpinnings to justify decisions and supported improved perceptions of change processes. The evidence also shows that implementation was impeded because of inadequate efforts to increase consensus and shared actions. For example, in Ethiopia, previous efforts in promoting fertilizer marketing reform were undermined as the project reduced the role of farmer cooperatives in importing fertilizers by focusing on government capacity (World Bank 2011). Institutional capacity development often goes hand in hand with coordination and cooperation among different levels of government. Cases with a larger share of positive reporting on institutional capacity development interventions also implemented a greater extent of interventions at the subnational (38 percent) and community (44 percent) levels. Of the cases with a lower share of successful reporting, few interventions focused at the subnational (1 percent) and community (7 percent) levels. For example, in a case that predominantly contained positive accounts on institutional capacity development from Rwanda, the government adopted a decentralization policy and initiated a multiphase decentralization program in which local government structures adopted clear roles, responsibilities, and functions, with clear decentralization arrangements in various sectors (World Bank 2018e). This finding suggests that it is important to connect to different spheres of government to enact institutional change—a point reinforced by research (Andrews, Pritchett, and Woolcock 2017; World Bank Group 2017). Risks to achieved development outcomes were reported in 90 percent of cases related to commitment, coordination, and cooperation. For example, in Tanzania, weakening government commitment to the established processes of dialogue meant that Public Expenditure Reviews were perceived as externally owned (World Bank 2013e). Furthermore, in a project in Kenya, weak coordination between the national and the local agencies meant that ownership was not strengthened, which hindered the expansion and institutional reforms and put at risk achievements of pilot projects (World Bank 2019). 14 Methodology For this EIN, we systematically analyzed the text contained in 64 IEG reports using three steps—identification, analysis, and synthesis. The identification step consisted of downloading documents, automated text search, and manual review. Thirty-five IEG validations of Completion and Learning Reviews and Country Program Evaluations for operations in Sub-Saharan Africa approved after July 2008 were downloaded. Thirty Project Performance Assessment Reports with new financing approved after July 2008 were also downloaded. An automated text search was conducted using Python, applying a taxonomy of 98 search terms of between one and five words for the dimensions of the ICAM framework. Of these, 25 were stand-alone terms, 67 were terms occurring in the same sentence, and 6 were synonyms of terms. A manual review was undertaken focusing on documents that had a lower number of hits against the taxonomy. Of the 65 reports, all except one Project Performance Assessment Report were found to have some content related to institutional capacity development.8 To give an indication of the extent of institutional capacity development at a portfolio level, we manually reviewed the objectives of 439 projects assessed by IEG in determining the efficacy in Implementation Completion and Results Report Reviews, approved from July 2008 and closed after 2015. We analyzed the text through coding of the text and objectives, statistical review of text counts, and content analysis. All text matching the taxonomy was extracted from the cases. All text extracts were systematically coded in NVivo based on ICAM characteristics applying a protocol. To support quality coding, a test phase helped refine the protocol, and two reviewers randomly checked samples of coding. Through this process, we found that all selected cases had relevant text on an intent to change institutional capacity development. In total, 749 text extracts were coded as identifying factors related to changes in institutional capacity development, whether positive or challenging. For the content analysis, relevant text was reread twice with similar responses clustered based on shared content; this process was informed by methods developed for meta-ethnography (France et al. 2019). We also conducted frequency analysis related to dimensions, instrument, fragile and conflict-affected situation Specifically, the evidence drew from 29 out of 30 Project Performance Assessment Reports, all 30 validations of country reviews, and all five 8  Country Program Evaluations. 15 status, size of project, and country portfolio. A correlation matrix was prepared to test the connection between different variables, such as percent positive in a case and outcome rating. We manually coded all project objectives assessed by IEG in deter- mining efficacy and also performed frequency analysis against the ICAM framework. Our synthesis drew together findings from quantitative and qualitative analyses. We reread text clusters with descriptions compared and contrasted to identify shared issues that enable and hinder institutional capacity development. We performed further interpretive analysis to elaborate themes from positive and challenging experi- ences of institutional capacity development and cross-referenced research literature. The findings from the qualitative and quantitative analyses were compared, and the shared findings were identified. References Andrews, Matt, Lant Pritchett, and Michael Woolcock. 2017. Building State Capability: Evidence, Analysis, Action. Oxford: Oxford University Press. Calhoun, Craig, ed. 2002. Dictionary of the Social Sciences. Oxford: Oxford University Press. France, Emma F., Isabelle Uny, Nicola Ring, Ruth L. Turley, Margaret Maxwell, Edward A. S. Duncan, Ruth G. Jepson, Rachel J. Roberts, and Jane Noyes. 2019. “A Methodological Systematic Review of Meta-Ethnography Conduct to Articulate the Complex Analytical Phases.” BMC Medical Research Methodology 19 (1): 35. Hopson, Rodney, and Fiona Cram, eds. 2018. Tackling Wicked Problems in Complex Ecologies: The Role of Evalua- tion. Redwood City: Stanford University Press. Kusek, Jody Zall, Marelize Görgens Prestidge, and Billy C. Hamilton. 2013. Fail-Safe Management: Five Rules to Avoid Project Failure. Washington, DC: World Bank. Lokshin, Michael. 2021. “The Highways and Side Roads of Statistical Capacity Building.” Policy Research Working Paper 9764, World Bank, Washington, DC. OECD (Organisation for Economic Co-operation and Development). 2008. Accra Agenda for Action. Paris: OECD. Otoo, Samuel, Natalia Agapitova, and Joy Behrens. 2009. The Capacity Development Results Framework: A Strate- gic and Results-Oriented Approach to Learning for Capacity Development. Washington, DC: World Bank. Reed, Mark S., and Hannah Rudman. 2023. “Re-thinking Research Impact: Voice, Context and Power at the In- terface of Science, Policy and Practice.” Sustainability Science 18 (2): 967–81. Smithers, Nicola. 2011. The Importance of Stakeholder Ownership for Capacity Development Results. Washington, DC: World Bank. 16 World Bank. 2005a. Building Effective States: Forging Engaged Societies. Washington, DC: World Bank. World Bank. 2005b. Capacity Building in Africa: An OED Evaluation of World Bank Support. Washington, DC: World Bank. World Bank. 2011. “Ethiopia—Fertilizer Support Project.” Project Performance Assessment Report 62834, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2012. “Malawi—Community-Based Rural Land Development Project.” Implementation Completion and Results Report ICR2265, World Bank, Washington, DC. World Bank. 2013a. Designing a Multi-Stakeholder Results Framework: A Toolkit to Guide Participatory Diagnostics and Planning for Stronger Results and Effectiveness. Washington, DC: World Bank. World Bank. 2013b. “Ghana—Country Assistance Strategy Completion Report Review for the Period FY08–12.” Country Assistance Strategy Completion Report Review 81029, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2013c. “Malawi—Community-Based Rural Land Development Project.” Project Performance Assessment Report 75556, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2013d. “Sierra Leone—National Social Action Project.” Project Performance Assessment Report 76197, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2013e. “Tanzania—Poverty Reduction Support Credits.” Project Performance Assessment Report 78188, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2014. “Nigeria—Country Partnership Strategy Progress Report for FY10–FY13.” Country Partnership Strategy Completion Report Review 87026, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2015. “Mali—Completion and Learning Review for the Period FY08–15.” Completion and Learning Review Review 101662, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2016. “Senegal: A Decade of World Bank Support to Senegal’s Nutrition Program.” Project Perfor- mance Assessment Report 110290, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2017a. “Ghana—Agriculture Development Policy Operations: Phase I–IV.” Project Performance Assessment Report 112622, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2017b. “What’s Next for Capacity Building in Africa.” Discussion Note, World Bank, Washington, DC. World Bank. 2018a. “Benin—Completion and Learning Review for the Period FY13–18.” Completion and Learning Review Review 127644, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2018b. CPIA Criteria 2017. Washington, DC: World Bank. 17 World Bank. 2018c. Institutional Development Roadmap Handbook: Effective Design, Implementation and Evaluation of Project-Led Institutional Development in the Transport Sector. Washington, DC: World Bank. World Bank. 2018d. Romania—Country Partnership Framework for the Period FY19–FY23. Washington, DC: World Bank. World Bank. 2018e. “Rwanda—Fourth Poverty Reduction Strategy Grant, Fifth Poverty Reduction Support Grant, Sixth Poverty Reduction Support Grant, and Seventh Poverty Reduction Support Financing.” Project Performance Assessment Report 129569, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2018f. “Sierra Leone—Integrated Public Financial Management Reform Project.” Project Performance Assessment Report 124660, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2019. “Kenya—Agricultural Productivity Program: KAPP I and II.” Project Performance Assessment Report 133838, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2021a. Annual Progress Report on the Reimbursable Advisory Services Program in Romania: January 1–December 31, 2020. Washington, DC: World Bank. World Bank. 2021b. “Guinea—Micro, Small, and Medium Enterprises Development Project.” Project Performance Assessment Report 162989, Independent Evaluation Group, World Bank, Washington, DC. World Bank. 2022a. CPIA Africa: Assessing Africa’s Policies and Institutions. Washington, DC: World Bank. World Bank. 2022b. The World Bank’s Early Support to Addressing COVID-19: Health and Social Response. Independent Evaluation Group. Washington, DC: World Bank. World Bank. Forthcoming. Assessing Results and Development Impact (ARDU) of the World Bank RAS Program in Russia: Strengthening Institutions to Support Development Priorities. Washington, DC: World Bank. World Bank Group. 2017. World Development Report 2017: Governance and the Law. Washington, DC: World Bank Group. This Evaluation Insight Note was produced by Stephen Porter. Substantial contributions were made by Kavita Mathur, Dawn Roberts, Yingjia Liu, and Xiaoxiao Peng. Guidance was provided by Galina Sotirova. People who provided useful comments or inputs include Amanda O’Brien, Ariya Hagh, Estelle Raimondo, Noah Yarrow, and Rasmus Heltberg. Corresponding author: Stephen Porter, sporter2@worldbank.org. 18