38915 Capacity Development BRIEFS S H A R I N G K N O W L E D G E A N D L E S S O N S L E A R N E D LINKING INDIVIDUAL, ORGANIZATIONAL, AND INSTITUTIONAL CAPACITY BUILDING TO RESULTS By Vinod Thomas, Director-General, Independent Evaluation Group Achieving rapid development calls for improved capacity in the public and private sectors to support development policies and projects. The World Bank recognizes that capacity building is a long-term pro- cess requiring a systemic approach. That is why many Bank projects in Africa and elsewhere include capacity development activities. But three drawbacks have limited the effectiveness of these efforts: · Many operations do not--but need to--take an integrated view of solutions involving the individual, organizational, and institutional contexts. · Individual, organizational, and institutional links vary greatly across sectors; not addressing these differences has led to less effective capacity building. · Capacity goals as they relate to this understanding of the individual, organizational, and institutional aspects have not been explicit. This Capacity Brief discusses and illustrates the importance of integrating capacity-building efforts at all three levels, addressing differences among sectors in their integration, while setting forth explicit capacity goals and monitoring and evaluating progress toward them. Donors have addressed human resource problems efforts are demand driven and integrated within a mainly by financing training of individuals. These coherent capacity-building effort. efforts almost always achieve their target numbers of The goals of training, furthermore, are usually not people to be trained. But training is only part of the defined in terms of the desired outcomes for an orga- solution to building human capacity. For example, low salaries and poor working conditions often contribute to high outflow of trained staff. More important, the record is mixed when it comes to the effectiveness of Vinod Thomas is the Director-General of the Independent organizations where the trained people work. Evaluation Group of the World Bank. IEG (previously the Operations Evaluation Department) is an independent unit The Independent Evaluation Group's (IEG's) evalu- within the World Bank that reports directly to the Bank's Board ation of the Bank's support for capacity building in of Executive Directors. IEG assesses what works and what does Africa (World Bank 2005a) found that individuals are not, how a borrower plans to run and maintain a project, and the too often trained for specific tasks before the organi- lasting contribution of the Bank to a country's overall develop- zational framework has been reformed to allow them ment. IEG evaluations help improve the Bank's work by identify- ing and disseminating the lessons learned from experience and to use the training effectively. The more successful by framing recommendations drawn from evaluation findings. DECEMBER 2006 NUMBER 19 nization's effectiveness. Many of the measurement tools currently in use measure the reaction of the individual participant, which is not necessarily a good Only about one-third of the proj- predictor of how much the training improved his or ects that IEG reviewed for its Africa her organization's performance. So it is indeed diffi- cult to draw links between training and organizational evaluation were clear about the rela- capacity. In IEG's evaluation of the Bank's support for tionships among individual, organi- capacity building in Africa, IEG recommended that the Bank reassess what role training could play in its zational, and institutional aspects capacity-building support, how training could be pro- of capacity and the need to make vided, and what the respective roles of a central train- ing unit and regional programs could be in any future changes in their sequence and combi- support in this area. nation that can be expected to deliver results. Need for an Integrated View An important consideration in the overall effective- ness of capacity building then is: programs are seen within the context of larger insti- · How well institutional rules of the game (for tutional reform effort to ensure that the skills will be example, relating to tax and spending regimes) retained and applied. are linked with sustained performance of both organizational entities (such as ministries and their departments) as well as individuals Varying Success across Sectors responsible for delivering results (for example, staff). In addition, success in enhancing the performance of the public sector and individual, organizational, and Only about one-third of the projects that IEG institutional links varies greatly across sectors. reviewed for its Africa evaluation were clear about the relationships among individual, organizational, · Not addressing differences in linking individual, and institutional aspects of capacity and the need to organizational, and institution capacity building make changes in their sequence and combination that has led to less effective results. can be expected to deliver results. For example, in the health sector in Mali and elsewhere, a lot of effort was For example, capacity development in road construc- devoted to fixing weaknesses in organizational capac- tion projects has been particularly successful, because ity, even though the main problem was poor incen- the operations combine human resource development tives for staff. In other cases, the evaluation found plans with changes in the institutional framework that individuals are too often trained for specific tasks and organizational processes. Road projects tend to without reforming the organizational framework to let be characterized by clear goals, stakeholders with a them use the training effectively. direct interest in change, known techniques that are In contrast, individual, organizational, and institu- transferable across countries, and readily measurable tional aspects running more in tandem have shown success. For example, El Salvador's success in electric- ity sector privatization is due in part to progress made in preparing institutional changes in the form of com- prehensive sectoral legislation and rules that allowed the organizational and individual improvements More effort is needed to derive capac- to deliver results (box 1). Recent capacity building ity building lessons along sectoral and reform operations, such as those in Ethiopia, Mozambique, Rwanda, and Tanzania (box 2), incor- dimensions and to promote country- porate some of the lessons learned in El Salvador. led capacity building planning within In these examples, an effort is being made to align what an individual ministry is trying to do with the sector programs. government's overall development goals, and training DECEMBER 2006 NUMBER 19 Box 1. El Salvador: Bank Group Work in a Box 2: New Multisectoral Projects in Africa Country Committed to Power Reforms Integrate Essential Capacity-Building Elements into Project Design A technical assistance project in El Salvador was delayed for two years, while the best structures of the power sector were Four new multisectoral capacity-building projects in Africa defined. During this period, divergent views existed within the address both inter- and intra-ministerial capacity building Bank on the extent of privatization and reform to carry out. In issues in an integrated way. The $41.2 million Tanzania the end, the project succeeded in: Public Service Reform Credit, $100 million Ethiopia Public Sector Capacity Building Program, $20 million Rwanda (a) Developing a legal and regulatory framework for the Multisectoral Capacity Building Project, and $26 million sector, including restructuring of the Comisión Ejecutiva Mozambique Public Sector Reform Project have four key ele- Hidroeléctrica del Río Lempa, the state-owned electricity ments in common: utility; organization of the Transactions Unit (Unidad de Transacciones); and initiation of design of a wholesale · A capacity needs assessment conducted by the ministries/ market for electricity departments with stakeholder participation (b) Drafting and enacting a new electricity law and creating · A strategic process that aligns objectives and improve- the sector regulator ments sought in individual ministries and departments to (c) Estimating the marginal costs the government's overall development goals and public expenditure policies (d) Preparing a sector environmental action plan and imple- menting an environmental impact assessment · A human capacity development program (training, equip- (e) Developing a least-cost expansion plan for the system ment, and resources) within the context of a larger insti- tutional civil service reform effort to ensure retention and (f) Training the state-owned utility and government staff in application of skills acquired new operations and technical aspects. · Development of monitoring and evaluation systems to As a result, sector reforms have led to an increase in service provide (a) interprogram tracking and coordination so coverage, reduction in system losses, and decrease in state outcomes of the capacity-building program and other subsidies. Progress has continued even after Bank assistance complementary reform efforts remain coherent and (b) ended. The four government-owned distribution companies ministerial tracking and reporting on capacity develop- were privatized in January 1998, and generation companies ment efforts. were privatized in 1999. The following lessons can be learned from the privatization Source: World Bank 2005a. experience in El Salvador: · Sector policy and regulatory reform must be well under- way before privatization in the sector, so that bidders feel they are entering a secure environment and will have a sound basis for calculating their bids. Much of El Salvador's success in privatization is due to progress made beforehand in preparing comprehensive sectoral generate institutional, organizational, and human legislation and rules. resource achievements. · The government's reform and privatization team benefits It is not easy, however, to replicate road construc- from being staffed with qualified top-level staff with tion project experience. Health and education proj- proven commitment to a reform and track record of get- ects, for example, have seen less success in integrat- ting things done under extremely tight deadlines. ing the three dimensions; health and education face · Full-time attention must be paid to constituency building, greater challenges, because they tend to be labor lest public resistance impede the process or threaten its intensive and decentralized, relying on thousands of results. dispersed frontline service providers. Public pressure · High-level political support is important for the success of for change is more diffuse and it is more difficult to the reform and privatization process. When the message measure and manage for results. The health sector from the top is clearly in favor of privatization, the pro- in Africa is further handicapped by the continuous cess moves ahead rapidly. outflow of trained staff due to low salaries and poor Source: World Bank 2003. working conditions. It is, therefore, much more diffi- cult to set monitorable goals and to measure progress in the health and education sectors, compared with roads. More effort is needed to derive capacity build- ing lessons along sectoral dimensions and to promote results. These factors foster the emergence of a coher- country-led capacity building planning within sector ent capacity development strategy that has helped programs. Moving toward Results institutions and collaboration with key sectors are also important in the national AIDS response. In To be effective, capacity-building efforts also need Brazil, a prison condom program required coop- well-specified objectives and indicators of success, in eration and inputs from the Ministry of Health, other words: Ministry of Justice, and Ministry of Interior. The first India National Aids Control Project created · An understanding of the integration of indi- the National Aids Control Organization, a subna- vidual, organizational, and institutional aspects tional institution under the Ministry of Health and of capacity building must relate to a clear results Family Welfare. agenda. As part of this, IEG recommends that operations that aim to build capacity be based on an adequate upfront assessment of capacity and incorporate ways IEG recommends that operations that to monitor and evaluate results. The knowledge base needs to be strengthened, so that staff and clients aim to build capacity be based on can understand and build on links among the indi- an adequate upfront assessment of vidual, the organizational, and institutional dimen- sions. capacity and incorporate ways to Worrying about outcomes and results is a challenge monitor and evaluate results. for many organizations. The Bank provided $9 bil- lion dollars in lending and almost another $1 billion in grants and administrative budget to support public capacity building in Africa during the 10 years up to 2004. This support unfortunately focused little on outcomes from these efforts; yet, if African countries · The interplay with political factors in getting are to achieve their goals of reducing poverty, acceler- results might get more attention. Political commit- ating economic growth, and providing better services ment and government conditions at all levels can to their citizens, they must see better outcomes in the affect the success of institutional development and form of improved performance of their public sectors. capacity building. Broad sector reform is a long- That in turn is necessary for sustaining the benefits of term goal requiring country-specific institutional projects. development and capacity building, particularly A focus on results could shift the discussion in the in the area of regulation. A lesson learned from following ways: El Salvador's success in electricity sector priva- tization is the importance of top-level political · A disaster project might want to assess the poten- commitment. Political commitment to reform was tial result of being prepared for the next disaster. important for the success of the reform and priva- Between 1958 and 1995, the Bank committed tization process. $120 million to Honduras, financing the construc- · Improved sector management would be seen as tion of 1,270 kilometers of highways and feeder essential to turning educational resources into roads (an approximately 50 percent increase from learning gains. Peru shares with many countries the total kilometers of existing roads in 1955). In a fundamental lack of capacity for managing a 1998 Hurricane Mitch destroyed 6,000 of 10,000 massive and highly spread out primary educa- km of existing roads in Honduras, almost five tion system where teacher and school account- times what the Bank had helped to build. Studies ability are nonexistent and the main reasons show that $1 spent on prevention (such as disas- for the poor quality of educational services. The ter preparedness) can save up to $40 of damages Peruvian case suggests that management capac- in the future. ity building from the Ministry of Education to · A health project might be assessed in terms of school classrooms needs to be a priority for gov- its impact on the capacity to solve problems. ernments and for agencies lending to improve Building and strengthening institutions and the results in primary education in developing coun- capacity of the ministries of health, the lead tech- tries. nical and implementation agencies in national AIDS responses, has been central to the success In Africa, the bottom line is to boost the perfor- of AIDS projects. Building capacity in subnational mance of the relatively weak public sector. As more DECEMBER 2006 NUMBER 19 on the intended uses of the information that such a Box 3: Chile's Whole-Government system will produce, whether used to assist resource Monitoring and Evaluation System allocation decisions in the budget process, help national and sectoral planning, aid ongoing manage- Chile's monitoring and evaluation system, which is man- ment and delivery of government services, or underpin aged by the powerful finance ministry, has developed pro- accountability relationships. The more ambitious gov- gressively with time, partly in response to fiscal pressures, partly to the changing landscape of public sector reforms, ernment systems endeavor to achieve two or more of and partly in an opportunistic manner. these desired uses. Major milestones include the following: The need to tailor capacity development, particularly development of evaluation capacity, closely to country · Ex ante cost-benefit analysis (1974) has been required circumstances is more of an art, than a science. Even for all government projects. developed countries have encountered some difficul- · Performance indicator information (1994) has been col- ties in building sustainable monitoring and evalu- lected for all government programs. Regular information ation systems supporting sound governance. These on about 1,600 indicators is being used in formal reports difficulties are not so much technical as institutional. prepared for the Congress and as key data for various types of evaluation conducted. Countries often lack incentives to conduct monitor- ing and evaluation and use those findings to support · Comprehensive spending reports (1996) have been pro- better management, government decision making, duced. and underpin accountability relationships. These chal- · Government program evaluations (1996) are basically lenges are only magnified in developing countries. program reviews. The approximately 160 conducted so far have clarified and led to agreement on detailed program objectives, preparation of a log frame analysis, conduct of Conclusion desk reviews, and analysis of existing data. They cost an average of about $11,000 each and usually take from four to six months to complete. It is evident from the preceding discussion that · Rigorous impact evaluations (2001) entail primary data capacity-building efforts require a more nuanced and collection and often use control groups. Fourteen have multilayered approach to produce results. Such efforts been completed to date, taking up to 18 months each to finish at an average cost of $88,000 each. About 60 per- should adopt a three-tiered approach, synchronizing cent of the government's budget has been evaluated by individual, organizational, and institutional compo- means of either these impact evaluations or the govern- nents. They also need to be tailored to suit particular ment program evaluations. geographic and sectoral considerations, as well as flex- · Comprehensive spending reviews (2002) review all pro- ible enough to take into account political factors that grams within a particular functional area and look at might influence the preconditions to initiating capac- issues of inefficiency and duplication of programs. The ity-building efforts. As the El Salvador case illustrates, five completed so far have included desk reviews and cost an average of $48,000 each. it might be better in some circumstances to delay a capacity-building project until the right preconditions Source: McKay 2006. are present. Furthermore, enhancing the effectiveness of capacity-building efforts to direct them more toward outcomes ultimately has the potential to transform the way development projects are conceptualized and implemented. resources are invested in capacity building, better References indicators would also need to track how outcomes are being affected by investments in capacity build- ing and public sector management. In Latin America, Keith Mackay. 2006. Institutionalization of Monitoring Chile offers an example of a successful case of a and Evaluation Systems to Improve Public Sector whole-government monitoring and evaluation system Management, ECD Working Paper Series No.15. that takes into account lessons learned from other IEG, World Bank, Washington, D.C. countries and has tailored them to suit local condi- World Bank. 2003. Private Sector Development in tions (box 3). Electric Power: A Review of World Bank Group No single best approach to a national or sectoral Experience. Operations and Evaluation Department monitoring and evaluation system, however, exists. (OED), Washington, D.C. Each country faces different starting and end points to ------. 2004. Evaluation Capacity Development: OED which they aspire. The design of the system depends Self-Evaluation. OED, Washington, D.C. ------. 2005a. Capacity Building in Africa: An Acknowledgements OED Evaluation of World Bank Support. OED, Washington, D.C. ------. 2005b. Committing to Results: Improving The author would like to thank Shampa Sinha and the Effectiveness of HIV/AIDS Assistance. OED, Bonnie Bradford for their assistance in developing and Washington, D.C. producing this note. ------. 2006a. From Schooling Access to Learning Outcomes: An Unfinished Agenda. IEG, Washington, D.C. ------. 2006b. Hazards of Nature, Risks to Development: An IEG Evaluation of World Bank Assistance for Natural Disasters. IEG. Washington, D.C. About World Bank Institute (WBI): Unleashing the Power of Knowledge to Enable a World Free of Poverty WBI helps people, institutions, and countries to diagnose problems that keep communities poor, to make informed choices to solve those problems, and to share what they learn with others. Through traditional and distance learn- ing methods, WBI and its partners in many countries deliver knowledge-based options to policymakers, technical experts, business and community leaders, and civil society stakeholders; fostering analytical and networking skills to help them make sound decisions, design effective socioeconomic policies and programs, and unleash the produc- tive potential of their societies. WBI Contact: Mark Nelson; Program Manager, Capacity Development Resource Center Tel: 202-458-8041, Email: mnelson1@worldbank.org Visit our website for more information and download the electronic copies of all Capacity Development Briefs: http://www.worldbank.org/capacity DECEMBER 2006 NUMBER 19