WORLD BANK OPERATIONS EVALUATION DEPARTMENT Improving the World Bank's Development Effectiveness What Does Evaluation Show? 2005 The World Bank http://www.worldbank.org/oed Washington, D.C. © 2005 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone 202-473-1000 Internet www.worldbank.org E-mail feedback@worldbank.org All rights reserved Manufactured in the United States of America The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in the work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, , World Bank, 1818 H Street NW Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Cover photo: Uzbekistan family by Curt Carnemark for the World Bank Photo Library. ISBN-10: 0-8213-6461-8 ISBN-13: 978-0-8213-6461-1 e-ISBN 0-8213-6462-6 DOI: 10.1596/978-0-8213-6461-1 Library of Congress Cataloging-in-Publication Data has been applied for. Ainsworth, Martha, 1955­ Committing to results : improving the effectiveness of HIV/AIDS assistance: an OED evaluation of the World Bank's assistance for HIV/AIDS control / Martha Ainsworth, Denise A. Vaillancourt, Judith Hahn Gaubatz. p. cm. -- (Operations evaluation studies) Includes bibliographical references. ISBN-13: 978-0-8213-6388-1 ISBN-10: 0-8213-6388-3 1. Economic assistance--Developing countries--Evaluation. 2. AIDS (Disease)--Economic aspects--Developing countries. 3. HIV infections--Economic aspects--Developing countries. 4. AIDS (Diseasse)--Developing countries--Prevention. 5. HIV infections--Developing countries--Prevention. 6. World Bank. I. Vaillancourt, Denise. . II. Hahn Gaubatz, Judith. III. Title. IV World Bank operations evaluation study. HC60.A4575 2005 362.196'9792'0091726--dc22 2005052329 World Bank InfoShop Operations Evaluation Department E-mail: pic@worldbank.org Knowledge Programs and Evaluation Capacity Telephone: 202-458-5454 Development (OEDKE) Facsimile: 202-522-1500 E-mail: eline@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 Printed on Recycled Paper Contents v Executive Summary ix Acronyms and Abbreviations 1 1 Progress on Poverty and Improved Living Conditions 7 2 The Bank's Record in Supporting Development 7 Project Performance Ratings Show Improvements over the Past 10 Years 9 Country Assistance Evaluations 14 Two Special Categories of Country Programs 14 Post-Conflict Countries 14 Four Lessons Learned 16 Middle-Income Countries 17 Global Programs 21 3 Crosscutting Lessons from Sector and Thematic Evaluations 27 4 Looking Forward 29 Annexes 31 A: Management Comments 43 B: Report to the Board from the Committee on Development Effectiveness (CODE) 47 Endnotes 49 References Figures 1 1.1 Poverty Reduction Shows Large Regional Variations (1981­2001) 2 1.2 Only One-Third of Developing Countries Are Converging 4 1.3 Income Diverged But Social Indicators Converged Overall 5 1.4 Policy and Institutional Improvements 5 1.5 Democracy Advanced and Conflict Declined in the Past Decade 8 2.1 The Outcomes of Bank-Supported Projects Have Been Improving iii I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? 8 2.2 Project Outcomes Improved in All But One Region 10 2.3 The Majority of Sectors Show Improvement 11 2.4 Almost One-Third of Country Programs Show Unsatisfactory Outcomes 14 2.5 Post-Conflict Countries Received Very Different Levels of Bank Support per Capita 16 2.6 Bank Commitment and Disbursement as a Ratio to Total Budget Are Declining Tables 9 2.1 Country and Project Outcome Ratings 12 2.2 Sectoral and Project Ratings for Fiscal Year 2001­03 CAEs 19 2.3 Only a Few Global Programs Address Global Public Goods iv Executive Summary O ur task is not to fix the blame for the past--but to help fix the course for the future. --John F. Kennedy The pace of change in the overall perform- sion. Many programs and policies have been re- ance of the developing world has not altered vised, with the objective of making the Bank more markedly over the past 20 years. The number effective in its support of the global fight against of people living in extreme poverty declined from poverty. The following key changes have been fre- 1.5 billion in 1980 (40 percent of population), to quently highlighted in internal communications and 1.2 billion in 1990 (28 percent of population), to external commentary on the Bank as strategic 1.1 billion in 2001 (21 percent of population). moves designed to improve the Bank's develop- Growth per capita has followed much the same pro- ment effectiveness: file. In the 1980s, only about two-thirds of devel- oping countries showed positive per capita income · Eradication of poverty as the Bank's central growth, and this percentage remains unchanged. mission statement · Shift to country focus combined with decen- Life expectancy and literacy indicators show tralization of staff overall improvements, but some Regions show · Expansion of global programs to address worrisome trends. There has been slow and global challenges in partnerships steady progress in overall development out- · Enhanced participa- comes during the period, but the speed and tory and commu- Since the mid-1990s, the scale of change remain static. These averages, of nity-based project World Bank has course, mask huge differences across Regions, approaches with very worrisome increases in poverty and · Increased attention to dramatically altered its continued low growth in Sub-Saharan Africa. governance and in- direction to emphasize stitutional change poverty reduction as the Since the mid-1990s, the World Bank has dra- · Greater country own- matically altered its direction to emphasize ership and donor institution's main poverty eradication as the institution's main mis- alignment through mission. v I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? There has been a steady Poverty Reduction Strat- percent satisfactory in the past four years. This is egy Papers (PRSPs) and indeed impressive progress, and it should be con- improvement over the other instruments tinued. But it does not necessarily indicate im- past decade in the ratings · Greater focus on re- proved development impact at the country level. of the outcomes of Bank sults and performance OED's Country Assistance Evaluation (CAE) rat- · New approaches to ings, which take a long-term perspective and can projects, but almost one- deal with fragile states-- better assess the impacts of World Bank support third of country programs for example, low-income to countries, show a satisfactory outcome in about are rated unsatisfactory. countries under stress two-thirds of the Bank's country programs. They (LICUS) also show that even when project ratings in a · Enhanced safeguard policies. sector are high, sectoral outcomes may not be sat- isfactory. Some of these initiatives are now well estab- lished, while others, such as the strengthened re- With one-third of country programs rated un- sults focus, were introduced more recently and satisfactory, there is substantial room for im- are still evolving. How have these initiatives im- provement in the Bank's development proved the Bank's development effectiveness? effectiveness through more coherent country programs tailored to country circumstances, as This paper uses recent evaluations to assess well as through further improvements in spe- the development effectiveness of the World Bank cific projects and programs. The PRSP initiative and how it could be improved. It covers three offers the low-income countries a framework questions: under which both country ownership can be en- couraged and donor interests can be incorpo- 1. What is the measurable progress in improved rated. This direction could also improve the living conditions in the developing world over outcomes of the Bank's country assistance pro- the last 10­15 years, and how does it compare grams. with the previous decade? 2. How effective has the World Bank been in The Bank is struggling to improve its effec- helping countries improve their living condi- tiveness at two ends of the development spec- tions through its various interventions and trum--in LICUS countries where state capacity programs? has collapsed or weakened considerably, and 3. Finally, what broad lessons emerge from OED in middle-income countries (MICs) that have evaluations on improving the Bank's devel- much greater access to other sources of capital. opment effectiveness? Greater focus and selectivity are also needed in the Bank's global programs to enhance their The Bank's development effectiveness can poverty impact and their benefits to developing be measured at the project or individual program countries. level, at the country level, and by looking at its global programs. The Bank also must focus on its costs of doing business, which have risen significantly over the The PRSP initiative offers There has been a last 10 years. For every dollar of its administra- steady improvement tive budget the Bank disbursed $13 in fiscal year promise, but the Bank is over the past decade in 1995, but only $9 in fiscal 2005, after a brief in- struggling to improve its the ratings of the out- crease in fiscal 1998­99 during the Asian crisis. effectiveness at two ends comes of Bank projects. Bank commitments have also fallen from $16 of the development Outcome ratings have in- per dollar of administrative budget in fiscal 1995 creased from around 65 to $11 in fiscal 2005. The average size of loans spectrum--LICUS and MIC percent satisfactory in has fallen from around $90 million in the mid- countries. the mid-1990s to over 75 1990s to around $80 million in the last two years.1 vi EXECUTIVE SUMMARY Lending has stagnated but the budget has · Improve monitoring and evaluation for re- continued to grow. Lending has dropped sharply sults, and start measuring what is important. to middle-income countries with access to · Improve coordination within the Bank and cheaper sources of finance. across the Bank Group. The Bank argues that some of the increased This review of the Bank's development effec- cost, such as safeguards or new community- tiveness points to a number of opportunities for based approaches, has helped improve its proj- further improvements. ects. Some of the increased cost is due to Some of these will re- Greater emphasis on expansion of knowledge activities, and some to quire changes in busi- growth is vital for lasting the financing of global programs. But smaller ness processes, such as poverty reduction. loans do not square with the Bank's objective of new ways of dealing with scaling up its poverty-reducing activities. It must LICUS and MICs and further improvements with also address why lending is declining at the Bank the PRSPs. Some will require changes in the or- and not in other Multilateral Development Banks ganization, to improve coordination within the (MDBs), and how effective all its non-lending ac- Bank and across the Bank Group to set the right tivities--the knowledge bank--are for support- incentives; consolidate the Bank's networks to- ing development. Knowledge and lending should ward the two pillars--the investment climate and complement each other and not be seen as sub- social inclusion; more focused use of knowledge stitutes. to address country needs, less mechanistic ap- plication of safeguards; OED findings and evaluations at the project, and a review of the ma- Greater selectivity and a sectoral and thematic, and country and global lev- trix structure, which more hard-nosed focus on els suggest that the following nine directions leads to multiplication results are needed. can improve the Bank's development effective- of tasks and works ness: against selectivity. · Understand and analyze comprehensively, Still others will require strategic choices for but act far more selectively. the institution, such as how to define its role · Emphasize institutional development and ca- more selectively, more focused on its core com- pacity building even more strongly. petencies in the global war on poverty, rather · Re-think areas of punctured optimism, such than trying to cover every aspect of develop- as growth, private provision of infrastructure, ment. A greater emphasis on growth is needed and turn-around situations. for lasting poverty reduction. · Tailor programs and projects to the circum- stances of each country, and adapt strategies The Bank has transformed itself significantly to the local political economy. in the past 10 years, and should be ready for fur- · Lend mainly to countries with improving poli- ther adjustments to current climate of rapid cies and institutions, but find ways to deal change. Greater selectivity, more flexibility, and with poor, misgoverned states. improved efficiency within its chosen areas of in- · Introduce greater flexibility into programs tervention are needed going forward if a global with well-managed MICs. institution such as the Bank is to remain useful · Make better use of the Bank's knowledge and and relevant and show concrete results in a fast- technical assistance. changing world. vii ACRONYMS AND ABBREVIATIONS AAA Analytical and advisory activities AFR Africa Region APL Adaptable Program Loan ARDE Annual Review of Development Effectiveness CAE Country Assistance Evaluation CAS Country Assistance Strategy CDD Community-driven development CEPF Critical Ecosystems Partnership Fund CGAP Consultative Group to Assist the Poor CGIAR Consultative Group on International Agricultural Research CPIA Country Policy and Institutional Assessment (ratings) EAP East Asia and the Pacific Region ECA Europe and Central Asia Region ED Education EMT Energy and mining ENV Environment EP Economic Policy ESMAP Energy Sector Management Assistance Program ESW Economic and sector work FIRST Financial Sector Reform & Strengthening Initiative FSAP Financial Sector Assessment Program FSP Financial sector FY Fiscal year GAVI Global Alliance for Vaccines and Immunization GDN Global Development Network GDP Gross domestic product GEF Global Environment Facility GIF Global Integrated Pest Management Facility GPP Global programs and partnerships GWP Global water partnerships HIPC Heavily Indebted Poor Countries HNP Health, nutrition, and population IBRD International Bank for Reconstruction and Development (World Bank) IDA International Development Association IDB Inter-American Development Bank IF Integrated Framework for Trade-Related Technical Assistance IFC International Finance Corporation IFI International financial institution infoDev Information for Development Program LAC Latin America and the Caribbean Region ix I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? LDC Least-developed country LICUS Low-income countries under stress LIL Learning and Innovation Loan M&E Monitoring and evaluation MDB Multilateral development bank MDG Millennium Development Goal MIC Middle-income country MIGA Multilateral Investment Guarantee Agency MLF Multilateral Fund for the Implementation of the Montreal Protocol MNA Middle East and North Africa Region ODA Official development assistance OED Operations Evaluation Department (World Bank) PPIAF Public-Private Infrastructure Advisory Facility PREM Poverty Reduction and Economic Management Network PRS Poverty reduction strategy PRSP Poverty Reduction Strategy Paper PSD Private sector development PSDD Private Sector Development Department PSG Public sector governance PSM Public sector management RBM Roll Back Malaria RDV Rural sector SAR South Asia Region SP Social protection SSA Sub-Saharan Africa SWAp Sectorwide approach TDR Special Program for Research and Training in Tropical Diseases TR Transport TSS Transitional Support Strategy UCW Understanding Children's Work UD Urban development UNAIDS Joint United Nations Program on HIV/AIDS USAID United States Agency for International Development WSP Water and Sanitation Program WSS Water supply and sanitation x 1 Progress on Poverty and Improved Living Conditions T he global fight against poverty shows progress--but at a very slow pace and with huge variations across Regions. The number of people in the developing world living in extreme poverty declined from 1.5 bil- lion in 1980 (40 percent), to 1.2 billion in 1990 (28 percent), and to 1.1 bil- lion in 2001 (21 percent). This progression could be construed as a slow-down in the rate of improvement (figure 1.1). Figure 1.1: Poverty Reduction Shows Large Regional Variations (1981­2001) 70 60 Percentage of population living on less than US$1 per day 50 40 30 20 10 0 East Asia & China Eastern Latin America Middle East & South Asia Sub-Saharan All LDCs Pacific, Europe & & Caribbean North Africa Africa excluding Central Asia China 1981 1990 2001 Source: 2004 Annual Review of Development Effectiveness (OED 2005a). Note: LDCs = least-developed countries. 1 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Poverty has increased Much of the decline tion of almost 3 billion people--half the world's sharply in the Africa and took place in East Asia-- population--continues to reduce poverty in the and most spectacularly coming decade, and to reduce it in a more Eastern Europe and in China. In South Asia, dramatic manner. But Sub-Saharan Africa would Central Asia Regions. which has the largest remain a Region of concern in the war on population of poor poverty. people, the number of poor people has begun to decline after almost a half-century of All the evidence points to a very close increase.1 correlation between poverty reduction and growth in income per capita. But growth Poverty has increased sharply in the Africa remains as elusive as it was in the 1980s. The and Eastern Europe and Central Asia Regions most dramatic reduction in poverty has and has shown no visible improvements in occurred in countries with very rapid growth in Latin America or the Middle East. Overall, some income per capita. Poverty reduction is almost 50 percent of developing countries showed an a mirror image of this growth. But in more than increase in the number of people below the two-thirds of developing countries, income per poverty line of US$1 per day during the 1990s. capita is growing more slowly than in the developed world. In the past decade Broad social indicators there has not been a The picture looks even less promising if one show more widespread measurable acceleration looks back to 1980: from 1980 to 2003, more convergence with the in the eradication of than three-quarters of the developing poverty in the develop- countries have grown more slowly than the developed world, with the ing world. This picture developed world. Convergence of income, exception of Sub-Saharan could change as Asia, except for Asia and some accession countries Africa. with a combined popula- in Eastern Europe, is only a distant possibility. Figure 1.2: Only One-Third of Developing Countries Are Converging 9.0 6.0 GDP per capita growth 2.1% Long-run growth in 3.0 U.S. GDP per capita 0.0 ­3.0 ­6.0 ­9.0 $100 $1,000 $10,000 $100,000 Income Source: SIMA­February 2005. GDP PC observations for 143 developing countries between 1990 and 2003. 2 PROGRESS ON POVERTY REDUCTION AND IMPROVED LIVING CONDITIONS Moreover, almost one-third of the developing the proposition that Institutional quality and countries have shown no increase in GDP per conflict and poverty capita since 1980. Once again, the growth seem to feed off each governance, which effect picture in Asia looks encouraging, but Sub- other. Indicators of people's lives, especially Saharan Africa remains mired in slow growth. democracy, which im- those of the poor, show proved sharply in the some improvements. In contrast to income and poverty indica- late 1980s and early tors, broad social indicators, such as life 1990s, continued to show steady but slower expectancy and literacy, show more widespread improvements during the past few years (see convergence with those in the developed figure 1.5). world, with the exception of Sub-Saharan Africa (figure 1.3). In every other Region there is To summarize, over the past decade, the convergence toward the developed country record of development is very mixed. Poverty figures for basic social indicators such as life has declined, but not at an accelerated pace, expectancy and literacy. But, again, Sub- except in East Asia. Growth remains as elusive Saharan Africa stands as an outlier on life as it was in the 1980s. Social indicators are expectancy, partly because of the widespread improving and show incidence of AIDS in the sub-continent. convergence, except on Conflict has not declined life expectancy in Sub- in the poorest quartile of Institutional quality and governance, which Saharan Africa, largely effect people's lives, especially those of the due to HIV-AIDS. countries. Conflict and poor, show some improvements (figure 1.4), but poverty feed off each there has been deterioration in several Other factors that other. components, including corruption and political affect people's lives-- instability. There is a perception that other such as governance and conflict--show some indicators, such as conflict and democratization, improvements, but again the poorest countries are deteriorating--but evidence does not do not show progress. The drive for more support this proposition. In fact, indicators of democratic government, which increased conflict show an improvement (less conflict), dramatically in the 1980s and early 1990s, although they remain at high levels for the continued to show small improvements. There poorest countries. Conflict has not declined in are reasons for hope--but also reasons for the poorest quartile of countries. This supports concern, especially in Sub-Saharan Africa. 3 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Figure 1.3: Income Diverged But Social Indicators Converged Overall (Regional as percentage of developed world) GDP per capita (constant 1995 US$) 30 25 Percentage 20 15 10 5 0 1980 1990 2000 Regions Life expectancy at birth (total years) 100 90 Percentage 80 70 60 50 1980 1990 2000 Regions Adult literacy (people ages 15 and above) 100 80 60 Percentage 40 20 0 1980 1990 2000 Regions EAP ECA LAC MNA SAR SSA Source: SIMA­April 2005. Developed world = high income, OECD countries. Note: EAP = East Asia and the Pacific; ECA - Europe and Central Asia; LAC = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia; SSA = Sub-Saha- ran Africa. 4 PROGRESS ON POVERTY REDUCTION AND IMPROVED LIVING CONDITIONS Figure 1.4: Policy and Institutional Improvements 3.8 3.7 3.6 3.5 CPIA ranking 3.4 3.3 3.2 3.1 3.0 2.9 2.8 Economic Structural Policies for social Public sector management policies inclusion/equity management and institutions 2003 1999 Source: 2003 Annual Review of Development Effectiveness (OED 2004a). Note: CPIA = Country Policy an Institutional Assessment. Figure 1.5: Democracy Advanced and Conflict Declined in the Past Decade Trends by armed conflict type, 1946­2004 Global democracy and autocracy, 1946­2004 100 1,000 Episodes of major armed conflict Units of democracy and autocracy 80 800 60 600 40 400 20 200 0 0 1950 1960 1970 1980 1990 2000 1950 1960 1970 1980 1990 2000 Inter-state war Revolutionary war Ethnic war Democracy Autocracy Source: Center for Systemic Peace. Note: The figure on the right sums Polity IV scores of institutional authority for democracy and autocracy for each independent state for each year. Each country is given annual scores (10-point scales) on each of two basic types of regime authority. Although the two types are opposing, many countries exhibit mixed authority traits (that is, they have middling values on each scale). The figure shows global changes in total "units of democracy" in contrast to total "units of autocracy" in the global system. 5 2 The Bank's Record in Supporting Development G iven the World Bank's enhanced profile in the fight against poverty in the past decade, how effective has it been in helping countries eradicate poverty? Critics of the Bank from one side argue that much of what the Bank does is immiserization. They say that countries that have followed advice from the World Bank--as in Sub-Saharan Africa--have not shown much progress. Where success has been achieved in the fight against poverty, as in Asia, it is largely the result of home-grown policies and programs, which often go against the advice offered by international agencies. Critics from another side argue that by Project Performance Ratings Show working largely with ineffective and corrupt Improvements over the Past 10 Years1 governments, the Bank not only has very little to show for its efforts, but has also helped prop Outcome up these governments and perpetuated Since fiscal year 2000, Project performance poverty. project performance as ratings show measured by outcome To answer these criticisms, it is useful to has shown continued improvements over the look objectively at the Bank's record over the improvement, except in past 10 years, and project past decade. In this section we look at the fiscal 2003: outcome outcomes improved in all Bank's performance at various levels of ratings dropped from 78 aggregation--at the project level, at the percent to 75 percent but one Region. country level, and at the global level. What satisfactory between fiscal 2002 and fiscal 2003, results are attributable to support from the as illustrated in figure 2.1. A partial cohort for Bank? Which programs and projects supported fiscal 2004 shows a resumption in the improve- by the Bank have shown good results? What ment in outcome ratings. This represents a processes and initiatives show promise to recovery from the fiscal 2003 drop, and may be build on for the future? indicative of a resumption of the upward trend 7 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Figure 2.1: The Outcomes of Bank-Supported Projects Have Been Improving 100 Percentage satisfactory outcome 80 60 40 20 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004* Exit fiscal year By project Weighted by disbursement Source: World Bank database, 2005. Note: 2004* Partial (dashed line). Figure 2.2: Project Outcomes Improved in All But One Region EAP Distribution of disbursements by Region (FY95­04*) SAR MNA SAR AFR 15% 13% Bankwide MNA LAC 6% ECA EAP 24% AFR 40 50 60 70 80 90 100 LAC 26% Percent satisfactory outcome (weighted by disbursement) ECA FY95­99 FY00­04* 16% Source: World Bank database, 2005. Note: 2004* Partial. EAP = East Asia and Pacific, SAR = South Asia, MNA = Middle East and North Africa, LAC = Latin America and Caribbean, ECA = Europe and Central Asia, AFR = Sub- Saharan Africa. The majority of the in project performance 2000­04 (partial) cohort compared with the sectors show that started in fiscal fiscal 1995­99 cohort. The East Asia and South 2000. Asia Regions are the top performers for the improvement in project fiscal 2000­04 (partial) cohort, exceeding the outcomes. Regional Performance Bank average of 80 percent. This is not Figure 2.2 presents the surprising, given the overall gains in growth percentage of satisfactory project outcomes, and poverty reduction in those Regions. The weighted by disbursement, for the fiscal Latin America Region, which was the best 8 THE BANK'S RECORD IN SUPPORTING DEVELOPMENT performer of the fiscal 1995­99 cohort, is the whether project interventions are addressing only Region that declined in performance for key development constraints. the fiscal 2000­04 (partial) cohort, partly reflecting the overall performance in the The Country Assistance Evaluation (CAE) is a Region. comprehensive evaluation of the Bank's program in a country Sectoral Performance that includes both In a third of country Compared with fiscal 1995­99 exits, the projects and analytical programs evaluated for outcome performance weighted by disburse- and advisory services. the period 1990-2003, ment for the fiscal 2000­04 (partial) exits Since CAEs assess the improved for 7 of 13 sector boards.2 overall Bank strategy, aggregate project including the size, outcomes were Figure 2.3 presents the sector boards' sectoral composition, satisfactory, but the outcomes in order of their performance. The and type of lending, the overall country assistance biggest gains in outcome ratings were in the country assistance out- water supply and sanitation and financial come may thus be programs were sectors. The greatest declines in performance unsatisfactory if, for unsuccessful. were in the social protection and economic example, there are policy sectors. Outcomes for economic policy, critical omissions in the Bank's overall energy and mining, environment, and water assistance strategy, even if the outcomes of supply and sanitation remain below the individual projects are rated satisfactory. Bankwide average for fiscal 2000­04 (partial). The Bank can have a satisfactory program, The overall improvement in lending even in countries that outcome indicators was not created by a shift are not doing well. The Bank can have a in the composition of lending toward adjust- Figure 2.4 presents the satisfactory program, ment loans--both adjustment and investment outcome ratings for 21 even in countries that are lending show improved indicators. While the countries that were improving trends in project performance are classified into three not doing well. noteworthy and creditable, they offer no groups: good perform- ground for complacency. They do not always ers, transition countries, and modest and poor show impacts and final results. Impact evalua- performers. tions offer one solution to assessing better the results of Bank support and are now being GDP per capita grew rapidly in the first initiated more vigorously by the Bank. But they group of six good-performing countries (Chile, also offer only a partial picture, and in any case China, Dominican Republic, India, Tunisia, and would cover only a part of the Bank's activities.3 Vietnam) from 1990 to 2003, ranging from 3.1 percent per annum in Tunisia to 8.1 percent in Country Assistance Evaluations China. This led to substantial reductions in the In a third of the country programs evaluated for the period 1990-2003, aggregate project outcomes were satisfactory, but the overall Table 2.1: Country and Project Outcome country assistance programs were unsuccess- Ratings (percent) ful (table 2.1). Evaluation at the country level yields a more complete picture of the outcome Country project Country outcome ratings of the Bank's assistance programs than do portfolio outcome Satisfactory Unsatisfactory evaluations of individual programs or projects. Satisfactory 53 33 Evaluation needs to capture critical dimensions Unsatisfactory 7 7 of country assistance programs, such as Source: OED databases and staff estimates. 9 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Figure 2.3: The Majority of Sectors Show Improvement Improvers TR PSG FSP RDV ED Bankwide WSS EMT 40 50 60 70 80 90 100 Percent satisfactory outcome (weighted by disbursements) Decliners PSD Bankwide SP ED = Education UD EP = Economic policy HNP EMT = Energy and mining ENV = Environment ENV FSP = Financial sector HNP = Health, nutrition, and population EP PSD = Private sector development PSG = Public sector governance 40 50 60 70 80 90 100 RDV = Rural sector Percent satisfactory outcome (weighted by disbursements) SP = Social protection FY95­99 FY00­04* TR = Transport UD = Urban development Disbursements by Sector Board (FY95­04*) WSS = Water supply and sanitation WS ED UD 4% 4% 8% TR 11% EMT 14% SP 5% ENV 2% RDV 12% EP 14% PSD 3% PSG FSP 6% HNP 12% 5% Source: World Bank database, 2005. Note: 2004* Partial. The sector classification provides a single rating for the entire project rather than rating individual project components that cover particular sectors or themes. The sector board classification applies to the whole project and enables outcomes to be matched to it. 10 THE BANK'S RECORD IN SUPPORTING DEVELOPMENT Figure 2.4: Almost One-Third of Country Programs Show Unsatisfactory Outcomes Highly Good performers Transition countries Modest and poor performers satisfactory 6 Satisfactory 5 Moderately satisfactory 4 Moderately unsatisfactory 3 Unsatisfactory 2 Highly unsatisfactory 1 0 ile Tu a ia nic am p. ia Lit nia M nia Bu lia ian a Cr d. M ia a il at n ala Le ru M ho Za o Zim ia e in i ov az Gu a c bw nis Re Ind Ru lgar Fe t mb Pe oc Ch go rd t oa Ch mi tn me a em so Br old hu ba Jo or on Do Vie an Ar ss Source: 2004 Annual Review of Development Effectiveness (OED 2005a), Appendix C, table C.1. percentage of the population living below the programs were more successful in Armenia and poverty line (income poverty), from 35 Lithuania than in Croatia and Moldova. percent in 1990 to 16 percent in 2001. But even in good performers, the outcome of the The third group, of eight modest and poor- Bank's program could be less than fully performing countries (Brazil, Guatemala, Jordan, satisfactory because of a few ill-chosen Lesotho, Morocco, Peru, Zambia, and interventions, as in India and the Dominican Zimbabwe), experienced either negative or low Republic. growth in GDP per capita for the period 1990­2003 and showed no improvements in The seven countries in the second group income poverty. But even in this category of experienced negative growth in the early stages countries, the Bank's program could have of their transition, starting in 1990. This was successful outcomes, as followed by a slow recovery, starting in 1993 in in Brazil, if interventions The Bank has had Armenia, Croatia, and Mongolia; 1994 in were chosen carefully. In successful outcomes in Lithuania; 1997 in Bulgaria; 1998 in Russia; and Brazil the Bank was health and education, but 1999 in Moldova. All the transition countries successful in supporting efforts in private sector experienced substantial increases in poverty fiscal reforms and in during their periods of recession, as well as regional programs in development, public considerable increases in inequality and deteri- Northeast Brazil, which sector management, and orating social indicators. While rapid economic lags behind other parts rural development have growth has brought about a decline in income of the country. poverty in these countries in recent times, There is no obvious been less successful social indicators have lagged. The Bank's correlation between overall. 11 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Table 2.2: Sectoral and Project Ratings for Fiscal Year 2001­03 CAEs CAE sector outcome ratings (% satisfactory)a Project outcomes Satisfactory Unsatisfactory Mixed Not assessed (% satisfactory)b Social protection 12 24 20 44 37.9 Rural 12 36 28 24 87.2 Private sector development 12 24 44 20 60.1 Environment 16 16 24 44 25.5 Financial sector 24 16 32 28 81.6 Public sector 24 32 28 16 99.9 Infrastructure 36 16 24 24 87.7 Health 40 20 8 32 86.5 Education 48 8 4 40 100.0 Source: World Bank data and OED staff estimates. a. These columns are ratings by CAE task managers and a desk review of CAEs of the outcomes by sectors. In most cases, explicit ratings by sectors were not included in the CAEs, and thus these ratings should be treated as indicative only. b. This is the percentage of satisfactory project outcomes by sector (commitment value) for the countries where CAEs were completed during FY01­03. successful country assistance ratings and had poor outcomes in the sector as a whole decentralization of Bank teams, although because: strong field presence is cited as a contributory factor to program success. Field presence, pari · Reforms in these sectors seem to face stronger passu, should improve country knowledge and opposition from vested interests and potential adaptation to local conditions. Greater field losers. presence is also welcomed by most Bank · Institutional capacity constraints take time to re- clients. But a successful country assistance solve and many projects in these sectors attempt strategy requires relevance and the astute use to improve upon the legal, institutional, and reg- of the Bank's skills in a specific set of country ulatory framework, but to implement change, in- issues where Bank support is needed. ertia and adverse incentives in the bureaucracies of many countries must be overcome. The Bank has had successful outcomes in · These sectors also suffer larger adverse effects health and education. But in private sector from exogenous events and macroeconomic development, public sector management, shocks. Improving outcomes in all sectors and rural development, the Bank's efforts would imply focusing more on measuring and have been less successful overall (table 2.2). supporting results-based indicators. These results are also largely reflected in aggregate project ratings for the same set What lessons can we draw for improving the of countries (table 2.2). In private sec- formulation and implementation of the Bank's Successful country tor development (PSD), country assistance strategies? First, a key rural development, and component of successful country programs is programs are tailored to public sector manage- that they are tailored to the country context, the country context, and ment (PSM), despite and a sound understanding of the political a sound understanding of the high proportion of economy of reform is essential. Domestic satisfactory outcomes politics and vested interests largely determine the political economy of at the project level, the the pace and content of reform. An insufficient reform is essential. Bank's program has understanding of the political economy of 12 THE BANK'S RECORD IN SUPPORTING DEVELOPMENT reform led the Bank to push reforms that stood track record in, im- The Bank needs to resist little chance of success. More active dialogue plementing reforms. pressures to persist with with parliaments, local governments, and stakeholders enhances the Bank's understand- There are, however, adjustment lending in the ing of political economy considerations. few cases when the absence of government Bank resisted such commitment to, and a Second, country knowledge is strongly pressures. In many associated with success. It is well understood countries, giving in to satisfactory track record that project interventions are more successful the pressure to lend in, implementing reforms. when they are based on in-depth analytical produced a number of work. The same finding conveys to the success unsatisfactory outcomes. More important, it or failure of country programs. The OED CAE weakened the incentive to reform. Retrospective (OED 2005b) found that in more than two-thirds of successful country Finally, there are several specific actions that programs, the Bank's analytical work was would serve to improve the outcomes of Bank timely. assistance programs: Analytical and advisory activities (AAA) can · Carry out more robust risk analysis to carefully also be an effective vehicle for engaging govern- assess borrower implementation capacity and ments in policy dialogue given to dissemination. commitment to reform. This work should be In many cases, dissemination has been informed by the analysis done by others and inadequate. by the feedback obtained through wider dia- logue with stakeholders. Poor dissemination can also be a feature of · Reduce the level of planned assistance when participatory analytical work, as findings still faced with clear evidence of policy slippage. may not be widely shared outside the govern- The Bank needs to set clear and meaningful ment ministries that collaborated in the analyt- triggers for its assistance programs and, when ical work with the Bank. confronted with slippage, cut back its level of assistance. Third, technical assistance and investment · Lend more prudently in turnaround situa- loans can help promote institutional develop- tions. This is especially important in the pres- ment and capacity building. But the sustainabil- ence of long-standing issues of implementation ity of benefits hinges on the incorporation of failure. Initial levels of assistance should be these operations within a broader macro prudent and calibrated to measurable out- stabilization and reform program. Linking comes and to meeting technical assistance and investment loans with concrete benchmarks. The Bank is struggling policy reforms supported by adjustment loans The outcomes of Bank with its model for also improves the probability of success. lending are better when lending goes up development effectiveness Fourth, adjustment lending can be success- the ladder with the re- in LICUS countries and ful, especially when combined with a strong form program, rather MICs. government commitment to macro stabiliza- than ahead of it. tion and structural reform. But adjustment lending in the absence of sustained progress In summary, optimistic projections or on stabilization and reform saddles the country expectations accompanied by inadequate risk with debt and weakens the incentive to reform. analysis often weaken the performance of The Bank needs to resist pressures to persist country strategies. Programs should be based with adjustment lending in the absence of on the most likely--not the best possible-- government commitment to, and a satisfactory forecasted outcomes. And, finally, country 13 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Figure 2.5: Post-Conflict Countries Received Very Different Levels of Bank Support per Capita 50 45 40 Per capita annual average 35 30 25 20 15 10 5 0 Bosnia and Timor Leste West Bank Rwanda Eritrea Cambodia Herzegovina (2000­03) and Gaza (1994­97) (1994­97) (1992­95) (1996­99) (1994­97) Source: World Bank Business database. strategies need to be flexible, not rigid and 38 post-conflict countries and areas. A fifth of narrow, with only one path to follow. Bank commitments were devoted to those countries in fiscal 2004. When Bank commit- Two Special Categories of Country ments to Bosnia and Herzegovina are Programs compared with the Bank's response in other There are at least two categories of countries post-conflict countries, the exceptional nature where the Bank is struggling with its model for of the Bosnia and Herzegovina case stands out development effectiveness: low-income starkly (figure 2.5). countries under stress and middle-income countries. OED is beginning work on LICUS Four Lessons Learned countries, and it is too Early engagement and a early to draw any Early Engagement in Post-Conflict Situations strong and continuous lessons for that group. The need for early engagement is strongly But OED has reviewed indicated by Bank experience. Time is of the field presence improve its CAEs for a subset of essence in post-conflict situations. Often, there performance. post-conflict countries are windows of opportunity for significant (some of these are also progress, but these opportunities can quickly LICUS) and has an evaluation of LICUS under pass. way. In general, the Bank has responded quickly Post-Conflict Countries and flexibly in initiating contact and preparing Since 1980, over 50 countries have experienced or participating in the preparation of strategy significant periods of conflict, with severe and and funding packages. Many factors constrain lasting effects through destruction of physical the Bank's early engagement in post-conflict assets, disruption in trade links, displacement situations, including the presence of "stateless of people, and loss of income and life (includ- societies." Outstanding debt and arrears to the ing genocide). The World Bank is engaged in Bank constitute an additional impediment to 14 THE BANK'S RECORD IN SUPPORTING DEVELOPMENT the Bank's early engagement. Since access to where conditions do not warrant a full CAS, the resources from the Bank and other interna- Bank has recently adapted this approach and has tional financial institutions (IFIs) is important prepared Transitional Support Strategies (TSS) for for most post-conflict countries' medium- and nine countries, including Afghanistan (2003), longer-term development, planning financial Angola (2003), Burundi (2002), Democratic normalization is crucial. Republic of Congo (2004), and Eritrea (2000), consistent with the Bank's post-conflict policy. Strong and Continuous Field Presence The importance of a strong and continuous In a post-conflict In a post-conflict setting, field presence is the second lesson to emerge setting, in particular, assistance programs from Bank experience. Post-conflict situations where there is a con- are complicated and involve a multitude of siderable uncertainty, should include capacity players and organizations. Field presence is assistance programs building and not be essential to monitor Bank programs and should not be overly overly ambitious. projects, to maintain coordination with other ambitious and their donors, and to respond quickly and flexibly to objectives should include capacity building. changes. Bank field missions need to be The Rwanda CAE recommends simple project strengthened to meet post-conflict roles. An design and allocation of sufficient Bank expansion of field presence and the devolution resources to ensure close project supervision. of authority to managers in the field has proved The recommendation is based on the CAE to be a critical aspect of Bank programs in the finding that project lending regularly failed for West Bank and Gaza and in Bosnia and two reasons: overly ambitious design and Herzegovina, and should be a primary consid- inadequate supervision, given the country eration for other countries where local need. dynamics are similarly complicated and in flux. Effective Aid Coordination and Partnerships In contrast, the absence of a strong field with Other Donors presence reduced Bank effectiveness in a Effective aid coordina- number of reconstruction programs. The lack tion in a reconstruction In a reconstruction of Bank representation in Haiti reduced the program improves Bank program, effective aid ability of the Bank to work with key donors, responsiveness to the coordination and such as the Inter-American Development Bank realities of post-conflict (IDB) and the U.S. Agency for International situations. At the re- partnerships with other Development (USAID), which were more quest of international donors are vital. decentralized than the Bank, as well as with donors, the Bank has government and civil society. And in Rwanda, played a key role in coordinating international the Bank's working relationship was mixed in aid in the West Bank and Gaza, Bosnia and the aftermath of the genocide because a fully Herzegovina, and other transitional situations staffed Bank country office was not in place through its Holst Fund (US$270 million until two years later. pledged by 26 donors) and other mechanisms. In contrast, had the Bank played a more formal Adapting Bank Services and Products role in the negotiations leading up to the El to Post-Conflict Situations Salvador Peace Accords of January 1992, a The uncertainties inherent in post-conflict clearer, more realistic picture of the costs of its situations suggest that a different approach is provisions might have emerged earlier. Partner- required, as it is not always feasible for the Bank to ships with the U.N. and other donor agencies define an overall Country Assistance Strategy (CAS) need to be forged as soon as possible, because a priori. Instead, an opportunistic approach is they play a critical role in delivering aid and needed that builds on what is feasible. In countries services in post-conflict situations. However, 15 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? high-level agreement should be reached early crisis. Commitments as a ratio of the adminis- on the respective roles of the main players, and trative budget also show a sharp decline (figure especially on who will have the lead role in 2.6). The average size of Bank loans has each sector. declined since the mid-1990s from around $90 million in the mid-1990s to around $80 million Middle-Income Countries in the last two years.4 The scaling-up of The Bank has played a interventions is not occurring and internal The stagnation in IBRD useful role in MICs, but incentives--including the matrix structure-- lending to MICs and it has struggled to be have led to proliferation of activities. Recent increased costs of doing relevant in these OED evaluations of MICs show that the Bank countries. For a variety continues to have an important role in MICs, business with the Bank of reasons, the Bank but that it needs to focus on the following raise questions about its has seen its role in the issues. overall efficiency. MICs diminish in recent years, and as a result, Helping the MICs Through Their Remaining IBRD (International Bank for Reconstruction Development Challenges and Development) lending has stagnated. The The Bank has played a positive role in helping stagnation in IBRD lending to MICs and the MICs through crisis--both as a contra-cyclical increase in the costs of doing business with the financier and in helping countries introduce Bank have raised questions about its overall better regulatory and institutional structures efficiency. and mitigate the social costs of crisis. A more systematic and articulated role for the Bank in Disbursements per dollar of administrative these crisis situations would be useful. The budget have declined from $13 in fiscal 1995 to Bank played an important role in a number of $9 in fiscal 2005--except for a brief period in countries, such as Turkey and Tunisia, but fiscal 1998 and fiscal 1999 due to fast-disburs- appeared to lack coordination in some crisis ing loans to help countries deal with the Asian situations, as in Argentina and Indonesia. Many Figure 2.6: Bank Commitment and Disbursement as a Ratio to Total Budget Are Declining 25.0 20.0 Ratio to budget 15.0 10.0 5.0 0.0 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Fiscal year Disbursement Commitment Source: World Bank database. Datasets for administrative budget, all project commitments, and disbursements by fiscal year for active and inactive projects­July 2005. 16 THE BANK'S RECORD IN SUPPORTING DEVELOPMENT MICs continue to find Bank support on a range understanding of how Dealing with regional of development issues--such as the environ- they may work in a inequality and pockets of ment, pension reform, banking and energy particular context. With regulation, social assistance, and subnational MICs' ability to access poverty is a key agenda institution building--very useful. other funding sources, in MICs. countries that are not Dealing with Regional Inequality and Pockets convinced of the benefits of such Bank policies of Poverty can proceed with projects where these policies With a few exceptions,5 the MICs do not have have become an issue using other funds. As large-scale problems of extreme poverty. Instead, with all other policy they typically have pockets of poverty and identi- matters, countries must There is a strong demand fiable regions that are less developed and poorer be convinced of the in MICs for the Bank's than other areas of the country. Where the Bank utility of safeguards, not knowledge and ideas. has consciously focused on these issues, it has have them imposed by played a constructive role, as in Brazil. There is outsiders. greater demand from MICs for Bank support in their less-developed regions than in better-off MICs' Global Role areas. Ironically, the Bank's sharper poverty focus The MICs' increasingly important and expand- has made its role in some MICs less relevant, as ing role on a range of global development they do not identify themselves as poor issues makes them key countries and are therefore reluctant to engage players in both the The Bank should give with the Bank. achievement of global collective action and in more importance to the Transferring Knowledge and Ideas enhancing the links MICs' global role. Most MICs still regard the Bank as a key source between the Bank's of knowledge and ideas and as a neutral source country and global programs. The larger MICs, of policy advice. But several MICs have such as Brazil, China, and India, are increas- expressed a desire for the Bank to provide less ingly vocal regarding a wide range of global generic knowledge and more help in finding institutions. The Bank needs to develop a solutions to their specific problems--more strategy to ensure that the voice of these MICs custom and country fit. Many MICs with deeper influences Bank global programs and is local capacity would also like Bank teams to internalized within country programs. work more closely in partnership with local institutions. Working with MICs also helps the Global Programs Bank gain ideas, which can be transferred to low-income countries. Global Programs Are an Important and Growing Line of Business for the World Bank Costs of Doing Business with the Bank The Bank manages by far the largest stock of MIC lending has declined at the World Bank, trust funds among international organiza- although not at the regional multilateral tions--$7.1 billion at the end of fiscal 2004 (not development banks (MDBs). This has raised including Heavily Indebted Poor Country the issue of the increased costs--both financial [HIPC] and International Finance Corporation and nonfinancial--of doing business with the [IFC] trust funds). Sixty-four percent of this Bank. Enhanced safeguard and fiduciary sum was committed to global and Regional policies are meant to benefit countries that programs. OED has completed a series of borrow, but some of these policies may be evaluations of these programs over the past 4 perceived as an external imposition, or be years and has presented the following findings applied in a rigid manner, without a full to improve their effectiveness. 17 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? The Bank's strategy for The Bank's strategy Diseases [TDR], Joint United Nations Program for global programs is on HIV/AIDS [UNAIDS], Stop TB, Roll Back global programs is poorly poorly defined. The Malaria, Global Forum for Health Research, and defined, and criteria for Bank has lacked, but Global Alliance for Vaccines and Immunization Bank involvement are too clearly needs, a global [GAVI] Global Research Funding) largely meet strategy that is de- the four Development Committee criteria for general. veloped in conjunction selectivity. Most global programs also largely with its key partners and draws on the capacity meet the approval and eligibility criteria for that exists in its central vice presidencies, Bank involvement. CGIAR does not meet the network anchors, and Regions to do so. The arm's-length criterion; the Bank did not involve strategy needs to address the coherence, or developing country stakeholders in CEPF's lack thereof, between global expectations establishment or its global-level governance; (particularly in the donor community) and the and the Bank did not do a thorough analysis of needs of developing countries. At its center, the the expected level of Bank resources required global strategy needs a clear focus on sustain- for the health programs, or of how to able, poverty-reducing growth in the Bank's implement and manage this new commitment. client countries; on global policy issues that It is time to move from "letting a thousand prevent such growth; and on mobilizing flowers bloom" to assessing which programs incremental, unrestricted funding to address deserve continuing Bank support, and which global issues that are high-priority for develop- do not. ing countries. Such a strategy will not simply emerge from improved selectivity or oversight Their Strengths of individual global programs--it must be Only a few global programs deliver global worked out. Furthermore, strengthening public goods (table 2.3). Global public goods oversight in the absence of an overall strategy programs (CGIAR, TDR, MLF, parts of GEF, and risks micro-managing the global program even some new global-health programs) rate portfolio. well in their impacts on reducing poverty or on focusing on the policy, institutional, infrastruc- The current criteria for Bank involvement in a ture, or technological constraints faced by global program are too general. This is largely developing countries in achieving sustainable because the criteria are broad and difficult to economic growth. Adding value on the ground apply precisely. For example, the first criterion-- in client countries is typically a joint product of "an international consensus that global action is global and country-level activities. For example, required," which all programs claim as their CGIAR, like TDR, has demonstrated impressive reason for being--provides no basis for selectiv- poverty-reducing impacts, in part because the ity, because the concept Bank, donors, and some governments made Only a few global of international consen- complementary investments at the country programs deliver global sus is amorphous and level. However, as country-level investments loosely applied. have shrunk, donors have tried to compensate public goods. by encouraging CGIAR to move downstream. While largely supply- They have offered restricted funding tied to driven, most Bank-supported global public research programs that demonstrate immedi- goods programs (Multilateral Fund for the ate impacts to push CGIAR toward more Implementation of the Montreal Protocol national- and local-level applied and adaptive [MLF], Global Environment Facility [GEF], work, and away from its core competency of ProCarbFund, Critical Ecosystems Partnership research. Fund [CEPF], Consultative Group on Interna- Global programs have revealed major gaps tional Agriculture Research [CGIAR], Special in investment. Evidence indicates that invest- Program for Research and Training in Tropical ments in health research have substantial 18 THE BANK'S RECORD IN SUPPORTING DEVELOPMENT Table 2.3: Only a Few Global Programs Address Global Public Goods Fostering Financing Financing Financing country-level country-level Strengthening Number global country-level approaches, technical country-level of investments investments standards assistancea capacity programs Delivering CGIAR (1972), GEF (1991), UNAIDS (1996), 0 global public TDR (1975) MLF (1991), RBM (1998), goods ProCarbFund (2000), Global Forum (1997), CEPF (2000) Stop TB (1999)c Delivering PostConFund (1998), CGAP (1995), WSP (1978), InfoDev (1995), 16 national GAVI (1999)b GIF (1996), ESMAP (1982), World Links (1998), public goods GWP (1997), IF (1997), GDN (1999), UCW (2000) PPIAF 1999), FSAP (1999), Cities Alliance (1999) FIRST (2002) Number of programs 2 6 8 5 5 26 Source: OED Global Programs Evaluation (OED 2004c). Note: Each program is classified by OED according to only one category, corresponding to its primary activity. Programs are listed chronologically by start date within each category. a. .With the intent of stimulating public or private investments in the sector. b. .The GAVI Vaccine Fund also finances research and development of new vaccines and promotes strategies to address the constraints to R&D investment. c.. Stop TB also has a small drug facility that is financing country-level investments in the form of drugs for the treatment of tuberculosis. poverty-reducing impacts. The current global Their Weaknesses Global programs have policy and aid environment has huge invest- The voices of developing ment gaps at the global level in the provision of countries, as well as those revealed major gaps in global health research, as well as gaps in of the Bank's operational investment and global complementary investments at the country Regions, are inadequately public policy. level. Health research, like agricultural represented in the inter- research, is a long-term activity that is unlikely national consensus. The case studies of corporate to be addressed by the private sector on the advocacy programs show that including develop- scale needed. ing country voices at the concept stage enhances program ownership, makes the organizational Global programs have also revealed gaps in design more effective, and increases program global public policy. Several global programs impacts. Based on the evidence OED has highlight global public-policy gaps--often provided so far, management has acknowledged involving industrial-country policies in trade, the need to strengthen the role of developing aid, finance, and intellectual-property rights-- countries and the Bank's operational Regions in that affect developing countries. Few global programs. programs regard it as within their mandate to The voices of both address these policy gaps. If changing the Governance is weak international ground rules is the objective of in several programs. developing countries and the programs, and if advocacy is the means to While pure shareholder the Bank's operational achieve it, then the programs should be models of program Regions are inadequately assessed on their ability to deliver changed governance are being policies or a changed global environment from replaced by stakeholder represented in the the perspective of the poor. models, programs are international consensus. 19 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? still struggling to balance partnerships for drug and vaccine develop- Governance is weak in legitimacy and accounta- ment. Given the opportunity cost of ODA several programs. bility for results with funds, the Bank's involvement in programs efficiency in achieving with important goals, but little demonstrated them. The permanent members of the value, should be reconsidered. In some cases, programs' governing bodies, who tend typically too close an association with the Bank has to be the major international organizations and hampered mobilization of other funds for these donors, have greater de facto responsibility, programs. relative to the rotating members, to ensure that programs are successful. But such responsibility The Bank deploys its comparative advantages and accountability are rarely clearly articulated. more at the global level than at the country level. Lack of effective governance and management The international consensus on the existence of constitutes a risk for the Bank. a global problem is usually strong; consensus on what collective action is required is often weak. Global programs Many global programs are implicitly (sometimes Global programs have have increased overall explicitly) established to promote consensus, to increased overall aid very aid very little. At the "harmonize" donor approaches to specific little. aggregate level, the problems, to delineate donor comparative global programs re- advantages in addressing those problems, and viewed have added little new money to official to give the donors specialized knowledge they development assistance. Exceptions include can use to deal with the problems. Capacity funds from private sources for the Prototype building in the recipient countries is secondary Carbon Fund, from the Gates Foundation for in such projects. There is very weak connection health, and small amounts from pharmaceuti- in strategy or implementation between the cal companies through new public-private Bank's global and country programs. 20 3 Crosscutting Lessons from Sector and Thematic Evaluations N ine crosscutting lessons emerge from recent evaluations--including the sector and thematic studies--that suggest directions to further im- prove the Bank's development effectiveness. Specific sectoral lessons are also available, but are not highlighted here. 1. Understand and analyze comprehensively, 2. Emphasize institutional development but act more selectively and capacity building even more strongly, but sequencing and prioritization are · A comprehensive reform strategy is needed necessary. in various sectors to produce results in a spe- cific area. In real life, no developing coun- · Institutional quality is especially important try has the capacity to implement all the for public sector reforms and private sec- needed reforms at the same time. tor development. Major institutional aims · The weak capacity of central and regional include the creation of competitive sys- agencies makes it difficult to implement tems, appropriate incentives for key stake- government-wide results-orientation. holders, appropriate regulatory and · The Bank and clients need to be more se- supervisory frameworks, transparency, and lective about the poverty indicators they accountability. choose to monitor under the PRSP. · The record of attempts to unbundle and · Carefully define governance requirements privatize infrastructure has been poor (es- for lending in countries with poor gover- pecially in electricity and water), unless it nance. Be selective. was accompanied by strong institutional · Global programs have weak selection cri- and regulatory frameworks. Much the same teria. Several programs do not address a is true of financial global public-good need, and poverty erad- sector reforms: Selectivity is vital. No ication, the Bank's main mission, does not strong prudential developing country has factor into program selection. Some, such regulations and su- as CGIAR, do meet these criteria, but many pervisory systems the capacity to implement new ones do not. are required for all the needed reforms. 21 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? The Bank needs to do success. The record of capacity building work comparable to that privatizing poorly run of its other main work. more careful work on the government corpora- · Strong institutions are clearly important link between institutions tions has been very for rapid growth, but high growth has and growth and poverty mixed, and the record of sometimes been experienced in countries voucher privatization has with weak institutions. The Bank needs to reduction. been the worst of all. do more careful work on the link between Rushing to privatize institutions and growth and poverty re- without adequate regulatory, prudential, duction. and incentive systems is a recipe for failure · It is time to start assessing the impact of the and for serious political and social conse- Bank's work on governance and to pro- quences. vide more specific solutions. · In social sectors such as education and health, the required reforms are not priva- 3. Tailor programs to the circumstances of tization, so much as better service delivery. each country and adapt strategies to the This, in turn, requires institutional changes local political economy. to empower users of services, and thus change the incentives for service providers. · While the Bank produces many toolkits The institutional changes and papers disseminating best practices The Bank needs an need to be tailored to in- in various sectors, experience shows that dividual country settings. what may be best practice in one setting inventory of good · Some promising ap- may fare poorly in another. The Bank needs practices, but these proaches are threatened an inventory of good practices, but these should be adapted to the by unsustainability aris- should be adapted to the local context. local context. ing from an absence of Pilot projects in different regions of a coun- strong institutions when try can test the waters and yield lessons for a project ends. Social funds, environmen- modifications before scaling up. tal management, forestry, and natural dis- · Privatization is a good example of an area aster management are examples. Lack of where processes and procedures were sustainability is an issue in CDD, urban re- often not adapted to the local context. It forms, and service delivery in general. succeeded where political commitment · Decentralization can create permanent and institutions were strong, but not else- local government institutions that are close where. Evaluations reveal a lack of cus- to the grassroots and can improve ac- tom-fitting in public sector reform, private countability and service delivery. However, sector development, infrastructure, social political decentralization (creation of local sectors, and fragile states. councils) needs to be accompanied by ad- · PRSPs are supposed to be designed by ministrative decentralization (right of local countries, to ensure strong ownership. Yet councils to hire and discipline staff) and fis- Washington has to sign off on these, which cal decentralization (giving local govern- means clients tend to see them as ways of ments tax powers and a share of central accessing cash. This conflicts with the goal government revenues through a revenue- of ownership. sharing formula). This · Lending to governments is inescapably po- Custom-fitting to the will give decentralized litical. The best technocratic designs can political economy is as institutions the where- wither in infertile political soil. OED eval- important as custom- withal to deliver results. uations show that the Bank needs to pay · The Bank has not yet much more attention to the political econ- fitting to the real established a knowledge omy of countries, assessing the political economy. base and guidance for risks, costs, and benefits as well as the eco- 22 C R O S S C U T T I N G L E S S O N S F R O M S E C T O R A N D T H E M AT I C E VA L U AT I O N S nomic implications. Custom-fitting to the other fragile states. Consensus building can political economy is as important as cus- Over-optimism also be gradual, but creates a tom-fitting to the real economy. characterizes ef- · Persuade rather than prescribe. Country forts to rehabilitate better basis for reform ownership of programs is key for success, persons displaced then radicalism imposed and conditionalities are no substitute. Yet by projects. from outside. Bank staff still harbor--and Bank documents · The Bank needs to still include--the assumption that the Bank go back to the drawing board to deal with has high leverage in aid-dependent coun- some of these issues. More research and tries. This is an illusion: leverage produces pilot projects may throw light on alterna- lip service, not country ownership. tive approaches. · Local political champions are essential for · Learning and Inno- The record shows pushing through reforms. The Bank needs vation Loans (LILs) persistent over-optimism to piggyback on the efforts of local cham- and Adaptable Pro- pions rather than press for change through gram Loans (APLs) on privatization, debt conditionalities. But local champions are designed to ac- sustainability, and not enough: their work needs to be but- commodate exper- growth. tressed by new institutions that change imentation and the balance of power in the political econ- innovation as implementation proceeds omy (such as elected local governments in or to test new ideas have proven much place of a monolithic central government). less successful than standard projects. · Where there are no local champions, and · Provide support in turnaround situations the political economy resists reform, the or in countries where the Bank engages Bank should reduce lending, and instead after a long absence in a carefully selected engage with a wide variety of stakeholders manner. Avoid rushing in with large vol- to create a consensus for reform. This can umes of lending. take time, effort, and patience. · Consensus building for reforms within a 5. Lend mainly to countries with improving country can be slow and gradual, but cre- policies and institutions, but find ways to ates a better basis for reform than radical- deal with poor, misgoverned states. ism imposed from outside. Often gradualists have achieved better results · Bank research shows that aid works well in than radical reformers. Experience shows countries with good policies and institu- that the timing, sequencing, and selection tions, and tends to fail elsewhere. With of reforms are crucial, and these may be this understanding, clear performance cri- better determined by a country's political teria for lending has become an important dynamics than by outside advice. principle: the Bank seeks to lend more to countries with a higher and improving 4. Rethink areas of punctured optimism. CPIA ratings. · Lending in post-conflict situations has no · The 10-year record shows progress in many clear and systematic criteria and is not areas, but also repeated setbacks arising linked to poverty. The poor countries in from persistent over-optimism. East Africa received much less Bank assis- · Privatization and structural adjustment have tance per capita than less-poor countries been plagued by over-optimism about po- in Europe and Central Asia or the Middle litical commitment and institutional capacity. East and North Africa. · The record shows persistent over-opti- · Several evaluations highlight the risks and mism about debt sustainability, export moral hazard of lending to fragile states growth, and GNP growth in HIPCs and with poor governance. Nonlending strate- 23 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? gies constitute an alter- with knowledge of international and Safeguards and fiduciary native approach. These cross-country experiences. requirements are applied include early and con- · The costs of doing business with the Bank are mechanically. tinued engagement, high and its procedures are very bureau- building a consensus for cratic. More intelligent application of safe- reform among a wide range of stakehold- guards and fiduciary requirements is needed. ers, creating enclaves of improved gover- nance and development through social 7. Make better use of the Bank's knowledge funds that bypass the normal state struc- and technical assistance. ture, adapting procedures and processes for the difficult conditions of fragile states, · The Bank has the largest operational ex- and improving aid harmonization. perience and research output among de- · But nonlending strategies do not deal with velopment agencies. Since the mid-1990s, the moral dilemma of withdrawing assis- it has sought to evolve into a "knowledge tance from desperately poor, misgoverned bank." OED evaluations suggest that the countries. One proposed solution is to shift Bank's knowledge initiative was timely and from soft loans to outright grants for weak appropriate, but was not linked tightly to states. Such a shift, which may blur the line core lending and nonlending processes. between humanitarian and development This has limited its impact. assistance, is proposed under IDA 14, but · The Bank must establish itself as a preem- will be evaluated after some time has inent center of excellence in key policy elapsed. and institutional areas. It needs to be rec- · Standards for lending to HIPCs have been ognized as an authority on good practices, diluted. Many HIPCs have yet to demon- their adaptation to country circumstances, strate the ability to put in place frame- and ways to replace templates with a menu works for reform, which is raising concerns of options bearing different risks. about outcomes and track records. This · Bank staff have a wide array of specialist risks a race to the bottom. skills, and a huge reservoir of intimate knowledge of the political economy in dif- 6. Introduce greater flexibility and innova- ferent countries. Yet this is not used widely tion in its dealing with MICs. enough for custom-fitting. · Such intimate knowledge should be · The Bank needs to provide less interna- brought more openly into Country Assis- tional best practice and more custom-fit tance Strategies. A challenge is to share policy advice. this knowledge of political economies ef- · The Bank needs to develop new and in- fectively among all Bank staff and with novative instruments for subnational other donors. financing. · OED evaluations highlight the need for · Old-style Bank lending does not mesh with the Bank to research and build knowledge fiscal constraints in the country, or even its on how to deal with fragile states, where institutional framework. traditional approaches have failed widely. Bank knowledge and · We have a very good · Evaluations suggest that Bank research lending activities are not knowledge of local needs to establish more clearly the links be- well aligned, and few conditions and the tween institutions and growth. It also needs efforts are made to track local market, but the to improve our understanding of when great asset of the and why reforms occur, and why the tim- the Bank's impact on Bank is having people ing and sequencing varies so much in dif- poverty. from different places, ferent countries. 24 C R O S S C U T T I N G L E S S O N S F R O M S E C T O R A N D T H E M AT I C E VA L U AT I O N S 8. Improve monitoring and evaluation not well integrated into normal govern- (M&E). ment operations. · A persistent theme running through several 9. Improve coordination within and across sectors is the inadequacy of M&E. While the Bank Group. strong M&E adds to costs, it is essential to throw light on the impact of projects and · Links between the There are enormous programs and to suggest remedial steps. global and country · Evaluations in the health and education programs of the opportunities to further sectors note that M&E is so weak that lit- Bank are very improve coordination tle is known of outcomes. In consequence, weak. within and across the Bank staff tend to measure performance · Coordination be- by inputs rather than by outputs or out- tween the Bank, Bank group. comes. the International · In the urban sector, few efforts have been Finance Corpora- made to track the Bank's impact on tion (IFC), and the Multilateral Investment poverty, so successes here have not re- Guarantee Agency (MIGA) on investment ceived the publicity they deserve. climate issues needs improvement. · At completion, almost 40 percent of lines · There are unexploited synergies between of credit had no information on repayment the agendas for private sector develop- rates of Bank-funded subloans. ment and public sector reform. · Environmental projects require Geo- · Pension reform requires close coordina- graphical Information Systems to track tion with financial sector work, but pension ecological trends. experts lack access to information. · Where pilots constitute the first stage of · Closer coordination is needed between projects, participatory M&E by stakehold- the Poverty Reduction and Economic Man- ers can provide valuable feedback that agement Network (PREM) and the Private greatly improves project design. Sector Development Department (PSDD) · Data on Millennium Development Goals on investment climate issues. (MDGs) is not available in a large number · Both the Bank and other donors tend to be of countries. organized by sector, fostering institutional · Despite some advances, M&E activities are "silos" that inhibit cross-sector dialogue still mainly donor-driven and -funded, and and planning. 25 4 Looking Forward T he fight against poverty has not yet turned the corner, despite all the effort. The past decade saw some progress on poverty eradication--but at a pace too slow to make a significant dent in global poverty. Better progress was made in Asia, which has the largest share of the world's poor, but significant increases in poverty in other parts of the world, especially in Sub-Saharan Africa, remain very worrisome. Poverty has also risen in Europe and Central Asia, objectives such as growth, poverty reduction, although this may be a more transitory phenom- and environmental sustainability are better enon, especially in Eastern Europe. Latin America measured--remain unsuccessful. Its global has also not seen great progress on growth and programs, when focused on global public poverty reduction and is shifting away from the goods such as agricultural research, show liberal market model it embraced enthusiastically positive results, but others are less useful. The two decades ago. A renewed emphasis on growth links between global and country programs is vital for poverty reduction. also remain weak. The Bank is still struggling A renewed emphasis on The World Bank has been dramatically to find a viable model growth is vital for poverty altered over the past decade, with the objective for working with poor, reduction. of making it more effective in supporting the misgoverned states, global fight to eradicate poverty. It has and its rising costs of doing business have revamped its over-arching mission and reduced its attractiveness to the middle- reorganized internally to scale up the focus of income countries, and this has led to a its activities from individual projects to country shrunken role for the Bank in global develop- and global programs. Ironically, during this ment finance. period the success rate of its projects has improved, but its country programs still show This review of the Bank's development considerable scope for improvement. effectiveness points to a number of opportuni- ties for further improvements. Some of these will Almost a third of the Bank's country require changes in business processes, such as programs--where overall impact on broader new ways of dealing with LICUS and MICs, and 27 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Greater selectivity, more further improvements networks toward the two pillars, the investment with PRSPs, as well as climate and social inclusion; and a review of the flexibility, and better less mechanistic applica- matrix structure, which leads to multiplication of efficiency--a more hard- tion of safeguards. Some tasks and works against selectivity. Still others will nosed focus on concrete will require changes in require strategic choices for the institution, such the organization, such as as how to define its role more selectively, rather results--are needed. the need to find and put than trying to cover every aspect of develop- in place organizational ment. The Bank has transformed itself signifi- structures to improve coordination within the cantly in the past 10 years, but in the current Bank and across the Bank Group to set the right climate of rapid change, it should be ready for incentives; a consolidation of the Bank's further adjustments. 28 ANNEXES ANNEX A: MANAGEMENT COMMENTS Introduction Bank's record in working to improve living con- Management welcomes this opportunity to com- ditions in borrower countries has been much ment on the recent synthesis of evaluation find- solider than OED reports, and it cites evidence in ings by the Operations Evaluation Department this note. (OED). The report raises challenging but well- known questions of both organizational and de- Coverage velopment effectiveness. Perhaps inevitably, given This note is organized around the three the nature of the review in trying to highlight questions OED's synthesis covers: areas for further work, it does not document the major achievements in recent years in terms of de- · What is the measurable progress in improved velopment in general or of the Bank's development living conditions in the developing world over support. It does not report on the encouraging ev- the last 10­15 years, and how does it compare idence of strong overall developing country growth with the previous decade? and poverty reduction, nor does it highlight the · How effective has the World Bank been in centrality of the country and country ownership helping countries improve their living condi- to recent successful growth and poverty reduction, tions through its various interventions and the Bank's leadership in helping to align external programs? assistance around country-owned programs, or · Finally, what broad lessons emerge from OED the Bank's success in channeling funds to well-per- evaluations on improving the Bank's devel- forming countries that can effectively use the re- opment effectiveness? sources provided. It downplays the substantial improvement in the development outcomes of Progress in Living Conditions Bank-supported operations and better manage- OED finds that "the pace of change in the over- ment of Bank program delivery, both of which all performance of the developing world has not management saw as necessary before moving, as altered markedly over the past 20 years." By it has, to further scale up the impact of the Bank's painting the picture in this way, OED opens the support and strengthen its results focus. For ex- door for critics to use this review to undermine ample, the report fails to mention that while in support for development assistance at a time 1995, one out of three Bank-funded operations when every dollar of development assistance is yielded unsatisfactory results, today, that ratio is very much needed--and is more likely than any down to one out of five.1 On a portfolio in which time in recent history to be used effectively. As projects costing some $40 billion are completed OED's Annual Review of Development Effec- each year, this means that over the last 10 years tiveness (ARDE)2 recognized last year, there is a about $5 billion of projects per year have moved real trend in better policy performance, and that from unsatisfactory to satisfactory status and are performance is reflected in better growth out- delivering on results. Similarly, the OED report comes in developing countries. On average since does not mention the Bank's increasing lending 1990, and especially in recent years, developing commitments and deeper engagement with the countries have grown faster in per capita terms neediest countries. Management considers that the than high-income countries. Over the past four 31 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Figure A.1: GDP per Capita Growth (average annual, %) 3.50 3.2 3.1 3.00 2.9 2.6 2.6 2.6 2.50 2.4 2.0 Percent 2.00 1.7 1.5 1.4 1.50 1.2 1.2 0.9 1.00 0.50 0.3 0.00 Low income Middle income High income 1980­85 1985­90 1990­95 1995­2000 2000­03 Source: Development Economics Department. Table A.1: Direction of per Capita Growth Rates Number of countries Share of people (percent) Direction 1980s 2002­03 1980s 2002­03 Positive 58 119 78 90 Negative 53 22 22 10 Source: Development Economics Department. years, high-income countries' income per capita about 2/3 of developing countries showed has grown at an average rate of 0.9 percent per positive per capita income growth, and this year, that of middle-income countries at 2.6 per- percentage remains unchanged." It is true that cent, and that of low-income countries at 3.2 since 1980, the median proportion of low- and percent (see figure A.1). This means that in- middle-income countries with positive annual creasing numbers and proportions of people rates of growth in GDP per capita has been 64 are living in countries with positive growth rates percent. But that proportion has risen from a (see table A.1). In Sub-Saharan Africa, a Region median value of 59 percent between 1980 and of special focus for the development community, 1990 to 72 percent in the period 1991 through per capita income grew at an annual rate of 1.2 2003 (see figure A.3). percent between 2000­2003 (figure A.2), and 15 countries have had a real per capita growth Outlook rate above 2 percent per year since 1995. The overall outlook for a decline in income poverty remains promising for the next decade. Median Growth Rates If current projections for growth hold, world- OED's report also states that "in the 1980s, only wide, the poverty headcount index will fall from 32 ANNEX A: MANAGEMENT COMMENTS Figure A.2: Average Annual GDP per Capita Growth (percent), by Region 8.0 6.9 6.1 6.2 6.0 4.4 4.0 3.6 3.6 3.3 Percent 2.0 1.6 1.6 1.0 1.2 0.0 0.0 ­0.5 ­0.1 ­1.0 ­1.1 ­2.0 ­1.2 ­1.8 East Asia Europe & Latin America Middle East South Asia Sub-Saharan & Pacific Central Asia & Caribbean & North Africa Africa 1980­90 1990­2000 2000­03 Source: Development Economics Department. Figure A.3: Proportion of Countries with Positive Annual per Capita Income Growth Rates 100 75 Percent 50 25 0 1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 Low-income growth > 0 Middle-income growth > 0 High-income growth > 0 Source: Development Economics Department. 28 percent in 1990 to 10 percent in 2015, and the world's poor people, but where income growth number of people living on less than $1 a day will has accelerated since 1990 and has remained fall from 1.22 billion to 622 million. 3 These high. In Europe and Central Asia, where the achievements will largely reflect successes in poverty rate is relatively low, the sharp drop in China and India, which contain most of the incomes of the early 1990s has been reversed. 33 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? Social Indicators reduced the mortality rate of children under age OED notes that "broad social indicators, such as 5 (although it is clear that countries in Sub-Saha- life expectancy and literacy, show more wide- ran Africa in particular have a long way to go). And spread convergence with those of the developed as figure A.5 shows, middle-income countries and world, with the exception of Sub-Saharan Africa." the countries in the East Asia and Pacific and Latin Management would add two additional indica- America and Caribbean Regions are close to uni- tors--under-5 mortality and primary education versal primary education. Although South Asia completion--on which developing countries have and Sub-Saharan Africa have the lowest percent- also shown encouraging signs of convergence ages of school-age children completing primary with developed countries. As figure A.4 shows, education, they have made good progress in the since 1980 developing countries have consistently last decade to record levels. Figure A.4: Under-Five Mortality Rate in Developing Countries (deaths per thouand) 250 200 150 100 50 0 High income Low income Middle income EAP ECA LAC MNA SAR SSA 1980 1990 1995 2000 2003 Source: Development Economics Department. Figure A.5: Primary Completion Rate (percent), by Region 100 80 Percent 60 40 20 0 Low income Middle income EAP ECA LAC MNA SAR SSA 1990 2002 Source: Development Economics Department. 34 ANNEX A: MANAGEMENT COMMENTS Continuing Efforts development effectiveness. Management While management is pleased to point out that agrees that development effectiveness at the progress in growth and poverty reduction has country level could be further improved, and it been positive in recent years, it is not saying has taken a number of steps in that direction. that all is well. The entire development com- However, management would like to note munity is united behind the Millennium Devel- certain problems with the methodology OED opment Goals (MDGs). Achieving them will uses. First, OED split the ratings of some of the require that developing countries further im- countries it cites--notably Russia, where OED prove their policy performance, but it will also rated the outcome since 1999 as satisfactory, require more--and even more effective--de- while rating the 1992 to 1998 period as unsatis- velopment assistance. The Bank will continue factory. A more fundamental point is that, in striving to improve its effectiveness (the sub- spite of long discussions, management and ject of the next section); but OED should not ig- OED have never agreed on the country nore its own evidence of the Bank's effectiveness program rating methodology. A major reason in recent years, or the Bank's work to scale up for these disagreements is that, before results- the impact of its assistance by harmonizing its as- based Country Assistance Strategies (CASs) sistance with that of other donors and aligning were introduced, country programs were not that assistance with country goals; Bank leader- designed to be evaluated. OED has long rated ship on results at the country, Bank, and donor the achievement of objectives, but objectives community levels; and the work to modernize and strategies to achieve them change from and simplify Bank operational tools for greater one CAS to the next in response to changing effectiveness. country circumstances, and it has never been clear to management which objectives OED World Bank's Effectiveness in Helping rated.6 Management has also identified some Countries Improve Living Conditions inconsistencies in the rating system across Management notes that it is difficult for any de- Country Assistance Evaluations (CAEs). Having velopment institution to truly measure its impact expressed this concern, management would on improving living conditions, since many ele- note that we are now working constructively ments--some of which are not well under- with OED to establish a clear understanding stood--affect growth and poverty levels.4 As last on the basis for ratings in the context of year's ARDE pointed out, the Bank has done results-based CASs, and management expects well in directing its financial support to countries a higher level of convergence of views in the with better policies, which can best use these re- future. sources for growth and poverty reduction.5 At the same time, it has also stepped up its support to Caveats low-income countries under stress (LICUSs), Management would also like to point out three with the goal of helping them achieve consen- things a reader should bear in mind in inter- sus on policy direction and over time improve preting the country program ratings OED re- their policy and institutional performance and ports. gain greater access to financing from IDA and other sources. With regard to the Bank's effec- · OED's country program rating is not a rat- tiveness, management has some concerns about ing of a country's progress in living stan- both the methodology OED used to reach its dards. OED evaluates a country assistance conclusions, and the conclusions themselves. program on how well it meets its particular ob- jectives, which are typically a subset of the Country Program Ratings client's development objectives. Thus the out- OED finds that with one-third of country come of the Bank's assistance could be fully sat- programs rated unsatisfactory, there is isfactory, yet the country's economy could be substantial room for improving the Bank's deteriorating. 35 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? · OED's rating of a country program is not Centrality of Growth a rating of the Bank's performance. OED's In the mid-1990s, while the IMF worked with evaluation methodology distinguishes be- clients on the macroeconomic front, the Bank tween the performance of a country pro- took a leading role in helping structure the pri- gram and that of the Bank. Therefore, having vatization of utilities and promote private par- one-third unsatisfactory outcomes in coun- ticipation in other infrastructure as a way to try programs does not translate into an un- promote growth, given the hige infrastructure satisfactory Bank performance in one-third gaps and the shortage of public funds (notably of country programs. A country program development assistance). In the late 1990s, the rating in OED's methodology is determined Bank was clearer in emphasizing that while pri- by the joint impact of four agents: the client, vatization could contribute to growth in many the Bank, partners and other stakeholders, countries, it needed to respond to local condi- and exogenous factors. Thus Bank per- tions; the key was not necessarily privatization formance (which OED does not rate) can be but private sector development. The Bank also good even when OED rates outcome as un- emphasized that macroeconomic stabilization, satisfactory. while fundamental, could not in itself create · Divergence between country outcome rat- thriving economic growth. Entrepreneurs ings and outcome ratings of Bank projects needed to be free to set up new firms--which is not significant. OED finds that while proj- meant appropriate business regulation. Thus, ect outcome ratings are 75 percent satisfactory, the Bank broadened its assistance to helping outcome ratings are satisfactory in two-thirds countries level the playing field for responsible (67 percent) of country evaluations. Man- business development. The joint World Bank­IFC agement would like to note that OED has Doing Business project has produced objective rated country-level outcomes in four countries measures of the cost of common business pro- that had a heavy concentration of Bank op- cedures in 145 countries. Similarly, the online In- erations in FY04. Country-level outcomes vestment Climate Survey database provides a were rated as moderately satisfactory in India rich set of data based on responses from over and satisfactory/highly satisfactory in Brazil, 27,000 firms. Both the Doing Business Reports China, and Vietnam (figure 2.4 in OED's syn- and Investment Climate Surveys have helped thesis report). OED rated project outcomes the Bank Group to develop clear, locally tailored in these same countries as moderately satis- strategies for supporting countries in under- factory or above. taking country-owned reform (an OED recom- mendation) and have created powerful domestic Growth for Poverty Reduction constituencies for removing obstacles to private OED's analysis of growth in this report and the sector­led growth. The agenda-setting World findings of its ARDE 2004 suggest that the Bank Development Report 20058 pulled together the has given insufficient emphasis to growth for state of the art on investment climate reforms. poverty reduction.7 OED cites countries' poverty At the same time, the broader development reduction strategies (PRSs) as evidence. While it community and the Bank have all recognized is true that early PRSs in low-income countries that there cannot be a long-term growth strategy were less focused on broader growth issues than that does not rely on human development. While on social services (although, as OED empha- a growth-oriented strategy is critical for reduc- sizes, better social services do contribute to ing poverty, long-term growth will fail if human growth), more recently low-income countries development is ignored. have placed greater emphasis on sustainable growth in developing their PRSs. Management Cost of Doing the Bank's Business also wants to make the point that, while the OED recommends that the Bank focus on its Bank supports countries in producing PRSs, they cost of doing business, pointing out that for are not Bank documents. every dollar of its administrative budget, the 36 ANNEX A: MANAGEMENT COMMENTS Bank disbursed $13 in FY95 but only $9 in FY05. of lending to Bank budget as a meaningful indi- It is unusual, to say the least, for an evaluation cator, it should also take into account the costs unit to imply that the Bank should measure ef- of this involvement in LICUSs, the Bank's fectiveness in this way. It is clear that if lending stepped-up engagement in middle-income coun- were the only measure of Bank contributions to tries (MICs), and the Bank's expanding leader- development outcomes--without concern for ef- ship in global programs--all of which are ficiency and effectiveness or debt sustainabil- discussed in great detail below. ity--the Bank could easily expand lending. However, it has been management's view that the · Composition of Lending Commitments. An key focus of Bank work should be on develop- important point that OED does not note is ment impact, not lending. In view of this, heavy that IDA commitments have increased by 60 emphasis has been placed over the last five years percent between FY95 and FY04, growing at an on core analytic work and improving project average of 16 percent a year during the FY00- quality. Having established a solid base for in- 05 period. IDA countries are the poorest in the creased lending, the Bank has expanded its lend- world and are key to achieving the Millennium ing over the past three years: new commitments Development Goals. IDA resources have fi- in FY05 of $22.3 billion are 10 percent higher than nanced projects and programs that have helped new commitments in FY04, and the highest since build and sustain the policies, institutions, in- the post-financial crises period of FY00­01 (fig- frastructure, and human capital needed for ure A.6).9 Moreover, it has increased lending equitable and environmentally sustainable de- without sacrificing quality. There have been note- velopment in low-income countries. IDA allo- worthy increases in the quality and results of cations are performance-based. Better policy Bank products and services, and a significant performance of IDA countries--as documented increase in the Bank's work program, particularly by OED in last year's AROE--has permitted this in LICUSs, all against a backdrop of a flat budget increase in IDA commitments and disburse- in real terms during FY95­04, signaling rising pro- ments. Since IDA credits and grants are gen- ductivity. Before OED attempts to use the ratio erally smaller than IBRD loans and are more Figure A.6: IBRD/IDA Commitments and Disbursements ($m), FY95­05 30,000 25,000 20,000 15,000 10,000 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 IBRD/IDA disbursements IBRD/IDA commitments Source: SAP/World Bank data. Note: FY05 includes guarantees. 37 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? expensive for each dollar lent (because of many opment impact; it must be complemented by factors, including smaller average country size analytic and advisory services. Reinforcing this and greater capacity constraints), the increase point, Bank operations are now supported by in IDA commitments and disbursements has more, stronger, better-monitored, higher-qual- contributed to reducing commitments and ity economic and sector work (ESW) programs, disbursements per dollar of administrative which also contribute directly to supporting budget. country outcomes. ESW deliveries have been · Results of Bank Lending. Results have im- ramped up: 734 pieces of ESW were delivered proved a great deal over the past 10 years: in in FY04 compared to 368 in FY95 (figure A.7). FY95, 67 percent of projects exiting the port- By the end of FY04, most active IDA borrow- folio were rated satisfactory for development ers were covered with up-to-date core diag- outcomes, compared with 78 percent in FY04 nostic reports (Poverty Assessments, Country (table A.2). If, as is expected, this trend con- Economic Memoranda, Public Expenditure Re- tinues through the rest of FY05, the Bank will views, Country Procurement Assessment Re- reach its target of 80 percent satisfactory out- ports, and Country Financial Accountability comes for the first time in a decade. As men- Assessments). This increase in ESW deliveries tioned earlier, this means that each year did not come at the expense of quality; in fact, projects costing about $5 billion are delivering the quality of individual pieces of ESW im- substantial results just because of the attention proved substantially from 73 percent satisfac- to quality. Evaluated commitments in FY04 tory or better in FY98-99 to over 90 percent also reached a record high of 85 percent sat- today. The likely impact (on client policies and isfactory, meaning that almost $1 billion more institutional reforms, on Bank thinking, and on in commitments achieved their planned de- the wider development community) of ESW velopment outcomes in FY04. also went up from 66 percent in FY98­99 to 86 · Delivery and Results of Economic and Sector percent in FY02 (table A.2). OED's synthesis Work. OED itself has often made the point , reaffirms the importance of ESW particularly that lending alone is not sufficient for devel- in MICs: "Most MICs still regard the Bank as a Table A.2: Examples of the Changing Bank, FY96­FY05 Factor FY96 FY05 Country-owned PRSPs None 46 full/52 interim Country Assistance Strategies (CASs) 1 38 (FY04) ESW deliveries 336 734 (FY04) Encouraging a comprehensive development framework None In 50+ countries Country directors in the field None 73 percent Post-conflict lending and advice 15 countries 40 countries and territories Civil society involvement <50 percent 72 percent Debt relief operations None 27 countries (receiving debt relief), 18 countries (completion point) Project preparation time 24 months 14 months Quality at entry 78 percent 85 percent (FY03) Quality of supervision 63 percent (FY97) 90 percent (FY03-04) Quality of ESW 73 percent (FY98-99) 90+ percent (FY02) Satisfactory project outcomes 69 percent 78 percent (FY04) 38 ANNEX A: MANAGEMENT COMMENTS Figure A.7: ESW Deliveries, FY95­04 800 700 600 500 400 300 200 100 0 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03 FY04 Source: SAP/World Bank data. key source of knowledge and ideas and as a and monitor progress with regard to this im- neutral source of policy advice" (page 17). portant agenda.10 · Client Ownership. The Bank has increasingly emphasized building client ownership by sup- Approach to MIC Clients and LICUSs porting countries in their formulation of coun- The OED report identifies two areas in which it try development strategies. The cumulative charges that the Bank is "struggling with its number of country-owned PRSPs completed model for development effectiveness"--MICs has increased from zero in FY96 to 56 in FY05. and LICUSs. Management acknowledges that The number of CASs completed increased from these groups of countries do present special 1 in FY96 to 38 in FY04, and these CASs are now challenges, but would note it has worked set up to support country-owned goals. diligently to develop approaches that help the · Alignment, Harmonization, and Results. The Bank better meet the needs of these countries. Bank has taken a leading role in working with other donors to align donor support around MICs. Lending to MICs is up in FY05, and there country priorities, harmonize donor require- is a strong loan preparation pipeline. These ments to lower costs to client countries, and trends result in large part from a number of put the focus on results. While this work en- changes the Bank has made in its approach to tails up-front budgetary costs, it will pay growth MIC clients. and poverty reduction dividends as this agenda is taken to the country level and implemented · It is increasing its use of well-performing coun- widely. Executive directors have repeatedly try systems in special fiduciary areas (notably noted that they expect the Bank to play a lead- financial management). In September 2004, ing role in ensuring progress. The Bank is cur- the Board endorsed the idea of pilot-testing the rently working with other development use of country environmental and social safe- agencies to establish baseline data, set targets, guard systems, and it is now considering pro- 39 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? posals. Following a review of the experience in recognition of the importance of main- with implementing the pilot projects, pro- taining analytical and capacity-building work posals for wider adoption of the approach in these countries, even while lending is low. would be considered. While the pilot is not lim- Bank budget allocated for ESW to LICUSs in- ited to one set of countries, many MICs are ex- creased by 85 percent between FY02­04, al- pected to benefit from it. lowing the Bank to rebuild its knowledge · It is expanding the use of sectorwide ap- base in LICUS, increasing readiness for fuller proaches (SWAps), which until now have fo- engagement, and enhancing the effectiveness cused mainly on social sectors, to also include of development interventions in the more infrastructure. stable LICUSs. · It is offering MIC clients more choices, espe- · The Bank has taken a leadership role in bring- cially by promoting the use of a range of IBRD ing donors together around a common plat- financial products. form for support to these countries. At the · The Bank Group is working to better exploit Senior Level Forum for Development Effec- synergies in supporting development efforts in tiveness in Fragile States, held in London on MICs: a Bank Group framework has been de- January 13­14, 2005, participants agreed to veloped for lending to financial intermediates, draft and field-test a set of principles for sup- including municipal funds; and a working port, and the Bank has been instrumental in group has been established to outline a pro- producing these draft principles. They are now posal for a Bank Group facility for subsovereign available and are being applied in LICUSs, no- lending. tably in conflict-affected countries.12 In addition, · Recent CAS documents for MICs--notably El the Transitional Results Matrix,13 a planning Salvador, India, Kazakhstan, Mexico, and Slo- tool that the Bank helped develop and put vakia--have customized support to country into operation, integrates the political, eco- circumstances and facilitated a more flexible nomic, security, and social dimensions of re- and rapid response to lending demand.11 covery; allows for close donor coordination around government-agreed priorities; permits Approach to LICUS. Until recently, the Bank flexibility; and supports the monitoring of was able to do very little in countries troubled progress. by conflict or other serious problems, where policy performance was too poor to allow Global Programs significant lending. But the Board has repeat- The findings of OED's global program review edly stressed that all member countries underscore the Bank's important role in the deserve some level of support from the Bank; area of global activities--for example, thus Bank resources are now going to country HIV/AIDS programs, the HIPC Debt Initiative, teams for work in countries emerging from observance of standards and codes, and efforts conflict and in LICUSs. Several developments to fight money laundering and combat the illustrate the dimensions of this work. financing of terrorism--and the need for more systematic attention to the management of · According to the FY06 Bank budget document, global programs. Management took into budgetary costs to support 13 of these coun- account OED's recommendations in preparing tries/country groupings have risen from $12 a report on a strategic framework for global million in FY95 to $53 million in FY04, with ap- programs. Each vice presidential unit is now propriately limited flow-through to lending explicitly accountable for strengthening the commitments. links of global programs and partnerships · An important institutional change for engag- (GPPs) to Bank country- and Region-based ing more effectively with LICUSs has been development strategies; periodically reviewing delinking administrative budgets for ESW and and adjusting global programs to ensure that technical assistance from lending volumes, they focus on getting results; ensuring stable 40 ANNEX A: MANAGEMENT COMMENTS funding for key GPPs and planning exit strate- procedures to allow projects to be processed gies ahead of declines in funding; strengthen- more quickly, reducing average project process- ing the Bank's oversight of its GPPs to reduce ing time from 18 months to 14 months. reputational risks to the Bank; and ensuring a properly functioning control environment for OED Recommendations programs that have in-house secretariats or are OED's findings and evaluations at the project, funded by the Development Grant Facility. New sectoral and thematic, and country and global business processes will allow the Bank to track levels lead it to make the following nine its GPPs from their start through to evaluation recommendations to improve the Bank's and impact assessment. These areas build on development effectiveness: the actions management previously took in · Understand and analyze comprehensively, but response to OED's Phase I Report on Global act far more selectively. Programs: for example, establishing a GPP · Emphasize institutional development and Council as the senior management committee capacity building even more strongly. overseeing the strategic direction and · Rethink areas of punctured optimism, such as operational policies for Bank involvement in growth, private provision of infrastructure, GPPs,14 and establishing a new GPP Group led turn-around situations. by a director reporting to the Vice President, · Tailor programs and projects to the circum- Concessional Finance and Global Partnerships. stances of each country, and adapt strategies to the local political economy. Continuing Efforts · Lend mainly to countries with improving poli- While management believes the Bank has cies and institutions, but find ways to deal with been--and is increasingly--effective in helping poor misgoverned states. to improve the living standards of people in · Introduce greater flexibility into programs with client countries, it never stops working to well-managed middle-income countries. increase that effectiveness. It is currently · Make better use of the Bank's knowledge and working with the Board to reform the Bank's technical assistance. instruments, policies, and processes to be more · Improve monitoring and evaluation for results responsive to clients. Development policy and start measuring what is important. lending now emphasizes country ownership of · Improve coordination within the Bank and the reform program, embraces consultation and across the Bank Group. participation of stakeholders, and requires greater systematic attention to the possible Management Response effects of policy changes on poverty and the Management has little difficulty with any of environment. The Bank is also reforming its these recommendations. They all represent tools for investment lending to increase good advice. Moreover, they are areas in which development impact. It has changed some of its management has already done a great deal of policies to support the new ways of doing work--with evidence of results. For example, business: for example, it has updated the policy results-based CASs are designed to analyze on expenditure eligibility to make it easier for broadly but pick selectively a subset of country the Bank to finance a broad array of its expendi- outcome goals for support, emphasize tures that borrowers need to make to governance and capacity building, and tailor implement agreed operations; it has updated its programs to country conditions; and flexibility procurement policies and is working on its has been a key element of recent country disbursement policies; it has streamlined its programs in support of MICs. In addition, approach to project auditing; and it has clarified coordination across the World Bank Group is its policy on participation in pooled financing to improving, notably because of joint CASs, joint support SWAps. With regard to processes, the work around support for subnational entities, Bank has introduced changes in documents and and support for guarantees. Management fully 41 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? intends to continue in these directions. thematic areas, and a longer-term goal is the However, management does not entirely agree results focus of global programs. The results with one OED recommendation--rethinking focus is an integral contributor to achieving Bank support in turn-around situations. selectivity in country programs, helping Successful turn-arounds have extremely high countries build appropriate capacity, tailoring payoffs in terms of growth and poverty programs to each country's circumstances, reduction. If Bank support is crucial to a turn- providing flexibility to middle-income around, management believes that the Bank countries, using knowledge and technical should take the risk. The key is to be open and assistance better, and improving measuring, clear about the nature of the risk internally and monitoring, and evaluation. Management will in discussions with executive directors. report to executive directors later in FY06 on (Tanzania is a good example of a country where, progress on the enhanced results focus and its by taking a risk in the 1990s, the Bank goals going forward. contributed to a successful turn-around.) Lessons Learned Focus on Results However, the principal lesson to be drawn from In management's view, the key to addressing all the OED report is that the Bank's work has of these recommendations is the Bank's made a significant difference to developing enhanced focus on results, which builds on countries, and its efforts to further increase its earlier successes in turning around operational impact are on the right track. Management is quality and delivery. The results agenda started confident that, as it continues pursuing the with a focus on results-based CASs and is initiatives it has begun, the Bank will continue increasingly moving to operations and ESW; the to increase its effectiveness in helping emphasis this year is on key sectoral and countries reduce poverty and enhance growth. 42 ANNEX B: REPORT TO THE BOARD FROM THE COMMITTEE ON DEVELOPMENT EFFECTIVENESS (CODE) On May 16, 2005, the Committee on Develop- · Improve coordination within the Bank and ment Effectiveness met to discuss the OED re- across the Bank group. port Improving the World Bank's Development Effectiveness: What Does Evaluation Show? The This review of the Bank's development ef- Draft Management Comments to the OED report fectiveness points to a number of opportunities were prepared and circulated after the meeting. for further improvements that will require OED Report. The report provides a broad changes in the business process, changes in the overview of the development context in which organization, and strategic choices for the insti- the development effectiveness of the Bank is tution, and it highlights the decline in the ratio being assessed by providing a summary of the of lending to the Bank's administrative budget record in developing countries of poverty in recent years (with perhaps some reversal this reduction, growth, policy and institutional year), and the perceived high costs of doing measures, and security. OED evaluation business with the Bank. findings at different levels of aggregation at Management comments. Since this the project, sectoral and thematic, and paper is not a new evaluation and attempts to country and global levels are provided. The summarize lessons from recent evaluations, report suggests that the following nine management felt that it has already responded directions can improve the Bank's develop- to the OED findings and recommendations. ment effectiveness: Management recognized that more progress could have been made in dealing with poverty, · Understand and analyze comprehensive, but but thought that the tonality of the OED paper act far more selectively. might be too pessimistic. While agreeing on the · Emphasize institutional development and ca- importance of growth for development and pacity building even more strongly. poverty reduction, management commented · Re-think areas of punctured optimism, such as that it is remarkable that developing countries growth, private provision of infrastructure, have significantly surpassed the industrial turn-around situations. countries' growth ratio in the past 6 to 7 years. · Tailor programs and projects to the circum- Regarding the disconnect between project and stances of each country, and adapt strategies country outcomes, management noticed that to the local political economy. OED evaluated CASs that were not conceived · Lend mainly to countries with improving poli- as evaluable instruments, which will be the case cies and institutions, while finding ways to deal for the results-based CASs. with poorly governed states. Overall conclusions. Members wel- · Introduce greater flexibility into programs with comed the opportunity to discuss the paper, an well-managed MICs. earlier version of which had served as a · Make better use of the Bank's knowledge and platform for some reflection at the last CODE technical assistance. Retreat. They commended OED for preparing a · Improve monitoring and evaluation for results, useful synthesis focusing on a number of key and start measuring what is important. questions that emerged by distilling the lessons 43 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? from recent evaluations. One member it had in the World Bank. Management informed wondered how to operationalize these lessons. that there is this year an increase in lending to Speakers found the paper well-written and the MICs. Other members added that transitional candid, and praised its format and length, economies, LICUS, and small states also need the which could serve as a model for other CODE Bank's engagement, and felt that the Bank needs papers in the future. Given the extensive scope to improve the staff incentive structure and man- of the paper despite its brevity, the comments agement of risky projects. were also wide-ranging and covered substan- Cost of doing business. Some members tive findings, methodological issues, as well as perceived that higher non-financial costs of implications for both for Bank strategy and doing business were making the Bank less CODE's own work. A few issues which competitive for many borrowing countries. One resonated with several participants were the of them stressed the importance of consultation following: (i) decline in lending volumes even and participatory processes in Bank projects but as the administrative budget has continued to cautioned against too much deference to grow; (ii) non-financial costs of doing business nongovernmental organizations (NGOs) that with the Bank; (iii) disconnect between project may contribute to undermining the authority of performance and country outcomes; (iv) need democratically elected governments. for a systematic review of policy advice on Disconnect between project and design and sequencing of reforms; and (v) country outcomes. Several members noted simplification and modernization of lending. the disconnect in several countries between Overall, most speakers welcomed the emphasis aggregate project ratings and country program on growth, which they saw as a necessary outcomes. Some suggestions were made (i) to condition for poverty reduction. review the efficacy of the Bank's development Among the specific issues raised by the Com- model rather than just outcomes of individual mittee were: programs and projects; (ii) to focus on sustain- Portfolio size. Some members expressed ability of country outcomes, including financing concern about the decline in the Bank's of recurrent costs; (iii) to consider the impact lending volumes even as the administrative of external factors on programs and projects; budget has continued to grow, which one and (iv) for OED to address lessons learned also member suggested could be a sign of increased from unsuccessful outcomes of assistance bureaucracy. They agreed with OED that the strategies as indicated by CAE ratings (which do increased cost is due to the rising share of not report on Bank performance). A member knowledge activities and other non-lending suggested that a synthesis of the findings of activities such as global programs. each year's CAEs be added to the ARDE. OED A few members added that the rise in ad- responded that the CAE retrospective, which ministrative budget could be explained by du- looked at 25 CAEs from FY0l to FY04, collected plication of activities created by the Bank's many lessons learned from a diverse group of decentralization, high cost of staff travel, and cases such as conflict-affected countries, small insufficient outsourcing. Management informed states, transitional economies, and countries that intra-regional staff travel has increased affected by external shocks. with decentralization, while outsourcing-type Poverty reduction and development activities were already in place in the Chennai strategies. Members concurred that there is back-office. Some members encouraged the strong correlation between economic growth Bank to reverse the decline of lending to MICs, and reduction of poverty. One of them felt that and to ensure more efficient use of resources in the analysis of the Bank's development knowledge activities including delivering more effectiveness should factor in the Bank's weight "just-in-time" analytic and advisory assistance (e.g., share of external flows) in assisting a (AAA). A member noted that lending to MICs by client country relative to other players. Another regional development banks had not declined as member felt that in some cases the CAEs 44 ANNEX B: REPORT TO THE BOARD FROM THE COMMITTEE ON DEVELOPMENT EFFECTIVENESS showed positive outcomes, while the growth effectiveness is marginal. Management noted performance of the countries had not been that new initiatives such as use of country positive, which questions the development systems could contribute to reduction of costs. effectiveness of the assistance. OED noted that Country programs. Many members there is high correlation between positive agreed that the Bank's programs should be country growth performance and satisfactory tailored to the specific circumstances of the outcomes of Bank programs. OED clarified countries including their political economy. They that these outcomes were measured against highlighted the importance of assisting countries the objectives set in the country programs. to build capacity for economic planning and Many speakers felt that a systematic review was policy formulation, improve governance, needed of policy advice on the design and strengthen environmental and social institutions, sequencing of reforms, and the expected and align the results framework of the CAS and interaction between public and private sectors, lending operations. OED commented that some especially in areas where results are mixed such programs aligned to Washington Consensus­ as privatization, or fiscal decentralization. type of reforms had not had the desired effect, Simplification and modernization of perhaps because they had not considered lending. A member asked for more simplifica- sufficiently the political economy of the tion of lending and enhanced use of country countries. Management added that three main systems. It was also suggested that the Bank issues--governance, capacity, and the should be selective in its assistance and exit country`s results management framework-- from programs where the Bank's development are addressed at each new CAS preparation. Chander Mohan Vasudev, Chairman 45 ENDNOTES Executive Summary 3. Global Monitoring Report 2005--Millennium 1. The average size of investment loans dropped Development Goals: From Consensus to Momentum from US$80 million in the mid-1990's to about US$65 (Washington, D.C.: The World Bank, 2005). million in the last two years. 4. The Global Monitoring Report 2005 drew on the most recent literature on growth to conclude that there Chapter 1 is no minimum set of reforms required to spur growth, 1. The increase in poverty head-count probably or a larger set that is sufficient to sustain it. The rela- stretches back to the beginning of the 20th century. tionships among growth and policies, aid, shocks, the external environment, and other factors are complex. Chapter 2 5. 2003 Annual Review of Development Effec- 1. This discussion is based on the 2004 Annual Re- tiveness, op. cit. view of Development Effectiveness: The Bank's Con- 6. OED notes that before commencing work on a tribution to Poverty Reduction. CAE, an Approach Paper (AP) is prepared that clearly 2. The analysis excludes the global information and sets out the objectives against which the country as- communications technology, poverty reduction, and sistance program will be evaluated. Each AP is sent to social development because OED has evaluated very OPCS and the Country Department for comment few of their projects. prior to its circulation to CODE for approval. There- 3. OED has carried out two detailed impact eval- fore, it is unclear to OED what is meant by the state- uations in education in Ghana (OED 2004b) and ment that "it has never been clear to management health and nutrition in Bangladesh (OED 2005c) and which objectives OED rated." is conducting another on rural poverty in India. OED 7. See 2004 Annual Review of Development Ef- is also reviewing all the impact evaluations under way fectiveness: The Bank's Contributions to Poverty Re- at the Bank. duction (R2005-0084, IDA/R2005-0061), April 12, 2005, 4. The size of investment loans declined from available at http://www.worldbank.org/oed/arde/2004/. US$80 million to US$65 million over the same period. 8. World Development Report 2005: A Better In- 5. China being an obvious one. vestment Climate for Everyone (Washington, D.C.: World Bank and Oxford University Press, 2004). Annex A 9. The years fiscal 1998­99 were extraordinary 1. OED notes that the first subsection of Chapter years for the World Bank, given the East Asian Crisis 2 (page 7) is entitled "Project Performance Ratings and support to Argentina. Show Improvements over the Past 10 Years," and that 10. Paris Declaration on Aid Effectiveness: Own- this section shows clearly that project ratings have in- ership, Harmonisation, Alignment, Results and Mu- creased (see, for example, figure 2.2). tual Accountability (DC2005-0002), April 1, 2005. 2. 2003 Annual Review of Development Effec- 11. Enhancing World Bank Support to Middle tiveness--The Effectiveness of Bank Support for Pol- Income Countries--Management Action Plan: icy Reform (Washington, D.C.: The World Bank, Progress Memorandum (Washington, D.C.: The World 2004). Bank, 2005). 47 I M P R O V I N G T H E W O R L D B A N K ' S D E V E L O P M E N T E F F E C T I V E N E S S : W H AT D O E S E VA L U AT I O N S H O W ? 12. Principles for Good International Engagement 14. Update on Management of Global Programs in Fragile States (Paris: Organisation for Economic Co- and Partnerships (Washington, D.C.: The World Bank, operation and Development, 2005). 2003). Two managing directors chair the Council, and 13. Search "Transitional Results Matrix" at its members are vice-presidents from Networks, Re- www.worldbank.org. gions, and corporate areas. 48 REFERENCES OED (World Bank Operations Evaluation De- of the World Bank's Approach to Global Pro- partment). 2005a. 2004 Annual Review of grams. OED Study Series. Washington, D.C.: Development Effectiveness: The World Bank's World Bank. Contribution to Poverty Reduction. OED OECD (Organisation for Economic Co-opera- Study Series. Washington, D.C.: World Bank. tion and Development). 2005. "Principles for ------. 2005b. Country Assistance Evaluation Good International Engagement in Fragile Retrospective: An OED Self-Evaluation. OED States." Report DCD(2005)8/REV2. Paris. Study Series. Washington, D.C.: World Bank. OECD/DAC. 2005. "Paris Declaration on Aid Ef- ------. 2005c. Maintaining Momentum to 2015? fectiveness: Ownership, Harmonisation, Align- An Impact Evaluation of Interventions to ment, Results and Mutual Accountability." Improve Maternal and Child Health and Paris. Nutrition in Bangladesh. OED Study Series. World Bank. 2005a. Global Monitoring Report Washington, D.C.: World Bank. 2005--Millennium Development Goals: ------. 2004a. 2003 Annual Review of Devel- From Consensus to Momentum. Washing- opment Effectiveness: The Effectiveness of ton, D.C. Bank Support for Policy Reform. OED Study ------. 2005b. "Enhancing World Bank Support Series. Washington, D.C.: World Bank. to Middle Income Countries--Management ------. 2004b. Books, Buildings, and Learning Action Plan: Progress Memorandum." Report Outcomes: An Impact Evaluation of World Bank No. 31469. Washington, D.C. Support to Basic Education in Ghana. OED ------. 2004. World Development Report 2005: Study Series. Washington, D.C.: World Bank. A Better Investment Climate for Everyone. ------. 2004c. Addressing the Challenges of Washington, D.C.: World Bank and Oxford Globalization: An Independent Evaluation University Press. 49