WORLD BANK OPERATIONS EVALUATION DEPARTMENT 1ED 22027 March 2001 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESIS ~~~~~~~~~~~~~~,Jk~~~~~~~~~~~~~~~~~~ s; ,L, ik5.9$ t.b ......... . OPERATIONS EVALUATION DEPARTMENT ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. WORLD BANK OPERATIONS EVALUATION DEPARTMENT (ONEDE From Strategy to Results Timothy Johnston with William Battaile 2001 The World Bank http://www.worldbank.org/html/oed Washington, D.C.U Copyright C 2001 The International Bank for Reconstruction and Development/THE WORLD BANK 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. All rights reserved Manufactured in The United States of America First edition March 2001 The opinions expressed in this report do not necessarily represent the views of the World Bank or its member governments. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. The material in this publication is copyrighted. The World Bank encourages dissemination of its work and will normally grant permission promptly. Permission to photocopy items for internal or personal use, for the internal or personal use of specific clients, or for educational classroom use is granted by the World Bank, provided that the appropriate fee is paid directly to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, U.S.A., telephone 978-750- 8400, fax 978-750-4470. Please contact the Copyright Clearance Center before photocopying items. For permission to reprint individual articles or chapters, please fax your request with complete information to the Republication Department, Copyright Clearance Center, fax 978-750-4470. All other queries on rights and licenses should be addressed to the Office of the Publisher, World Bank, at the address above, or faxed to 202-522-2422. Cover photo: Pakistan school (Aga Khan Foundation). Photo by Curt Carnemark. ISSN 1019-4363 ISBN 0-8213-4929-5 7< Printed on Recycled Paper CONTENTS v Acknowledgments vii Foreword xi Executive Summary xxiv Abbreviations and Acronyms 1. From Strategy to Results 1 A Complex Operating Environment 1 The Bank's Response 2 Challenges and Tensions 3 The 2000 ARDE 5 2. Trends and Lessons in Project Performance 7 Trends in Outcome, Institutional Development, and Sustainability 10 Performance by Region, Sector, and Instrument 14 Risks and Rewards of Operations 14 Determinants of Project Success 19 3. Adapting to the Country Context 20 What Influences Country Program Outcomes? 24 Do Instruments Match Objectives? 26 How Are Resources Allocated among Countries? 28 Are Bank Country-Level Partnerships Strategic? 29 Different Roles for the Bank? 31 4. Toward Global Results: Sector and Thematic Dimensions 31 What Affects Sector Performance? 34 Challenges in Achieving Sectoral and Thematic Objectives 37 Design and Implementation of Sector Strategies 39 Implications for Sector Strategy Papers 43 5. Corporate and Policy Implications 44 Addressing Tensions and Challenges 46 Corporate Management Implications 53 Annexes 47 Annex A: Relevance and Outcome of Country Programs 49 Annex B: Recently Evaluated Sector and Thematic Strategies 51 Annex C: Statistical Annex 51 Table C. 1. Outcome, Sustainability, Institutional Development (ID) Impact, and Aggregate by Sector, Network, Lending Type/Source, Region, and WDI Income Group for Exit Fiscal Year Groups 1990-94, 1995-98, 1999-2000 (by projects) 52 Table C.2. Outcome, Sustainability, Institutional Development Impact, and Aggregate by Sector, Network, Lending Type/Source, Region, and WDI Income Group for Exit Fiscal Year Groups 1990-94, 1995-98, 1999-2000 (by disbursements, FY96 US$) iii 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS 53 Annex D: Managing Development Effectiveness: An Overview from the CODE Chairperson 56 Endnotes 59 Bibliography Table 20 Table 3.1. How Country Program Outcomes Stack Up Against Portfolio Performance Figures 6 Figure 2.1. Approval Years of Recently Evaluated Projects 8 Figure 2.2. Trends in Satisfactory Project Outcomes 9 Figure 2.3. Trends in Institutional Development Impact 10 Figure 2.4. Trends in Sustainability 11 Figure 2.5. Trends in the Aggregate Project Performance Index (APPI) 11 Figure 2.6. Regional Trends in Satisfactory Operations, by Projects 13 Figure 2.7. Sectoral Trends in Satisfactory Operations 15 Figure 2.8. Risks and Rewards, by Sector and Region, FY99-00 and Selected FY95-98 Averages 16 Figure 2.9. Institutional Quality, Policy, and Portfolio Performance 24 Figure 3.1. Country Performance Influences Instrument Performance 27 Figure 3.2. Per Capita IDA Lending by CPIA Ratings 27 Figure 3.3. Per Capita IBRD Lending by CPIA Ratings 32 Figure 4.1. Sector Project Outcomes Vary by Country Income 39 Figure 4.2. Sector and Thematic Strategy Ratings 48 Figure A. 1. Relevance of Bank Strategy and Effectiveness of Country Program Implementation Boxes 7 Box 2.1. How Project Results Are Measured 12 Box 2.2. Effects of Country Conditions on Project Performance in Russia 13 Box 2.3. Assessing Adjustment Lending: Timing Matters 22 Box 3.1. The Bank and the Client May Disagree 23 Box 3.2. Sources of Over-Optimism 26 Box 3.3. Instrument Mix and Building Consensus for Reform 29 Box 3.4. Moving to a New Level in Partnerships 35 Box 4.1. Stakeholder Perceptions of Bank Strengths and Weaknesses 40 Box 4.2. Safeguard Policies iv ACKNOWLEDGMENTS T his report was prepared by a team led by Quesada, Enrique Rueda-Sabater, Susan Stout, Timothy Johnston. William Battaile was Gene Tidrick, Joseph Wambia, and Irene S. responsible for Chapter 2 on project per- Xenakis. formance trends. Other team members included The Review was initiated under the guidance Mita Marra, Carlos Reyes, Kimberly Smith, and of Wendy Jarvie, former manager, OEDCM, and S. Ramachandran. The team received excellent completed under Victoria Elliott, the current support from Annisa Cline-Thomas, Parveen manager. Elizabeth Campbell-Page, Caroline Moses, and Julia Ooro. The report benefited McEuen, William Hurlbut, and Bruce Ross-Larson from extensive contributions from OED staff, and provided editorial support. drew on a wide range of recent OED country, The study was published in the Partnerships sector, and thematic evaluations. Background pa- and Knowledge Group (OEDPK) by the Out- pers and supplemental analyses were provided reach and Dissemination Unit. The task team in- by Manuel Hinds, Luis Landau, Federico Mini, cludes Elizabeth Campbell-Page (task team leader), and Kevin Morrison. Caroline McEuen (editor), and Juicy Qureishi- The comments and guidance from peer re- Huq (administrative assistant). viewers are gratefully acknowledged, including Aysegul Akin-Karasapan, Robert Buckley, Director-General, Operations Evaluation: RobertPicciotto Stephen Commins, Ishac Diwan, David Dollar, Director, Operations Evaluation Department: GregoryIngram Catherine Gwin, John Heilbrunn, Wendy Jarvie, Manager, Corporate Evaluation and Methods: Victoria Elliott Ashoka Mody, Linda Morra, Manuel Penalver- Task Manager: TimothyJohnston v FOREWORD PREFACIO AVANT- - E i PROPOS The World Bank is uniquely El Banco Mundial se encuen- La Banque est dans une po- placed to provide finance, knowledge, tra en situaci6n privilegiada para ofre- sition ideale pour fournir des services connectivity, and capacity building cer financiamiento, conocimientos, dofinancement, d'information, de con- services to its developing member posibilidades de conexi6n y servicios nectivite et de renforcement des ca- countries. As the premier multilateral para el desarrollo de la capacidad de pacites a ses pays membres en financial institution, it is mandated to sus paises miembros en desarrollo. developpement. En sa qualite d'insti- help its developing country members En cuanto primera instituci6n fi- tution financiere multilaterale de pre- hook into the global economy. As a nanciera multilateral, tiene como mier plan, elle a pour mission d'aider global development agency, its mis- mandato ayudar a sus paises miem- les pays en developpement a prendre sion is to reduce poverty through so- bros en desarrollo a integrarse en la pied dans l'economie mondiale. En sa cially and environmentallysustainable economia mundial. En cuanto organ- qualite d'organisme de developpe- growth. This twin mandate must be ismo mundial de desarrollo, su misi6n ment mondial, elle a pour mission de implementedinademandingoperating es reducir la pobreza mediante un reduire la pauvret6 par le biais environment characterized by finan- crecimiento socialmente y ecol6gi- d'une croissance socialement et cial instability, rapid technological camente sostenible. Ese doble ecologiquement durable. Elle doit change, and growing environmental mandato debe ponerse en practica en poursuivre ce double objectif dans un fragility. una coyuntura caracterizada por la in- cadre op6rationnel difficile, carac- The Anniual Review of Develop- estabilidad financiera, el rapido cam- terise par une situation financiere in- ment Effectiveness 2000 builds on bio tecnol6gico y una creciente stable, des progres technologiques those of past years. The 1998 Review, fragilidad ambiental. rapides et un environnement de plus released in a hostile environment of El Examen anual de la eficacia en plus fragile. financial crisis, concluded that im- en terminos de desarrollo, 2000 esta L'Examen annuel de l'efficacite provements in project performance basado en las ediciones de anos an- du developpement 2000 s'inscrit are not enough-there must also be teriores. En el Examen de 1998, pub- dans le prolongement des examens improvements at the higher plane of licado en el contexto hostil de la anterieurs. L'Examen de 1998, pub- program and country performance. crisis financiera, se llegaba la con- lie alors que le monde etait en proie The 1999 Review described the clusi6n de que las mejoras con- a la crise financiere, notait en con- dilemmas and implementation chal- seguidas en los resultados de los clusion qu'il ne suffirait pas lenges of implementing the Com- proyectos no son suficientes: hay d'am6liorer les resultats des projets prehensive Development Framework. que lograr tambien mejoras en el - it fallait viser plus haut, et It identified several promising plan mas elevado de los programas ameliorer les resultats des pro- practices for dealing with those y paises. En el Examen de 1999 se grammes et des pays. L'examen de challenges-all predicated on strong describian los dilemas y desafios 1999 decrivait les dilemmes et les country commitments to poverty re- que presentaba la aplicaci6n del problemes poses par la mise en duction and sustainable growth. Marco Integral de Desarrollo. Se ceuvre du Cadre de developpement This year's Review finds that identificaban varias practicas prom- integre. Il identifiait plusieurs meth- progress in 2000 was solid on a broad etedoras para superar esos desafios, odes prometteuses pour faire face a front, and that further progress is likely. todas ellas basadas en el firme com- ces problemes, qui necessitaient Portfolio performance is likely to ex- promiso de los paises con la reduc- toutes que les pays s'engagent fer- ceed the Strategic Compact target of 75 ci6n de la pobreza y el crecimiento mement a reduire la pauvrete et a as- percent satisfactory outcomes. Sus- sostenible. surer une croissance durable. vii 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS tainability and institutional En el Examen de este anio Cette annee, I'Examen fait development ratings have also se comprueba que los pro- etat des solides progres improved. Given the increased gresos conseguidos en el afno accomplis dans de nombreux complexity, riskiness, and - 2000 fueron s6lidos en un domaines et prevoit la pour- demandingness of Bank op- - amplio frente, y que es prob- - suite de cette evolution erations, especially in low- able que se realicen nuevos - positive. La performance du income countries, this progress avances. El desempenxo de portefeuille devrait &tre is commendable. la cartera superara, proba- superieure A l'objectif fixe The Review examines four ten- blemente, la meta del 75% de re- dans le cadre du Pacte strategique, sions the Bank currently faces. As it sultados satisfactorios establecida en soit l'obtention de resultats satis- seeks to implement an increasingly el Pacto Estrategico. Han mejorado faisants pour 75 % des projets. Les crowded agenda, the Bank must leam las calificaciones con respecto a la notes obtenues au plan de la dura- to reconcile client and corporate pri- sostenibilidad y el desarrollo insti- bilite et du developpeement institu- orities; to adapt global prescriptions tucional. Este progreso es especial- tionnel se sont egalement am6liorees. to local conditions; to balance coun- mente encomiable si se tienen en On ne peut que s'en feliciter, etant try performance and poverty inci- cuenta las mayores exigencias, donne la complexite, les risques et dence in allocating its resources; and riesgo y dificultad de las operaciones A la difficulte des operations de la to achieve efficiency and selectivity del Banco, sobre todo en los paises Banque, notamment dans les pays A while seeking to implement a long- en desarrollo. faible revenu. term, holistic vision of development. En el Examen se analizan cuatro L'Examen passe en revue quatre The Bank has also taken it upon it- tensiones existentes actualmente en domaines dans lesquels la Banque self to generate and disseminate de- el Banco. A medida que intenta doit actuellement proceder a des ar- velopment knowledge, engage its aplicar un programa cada vez mas bitrages. Alors meme qu'elle s'efforce partners in global policy debates, and ambicioso, el Banco debe aprender de poursuivre un programme de plus deal with global development chal- a compaginar las prioridades de los en plus volumineux, l'institution doit lenges that cannot be tackled effec- clientes y de la instituci6n, a adap- apprendre A r6concilier les priorites tively at the country level. tar las soluciones mundiales a las de ses clients et ses priorites institu- The Review concludes that the condiciones locales, a equilibrar el tionnelles ; adapter aux conditions lo- Bank can further improve its de- desempeno de los paises y la inci- cales des remedes formules au plan velopment effectiveness by directly dencia en la pobreza al asignar sus mondial ; allouer ses ressources en confronting the dilemmas it faces. recursos, y a garantizar la eficiencia fonction non seulement des resul- First, Bank strategies should ex- y la selectividad al mismo tiempo tats obtenus par les pays mais aussi plicitly acknowledge differences que intenta alcanzar una visi6n a de l'incidence la pauvrete ; et faire between client and Bank priorities, largo tiempo y global del desarrollo. preuve d'efficacite et de selectivite and Bank instruments should be El Banco ha asumido la tarea de tout en s'efforcant de promouvoir mobilized more strategically to generar y difundir los conocimien- une vision globale du developpe- build consensus at the country and tos relacionados con el desarrollo, ment A long terme. La Banque a global levels. Second, the Bank implicar a sus socios en los debates egalement entrepris de generer et de should concentrate on the effec- sobre politicas de alcance mundial diffuser des connaissances dans le do- tive implementation of its policies y hacer frente a los desafios del de- maine du developpement, de par- and strategies through judicious sarrollo que no se pueden resolver ticiper avec ses partenaires A des adaptation to diverse institutional, de manera eficaz con intervenciones debats d'orientation au plan mon- social, and political environments. limitadas unicamente a los paises. dial, et de relever les defis que pose Third, the Bank should clarify and En el Examen se llega a la con- le developpement et qui ne peuvent strengthen its approach to poor- clusi6n de que el Banco puede etre releves de maniere efficace au performing countries and to middle- mejorar todavia mas su eficacia en niveau national. income countries with access to terminos de desarrollo si afronta di- L'Examen conclut que la Banque capital markets. rectamente los dilemas que se le pourrait encore accroitre son im- viii FOREWORD The Bank will also have plantean. En primer lugar, las pact sur le developpement to adapt its organizational - estrategias del Banco de- en s'attaquant directement structure, business practices, berian reconocer de forma aux dilemmes auxquels elle and operational instruments expresa las diferencias entre se trouve confrontee. Pre- to reduce the enormous stress las prioridades de los clientes - mierement, ses strategies de- the organization is suffering y las del Banco, y los instru- vraient explicitement prendre from-and which, unless it mentos de este deberian en compte les diff6rences qui is remedied, could undermine movilizarse de manera mAs existent entre ses priorites et its development effectiveness. First, estrategica para conseguir el con- celles de ses clients, et ses instru- Bank strategies, programs, budgeting senso en los paises y en todo el ments devraient Wtre mobilises de processes, and staff incentives must mundo. En segundo lugar, el Banco maniere plus strategique pour forger be linked to the achievement of re- deberia concentrarse en la aplicaci6n des consensus au niveau national et sults, and budget processes reformed eficaz de sus politicas y estrategias mondial. Deuxiemement, la Banque to align corporate strategies with re- mediante la sabia aceptaci6n a los devrait essentiellement viser a ap- source allocations. Second, the Bank diferentes entornos institucionales, pliquer efficacement ses grands should give greater priority to knowl- sociales y politicos. En tercer lugar, principes et strategies en les adap- edge creation, at both the country el Banco deberia aclarar y consoli- tant de maniere judicieuse aux dif- and global levels, and establish dar su posici6n con respecto a los ferents cadres institutionnels, sociaux quality monitoring and tracking paises con resultados poco satisfac- et politiques. Troisiemement, la processes to strengthen the quality torios y a los paises de ingreso me- Banque devrait preciser et renforcer and coherence of its activities. Third, diano, que tienen acceso a los la demarche qu'elle entend pour- the Bank should continually assess mercados de capital. suivre dans le cas des pays obtenant its comparative advantage relative El Banco tendri que adaptar tam- de pietres resultats et des pays a to international and country part- bien se estructura organica, practicas revenu interm6diaire qui ont acces ners-and achieve responsible se- e instrumentos operacionales con el aux marches financiers. lectivity through businesslike fin de reducir la enorme presion a La Banque devra egalement partnerships. que se ve sometida la organizaci6n adapter sa structure organique, ses -y que, si no se pone remedio, po- procedures et ses instruments opera- dria echar por tierra su eficacia en ter- tionnels pour reduire les pressions minos de desarrollo. En primer lugar, considerables auxquelles elle est as- las estrategias, programas, procesos sujettie et qui, si rien n'est fait, pour- presupuestarios e incentivos de per- raient nuire a l'efficacite de son action sonal del Banco deberian estar vin- en faveur du developpement. Pre- culados al logro de resultados, y los mierement, les strategies, les pro- procesos presupuestarios deberian grammes, les processus budgetaires reformarse para que las estrategias in- et les incitations donnees au per- stitucionales vayan acompahadas de sonnel de la Banque doivent Wtre las correspondientes asignaciones fonction des resultats obtenus, et les de recursos. En segundo lugar, el processus budgetaires doivent etre Banco deberia otorgar mayor prior- revus de maniere a ce que les allo- idad a la creaci6n de conocimientos, cations de ressource refletent mieux tanto en los paises como en todo el les strategies institutionnelles. Deux- mundo, y establecer procesos de iemement, la Banque devrait ac- seguimiento y supervisi6n de la cal- corder une plus grande priorit6 a la idad con el fin de elevar esta y lo- creation de savoir, au niveau na- grar una mayor coherencia de sus tional comme au niveau mondial, et actividades. En tercer lugar, el Banco mettre en place des procedures de deberia evaluar ininterrumpidamente suivi et d'evaluation qualitative pour ix 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS su ventaja comparativa con renforcer la qualite et la co- - respecto a los asociados in- herence de ses activites. ternacionales y nacionales, y Troisiemement, la Banque de- lograr una selectividad re- vrait systematiquement eval- sponsable mediante asocia- - uer 1'avantage comparatif ciones eficaces. dont elle jouit par rapport a ses partenaires internationaux et nationaux, et se montrer judicieusement selective en consti- tuant des partenariats productifs. Robert Picciotto Director-General, Operations Evaluation x EXECUTIVE RESUMEN RESUME SUMMARY - ANALYFIQUE The World Bank's mandate has El mandato del Banco La mission de la Banque mon- expanded and deepened over the past Mundial se ha ampliado y profundizado diale s'est diversifiee en mime temps decade. Changes in the international Ie el ultimo decenio. Los cambios qu'elle s'est approfondie au cours des economy and the development system ocurridos en la economia internacional dix dernieres annees. La transforma- have challengedthe Bankto give more y en el sistema de desarrollo le han tion de l'economie internationale et emphasis to institutional development, obligado a hacer mayor hincapie en el du systeme de developpement a amene good governance, social development, desarrollo institucional, el buen gob- la Banque a mettre davantage l'accent sur le developpement institutionnel, la bonne gestion des affaires W 11 - s publiques, le developpement social, la reconstruction des pays a l'issue d'un conflit, I'allegement de la dette et A S | i . > > les biens publics mondiaux. De nou- veaux defis doivent donc etre releves pour assurer 1'efficacite du developpe- f- , -^ ment. . cette fin, l'institution a lance, il y a cinq ans, un processus de reno- vation pour se doter de nouvelles competences, renouveler les sys- .4. g f , temes et les processus, mettre en place de nouvelles modalites de ges- tion et reorienter ses valeurs et sa culture. Dans le meme temps, elle a recentr6 sa mission sur la reduction post-conflictreconstruction,debtrelief, ierno, el desarrollo social, la recon- de la pauvrete reaffecte des and global public goods. As a result, strucci6n despues de los conflictos, ressources budgetaires aux activites fresh development effectiveness chal- el alivio de la deuda y los bienes publi- de gestion du portefeuille; renforce lenges emerged. cos mundiales. En consecuencia, la le cadre des mesures de sauvegarde To meet them, the institution eficacia en las actividades del desar- environnementales, sociales et fidu- launched a renewal process five rollo se ha visto expuesta a nuevos ciaires ; et encourage la poursuite years ago to build new competen- desafios. d'un processus participatif dans le cies, update systems and processes, Para hacerles frente, la institu- cadre de l'execution des projets et establish new management arrange- ci6n puso en marcha hace cinco de la formulation des grandes ori- ments, and reorient its values and afios un proceso de renovaci6n para entations. culture. In parallel, the Bank conseguir nuevas competencias, ac- Si ces transformations ont per- refocused its mission toward poverty tualizar los sistemas y procesos, es- mis d'ameliorer l'impact de l'aide reduction; redeployed budget tablecer nuevos mecanismos de de la Banque sur le developpement, resources toward portfolio manage- gesti6n y revisar sus valores y cul- elles ont egalement accru la com- ment; strengthened its environmen- tura. Paralelamente, el Banco reori- plexit6 des operations de l'institution tal, social, and fiduciary safeguard ent6 su misi6n hacia la reducci6n de et le couit qu'elles representent pour xi 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS policy framework; and en- la pobreza; transfiri6 recursos les pays, et cree des situations couraged participation in presupuestarios hacia la de tension au niveau des project implementation and gesti6n de la cartera; con- structures, des processus et policy development. solid6 su marco de politicas des personnes. La Banque While these enhance- ambientales, sociales y de devra choisir entre divers ob- ments have increased the de- salvaguardia fiduciaria y - jectifs, et proceder a des ar- velopment rewards of Bank alent6 la participaci6n en la bitrages entre ses objectifs et assistance, they have also in- ejecuci6n de los proyectos y ceux de ses clients, la per- creased the complexity of Bank op- la formulaci6n de politicas. formance des pays et l'incidence la erations, raised the cost of doing Si bien estas mejoras han hecho pauvrete, et enfin les mesures pre- business with the Bank, and gener- posible que la asistencia del Banco scrites a l'echelle mondiale et leur ated stress on structures, processes, contribuya en mayor medida al de- adaptation aux conditions locales. and people. The Bank faces trade- sarrollo, han aumentado tambien la Cet Examen annuel de l'efficac- offs among diverse goals and must complejidad de sus operaciones, el- ite du developpement (ARDE 2000) reconcile tensions, including those evado el costo de las operaciones se base sur les conclusions des between corporate and client ob- con 6l y generado una mayor pre- dernieres evaluations du departe- jectives, country performance and si6n para las estructuras, procesos y ment de l'evaluation des operations poverty prevalence, and global pre- personas. El Banco tiene que tomar (OED) pour guider les arbitrages scriptions and local adaptation. soluciones de compromiso entre mentionnes plus hauts. Le rapport This Annual Review of Develop- metas distintas y reconciliar las ten- s'articule autour des trois grands 61e- ment Effectiveness (ARDE 2000) siones, en particular las existentes ments des evaluations de l'OED: les draws on the findings of the Oper- entre los objetivos de la instituci6n projets acheves par la Banque, ses ations Evaluation Department's y de los clientes, el desempenio de programmes d'aide aux pays, et ses (OED) most recent evaluative work los paises y la difusi6n de la pobreza, activites sectorielles et thematiques. to help guide management of the y las soluciones de alcance mundial II aboutit a plusieurs conclusions above tensions. The report is struc- y la adaptaci6n local. importantes et generales: tured around the three main objects Este Examen anual de la eficacia * Les reformes et les programmes of OED's evaluations: the Bank's en terminos de desarrollo esta de developpement ont plus de completed projects, its country as- basado en las conclusiones de los es- chance d'etre durables lorsqu'ils sistance programs, and its sector and tudios de evaluaci6n mas recientes font l'objet d'un consensus dans thematic activities. Key cross-cutting del Departamento de Evaluaci6n de le pays et lorsque la Banque et le messages are: Operaciones (DEO), y su objetivo es pays poursuivent les memes ob- * Reforms and development pro- ayudar a controlar esas tensiones. El jectifs. Au niveau national, la grams are more likely to be sus- informe esta estructurado en torno Banque doit utiliser ses instru- tained if there is consensus within a los tres objetivos principales de las ments de prets et hors prets de the country and a shared vision evaluaciones del DEO: proyectos maniere plus systematique pour between the Bank and the coun- terminados del Banco, sus progra- forger un consensus en accrois- try. At the country level, the Bank mas de asistencia a los paises y sus sant la participation locale et en needs to use its lending and non- actividades sectoriales y tematicas. diffusant plus largement les etudes lending tools more deliberately to Pueden extraerse varias ensenianzas realisees ; elle doit plus souvent foster consensus through in- interdisciplinarias: monter des projets pilotes pour creased local involvement and * Los programas de reforma y de tester de nouvelles approches, et more widespread dissemination desarrollo tienen mayores prob- recourir plus largement a des serv- of analytic work, more systematic abilidades de recibir apoyo si hay ices de conseils et a des instru- use of pilot projects to test new ap- un consenso dentro del pais y ments de prets flexibles pour proaches, and greater use of ad- una visi6n compartida entre el promouvoir les processus de visory services and flexible lending Banco y el pais. En los paises, el rdorientation des politiques. Au instruments to nurture policy Banco debe utilizar sus instru- niveau des strategies sectorielles xii EXECUTIVE SUMMARY change processes. In sector mentos crediticios y no cred- et institutionnelles, la Banque and corporate strategies, the iticios de manera mAs delib- doit reconna^itre les terrains de Bank should acknowledge erada para fomentar el d6saccord, identifier les pri- areas of disagreement and consenso mediante una orites relatives a la creation de identify priorities for knowl- mayor participaci6n local y savoir, au partage de l'ex- edge creation, cross-country una difusion mas amplia de perience et A l'utilisation learning, and strategic use of sus estudios analiticos; un strategique de son role mo- its convening role. uso mas sistematico de los bilisateur. • Development effectiveness de- proyectos piloto para comprobar . L'efficacite du developpement pends critically on adapting in- nuevos planteamientos, y una depend dans une tres large stitutional priorities and programs mayor utilizaci6n de los servicios mesure de l'adaptation des pri- to individual countries' constraints de asesoramiento e instrumentos orites institutionnelles et des pro- and opportunities. Such adapta- crediticios flexibles para fomen- grammes compte tenu des tion requires an up-to-date tar los procesos de cambio en obstacles et des possibilites qui knowledge base about country las politicas. En las estrategias existent dans chaque pays. Pour operating contexts and the en- sectoriales e institucionales, el ce faire, il est important de dis- abling environment. Despite an Banco deberia reconocer las Areas poser d'informations A jour sur le increased focus on knowledge de desacuerdo e identificar pri- cadre des affaires et les conditions management, current resource al- oridades para la creaci6n de dans lesquelles se deroulent les location patterns do not give suf- conocimientos, el aprendizaje in- activites. Bien qu'une importance ficient priority to economic and terdisciplinario y el uso es- accrue soit accordee a la gestion sector work. trategico de su capacidad de des connaissances, la structure * Countries with stronger policy convocatoria. actuelle des allocations de frameworks receive more lending * La eficacia para el desarrollo de- ressources n'accorde pas une pri- resources per capita. But for pende fundamentalmente de la orite suffisante aux etudes each individual country, espe- adaptaci6n de las prioridades y economiques et sectorielles. cially those with poor policies programas institucionales a los * Les pays dans lesquels l'action and insstitutions, the Bank's ef- obstaculos y oportunidades de publique s'inscrit dans un cadre fectiveness depends not solely cada pals. Esta adaptaci6n re- solide recoivent un volume de on lending volume but also on quiere una base actualizada de prets plus eleve par habitant. the mix of instruments, the tim- conocimientos sobre la situaci6n Toutefois, dans tous les pays, ing of policy-based loans, and real de los paises y las circun- mais surtout dans ceux dont les the judicious use of partnerships. stancias que podrfan resultar mAs institutions sont precaires et les • The Bank has initiated a wide ventajosas. A pesar de una cre- politiques peu judicieuses, l'effi- range of productive partnerships, ciente atenci6n a la gesti6n del cacite de l'action de la Banque but will only realize their full po- conocimiento, las pautas actuales depend non seulement du vol- tential by being more businesslike de asignaci6n de los recursos no ume des prets mais aussi de and selective about what they are otorgan la suficiente prioridad a 1'ensemble des instruments util- intended to achieve, what risks los estudios econ6micos y secto- ises, du moment ou des prets they entail, and how they can riales. sont consentis A l'appui de re- yield operating economies for the . Los pafses con un marco normativo forme et du recours judicieux a Bank through sharper delineation mAs s6lido reciben mayor finan- des partenariats. of its comparative advantage. ciamiento per capita. No obstante, * La Banque a entrepris de former en lo que se refiere a cada uno de un large eventail de partenariats Improved Project Results los paises, en particular los que productifs, mais ne pourra pleine- Despite the challenging business presentan politicas e instituciones ment exploiter leur potentiel environment, the Bank's long-stand- mAs deficientes, la eficacia del qu'en se montrant pragmatique et ing efforts to improve project qual- Banco depende no solamente del en determinant soigneusement xiii 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS ity have borne fruit. Portfo- volumen de los prestamos sino les objectifs qu'ils doivent lio performance has im- tambien de la combinaci6n permettre d'atteindre, les proved over the past five de instrumentos, el marco risques qu'ils posent et la years, and is likely to exceed cronol6gico de los prestamos maniere dont ils peuvent per- the Strategic Compact target orientados a la reforma de las - mettre a la Banque de realiser of 75 percent satisfactory in politicas y la sabia utilizaci6n - des economies au niveau de FYOO. Sustainability and in- de las alianzas. ces operations en lui perme- stitutional development rat- * El Banco ha iniciado una ttant de definir plus precise- ings have also improved. Project gran variedad de asociaciones ment ces avantages comparatifs. performance remains disappointing productivas, pero s6lo hara ple- in Africa, but Quality Assurance namente realidad su potencial si Amelioration des restiltats des Group (QAG) data from the cur- se muestra mas profesional y se- projets rent portfolio could signal a prom- lectivo al determinar los objetivos Malgre l'environnement difficile dans ising turnaround in response to que se quieren conseguir, los ries- lequel prennent place ses opera- portfolio improvement efforts. gos que implican y la manera en tions, les efforts deployes de longue These gains have been achieved que pueden conseguir ahorros date par la Banque pour ameliorer even though projects are becoming en las operaciones del Banco la qualite des projets portent leurs increasingly complex and demand- mediante una delineaci6n mas fruits. La performance du portefeuille ing, especially in low-income coun- nitida de su ventaja comparativa. s'est amelioree au cours des cinq tries. More than four of every five dernieres annees et devrait etre recently evaluated projects were sub- Mejora en los resultados de superieure a l'objectif indique dans stantially demanding for the bor- los proyectos le Pacte strategique, qui fixait a 75 % rower. Since more than one in three A pesar de las dificiles condiciones la proportion des projets pouvant projects show unsatisfactory bor- econ6micas, los prolongados es- etre juges satisfaisants durant l'exer- rower implementation, further fuerzos del Banco por mejorar la cice 00. Les notes attribuees au titre progress will require parallel im- calidad de los proyectos han dado de la perennite des projets et du provements in capacity building and fruto. El resultado de las operaciones developpement institutionnel se sont adapting each project's design to de la cartera ha mejorado en los egalement ameliorees. Les projets borrower capacity. cinco ultimos anios y es probable ont continue de donner des resultats Overall lending ratings improved que supere la meta del Pacto Es- decevants en Afrique, mais les in- to 75 percent satisfactory for the trategico de un 75% de proyectos formations du Groupe de contr6le de partial FY00 sample, after plateauing satisfactorios en el ejercicio de 2000. la qualite (QAG) sur le portefeuille near 70 percent for several years. Las calificaciones han mejorado tam- actuel semblent indiquer une reprise Performance of adjustment opera- bien en lo que se refiere a la sosteni- prometteuse par suite des efforts tions has improved steadily over the bilidad y el desarrollo institucional. d'amelioration du portefeuille. past five years. But findings from Los resultados de los proyectos con- Ces progres ont ete accomplis project audits and country evalua- tinuian siendo decepcionantes en alors meme que les projets deve- tions suggest that the achievement Africa, pero los datos del Grupo de naient de plus en plus complexes et of immediate policy objectives does garantia de calidad correspondientes difficiles, en particulier dans les pays not necessarily translate into long- a la cartera actual podrian revelar un a faible revenu. Plus des quatre term impact on incomes and giro prometedor en respuesta a las cinquiemes des projets qui viennent poverty. Such impact requires gov- medidas de mejora de la cartera. etre evalues exigeaient beaucoup ernment commitment and a suffi- Estos progresos se han conseguido des emprunteurs. Comme plus d'un cient consensus to sustain reform. a pesar de que los proyectos son tiers des projets ont ete juges ne Outcomes can also be under- cada vez mas complejos y dificiles, pas avoir ete executes de maniere mined by exogenous factors. As the sobre todo en los paises de ingreso satisfaisante par ces derniers, il fau- Bank moves toward a results-based bajo. Mas de cuatro de cada cinco dra, pour pouvoir continuer de pro- framework, the objectives of adjust- proyectos recientemente evaluados gresser, renforcer les capacites et xiv EXECUTIVE SUMMARY ment programs should in- representaban unas exigen- adapter la conception de creasingly be framed in steps cias considerablemente may- chaque projet aux capacites toward explicit development ores para el prestatario. Como des emprunteurs. Les notes outcomes, with clear delin- - en mas de uno de cada tres des projets d'investissement eation of accountabilities proyectos se observa una - se sont ameliorees puisque among development actors. aplicaci6n poco satisfactoria - 75 % des projets inclus dans por parte del prestatario, para l'echantillon partiel constitue Adapting to Country conseguir nuevos progresos pour l'exercice 00 ont et Contexts to Achieve Results se requeriran mejoras paralelas en el juges satisfaisants alors que ce pour- In recent years, the Bank has sought desarrollo de la capacidad y la centage etait reste de l'ordre de 70 to shift its focus from individual proj- adaptaci6n de cada proyecto a la % les annees precedentes. Les re- ects to the "higher plane" of coun- capacidad del prestatario. sultats obtenus pour les operations try programs. The outcomes of Las clasificaciones satisfactorias de d'ajustement s'ameliorent progres- country programs, as evaluated in los proyectos de inversi6n alcanzaron sivement depuis cinq ans. Les con- OED Country Assistance Evaluations un 75% en la muestra parcial del ejer- clusions des audits des projets et (CAEs), frequently differ from the ag- cicio de 2000, despues de haberse es- des evaluations sur le terrain sem- gregate portfolio performance in the tabilizado en cifras pr6ximas al 70% blent toutefois indiquer que la real- country. Nor are country program durante varios anios. Los resultados de isation des objectifs d'action outcomes necessarily determined by las operaciones de ajuste han mejo- immediats n'a pas necessairement per capita income. Instead, devel- rado de forma constante en los cinco un impact a long terme sur les opment effectiveness at the country uiltimos afios. Por otro lado, las com- revenus et la pauvrete. Pour que cet level depends on the way in which probaciones relativas a las auditorias impact se materialise, il faut que les strategy and instrument mix- de proyectos y evaluaciones de los pouvoirs publics soient determines including partnerships-are selected, paises revelan que el logro de los a poursuivre les r6formes et que and adapted to country conditions. objetivos normativos inmediatos no se celles-ci fasse l'objet d'un consensus. The level of country consensus traduce necesariamente en conse- Les resultats peuvent en outre souf- for reform is a critical determinant of cuencias a largo plazo en los ingre- frir de l'effet de facteurs exogenes. the relevance and outcome of sos y la pobreza. Estas consecuencias Maintenant que la Banque replace country strategies. Inadequate bor- requieren el compromiso guberna- son action dans un cadre base sur rower commitment to the Bank's mental y un consenso suficiente en les resultats, les objectifs des pro- country program and to policy re- apoyo de la reforma. En algunos grammes d'ajustement devraient, de forms was the major cause of poor casos, los resultados se han deterio- plus en plus, &tre poursuivis par implementation. CAEs find a pat- rado por factores ex6genos. A medida etapes pour aboutir a des resultats tern of over-optimism about bor- que el Banco avanza hacia una es- de developpement explicites, et les rower governments' receptivity to tructura basada en los resultados, los responsabilites incombant aux dif- Bank advice, willingness to under- objetivos de los programas de ajuste ferents agents du developpement take difficult reforms, and capacity deberan concebirse cada vez mas devraient etre clairement definies. to implement recommended meas- como avances graduales hacia resul- ures. In some cases the Bank has tados relacionados expresamente con Amelioration des resultats par lent to support reforms-either el desarrollo, con una clara definici6n I'adaptation des operations through adjustment or investment de las responsabilidades entre los re- aux conditions en vigueur lending-before sufficient consensus sponsables del desarrollo. dans les pays had emerged. Stakeholder analysis Depuis quelques annees, la Banque remains the exception rather than Adaptaci6n al contexto de los s'efforce de mettre moins l'accent sur the rule. paises para conseguir la poursuite de projets independants The Bank should deploy lending resultados pour privilegier la formulation de and nonlending instruments more En los ultimos anios, el Banco ha programmes par pays. Les resultats strategically in relation to country cir- tratado de desviar su atenci6n de los de ces programmes, tels qu'ils fig- xv 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS cumstances. Adjustment lend- proyectos individuales hacia urent dans les evaluations de ing can support countries el "plano superior" de los l'aide aux pays realisees par seeking to improve macro- programas de alcance na- l'OED different frequemment economic or sectoral policies, cional. Los resultados de des resultats obtenus pour but only when stakeholders estos, seguin las evaluaciones 1 l'ensemble du portefeuille are convinced of the need for de asistencia a los paises, di- d'un pays donne. Par ailleurs, reforms. Complementary sup- fieren con frecuencia de los les resultats du programme port for economic and sector resultados agregados de la d'un pays ne sont pas neces- work, capacity building, and sector cartera de un pais, y no estan nece- sairement determines par le revenu investments are often necessary to sariamente determinados por el in- par habitant. En fait, 1'efficacite du maximize the impact of policy-based greso per capita. Mas bien, la eficacia developpement au niveau national lending. Where consensus for reform para el desarrollo en cada pais de- depend de la maniere dont la is weak, pilot projects, dialogue, and pende de la forma en que se selec- strategie et les instruments utilises - economic and sector work can help cionan las estrategias e instrumentos, y compris les partenariats - sont build consensus-but local involve- incluidas las asociaciones, y se adap- choisis et adaptes aux conditions en ment and widespread dissemination tan a las situaciones concretas. vigueur. of findings is essential. El consenso de los paises en La mesure dans laquelle un pays Most CAEs rate the Bank's con- relaci6n con la reforma es un de- accepte la necessite des reformes tribution to institutional develop- terminante decisivo de la pertinen- est un facteur determinant de la ment at the country level as modest. cia y resultados de las estrategias. La pertinence et des resultats des They confirm that institutional and falta de compromiso de los prestatar- strategies formulees pour ce pays. risk analysis are typically the weak- ios con el programa del Banco en un Les pietres resultats observes au est part of diagnostic work, and that pais y con las medidas de reforma niveau de l'execution des projets Bank projects and policy recom- fue la principal causa de una ejecu- ont essentiellement tenu au manque mendations frequently are still too ci6n poco satisfactoria. En las eval- de determination manifeste par les complex and demanding, particu- uaciones de asistencia a los paises parties prenantes a l'egard des pro- larly in countries with weak institu- se observa una tendencia de exce- grammes de la Banque et des re- tions and capacity. sivo optimismo acerca de la apertura formes. Les evaluations de l'aide Development assistance is gen- de los pafses al asesoramiento del aux pays montrent que la Banque erally more effective in countries Banco, su disposici6n a entablar re- surestime la mesure dans laquelle with a stable macroeconomic frame- formas dificiles y su capacidad de les gouvernements emprunteurs ac- work and an effective public sector. aplicar las medidas recomendadas. ceptent ses conseils, leur determi- On a per capita basis, IDA lends al- En algunos casos, el Banco ha otor- nation a entreprendre des reformes most twice as much to countries gado prestamos en apoyo de las re- difficiles et leurs capacites a exe- that receive higher performance rat- formas -mediante financiamiento cuter les mesures recommandees. ings on the Bank's Country Policy para el ajuste o la inversi6n- antes Dans certains cas, la Banque a con- and Institutional Assessment (CPIA) de que se hubiera llegado a un con- senti des prets a l'appui de reformes as it does to countries with rela- senso suficiente. El analisis de las -sous forme de prets d'ajuste- tively low ratings. A similar pattern partes interesadas continua siendo la ment ou de prets d'investissement holds for International Bank for Re- excepci6n, mas que la norma. - avant que ces reformes n'aient construction and Development El Banco deberia desplegar sus fait l'objet d'un consensus. Ce n'est (IBRD) lending, despite the lack of instrumentos crediticios y no cred- qu'a titre exceptionnel que des formal performance-based lending iticios de manera mas estrategica en analyses sont consacrees aux par- mechanisms. These linkages will be relaci6n con las circunstancias de ties prenantes. enhanced once the Bank begins to los paises. Los prestamos para el La Banque devrait recourir de share its performance assessments ajuste pueden ayudar a los paises a maniere plus strategique a ses in- with IDA borrower authorities, as tratar de mejorar las politicas macro- struments de pret et hors pret, en is planned for FY01. econ6micas o sectoriales, pero uni- fonction des conditions en vigueur xvi EXECUTIVE SUMMARY A special challenge is to camente cuando las partes dans les pays. Les prets d'a- find ways to reduce poverty - interesadas esten convenci- justement peuvent aider les in low-income countries with das de la necesidad de la re- pays qui souhaitent ameliorer weak governance and policy forma. En muchos casos, se leur politique macroe- frameworks. The Bank needs requiere apoyo complemen- - conomique ou sectorielle, to strengthen and clarify its tario para los estudios mais seulement lorsque les approach. It should focus on econ6micos y sectoriales, el parties prenantes sont con- diagnosing the factors affect- desarrollo de la capacidad y vaincues de la necessite des ing weak performance and use its las inversiones sectoriales con el fin reformes en question. II est souvent convening role to build consensus de multiplicar los efectos del finan- necessaire de fournir un appui com- for reform. Adjustment lending ciamiento para la reforma de las plementaire pour realiser des etudes should be avoided pending concrete politicas. Cuando el consenso sobre economiques et sectorielles, ren- actions that signal commitment to la necesidad de la reforma es insu- forcer leurs capacites et proceder a change. Depending on country cir- ficiente, este se puede reforzar me- des investissements sectoriels de cumstances, limited, targeted grants diante proyectos piloto, el dialogo y maniere a maximiser l'impact des or lending operations to strengthen estudios econ6micos y sectoriales, prets a l'appui de reformes. Lorsque public sector accountability systems pero en cualquier caso es funda- les reformes ne font pas l'objet d'un or to provide support to targeted mental la participaci6n local y una consensus, il peut etre souhaitable and closely monitored poverty pro- difusi6n muy amplia de las conclu- de proc6der a des operations pi- grams may be appropriate. siones. lotes, d'engager le dialogue et de The Bank's process for allocating La mayor parte de las evalua- realiser des etudes economiques et its administrative resources could be ciones sobre la asistencia a los paises sectorielles, mais il est, dans tous enhanced to foster development ef- otorgan calificaciones modestas a la les cas, essentiel d'obtenir la partic- fectiveness. Given the large number aportaci6n del Banco al desarrollo ipation des populations locales et de of country departments and their institucional de los paises. Confirman diffuser largement les conclusions dominant role in budget manage- que el analisis institucional y de des efforts accomplis. ment, resource redeployment is hard riesgo suele constituir la parte mas Selon la plupart des evaluations to achieve. Country program budget debil de la labor de diagn6stico, y de l'aide aux pays, la contribution de allocations are relatively invariant que los proyectos y recomenda- la Banque au developpement insti- with respect to country policy and ciones normativas del Banco suelen tutionnel semble reduite. Ces eval- project performance, and are not al- ser todavia demasiado complejos y uations confirment le fait que ways fully aligned with the Country exigentes, en particular en los paises I'analyse des institutions et des Assistance Strategy. The high share con poco desarrollo de las institu- risques est generalement la partie la of fixed costs within a country pro- ciones y de la capacidad. moins solide des etudes de diag- gram-especially where there are La asistencia para el desarrollo nostic, et que les projets et les field offices-further limits flexibility. suele ser mas eficaz en paises con un recommandations de la Banque sont Partnerships in support of coun- marco econ6mico estable y un sec- frequemment trop complexes et ex- try programs flourish when the Bank tor publico eficaz. En cifras per cdpita, igeants, en particulier dans les pays and client country have a well- la AIF presta casi dos veces mas a los dont les institutions sont precaires et specified and agreed policy frame- paises que reciben calificaciones el- les capacites limitees. work "owned" by the borrowing evadas en la evaluaci6n de las politi- L'aide au developpement a country. Clearly specified policy cas e instituciones nacionales que los generalement davantage d'impact frameworks facilitate the develop- paises con calificaciones relativamente dans les pays dont le cadre macroe- ment of two critical elements of suc- bajas. Algo semejante se observa en conomique est stable et le secteur cessful partnership: a clearly defined los prestamos del Banco Intemacional public efficace. L'IDA prete presque role for each partner, and a shared de Reconstrucci6n y Fomento (BIRF) deux fois plus, par habitant, aux understanding of each partner's dis- a pesar de la falta de mecanismos pays dont la performance est jug6e tinct accountability. formales de prestamos basados en el satisfaisante dans le cadre des eval- xvii 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS But the Bank's matrix man- desempefno. Estas vincula- uations de la politique et des agement structure complicates - ciones se intensificaran una institutions nationales (EPIN), commitment to global goals vez que el Banco comience a qu'a ceux dont la perform- and related resource alloca- compartir las evaluaciones del ance est jugee relativement tions to country programs. At desempefno con los prestatar- faible. On observe une dif- the sector and corporate ios de la AIF (en el ejercicio ference similaire pour les levels, the establishment of de 2001, segun las previ- prets de la Banque interna- more businesslike partnership siones). tionale pour la reconstruc- arrangements is a priority. Networks Un problema especial es encon- tion et le developpement (BIRD), need to clarify the Bank's core com- trar la manera de reducir la pobreza bien que celle-ci n'ait pas de me- petencies and comparative advan- en los paises de ingreso bajo con sis- canisme de pret formellement base tage in relation to its partners, and temas de gobierno y marcos nor- sur la performance. Ces liens seront establish businesslike alliances with mativos deficientes. El Banco debe encore renforces lorsque la Banque appropriate sharing of responsibility. reforzar y aclarar su planteamiento. commencera a communiquer ses This includes streamlining the num- Debera tratar de diagnosticar los evaluations de la performance aux ber of partnership arrangements, as- factores que condicionan el desem- autorites des pays qui empruntent suring that any agreements are backed penio, y utilizar su capacidad de con- a l'IDA, comme elle est censee le by corporate commitments, with a vocatoria para lograr mayor faire a partir de l'exercice 01. full recognition of resource implica- consenso en torno a la reforma. De- La Banque doit resoudre un tions, and letting go of activities that berian evitarse los prestamos para el probleme particulier, qui consiste a can be implemented by other actors. ajuste mientras no se adopten me- trouver le moyen de reduire la pau- didas concretas que demuestren el vrete dans les pays a faible revenu Toward Global Results: Sector compromiso con el cambio. Segun dans lesquels la gestion et le cadre and Thematic Dimensions las circunstancias del pais, podrian des affaires publiques presentent The Bank's ability to articulate a broad ser aconsejables donaciones o pres- des carences. Il lui faut renforcer et strategic vision is one of its strengths, tamos otorgados de forma selectiva preciser sa demarche. Elle doit s'- and it has an obligation to provide para reforzar los sistemas de rendi- efforcer de diagnostiquer les causes consistent advice across countries and ci6n de cuentas del sector publico de ces carences et utiliser son pou- Regions. Sector and thematic strate- o apoyar programas focalizados y es- voir de mobilisation pour forger un gies are important vehicles for artic- trechamente supervisados contra la consensus en faveur de reformes. II ulating corporate objectives and pobreza. lui faudra eviter de consentir des achieving international development El proceso del Banco para asig- prets d'ajustement tant que le pays targets, but must be reconciled with nar sus recursos administrativos po- n'aura pas pris de mesure concrete the increasingly demand-driven ap- drfa modificarse con el fin de montrant qu'il est determine a proach of Bank country programs. fomentar una mayor eficacia en ter- changer la situation. Selon les con- Agreement or consistency between minos de desarrollo. Dado el gran ditions en vigueur, il pourra etre the priorities identified in the Bank's numero de departamentos geogra- approprie de poursuivre des opera- strategy and those of the borrower- ficos y su influencia dominante en tions de prets ou de dons cibles or other stakeholders-fosters the im- la gestion del presupuesto, es difi- d'envergure limitee pour renforcer plementation of Bank sector or cil conseguir una redistribuci6n de les systemes de responsabilisation thematic strategies. But few Bank los recursos. Las asignaciones pre- du secteur public ou fournir un strategies and policies provide ex- supuestarias para los programas en appui a des programmes de pau- plicit guidance on what instruments los paises han cambiado relativa- vrete focalises et faisant l'objet d'un to use and what posture to take in the mente poco con respecto a la suivi attentif. country dialogue if governments do politica nacional y los resultados de Le processus d'allocation des not want to address an issue of global los proyectos, y no siempre estan en ressources administratives de la interest-for example, gender equity total consonancia con la estrategia de Banque pourrait etre renforce de or forest conservation. asistencia adoptada. La elevada pro- maniere a accroitre l'efficacite du xviii EXECUTIVE SUMMARY Customizing strategies and porci6n de los costos fijos developpement. Etant donne policy recommendations is dentro de los programas en le grand nombre de de- an important determinant of los paises -sobre todo en partements-pays et le r6le local program success. Coun- los que tienen oficinas del primordial qu'ils jouent au try-specific analytic work is Banco- limita todavia mas la - niveau de la gestion budge- crucial to support such adap- R flexibilidad. taire, il est difficile de reaf- tation; the decline in funding Las asociaciones en apoyo fecter les ressources. Les for economic and sector de los programas prosperan allocations budgetaires aux work is a concern. So is the chronic cuando el Banco y el gobiemo tienen programmes des pays sont relative- weakness observed in monitoring un marco de politicas bien especifi- ment independantes des resultats and evaluation arrangements-es- cado y concordado, con el que se obtenus par ces derniers au plan sential for adapting and updating siente identificado el pais prestatario. des politiques et des projets, et ne strategies. Unos marcos normativos claramente sont pas toujours vraiment adaptees Operational policies guide staff especificados facilitan la aparici6n a la strategie d'aide aux pays. L'am- about implementation of corporate de dos elementos criticos de una pleur relative des coOts fixes du pro- and sector priorities. But ambiguities asociaci6n eficaz: clara delimitaci6n gramme d'un pays, en particulier or gaps in the Bank's policy framework del papel de cada uno de los socios, lorsque que la Banque a un bureau have made Bank staff and manage- y una opini6n compartida acerca de dans ce dernier, reduit encore cette ment, clients, and partners uncertain la responsabilidad de cada uno de flexibilite. about requirements. Enforcement of ellos. Por otro lado, la estructura de Il est aise de constituer des parte- Safeguard Policies has been inconsis- gesti6n matricial del Banco dificulta nariats a l'appui du programme for- tent because of a combination of highly el compromiso con metas mundi- mul6 pour un pays lorsque la decentralized accountability and weak ales y las correspondientes asigna- Banque et le pays client ont un central coordination and oversight. ciones de recursos a los programas cadre de politique bien determine Bank management has taken steps to en los paises. En lo que se refiere a auquel le pays emprunteur ' ad- clarify the Bank's policy framework, los sectores y empresas, es prioritario lere,, pleinement. Lorsque le cadre but further progress is needed to en- el establecimiento de mecanismos de l'action publique est clairement hance accountability and harmonize de asociaci6n mas rigurosos. Las defini, il est plus facile de reunir les Bank policies and Safeguard require- redes deben aclarar las competencias deux conditions essentielles ia un ments with those of client countries basicas y la ventaja comparativa del partenariat fructueux : la definition and international partners. Banco, y establecer alianzas formales d'un r6le precis pour chaque parte- With the introduction of Sector con el correspondiente reparto de re- naire, et une r6partition claire et Strategy Papers (SSPs) in 1996, the sponsabilidades. Ello supone la nette des responsabilites entre les Bank began to standardize the racionalizaci6n de un numero de partenaires. Or, en raison de la process of sector strategy develop- acuerdos de asociaci6n, la garantfa structure de gestion matricielle de ment and to address some of the de que todos los acuerdos esten la Banque, il est plus difficile de weaknesses observed in the earlier respaldados por compromisos insti- s'engager a poursuivre des objectifs policy development process. Most tucionales, la atenci6n a sus reper- globaux et d'affecter les ressources budgetary resources and decision- cusiones en los recursos, y el requises aux programmes des pays. making, however, reside with coun- abandono de las actividades que Au niveau sectoriel et institutionnel, try departments. As a result, none of pueden ser realizadas por otros. il importe de donner la priorite a the sector strategies evaluated in- l'adoption de structures de parte- cluded an explicit and verifiable plan Hacia resultados de alcance nariat plus pragmatiques. Les for implementation. Recently some mundial: dimensiones reseaux doivent determiner claire- SSPs have given greater attention to sectoriales y tematicas ment les competences et avantages the business implications of sectoral La capacidad del Banco de articular comparatifs de la Banque pas com- and thematic goals, but the continued una amplia visi6n estrategica es uno paraison a ses partenaires, et forger imbalance in the matrix structure ren- de sus puntos de fuerza, e implica des alliances productives donnant xix 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS ders results-oriented man- una obligaci6n de ofrecer as- lieu un partage approprie agement of sector and the- esoramiento coherente a los des responsabilites. Pour ce matic strategies difficult to distintos paises y regiones. faire, il importera de ratio- achieve. Las estrategias sectoriales y naliser les differentes modal- A preliminary stocktaking tematicas son vehiculos im- - ites de partenariat, de on SSPs carried out by Bank portantes para articular ob- s'assurer que tous les ac- management notes steady im- jetivos institucionales y para cords beneficient de l'en- provements, but many of the el logro de las metas inter- gagement de l'institution, issues identified above remain rele- nacionales de desarrollo, pero deben compte duiment tenu de leurs im- vant. OED's analysis of Bank expe- compaginarse con el planteamiento plications au plan des ressources, et rience with the design and de los programas del Banco en los de cesser de poursuivre des activ- implementation of sector strategies paises, basado cada vez mas en la ites qui peuvent etre realisees par and policies suggests that further at- demanda. El acuerdo o coherencia d'autres acteurs. tention to four elements of success- entre las prioridades senaladas en ful strategies-Policy, Performance, la estrategia del Banco y las del La recherche de resultats Program, and Partnership-could prestatario -u otras partes intere- globaux: une dimension further strengthen the efficacy of the sadas- favorece la aplicaci6n de sectorielle et thematique Bank's sector and thematic strate- las estrategias sectoriales o tematicas La Banque a le grand avantage de gies. del Banco. Pero pocas de sus es- pouvoir definir des strategies trategias y politicas ofrecen aseso- generales, et a l'obligation de fournir Corporate Management ramiento en forma expresa sobre los des conseils refletant une continu- Implications instrumentos que se deben utilizar y ite de vue d'un pays et d'une region The findings in this review have im- la postura que se debe adoptar en el a l'autre. Les strategies sectorielles plications for the Bank's corporate dialogo con los paises si los gobier- et thematiques sont importantes car management. nos no desean plantearse un tema de elles permettent de definir les ob- * Bank strategies, programs, and interes mundial -por ejemplo, la jectifs institutionnels et d'atteindre budgeting processes should be igualdad de la mujer o la conser- les objectifs de developpement in- more explicitly linked to the vaci6n de los bosques. ternational; elles doivent toutefois achievement of results, consistent La adaptaci6n de las estrategias y &tre reconciliees avec le fait que la with the Bank's mission and com- recomendaciones normativas es un les programmes formules par la parative advantage. This implies determinante importante para el Banque pour les pays sont de plus that budget processes should al- exito de los programas locales. Los en plus impulses par la demande. locate resources in alignment with estudios analiticos por paises con- L'execution des strategies secto- corporate strategies. Approved cretos son fundamentales para con- rielles ou thematiques est d'autant Country Assistance Strategies, sec- seguir esa adaptaci6n, por lo que plus facile que les priorites identi- tor strategies, and global partner- resulta preocupante el descenso del fi6es dans la strategie de la Banque ships should be fully costed, and financiamiento para los estudios et celle de l'emprunteur - ou the costs linked to the annual econ6micos y sectoriales. Lo mismo d'autres parties prenantes - coin- budget process. Staff incentives cabria decir de la cr6nica debilidad cident ou sont compatibles. Toute- should be linked to results and de los mecanismos de seguimiento fois, rares sont les. strategies et performance, not inputs and y evaluaci6n, imprescindibles para politiques de la Banque qui four- processes. adaptar y actualizar las estrategias. nissent des directives explicites sur * Maintaining the Bank's knowledge Las politicas operacionales les instruments qu'il convient d'u- base is critical. The Bank should orientan al personal acerca de la tiliser et les positions a prendre dans strike a more appropriate balance aplicaci6n de las prioridades insti- le cadre du dialogue mene avec un between knowledge management tucionales y sectoriales. Pero las am- pays lorsqu'un gouvernement ne and knowledge creation, at both biguiedades o lagunas en el marco veulent pas aborder un probleme the country and global levels. It de polfticas del Banco han provo- qui depasse largement le cadre de xx EXECUTIVE SUMMARY should monitor and track the cado entre su personal y ad- ses frontieres - comme la quality and coherence of all ministraci6n, los clientes y parite hommes-femmes ou Bank knowledge activities. otras partes interesadas una la preservation des for&ts. Achieving selectivity will considerable incertidumbre Le succes des programmes require the Bank to seri- sobre cuales son los requisi- a l'echelon local est large- ously and continuously tos. La aplicaci6n de las ment fonction de l'adapta- assess the comparative ad- politicas de salvaguardia ha tion des strategies et des vantage of the institution rel- sido poco coherente por una politiques recommandees. ative to international and client combinaci6n de excesiva central- Pour pouvoir proc6der a cette adap- country partners. This may entail izaci6n en la rendici6n de cuentas tation, il est essentiel d'analyser la de "letting go" of activities that other y escasa coordinaci6n y supervisi6n chaque pays; la diminution des partners can carry out. central. La administraci6n del Banco ressources affect6es aux etudes ha adoptado medidas para aclarar el economiques et sectorielles ne marco de politicas, pero se requieren manque donc pas d'etre preoccu- nuevos progresos para lograr una pante, de meme que la faiblesse mayor rendici6n de cuentas y ar- chronique des modalites de suivi et monizar las politicas del Banco y d'evaluation - qui sont pourtant los requisitos de salvaguardia con los essentielles a l'adaptation et a la de los paises clientes y los asociados mise a jour des strategies. internacionales. Les politiques operationnelles A raiz de la introducci6n de los guident la mise en ceuvre des documentos de estrategia sectorial en priorit6s institutionnelles et secto- 1996 el Banco comenz6 a unificar el rielles par les services de l'institu- proceso de elaboraci6n de estrategias tion. Le cadre de l'action de la sectoriales y a abordar algunas de las Banque presente toutefois cer- deficiencias observadas anterior- taines ambiguites ou lacunes de mente en el proceso de elaboraci6n sorte que le personnel, la direction, de politicas. No obstante, la mayor les clients et les partenaires de l'in- parte de los recursos presupuestar- stitution ne sont pas certains des ios y de la toma de decisiones es conditions a remplir. L'application competencia de los departamentos des mesures de sauvegarde n'est geograficos por paises. En conse- pas systematique parce que la re- cuencia, ninguna de las estrategias sponsabilite de cet effort a et sectoriales evaluadas contenia un placee a un niveau tres decentral- plan explicito y verificable para su ise et que les activites de coordi- ejecuci6n. Recientemente, algunos nation et de suivi au niveau central documentos de estrategia sectorial sont restees limitees. La direction han otorgado mayor atenci6n a las de la Banque a pris des mesures consecuencias financieras de las pour pr6ciser le cadre de ses poli- metas sectoriales y tematicas, pero el tiques mais il lui faudra renforcer constante desequilibrio de la estruc- le processus de responsabilisation tural matricial hace muy dificil la et harmoniser ses politiques et ob- consecuci6n de una gesti6n de las ligations en matiere de sauvegarde estrategias sectoriales y tematicas ori- avec celles de ses pays clients et entada a los resultados. des partenaires internationaux. Las ensenianzas preliminares Depuis qu'il a ete decide, en obtenidas por la administraci6n del 1996, de realiser des etudes sur la Banco tras la aplicaci6n de los doc- strategie sectorielle, la Banque a xxi 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS umentos de estrategia secto- entrepris de standardiser le rial revelan una constante processus de formulation de mejora, pero continuian ex- ces strategies et de rem6dier istiendo muchos de los prob- a certaines des carences ob- ;- lemas mencionados mas servees anterieurement au arriba. El analisis del DEO niveau de la formulation des sobre la experiencia del politiques. Or, les departe- Banco en el diseno y apli- ments-pays ayant l'essentiel caci6n de estrategias y politicas sec- des ressources budgetaires et regis- toriales revela que una mayor sant en grande partie le processus atenci6n a los cuatro elementos de de decision, aucune des strategies una estrategia eficaz -politicas, de- sectorielles evaluees n'6tait assortie sempefio, programas y asocia- de plans d'execution explicite et ciones- podria multiplicar la eficacia verifiable. Certaines recentes etudes de las estrategias sectoriales y temati- sur la strategie sectorielle pretent cas del Banco. une plus grande attention aux im- plications pour l'institution des ob- Repercusiones para la gesti6n jectifs sectoriels et thematiques mais, de la instituci6n en raison du desequilibre qui per- Las comprobaciones de este exa- siste au niveau de la structure ma- men tienen repercusiones para la tricielle, il reste toujours tres difficile gesti6n del Banco. de gerer les strategies sectorielles et Las estrategias, programas y pro- thematiques sur la base des resul- cesos presupuestarios del Banco tats. deberian estar mas expresamente La direction de la Banque a vinculados con el logro de re- procede a un examen preliminaire sultados, de conformidad con la de ces etudes qui l'a amene a con- misi6n y la ventaja comparativa clure que la situation s'ameliorait del Banco. Ello significa que los progressivement mais que nombre procesos presupuestarios de- des problemes identifies ci-dessus berian asignar recursos de continuent de se poser. L'analyse acuerdo con las estrategias insti- par l'OED des accomplissements de tucionales. Deberian calcularse la Banque dans le cadre de la con- atentamente los costos de las es- ception et de l'execution des strate- trategias de asistencia a los gies et politiques sectorielles laisse paises, las estrategias sectoriales penser que, pour accroitre l'efficac- y las asociaciones mundiales ite des strategies sectorielles et the- aprobadas, y esos costos de- matiques de la Banque, il faudra berian vincularse al proceso pre- mettre l'accent sur les quatre ele- supuestario anual. Los incentivos ments necessaires au succes d'une al personal deberian estar asoci- strategie, a savoir les politiques, les ados a los resultados y al de- resultats, les programmes et les sempenio, no a los insumos ni a partenariats. los procesos. * El mantenimiento de la base de Implications pour la direction conocimientos del Banco es de- de la Banque cisivo. El Banco deberia lograr Les conclusions de cet examen un equilibrio mas adecuado entre prcsentent certaines implications xxii EXECUTIVE SUMMARY gesti6n de los conocimientos pour la direction de la y creaci6n de conocimien- Banque. tos, tanto a escala nacional * Les strategies, programmes como mundial. Deberia en- et processus budgetaires de la cargarse de la supervisi6n y - Banque devraient etre plus seguimiento de la calidad y - explicitement lies a l'obten- coherencia de todas sus ac- tion de resultats, compte tenu tividades en relaci6n con los de la mission et des avan- conocimientos. tages comparatifs de l'institution. * Para lograr una mayor selectivi- Les processus budgetaires de- dad, el Banco debera evaluar de vraient donc allouer les ressources forma seria y constante la ventaja sur la base des strategies de la comparativa de la instituci6n con Banque. Le cout total des strate- respecto a los paises clientes y los gies d'aide aux pays, des strategies socios internacionales. Ello puede sectorielles et des partenariats significar el "abandono" de ac- mondiaux qui ont et approuves tividades que pueden realizar devrait etre calcule et pris en otros socios. compte dans le cadre du proces- sus budgetaire annuel. Les incita- tions fournies au personnel de l'institution devraient etre fonc- tion des r6sultats et des perform- ances et n'ont pas des intrants et des processus. • IL est crucial de maintenir la base de connaissance de la Banque. L'institution devrait s'efforcer de mieux equilibrer la gestion et la creation de savoir, aussi bien au niveau national qu'au niveau mondial. Elle devrait suivre et evaluer en permanence la qual- ite et la coherence de toutes ses activites en ce domaine. . Pour pouvoir se montrer selective, la Banque devra soigneusement et systematiquement evaluer ses avantages comparatifs par rap- port a ceux de ses partenaires in- ternationaux et des pays clients. Cela pourrait signifier ' l'aban- don - d'activites dont d'autres partenaires peuvent se charger. xxiii ABBREVIATIONS AND ACRONYMS AfDB African Development Bank AFR Africa Region APL Adaptable Program Lending ARDE Annual Review of Development Effectiveness CAE Country Assistance Evaluation CAS Country Assistance Strategy CDF Comprehensive Development Framework CFA Communaut6 Financiere Africaine (Africa Financial Community) CPIA Country Policy and Institutional Assessment DAC Development Assistance Committee DGO Director-General, Operations Evaluation EAP East Asia and Pacific Region ECA Europe and Central Asia Region ESW Economic and sector work GDP Gross domestic product HIPC Highly Indebted Poor Countries HNP Health, Nutrition, and Population IBRD International Bank for Reconstruction and Development ID Institutional development IDA International Development Association IFC International Finance Corporation IMF International Monetary Fund LCR Latin America and the Caribbean Region LIL Learning and Innovation Loan MNA Middle East and North Africa Region NGO Nongovernmental organization OED Operations Evaluation Department OP Operational Policy PRSI Poverty Reduction Strategy Initiative QAG Quality Assurance Group SAL Structural Adjustment Loan SAR South Asia Region SECAL Sectoral Adjustment Loan SSP Sector Strategy Paper UNDP United Nations Development Program UNICEF United Nations Children's Fund xxiv FROM STRATEGY TO RESULTS To X he World Bank's mandate has expanded and deepened over the past decade. In response to changes in the international economy and the development system, the Bank is giving more emphasis to institutional development, good governance, social development, post- conflict reconstruction, debt relief, and global public goods. These insti- tutional responses, in turn, have given rise to tensions and tradeoffs among goals. Evaluation results shed light on how development effec- tiveness can be improved in a more complex, demanding, and risky op- erating environment. A Complex Operating Environment borrowers have always been diverse, but the in- Changes in development thinking, an expanding creased level and volatility of private capital flows and changing group of client countries, and de- under globalization are changing the Bank's role mands from shareholders, borrowers, private cor- in middle-income countries. At the same time, porations, and civil society organizations have led the rising number of failed states and conflict- the World Bank to take on new activities. The ridden countries has induced new approaches in Bank's authorizing environment has become more post-conflict countries, and disappointing growth complex and contentious. Member governments and poverty reduction trends (especially in highly are the Bank's primary clients, but the Bank must indebted countries) have spawned new assistance also reckon with an ascendant civil society and strategies for low-income countries. an expanding transnational corporate sector. These diverse stakeholders have strong and often The Bank's Response opposing views about the Bank's role, and the In response to changes in borrowers' needs and Bank has become a favorite target of both critics partners' expectations, the Bank launched a debt and supporters of globalization. The criticisms reduction program, strengthened its alliance converge on one imperative-the Bank must with the International Monetary Fund (IMF); made keep improving its development effectiveness. increased use of new investment lending Doing so requires judicious adaptation to greatly instruments, including Adaptable Program Loans differing country settings. The needs of the Bank's (APLs) and Learning and Innovation Loans (LILs); 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS and laid the foundations for new forms of pro- are embedded in the Poverty Reduction Strategy grammatic lending. In particular, with the advent Initiative (PRSI). Both the CDF and the PRSI aim of special adjustment operations geared to social to strengthen the ownership of pro-poor policies needs and structural change, the Bank was able and improve the quality of aid to the world's poor- to respond in the late 1990s to a countercyclical est countries. On the policy research front, the demand for funding in middle-income countries. World Development Report 2000/01: Attacking Finally, nonlending serv- Poverty has proposed a comprehensive approach ices have assumed more to poverty reduction based on opportunity, em- Changes to make importance across the powerment, and security (World Bank 2000a). It Bank efforts more board and the number of has yet to be translated into operational guidelines. relevant and partnerships has grown responsive have exponentially. Challenges and Tensions The Bank has also Changes to make Bank efforts more relevant increased costs and sought to enhance the and responsive (for example, through decen- expanded the range development impact of tralization) have increased costs and expanded of skills required. its traditional activities. It the range of skills required for Bank involvement. has strengthened the Paradoxically, partnership has contributed to poverty focus of its lend- the overload, rather than to greater burden shar- ing and nonlending services, established "do no ing and selectivity. New lines of business have harm" environmental and social Safeguards, opened, but few old ones have closed. The strengthened financial management and fiduci- Bank's administrative budget has also come ary Safeguards for investment lending, and under pressure from the increased demands of strengthened participatory approaches to proj- improved project design and supervision, non- ect implementation and policy development. lending services, global activities, grant facilities, These enhancements have increased the de- the CDF, and the PRSI. These trends have led to velopment rewards of Bank operations, while a remarkable increase in organizational and staff increasing complexity and adding substantially stress. to the cost of doing business with the Bank. Nevertheless, efforts to improve project qual- In parallel, the institution launched a re- ity have borne fruit. Portfolio performance has newal process to build new competencies, up- improved over the past five years, and, in terms date systems and processes, establish new of project outcomes, is likely to exceed the management arrangements, and reorient its val- Strategic Compact target of 75 percent satisfac- ues and culture. Although lending operations tory in FY00. But there is still major scope for have remained key building blocks of assistance improvements in sustainability and institutional programs, the Bank has given more emphasis development ratings, and project performance to knowledge dissemination and other non- remains disappointing in Africa. Furthermore, lending services, and has sought to strengthen achieving results at the country and sector its strategic focus by making the Country levels-where the Bank is only one of many Assistance Strategy (CAS) the core instrument players-remains a tough challenge. Safeguard for operational programming. The Bank also Policies have established minimum standards established sectoral and thematic networks and for development effectiveness ("do no harm"), initiated the preparation of Sector Strategy but a few visible cases of inadequate compliance Papers (SSPs). have damaged the Bank's reputation. Finally, the President's Comprehensive De- The evaluative evidence underlying this year's velopment Framework (CDF) initiative has chal- Annual Review of Development Effectiveness lenged the Bank and its partners to address a (ARDE) points to four tensions in the Bank's work holistic, long-term development agenda and to with its clients-balancing global prescription and adopt higher standards of cooperation, partici- local adaptation, reconciling client ownership and pation, and results orientation. CDF principles corporate priorities, balancing country performance 2 FROM STRATEGY TO RESULTS and poverty in resource allocation decisions, and remains considerable unrealized potential to make achieving selectivity through partnerships. These judicious use of partnerships. While partnerships tensions are faced by the Bank when it decides are essential to achieve selectivity, partnerships that whether to provide assistance, what assistance it are not aligned with the Bank's core mission can provides, and how it provides it. undermine selectivity. Im- proving development ef- Balancing Performance and Poverty fectiveness thus requires Bank projects in Resource Allocation establishing priorities and and policy Evaluation results confirm that development clearly defining the Bank's recommendations assistance is more effective in countries with a role and comparative ad- . stable macroeconomic framework, effective pub- vantage in relation to its gY P lic sectors, limited corruption, independent ju- public and private part- and demanding, diciaries, and government commitment to poverty ners. particularly in reduction. Many of the countries in which the poorest live lack these attributes. How, and how The 2000 ARDE countries with weak much, should the Bank assist countries with To assess how the Bank institutions and deep poverty but flawed policy environments? has addressed these ten- capacity. And how can the Bank better adapt its assistance sions and what lessons to such environments? can be drawn from ex- perience, this review draws on evaluation find- Reconciling Client Ownership and Corporate ings from: Priorities * Audits and evaluation summaries for 269 The Bank has made poverty reduction its mis- projects evaluated in the past year. In addi- sion and is committed to environmental protec- tion, more than 5,000 previously evaluated tion, participation, good governance, and gender projects in OED's database allow assessments equity. But the Bank also strives to "put the of long-term performance trends and statis- country in the drivers' seat"-which creates a tical analyses of the determinants of project dilemma when Bank objectives are not aligned success. with borrowers' priorities and capacities. The * Country Assistance Evaluations (CAEs) of 38 growing role of civil society in development borrowing countries, especially the 15 most further complicates the balancing act. recent. * Studies of 12 sectors and thematic areas.' Adapting Global Knowledge to Local The 2000 ARDE builds on the findings and Conditions themes of its predecessors. The 1998 ARDE The Bank synthesizes and disseminates global drew the lessons of the 1997 financial crisis for development knowledge, but development ef- development effectiveness. It concluded that fectiveness requires adapting strategies and poli- stable macroeconomic policies were not cies to national and local contexts. How can Bank enough, and that weaknesses in institutions- operations respond to country needs, while particularly in the financial and social sectors- maintaining equitable standards and leveraging can undermine prospects for growth and global development experience? poverty reduction. The 1999 ARDE examined evaluation evidence to identify the tensions Achieving Selectivity Through Partnerships and dilemmas likely to be involved in imple- Evaluation results show that Bank projects and pol- menting the CDF. The central focus of this icy recommendations are increasingly complex year's review is on the effective implementa- and demanding, particularly in countries with tion of operational strategies to deliver devel- weak institutions and capacity. Moreover, there opment results. 3 -I- TRENDS AND LESSONS IN PROJECT PERFORMANCE ending operations are important instruments for achieving develop- L ment effectiveness. OED evaluations in the past year confirm quality improvements in Bank-supported development projects since the mid-1990s. Preliminary results for FY00 exits show an increase to 77 per- cent satisfactory outcomes from 72 percent in FY99. This suggests that ef- forts to enhance quality over the past five years are yielding results. Adjustment lending continues to perform strongly. OED project evaluations also forecast longer-lasting net benefits and greater institutional develop- ment impact, but there remains substantial room for improvement in both Bank performance and borrower capacity development. World Bank lending has changed in focus and proved in the early/mid-1990s (figure 2.1). intensity over the past 10 years to address new Overall, nearly one in four were approved in priorities and development challenges in client the last five years. These evaluations, together countries. OED project evaluations provide in- with a stock of more than 5,000 project eval- sight into the Bank's lending effectiveness in uations, constitute a rich reservoir of lessons this changing environment. Since last year's for current and future lending (see box 2.1 for ARDE, OED has evaluated 269 completed proj- evaluation methods). ects that exited in FY99-00, covering over $18 The complexity and demandingness of evalu- billion in Bank disbursements. These projects ated projects have increased by more than 50 per- reflect the Bank's movement into social sector cent in the past 10 years, reflecting an increased support, increased adjustment lending, transi- focus on institutional and policy change in both tory crisis lending, and greater assistance to investment and adjustment lending.1 Three of transition economies. The majority of the re- four recently evaluated projects were seen as cently evaluated adjustment operations were substantially complex by OED evaluators, and approved in the past three years, while the bulk more than four of five as substantially demand- of evaluated investment projects were ap- ing for the borrower. More than one in three 5 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS show unsatisfactory borrower implementation. taking of portfolio quality and creation of the While there is no simple correlation between Quality Assurance Group (QAG) in 1996. At-risk project complexity and outcomes, OED evalua- projects have been halved since the Portfolio tions consistently point to problems of excessive Improvement Program was introduced in 1996. complexity, particularly Recent OED evaluations signal that these efforts Attention to quality in low-capacity environ- are showing results. The Strategic Compact tar- ments. For borrowers with get of 75 percent satisfactory ratings is likely to has increased since weak institutions, bor- be achieved for FY00 exiting projects. Based on the early 1990s, rower implementation QAG assessments of the portfolio under im- stimulated by an performance is more plementation, further gains may be in the off- likely to be unsatisfactory ing, but lasting results will require both a institutional stock- for projects that were sub- sustained effort to improve Bank and borrower taking of portfolio stantially or highly de- performance and a sharper focus on risk as- quality. manding. While improved sessments, participation, and institutional de- portfolio performance has velopment (ID). Moreover, development impact been achieved despite in- depends on the relevance of overall Bank coun- creased complexity, further progress will require try and sector strategies, synergies between better adaptation of project design to institutional lending and nonlending services, and comple- capacity, greater attention to sequencing of re- mentarity with domestic and international part- forms, and more effective enhancement of bor- ners. The Bank faces the challenge of balancing rower capacity. continued focus on operational quality with Attention to quality has increased since the increased attention to "higher-level" country, early 1990s, stimulated by an institutional stock- sector, and global policy issues. Number of projects evaluated * Adjustment El Investment 50 40- an 0 30 Pre-1990 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 Approval fiscal year Note: The sample comprises 269 projects evaluated since ARDE 1999. 6 TRENDS AND LESSONS IN PROJECT PERFORMANCE Trends in Outcome, Institutional unsatisfactory oil and gas projects in Russia, and Development, and Sustainability it may not persist in the second half of FY00. But the volatility of disbursement-weighted Outcome portfolio performance may continue, given the Bankwide outcome performance of exiting great variation in loan sizes-ranging from projects plateaued during FY97-99 (figure 2.2), small Learning and Innovation Loans to in- in part because of loan and credit closures creasingly large adjustment and programmatic and associated cancellations for a large num- loans. ber of problem projects.2 The FY00 preliminary Project outcomes by approval year are im- results-based on Implementation Completion proving, although not enough of the FY96-00 Reports for 45 percent of FY00 exiting proj- approvals have closed to permit a representa- ects-show a rise to 77 percent satisfactory tive estimate of performance. OED has evalu- outcomes. This appears representative of the ated only 132 of more than 1,200 projects entire cohort of exiting projects, indicating that approved in FY96-00. Ninety percent of these the Strategic Compact target of 75 percent sat- operations have satisfactory outcomes, but this isfactory outcomes is likely to be met or is a small and unrepresentative sample: it in- exceeded. cludes a large share of adjustment operations, Weighted by disbursements, satisfactory out- and projects evaluated early for each approval comes deteriorated from 83 to 73 percent sat- year are typically the best performers. The out- isfactory between FY99 and FY00. This come of the first 10 percent of FY92-95 ap- downturn largely reflects the exit of two large, provals, for example, showed 86 percent of Box 2.1 _ Operational staff prepare completion reports for all projects * ID impact assesses achievements in addressing institutional within six months of loan closing. All self-evaluations are re- constraints to development Overcoming such constraints- viewed by OED to validate performance ratings. OED carries out through capacity building, for example-creates direct ben- field performance audits for 25 percent of all operations one to efits and better operating environments for other development seven years after project completion, with a further reassess- activities. ment of performance ratings and active interaction with bor- * Sustainability assesses the durability of net project benefits. rowers, cofinanciers, and beneficiaries. While two recent Forexample,thelikelihoodthattheywill bemaintainedorex- independent reviews found OED methods to be closest to best ceeded overthe long haul. While the outcome rating is the best practice among multilateral banks, OED continues to review estimate of whether an operation justifies the use of scarce and adjust its evaluation methodology and practices while en- resources, the sustainability criterion reflects the evalua- couraging the Bank and its partners to remedy chronic weak- tor's judgment of its ability to continue producing net bene- nesses in monitoring and evaluation systems. fits in the face of risk and uncertainty. Three key performance criteria capture likely development Evaluation ratings must be interpreted with caution. Even a results-the project's outcome, institutional development (ID) im- decade or more following approval they are forward-looking- pact, and sustainability. exogenous events, policy reversals, or other factors can change * Outcome, the primary measure of project performance, sum- expected outcomes. Such changes are more likely to affect marizes the likelihood thatthe project will achieve its major completion reviews than audits, and adjustment lending more relevant objectives (do the right things) efficiently (do things than investment lending. Ratings also focus only on specific right). The relevance check assures that achievements project activities. To capture the full development impact of rated satisfactory address the country's current development lending interventions, higher-scale evaluations such as Coun- needs and Bank operational priorities. Outcome captures try Assistance Evaluations and sector reviews are necessary to both accomplishments and the expected level of future net assess the indirect and secondary effects and the synergy with benefits. other interventions (see Chapters 3 and 4). 7 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS projects as satisfactory, which indicates some Bank interventions to address institutional con- improvement in the evaluated FY96-00 straints are of two main types: efforts to change approvals. the "rules of the game" at the national or sector levels and strengthening organizational effective- Institutional ness through the development of administrative Development Impact and management skills or restructuring. Adjustment Increased attention Recognizing and address- operations typically focus on the former, and in- to ID appears to ing institutional constraints vestment projects on capacity building, although have contributed to is critical to the develop- the reform content of investment operations has tead Progress in ment effectiveness of Bank increased. The municipal development projects in lending. Since the mid- Brazil and the Philippines sought to combine ID impact. 1990s, the Bank has changes in institutional arrangements with tar- geared more of its lending geted capacity building. These projects signifi- to address such constraints. The share of evalu- cantly enhanced the fiscal autonomy of local ated lending focused primarily on ID increased municipalities by decentralizing decisionmaking in from 18 percent of projects exiting in 1996 to revenue collection and budget management. nearly 30 percent for 1999-00 exits, and most The improvements in property tax collection projects include some components aimed at ID or helped pay off existing obligations, balance the capacity building. Increased attention to ID appears budget, and support entrepreneurial man- to have contributed to steady progress in ID im- agement. The focus on organizations can be found pact, although there remains substantial room for in the cotton sector in Benin, where a number of improvement (figure 2.3). Forty-three percent of semiautonomous state agencies-responsible for FY99-00 exiting projects had substantial ID impact, production, ginning, marketing, and export- and the share of Bank-supported projects with neg- were consolidated within a new institution (SON- ligible ID impact has fallen consistently since APRA) under the first Borgou Project. The prior FY95. complex distribution of responsibilities had created Percent satisfactory 90 - - - - - - 80 By disbursements 60~~~~~~~~~~~~~~~~~~~B project 60 50 I I I I I I I i 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Exit fiscal year 8 TRENDS AND LESSONS IN PROJECT PERFORMANCE an inefficient, high-cost operation. SONAPRA was ina Faso supported the training and visit system a substantive step toward institutional simplifica- of agricultural extension, which is character- tion and efficient restructuring of the sector. ized by a rigid, top-down management style, Many ID projects fail because they are poorly without considering two other types of exten- adapted to the local environment. Some are sion and adaptive re- overly complex. The Ghana Economic Man- search already in place. Sustainability has agement Support Project, for example, aimed to The result was dispersion enhance analytical and administrative capacity of scarce administrative improvedfrom 48 through improved economic statistics, a com- and budget resources percent likely puterized budget and accounting system, and and a mismatch between sustainability in civil service reforms. But the wide-ranging ob- project activities and FY95 98 it t 57 jectives of the project and the lack of political farmers' needs. Better and managerial support presented a formidable customization of ID de- percent in FY99-00. management task that exceeded the supervisory sign holds the key to capacity of both the implementing agencies and high-quality outcomes. the International Development Association (IDA), and project achievements were limited. Sustainability Better long-term sequencing would have facil- Sustainability has improved from 48 percent likely itated cumulative learning and a gradual tran- sustainability in FY95-98 exits to 57 percent in sition to more complex operations. In other FY99-00 (figure 2.4). Gains were most pronounced cases, a program design that has succeeded in in the Latin America and South Asia Regions. The one context proves a poor fit in another. For in- share of projects likely to be sustainable declined stance, the Agricultural Services Project in Burk- in East Asia to roughly the Bankwide average, Figure 2.3 Percent 100 80- 60- 40- 20- 0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Exit fiscal year * Negligible D- Modest USubstantial 9 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS pardy as a result of the financial crisis and its af- Performance by Region, Sector, termath. Banking systems have high levels of non- and Instrument performing loans and there is a large share of insolvent firms. This poses continued risk to the Regional Performance development benefits of Outcomes have improved across most Regions The Aggregate lending to the East Asia (figure 2.6). The Europe and Central Asia Region Region (World Bank (ECA) is again the top performer, with 84 per- Project Performance 2000b). cent satisfactory project outcomes. The Latin Index has shown America and Caribbean Region (LCR) follows consistent gains, both Aggregate Project with nearly 80 percent, while the East Asia and Performance Index Pacific Region (EAP) and the South Asia Region by projects and by The Aggregate Project (SAR) both reached 79 percent satisfactory. EAP disbursements. Performance Index (APPI) is the only Region with performance for FY99-00 combines information exits below its FY90-94 average. The Africa Re- on outcome, sustainability, and ID impact gion (AFR) is again the lowest performer in all into a single index (figure 2.5). Since the three dimensions. While performance in all other FY94 exit cohort, the APPI has shown con- Regions has consistently averaged 75 percent sat- sistent gains, both by projects and by dis- isfactory in the 1990s, AFR has averaged 56 per- bursements. This positive trend reflects overall cent. But QAG data show improvement in the project improvement, with the slight decline active FY00 AFR portfolio to near the Bankwide in the outcomes of FY98 and FY99 exits off- average in projects at risk. The gains suggested set by increases in ID impact and sustain- by QAG data are largest in Kenya, Senegal, and ability.3 Uganda. Of the sectors, improvements are most Percent 100 . 80- 60- 40- 20- 0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Exit fiscal year * Unlikely E Uncertain * Likely or better 1 0 TRENDS AND LESSONS IN PROJECT PERFORMANCE Trends in the Aggregate Project Fi|g ur e 2 .5 Performance Index (APPI) APPI average 9.00 8.00 7.00 ~ - By disbursements - - : -- 6.00 .. .... _ _____ By projects,. ___ . 5.00 I I I I 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Exit fiscal year ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. .. ................... . .._ . .... Fig u re 2 .6 Regional Trends in Satisfactory Operations, by Projects Percent satisfactory * Exit FY 90-94 100 - Exit FY 95-98 * Exit FY 99-00 80 -7 60 40 20 0 ECA LCR EAP SAR MNA AFR Bankwide noticeable in water; power; and health, nutrition, operations in Russia (Oil Rehabilitation I and II) and population.4 are responsible for roughly two-thirds of overall Weighted by disbursements, LCR and EAP con- unsatisfactory net disbursements (box 2.2). tinue as top performers, with over 88 percent sat- Improvements in sustainability and ID impact isfactory. ECA's outcome performance decreased were shared broadly among Regions. Between to 74 percent; two recently evaluated investment FY95-98 and FY99-00, the share of projects rated 11 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS likely to be sustained increased by more than 10 enhancements. QAG data suggest that future percentage points in LCR and SAR, and by 8 performance in PSM appears likely to remain percentage points in AFR. In ID impact, there was above average, but not at such an exception- a 9 point increase in the Bankwide average for ally high level. Performance declined most for substantial or better ID agriculture, multisector, and extractive industry- Adjustment performance between related projects. Water sector performance re- J FY95-98 and FY99-00, mains weak, although the preliminary FY00 lending outcomes from 34 to 43 percent. data suggest an upward trend in satisfactory have remained Key gains were reflected outcomes. This is supported by QAG data on strong since the in the MNA and SAR Re- the active water sector projects. On the down gions. Preliminary data for side, QAG data shows that finance is still a mid-1990s, rising the FY00 exits suggest fir- problem area-22 percent of active projects are from 68 percent ther improvements across at risk, the second-highest percentage among satisfactory in Regions in the near term, all Bank sectors. FY90-94 to 86 with fully half of the sample receiving sub- Adjustment Lending Performance percentfor stantial or higher ID rat- Adjustment lending outcomes have remained FY99-00 exits. ings Bankwide. strong since the mid-1990s, rising from 68 per- cent satisfactory in FY90-94 to 86 percent for Sectoral Performance FY99-00 exits. Weighted by disbursements, sat- There are large variations in sector outcomes isfactory outcomes rise to 97 percent, although between FY95-98 and FY99-00 (figure 2.7). Im- heavily influenced by several large crisis loans. provements in outcomes between these periods But the numbers need to be interpreted with are particularly strong in public sector man- caution (box 2.3). The improved adjustment op- agement (PSM), with a rise of 20 percentage eration outcomes reflect greater success in points in satisfactory outcomes (to 94 percent), achieving relevant stated objectives. These typ- and urban development, which gained 15 per- ically take the form of intermediate outputs, centage points (to 80 percent). The PSM gains such as policy reforms adopted, legislation ap- appear to reflect a narrower focus, with move- proved, or privatizations achieved. The adjust- ment toward public finance objectives, and ment reform agenda has shifted from an away from more general policy and planning emphasis on macroeconomic stabilization to Since Russia became a member of the World Bank Group, OED verely curtailed the scope and financial viability of the projocs i has evaluated and rated the outcome of six operations, five of Returns were also affected by the 1998 fall in international oil , these were completely disbursed before the 1998 financial cri- prices. Implementation Completion Reports and Evaluation Sum- sis. Three of the six, representing 38 percent of disbursements, mariesfor both projects found they had unsatisfactory outcomes, received unsatisfactory performance ratings. unlikely sustainability, and modest ID impact. Oil Rehabilitation I (13623) and 11 (13768), totaling $1.1 billion, The August 1998 financial crisis in Russiafurther exacerbated were the largest investment operations approved by the Bank dur- implementation constraints throughout FY00. At the end of FY98, ing 1993 and 1994 respectively. Their objectives were to slow the 32 percent of the Bank's projects were at risk, and 18 percent decline in oil production while fostering private sector invest- were considered problem projects. Two year later, by the end ment in the upstream oil sector through legal, tax, price, and in- of FY00, 39 percent of the Bank's active projects in Russia were stitutional reform. Implementation delays in the technical at risk, and 29 percent were considered problem projects. One assistance components, together with 1995 tax changes, se- of every two dollars committed was at risk. 12 TRENDS AND LESSONS IN PROJECT PERFORMANCE Fig u re 2.7 Sectoral Trends in Satisfactory F p e r a t i o n s Percent satisfactory * Exit FY 95-98 100 _ Exit FY 99-00 80 60 40 20 0 . 1 141 A |Y '1 1 | Note: Only sectors with at least 10 exiting projects in FY99-00 are included. Assessing Adjustment Lending: Box 2.3 Timing Matters Reliable rating of adjustment lending is difficult, especially countrys large conglomerates (chaebols) have neither been re- when the evaluation is done soon after disbursements are com- structured nor sold. pleted. Recent experience with Bank londing to East Asia cri- Indonesia obtained $21 billion in three SALs (Policy Reform sis countries confirms thatthe momentum of reform triggered by Support Loan I, $1 billion, in 1998; Policy Reform Support Loan an adjustment loan may not be durable. The Bank responded 11, $500 million, and the Social Safety NetAdjustment Loan, $600 quickly to the 1997 financial crisis in East Asia, along with the million, in mid-1999). OED has rated only the first of these IMF. The crisis loans" to the Republic of Korea and Indonesia loans and found the outcome satisfactory but with uncertain have since closed, and over the past two years OED has re- sustainability. While loan conditions were met, some of the viewed the completion reports. Although the worst of the crisis measures have not made a difference. Despite new bank- has passed and output in both countries began to increase, their ruptcy laws, for example, there are few bankruptcy filings in economic structure has proved more difficult to change. Indonesia despite the large unpaid claims. The blanket pro- Korea's three loans ($3 billion Economic Reconstruction Loan tection of banking depositors has doubled the public debt. approved in 1997, and SAL I and 11, of $2 billion each, in 1998) were Even if these debts are sustainable, the major redistribution evaluated as having satisfactory outcomes, gains likelyto be sus- of wealth may adversely affect the poor through the future tax tainable, and moderate to substantial ID. Although Korea's eco- burden. nomic decline was halted and growth has resumed, the bulk of Work is underway in OED to augment and refine approaches unserviceable domestic bank loans remains unresolved. The to assessing adjustment lending. 13 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS reforming public sector management and the or safety nets, and the impact of reforms on the financial and private sectors, medium-term in- poorest is rarely considered. This partly reflects stitution building, and ameliorating social im- difficulties in assessing tradeoffs among the macro pacts. Among adjustment instruments, a larger and micro implications of adjustment programs share of sectoral adjustment loans (SECALs) (Evans 2000) and highlights the importance of en- are rated satisfactory than are structural ad- hanced poverty focus in the evaluation of both justment loans (72 percent) for FY99-00 exit- adjustment and investment lending. ing operations.5 Over the past decade, SECAL performance has improved fairly steadily, while Risks and Rewards of Operations SAL performance has been more variable. To manage development risk, the Bank should Several factors explain this improved adjust- consider the expected rewards of various types of ment performance. First, the Bank has incorpo- operations and the likelihood, or risk, that the ex- rated lessons learned from previous adjustment pected development returns will not materialize experience, including increased attention to bor- as intended. Past operational results can be a use- rower ownership, fewer conditions, and an fuil indicator of the risks and rewards of projects increased number of single-tranche operations- in different Regions and sectors. Figure 2.8 shows which disburse based on actions taken rather than the rewards (as measured by the average APPI) future promises. Second, adjustment lending has and the risks (as measured by the APPI's standard shifted toward better-performing countries. Over deviation) of the evaluated FY99-00 exiting a quarter of the improvement in adjustment lend- projects. ing performance is explained by the lower share The figure allows comparison of the average of evaluated adjustment operations in the Africa project results and the dispersion of results around Region, which has ranked last among Regions in the corresponding Bankwide values. This relative adjustment lending performance. The Africa Re- risk-reward framework illustrates potential weak- gion's share of evaluated adjustment lending de- nesses and strengths of Bank interventions in clined from 44 percent in the FY90-94 period to terms of specific loan and credit portfolios. only 20 percent for FY99-00, with an increasing The figure highlights with arrows the sectors and share exiting from the ECA Region. Third, over- Regions whose risks and rewards changed sig- all borrower performance improved, particularly nificantly between FY95-98 and FY99-00. Project among upper-middle-income countries. rewards improved significantly in the transport, so- Adjustment lending has improved achieve- cial, and PSM sectors. The average rewards of fi- ment of the direct objectives of macro or sectoral nancial sector projects have not significantly reform, but less progress has been made in inte- changed, but their variation has been reduced grating poverty concerns. OED's 1999 poverty from very high levels in FY95-98 to average lev- study highlighted the need to take account of the els in FY99-00. Regionally, ECA and MNA have effects of adjustment lending on the poor and the achieved significant declines in risk, while SAR per- environment. Although adjustment lending is formance has moved to higher reward but also often criticized for reducing social expenditures, higher risk. These movements in the relative risk- most of the loans reviewed successfully reward framework can help sector and country protected-and sometimes increased-social sec- managers identify priorities for improving portfo- tor spending (Evans 2000; Hassan 1999). But they lio management as well as providing a tool for en- were less successful in reorienting spending within hancing selectivity. Risk and reward at the project the social sectors to priority areas (primary health, level should be complemented by recognition of basic education) or to an improved focus on re- tradeoffs at the country and sector levels (see sults of social spending. Although a number of op- Chapters 3 and 4). erations included safety net provisions, the record of success was mixed. For most adjustment op- Determinants of Project Success erations, the focus on public expenditures out- The performance of evaluated projects can pro- weighs that on addressing economic distortions vide lessons for the implementation of the Bank's 14 TRENDS AND LESSONS IN PROJECT PERFORMANCE Figure 2.8 Risks and Rew FY99-OO and -3 Risk 0.6 -O Selected FY 95-98 0.5 - Finance * FY 99-00 Electric power & other energy S 0.4 0.3- 0.2- Water supply Urban development 0.1 - -1.0 HNP * S 1.0 0.0- - _____Multisecto Reward 0 0 Agriculture -01 Finance Extract. ind Multisector Education Social sector -0.2- \ PSM * Multisector L Social sector Transpo rtation * Transportation -0.4 - Environment Risk 0.3 SAR 0.3 0.2 0.2 - AFR ECA 0.1 0.1 -1.0 MNA 1.0 0.0 Reward .1 ~~~~ECA 0EAP LC -.1 SAR N -0.2 -0.2 Note. The axes measure differences relative to Bankwide averages. Hence, each quadrant, starting from the upper-left and moving clockwise, corresponds to one of the four classes: higher risk-lower reward, higher risk-higher reward, lower risk-higher reward, and lower risk-lower reward. Reward is represented by the average Aggregate Policy Performance Index (APPI) for the sector or region. Risk is proxied by the standard deviation of the APPI. 1 5 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS approximately 1,600 active projects, as well as institutional quality for borrower countries.7 A the design of new projects. This section updates particular value of the current exercise is the in- the model of investment-project outcome clusion of project-specific performance assess- determinants used in previous ARDEs. It uses the ments, testing for their significance while project as the unit of controlling for the country's socioeconomic en- Sound macroeconomic account and includes vironment.8 For evaluated FY99-00 exits, Bank policy and higher-quality three main groups of ex- effort in bringing the project to approval and dur- institutions increase the planatory factors-Bank ing supervision are significant; the effort during performance, borrower supervision shows greater impact on the prob- effectiveness ofpublic performance, and coun- ability of satisfactory outcome for recent evalu- sector investments. try factors exogenous to ations relative to FY95-98 exits. Quality at entry the project-while con- has the greatest effect on the likelihood of proj- trolling for other characteristics, such as size, sec- ect success for FY99-00 exits. Borrower imple- tor, and location.6 mentation performance and compliance with Updated econometric testing confirms and legal covenants are also significant determinants extends the results of past ARDEs. The country of project success. Borrower implementation environment has a significant impact on project performance has nearly the same effect as Bank outcomes. Two dimensions of country conditions supervision, followed by compliance. are modeled: macroeconomic policy perform- ance (proxied by average growth during project Country Factors implementation) and Bank staff assessments of Country conditions are critical to the success of projects, as figure 2.9 illustrates. Countries are classified as high or low for both macroeco- _p5~pflUUppE,Upt~UinS~IE.UIpp' a.; jm ~ nomic policy and institutional quality, and port- -*XX ,X .- - , .folio performance is the average percentage of satisfactory projects across countries in each of Satisfactory portfolio performance the four groups. Both sound macroeconomic (percent) policy and higher-quality institutions increase the effectiveness of public sector investments, with institutional quality slightly more influen- tial than sound macro policy. Project-level econometric testing confirms this 100 analysis. Both macroeconomic policy performance 90 and institutional quality are significant explana- 80 // 00000 | | tory variables for project outcomes. While con- 70 trolling for aggregate macro performance and country policies (proxied by the CPIA), the im- 60 portance of debt servicing on project perform- 50 ance was explored, confirming the hypothesis 40 that larger debt service payments are associated 30 / / i401 | W with poor investment-project performance. A 20 High one percentage point increase in the debt serv- 20 | /Institutional ice as a share of GDP is associated with more 10 - Low quality than a two percentage point decline in likely sat- 0 L isfactory outcome. Care must be taken in draw- High Low ing causal inferences, but the findings suggest Policy performance that debt relief under HIPC could make a mod- est contribution to improved project performance in the long term. 16 TRENDS AND LESSONS IN PROJECT PERFORMANCE Bank Performance formance in Africa and that in the other Re- The determinants model also confirms the influ- gions has widened over the last two years, from ence of Bank performance on project outcomes. 10 percentage points for FY96-98 exits (67 per- Satisfactory quality at Board approval is associated cent satisfactory for Africa compared with 77 per- with a 33 percentage point greater probability of cent for other Regions) to 24 percentage points satisfactory outcome than projects with unsatis- in FY99-00. Overall Bank performance in Africa factory quality at entry. Satisfactory Bank super- has fallen to 58 percent satisfactory, while the av- vision had a 21 percentage point marginal effect erage for the other Regions has improved to 82 on project outcome. Overall Bank performance av- percent. The pattern is the same for disburse- eraged 75 percent satisfactory for evaluated ment-weighted results. But recent QAG results FY99-00 exits, remaining effectively unchanged suggest that ongoing Africa Region efforts to from 74 percent for evaluated FY96-98 exits. improve portfolio quality could yield substantial Weighted by disbursements, overall Bank per- improvements. formance shows more improvement over the same time periods, rising to 83 percent from 78 Borrower Project Performance percent satisfactory. Preliminary FY00 data show Seventy percent of FY99-00 exits had satisfac- the significant upturn of quality at entry associated tory overall borrower project performance, near with the recent improvements in project out- the average for FY96-98 exits. Weighted by dis- comes, from roughly 60 to 70 percent satisfactory bursements, performance improved over the between FY93-96 and FY97. Self-evaluation as- same time periods, rising from 75 to 80 percent sessments on the quality at entry of the active port- satisfactory. As with Bank performance, prelim- folio suggest that future exits will build on past inary FY00 data signal significant improvement. performance progress-nearly 9 of 10 projects Borrower project performance comprises three assessed by QAG from the FY99 approval cohort dimensions-the quality of the borrower's con- were rated satisfactory or better. While satisfactory tribution to preparation, implementation, and Bank supervision has stagnated at 76 percent sat- compliance with legal covenants. The quality of isfactory (FY96-98 to FY99-00), the preliminary re- preparation and compliance remained unchanged, sults for FY00 exits suggest an improvement. at 80 percent and 74 percent satisfactory respec- Supervision for 85 percent of FY00 exits was sat- tively, between FY96-98 and FY99-00. Borrower isfactory. If maintained for the entire FY00 exit co- compliance for the partial FY00 sample notched hort, this would be the first exit year group to top down, with only 72 percent of the projects rated 80 percent satisfactory in supervision. Gains for the satisfactory. Satisfactory borrower implementa- partial FY00 sample were made in all Regions. tion has remained between 59 and 63 percent These results are consistent with the FY99 qual- since FY93 and continues to be the poorest per- ity of supervision assessments by QAG, which re- forming dimension of borrower performance. ported 84 percent satisfactory performance for The stagnation in borrower implementation per- the sample reviewed. formance underscores the need for capacity build- Despite an improvement in the evaluated ing, and to further adaptation of project design FYOO exits in Africa, the gap between Bank per- and Bank procedures to borrower capacity. 1 7 ADAPTING TO THE COUNTRY CONTEXT In recent years, the Bank has sought to shift its focus from individual projects to the "higher plane" of country programs. This chapter uses OED Country Assistance Evaluations (CAEs), project evaluation data, and other evidence to assess the relevance and efficacy of Country As- sistance Strategies (CASs). The two most important determinants of coun- try program outcomes are the level of borrower commitment to the country program and the extent to which Bank strategies, programs, processes, and partnerships are adapted to the country setting. Lending operations remain the basic building blocks of country programs, but Bank nonlending services also influence the effectiveness of lending and over- all program outcomes. The Bank is exercising more selectivity by relat- ing lending volumes to country performance ratings. But continued lending despite poor performance contributes to unsatisfactory outcomes. Project-level performance is an important indi- country program, as illustrated by table 3.1. For cator of the Bank's contribution at the country 15 recently completed CAEs, only 7 lie on the level, but it tells only part of the story. The Bank diagonal from the upper-left to the lower right also provides nonlending services and interacts of the table.2 Thus, ratings of country program with the borrowing country and its partners to outcomes can differ from those of the aggregate overcome structural and social obstacles to de- country portfolio performance. For example, velopment. OED therefore supplements its proj- program outcomes were fully satisfactory in ect evaluations with CAEs, which assess the Ghana and Uganda, where portfolio performance overall effectiveness of the Bank's country pro- was acceptable but not outstanding. Conversely, grams, usually over 5-10 years.1 Project portfo- all projects completed during the 1990s were lio performance tends to be positively but weakly rated satisfactory in Costa Rica, but OED rated associated with OED's overall rating of the Bank's the Bank's program outcome unsatisfactory.3 19 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS e 0 * 0 _ lll_ S l _XEfi_fi Il_ Country program outcome Portfolio performance Partially (% projects satisfactory) Fully satisfactory satisfactory Marginal Unsatisfactory >85 | . Kazakhstan (1991-99) Costa Rica (1990-00) U0ruuy (1990- Burkina Faso (1989-99) 70-85 Mexico (1989-99) Indonesia (1993-98) Paraguay (1990-00) Tanzania (1996-00) 50-69 Ghana(1995-00) Egypt (1990-00) TaIania (198-. Uganda (1986-99) India (1990-00) <50 Cameroon (1996-00) Cameroon (1982-95) Papua New Guinea (1989-00) Note: Income: bold=upper-middle; standard=lower-middle; italic=low. Evaluation and research results suggest that What Influences Country Program development effectiveness at the country level Outcomes? depends not only on the outcome of individual A country strategy can be relevant and the coun- projects but also on the relevance of the Bank's try program outcome satisfactory even for country strategy; the effectiveness of nonlending countries with poor policies or weak institu- services, including the synergies between lending tions. What is important is the extent to which and nonlending activities; the efficacy of partner- strategy, instrument mix, and efficacy of pro- ships; and the overall quality of country policies, gram implementation-including partnerships- institutions, and governance. External shocks and respond to country conditions and move the initial conditions (including the extent of indebt- country forward. As table 3.1 shows, country edness) also play a role. Finally, political instabil- program outcomes are not strongly associated ity or conflict can undermine program outcomes. with country income. More important to the suc- At the corporate level, development effectiveness cess of country strategies are the government's is influenced by the policy framework and by the commitment to development and implementation allocation of lending and nonlending resources of the objectives selected for the Bank-spon- among countries. To assess development effec- sored program and how well the Bank's strategy tiveness at the country level, this chapter asks and program address the country's most impor- four questions: tant development constraints. OED reviews of * What factors influence the relevance and out- completed CAEs, including a detailed assess- come of Bank country programs? ment of 15 recently completed CAEs,4 identify sev- * What is the role of partnerships at the coun- eral key determinants linking program outcomes try level? with Bank performance. * Does the Bank's choice and mix of instru- ments match the strategic objectives and Diagnosis country context? The Bank's diagnosis of major development is- * Does the Bank's allocation of lending and sues and constraints-what needs to be done- administrative resources among countries con- is generally good, averaging fully satisfactory tribute to improved development effectiveness? across the sample of 15 CAEs. But the quality of The chapter concludes by drawing implica- diagnosis varies by issue and sector. Diagnosis tions for the Bank's role and effectiveness in dif- of macroeconomic issues and constraints is usu- fering country contexts. ally strong, but the links between stabilization and 20 ADAPTING TO THE COUNTRY CONTEXT growth are not always well articulated. The qual- Balancing Corporate and Client Priorities ity and coverage of ESW explain much of the vari- The Bank has improved its client focus. But client ation in the diagnosis within and among countries. ownership does not eliminate divergence be- For example, the relevance and outcome of the tween Bank and borrower priorities, or contrary Bank's program in Kazakhstan was less than sat- views regarding the "right" isfactory because of inadequate up-front sector policies (Williamson 1999). Two-tbirds of FY99 CASs work-leading to a less than satisfactory program OED evaluators identi- reviewed were rated outcome despite good project performance. Con- fied inadequate borrower satisfactory or beterfor versely, high-quality and timely ESW in Uganda commitment as the most has strengthened the relevance of Bank pro- important reason for poor treatment of poverty. grams and contributed to a fully satisfactory pro- policy or project imple- gram outcome, despite a mixed investment mentation (box 3.1). Differences may arise about project performance (see table 3.1). The overall overarching priorities-such as poverty reduction impact of public expenditure reviews and poverty or improved governance-or specific policies assessments is strongly correlated with their qual- and programs.6 OED country evaluations consis- ity-as well as the degree of local involvement, tently emphasize the level of country consensus consultation, and timeliness (OED 1999a). about reform as critical to program outcome (Costa Rica, Kazakhstan, Paraguay). Even in coun- Poverty Focus tries with good macroeconomic frameworks and CAEs, the recently completed OED poverty institutions, structural constraints remain because study, and the Bank's 2000 CAS retrospective changing them is politically or institutionally dif- confirm that the Bank has strengthened the ficult, or because stakeholders have not agreed poverty orientation of its country programs. on how they should be changed. CAEs point to Economic growth-a necessary if not suffi- numerous examples of Bank lending to promote cient condition for poverty reduction-is at reforms-either through adjustment or invest- the center of nearly all country strategies, and ment lending-before enough consensus had most emphasize provision of basic social serv- emerged (Cameroon, Costa Rica, Papua New ices (World Bank 2000a; Evans 2000). The 2000 Guinea, Paraguay). While in some countries the CAS retrospective reported that two-thirds of lack of consensus reflected severe social tensions FY99 CASs reviewed were rated satisfactory or and governance weaknesses (Cameroon, Papua better for treatment of poverty, with significant New Guinea), in others it was the consequence improvements since FY96. But the linkages of democratic institutions based on a strong tra- between Bank country programs and poverty dition of gradual consensus building (Costa Rica, outcomes were not always well articulated. Uruguay). OED analysis and emerging research inside and outside the Bank-including the recent Assessing Ownership World Development Repori-suggest that equity Priorities are likely to differ among ministries and affects the poverty impact of growth, and that agencies within governments, between govern- safety nets and empowerment for the poor ment and civil society, and among different deserve increased attention (World Bank groups and civil society. The Bank and govern- 2000a). More needs to be done to integrate ment may agree on broad objectives but disagree broad-based poverty reduction strategies into regarding the measures and timetables to achieve macroeconomic and sectoral strategies and in- them (Costa Rica, India, Mexico). Concrete ac- terventions.5 Inadequate attention by the Bank tions and track records are better indicators of and other donors to poverty monitoring and ownership than statements of intent. But the evaluation has meant that significant knowledge ownership of policy reforms or programs can be gaps persist regarding the linkages between assessed based on the degree of intellectual policies and programs commonly supported by consensus among decisionmakers, the political the Bank and poverty outcomes (Evans 2000). commitment of national leaders, the locus of pol- 21 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS The client and the Bank may have different values or priorities Even where governments share the Bank's overall objec- about the country's development: tives and priorities, they may not necessarily wish to undertake * In Kazakhstan and some large states in India, for example, the the kind of lending operations the Bank proposes, or may wish authorities did not place high priority on poverty reduction. to forestall the Bank's involvement in sensitive areas. *Client eountries may resist Bank involvement in such pa- In some cases, the Bank's desire to sustain lending has moved litically sensitive issues as corruption. Indonesian author- proposed loans to an advanced stage of preparation despite lim- ities refused to address widespread corruption, and the ited government interest While cancellation of nonperforming Bank did not raise the issue forcefully until after the 1998 loans is part of good portfolio management a high incidence of crisis. cancelled loans may indicate weak borrower commitnent" and * In Paraguay, despite the vulnerable and undercapitalized contribute to higher average costs. Cancelled loans are more bankingsystem, authoritieshavenotbeen interested in struec- common in countries with access to aternative sources of fi- tural reform. nancing. In Egypt. for example, over a five-year period more Bank - Gender equality is a Bank priority that does not spark the in- resources were spent on preparing projects that were eventually terest of all governments. dropped than on projects that were eventually approved. .,____.._____.. ..... ....-- - ----. .... ...... .......... ................... . . .. _ ---- -- ---- . . . __.. ..... .... .... ......... ........ .. .. ... ..... .- ........... -- - icy development (for example, local or external), ect documents. These efforts are starting to and the participation of major stakeholders (John- show at the country level, but further progress son and Wasty 1993). Instruments for assessing is needed to articulate comprehensive institu- ownership, including stakeholder analysis, are tional development and governance strategies coming into wider use (World Bank 1999), but (Burkina Faso, Kazakhstan, Tanzania). they are still not systematically mainstreamed into project and program preparation. Over-Optimism The Burkina Faso, Costa Rica, India, Indonesia, Institutional Development Paraguay, and Uruguay (in the earlier years) The Bank's contribution to institutional devel- CAEs found a pattern of over-optimism about bor- opment at the country level has been modest, rower governments' receptivity to Bank advice, both in middle- and low-income countries. This willingness to undertake difficult reforms, and ca- reflects the inherent difficulties of institutional pacity to implement recommended measures. change and capacity building, as well as short- Over-optimism is largely the result of inadequate comings in the Bank's approach. The Bank's di- risk analysis, or reluctance to face up to risks in agnosis of institutional, political, and governance program design. The Bank often underestimates constraints generally has been weaker than its the political difficulties and the obstacles to rapid economic and technical diagnosis. This reflects change of long-established socioeconomic sys- both the professional orientation of Bank staff- tems, especially in transition economies (Kaza- mostly economists and sector specialists-and khstan).7 Risk analysis was rated as the weakest the Bank's traditional reluctance to address aspect of CAE diagnosis; reviews by QAG have these issues. Recently, the Bank has displayed also called for increased attention to risk analy- increased attention to governance and institu- sis (QAG 2000). Some of the reasons for chronic tional issues in its policy research and country optimism are listed in box 3.2. strategies (World Bank 1997a), and is piloting Institutional Governance Reviews in several Adaptation countries. QAG data suggest some improve- The Bank does not always adapt its programs ments in quality of institutional analysis in proj- to the prevailing context, especially in low- 22 ADAPTING TO THE COUNTRY CONTEXT .... ........... ....... .. _ Box 3 . 2 Sources of Over-Optimism Some of the reasons for excessive optimism are identiied in CAEs: sequently incorporated these lessons into its country strategy. * Inadequatetoolsforriskanalysiscontributetoatendencytoun- * There may be a perceived need to justify new loans to main- derestimate risks and to overestimate the influence of pro- tain a positveflow of net disbursementsto countules with large posed risk-mitigaton measures (Indonesia, Papua New Guineal. repayment obligadons on existing loans (Argentina, Mexico). * A bhalo effect"from previous good perfonmance can cloud per- * Outside observers-including Bank staff-may not fully ap- ceptions about emerging sructural problems lndonesia, Mex- preciate the political difficulties and social costs of deep ico). In Indonesia, warnings by Bank staff regarding changes in transition economies (Kazakhstan, Ukraine). weaknesses in the financial sector were not adequatety con- * Proposed timetables for project preparation or policy imple- veyed by Bank managementin county dialogue, patly because mientation may not account for delays and complications that of the country's good macroeconomic record. The Bank sub- arise in the course of consultative and participatory processes. income countries with weak implementation Selectivity capacity. Project design and recommended At the country level, selectivity requires iden- policy reforms may be too complex and in- tifying interventions that are consistent with adequately adapted to country capacity. the Bank's comparative advantage relative to Uneven ESW and insufficient engagement with partners. The CAE review, as well as the local counterparts can contribute to promul- Bank's recent CAS retrospective, suggest that gation of standard policy prescriptions. The the Bank is getting better at establishing pri- OED poverty study found "surprisingly limited orities for country programs, but that there is differentiation in the policy agenda across room for improvement. The Bank success- CASs over the past decade," although it noted fully exercised selectivity in Argentina, promising new signs of diversity, particularly Uganda, and Uruguay. In Argentina, for ex- in ECA (Evans 2000).8 ample, the Bank's program has evolved in response to changing development needs, Strategy and Program Deficiencies from public sector reform and privatization to Good, well-executed country strategies pro- provincial reform and social sector issues- duce good results (Argentina, Uganda). But issues that attract limited private capital. But about half of the 15 CAEs reviewed pointed to in a number of low-income countries, in- some deficiencies in the CAS. The most com- cluding Bolivia, Burkina Faso, and Tanza- mon shortcomings were excessive focus on nia-where many donors are active-the Bank macroeconomic issues; neglecting nonlending and donors are involved in too many sectors services; avoiding complex, sensitive gover- and have not sufficiently established priorities. nance issues; and overestimating borrower Donor overload is a particular problem for ownership and implementation capacity. Con- "popular" sectors, such as education, health, versely, if the CAS was sound but the subse- and agriculture.9 Justifying the Bank's presence quent program deviated from it (Morocco), the in many sectors by its convening role and ca- results were generally disappointing. But when pacity for policy analysis may underestimate the CAS is flawed (Jamaica), or if country cir- the capacity of other actors or take into ac- cumstances change significantly (Indonesia), count the need to use Bank administrative re- deviating from the CAS may be desirable. This sources selectively to achieve results. The suggests that flexibility in CAS implementation Tanzania CAE, among others, suggests that the is necessary, but also that significant changes Bank should work with UNDP to move the should be clearly justified. donor selectivity process forward. 23 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Do Instruments Match Objectives? above-average performance ratings, but only The choice and mix of instruments should be half are satisfactory in countries with lower rat- matched to the specific development objectives, ings. Investment loans fare slightly worse, which the level of consensus for reform, and the reflects the well-known negative influence of country's policy, institutional, and political con- weak policy environments for investments, as text.10 But evidence from well as inadequate adaptation of project design Country strategies CAEs and the CAS retro- and Bank processes to weak contexts. SECALs rarely state the spective suggest that perform well in both high- and medium-rated country strategies rarely borrower countries, but the worst for low-rated strategic rationale for state the strategic ration- countries-possibly reflecting weaker sectoral choosing one ale for choosing one in- capacity and the complexity of many sectoral instrument over strument over another. reforms in poor performers. SALs seek changes in macroeconomic policies, but evaluation ev- another Lending Instruments and idence suggests that "stroke of the pen" changes Country Context can be implemented even in weak institutional On average, all lending instruments perform bet- environments if key senior officials are ter in countries with better policy and institu- committed (Killick, Gunatilaka, and Marr 1998; tional environments-as measured by the Nelson and Eglinton 1993). Bank's Country Policy and Institutional As- sessment (CPIA) ratings." But the sensitivity ESW and Country Context varies by instrument (figure 3.1). Technical as- The Bank has begun to rely on borrowers' in- sistance loans, which support targeted capac- stitutions to carry out analytical work, particu- ity building, perform well in countries with larly for large borrowers and middle-income Percent satisfactory outcomes 100 _ 90 1 80 70- 60- 50- 40- 30- 120 14 10 - -- Emergency & Structual Technical Investment Sectoral rehabilitation adjustment assistance adjustment * High CPIA El Medium CPIA * Low CPIA 24 ADAPTING TO THE COUNTRY CONTEXT countries. This was done effectively in Argentina, Investment Lending India, and Mexico, which have skilled govern- Investment lending is less likely to attract ment officials, reputable think tanks, and well- controversy. It can allow experimentation and trained social scientists. The Bank can joint problem solving for specific development complement and expand on local findings with constraints and is more relevant international experience and bring an appropriate for targeted The Bank has begun to independent perspective to country analysis. capacity building. Some rely on borrowers' But the same problems that influence lending borrowers-such as China institutions to carry out in weak institutional and policy environments and India-therefore pre- can have a negative impact on analytic work. fer investment loans. But analytical work. According to QAG data, quality ratings for ESW investment projects are are highest in countries with high CPIA ratings, more likely to be hampered by shortfalls in but less than 60 percent satisfactory for low counterpart funds or weak public sectors. Par- CPIA countries. The average cost per ESW task ticularly in middle-income countries or large is also higher in weaker performers. This sug- borrowers, the wider impact of investment lend- gests the need for selectivity and a more ing depends on "spin-off' effects on other gov- differentiated approach to ESW in weaker en- ernment programs-but few investment projects vironments, including strong supervision, qual- are designed with this explicitly in mind. The pro- ity assurance, and striking an appropriate curement and other due diligence requirements balance between analytic quality and the local associated with investment lending can help involvement needed to improve the relevance strengthen borrower accountability and capac- and ownership of findings. ity, but are not always well adapted to borrower procedures. Adjustment Lending Adjustment lending can support policy changes Instrument Mix and reforms at the macroeconomic or sectoral Combining adjustment lending with other in- levels, but only when a critical mass of stake- struments-such as analytic and advisory services, holders become convinced of the need and technical assistance, and investment loans-may direction for reform.'2 Conditionality can be improve the policy environment, and hence out- effective as an instrument of mutual commit- comes. Complementary investments to build ment, shared objectives, and reciprocal obli- capacity, improve service delivery, or develop gations, but not as a coercive tool (Branson and supportive infrastructure often are necessary to Hanna 2000). Cash-strapped poor countries achieve the benefits of policy change.'3 Adapt- may be coaxed into accepting externally im- able program lending may be appropriate for posed conditions, but it is likely that such poli- long-term public sector reform, depending on au- cies will be poorly implemented or subsequently thorities' commitment to reform and the quality reversed (Killick, Gunatilaka, and Marr 1998; of the trigger points and evaluation benchmarks Devarajan, Dollar, and Hohmgren 1999). Ad- selected. In Tanzania, for example, a new 12-year justment lending has traditionally supported programmatic public sector reform credit repre- "stroke of the pen" reforms involving limited sents an encouraging departure from previous public consultation, but is increasingly being piecemeal efforts, although the CAE expresses used to support more complex "second- concern that the triggers are poorly defined. generation" reforms, involving many stake- holders. Bank efforts to promote civil service Building Consensus reform through adjustment have been mostly When consensus is weak, adjustment is unlikely unsuccessful (Girishankar 1999), but the Bank to be effective. Pilot projects can help test pos- is beginning to experiment with more flexible sible approaches and build consensus-if key and consultative approaches. These efforts are stakeholders are engaged in the design process, promising (Kazakhstan, Uganda). the pilot is adequately evaluated, and the find- 25 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Recent OED CAEs for Burkina Faso and India show that the In India the government has always insisted that it remain Bank can facilitate complex reforms through careful se- fully in charge of its policies and the Bank's program. The Bank's quencing of analytic and advisory work, consensus building program initially was oriented toward supporting the govern- and policy dialogue, and lending operations. In Burkina Faso, ment's country strategy through a wide range of project loans. a World Bank Public Expenditure Review in the early 1990s The Bank eventually realized that its lending-and India's de- identified very high teacher salaries as a significant con- velopment-were being undermined by distorted macroeco- straint on the expansion of basic education. The findings fed nomic policies and inefficiencies in the public sector. In the late into discussions among the government, civil society, unions, 1980s, the Bank sponsored a series of economic and sector and donors, and government eventually negotiated a deal studiesfocusing on macroeconomic and public sector issues and with the teachers' union to "grandfather' existing teacher sought to engage government officials and Indian specialists in salaries and to hire new teachers at a reduced pay scale. a dialogue on reform. The studies helped build technical con- There were several times more applicants than positions sensus between Bank staff and country colleagues, and the fi- available. These reforms, together with support from a World nancial crisis in 1991 created political space for economic Bank basic education loan, helped increase primary school reforn. The Bank was well-positioned to provide a series of ad- enrollment. justment loans to support agreed policy changes. ings are discussed and disseminated (box 3.3). criteria, and-to a lesser extent-GDP per capita. Bank nonlending services also can be an essential The performance criteria reflects Bank staff as- component of a consensus-building strategy sessments of country performance-captured by (OED 1999a). Most CAEs rated Bank analytic and the Country Policy and Institutional Assessment advisory services as technically well done, but (CPIA) rating-and previous portfolio perform- noted that they would have had greater impact ance. Lending levels may be reduced by up to with increased country involvement and more one-third for countries with severe governance timely dissemination, which helps to build con- problems. The process allows for flexibility and sensus and promote capacity building. Of the 15 judgment in taking account of country circum- CAEs reviewed, local involvement and dissem- stances. Each CAS-whether for IDA or IBRD bor- ination were considered fully satisfactory in only rowers-specifies a "base case" level and possible three countries. range of lending, taking assessed and prospec- tive performance into account. While there is nor- How Are Resources Allocated mally no explicit linkage to "norm" allocations or among Countries? CPIA ratings in the CAS, the evidence is that, in the past decade, IDA has reduced its lending per Lending Resources capita to countries with poor performance ratings Development projects are more effective in coun- and lent more per capita to the better-rated per- tries with good basic policies, strong institutions, formers (see figure 3.2). IBRD lending is not ra- and good governance. But when poor countries tioned, and the Bank has no formal mechanisms lack good institutions and policies, the Bank for allocating IBRD lending based on country per- faces a dilemma: projects tend to be less effec- formance ratings, except as it may affect the tive, but poverty is more pervasive. How does the Bank's creditworthiness. Nevertheless, it appears Bank manage the tension between country per- that, since the early 1990s, the Bank has lent more formance and its poverty reduction mandate in per capita to IBRD borrowers with relatively allocating lending and administrative resources? good performance ratings (figure 3.3). IDA lending to low-income countries is in- Although figures 3.2 and 3.3 suggest that the fluenced by population size, country performance Bank shifted toward a greater performance ori- 26 ADAPTING TO THE COUNTRY CONTEXT entation in its lending decisions, OED country evaluations point to a number of instances Figure 3.2 Per Capita IDA Lending where the Bank has continued to lend despite by C P I A R at i n g s poor country performance-sometimes despite commitments in the CAS that the Bank would Dollars per capita reduce lending if performance did not improve. Cameroon (1982-95), Papua New Guinea (1989-00), and Tanzania (1986-95) received marginal or unsatisfactory outcome ratings be- 14 cause the Bank continued to lend despite these 14 shortcomings (see table 3.1). This can be at- 12 tributed to poor country performance, but also to the Bank's failure to reorient its country pro- 10 gram on a timely basis. The Bank must take cal- 8 culated risks when conditions seem promising, but sometimes the sheer momentum of project 6 processing, combined with over-optimism, over- whelms prudential considerations. Nimble, flex- 4 High ible management of Bank programs is important, Medium 2 but so is effective quality assurance. The Bank has taken an important first step to- 0 ward greater transparency of the performance 1992-94 1995-97 1998-00 measurement system: deciding to discuss the recent CPIA ratings with borrower governments. The ratings still are not made public. Greater transparency would encourage more rigor in performance measurement and stimulate de- P e r C a p i t a I B R D L e n d i n g bate regarding reform within member countries. F i g u r e 3 3 by CPIA Ratings it also could increase the costs of the exercise and entail adjustments in the indicators to im- Dollars per capita prove their credibility and acceptance. In federal states, performance factors at the subnational level may influence the direction of lending. In Argentina and Brazil, commitment to poverty reduction and reform at the provincial 14 level has been significant in the design of lend- ing programs. In India, the Bank has allocated 12 two-thirds of total lending to states that demon- 1 0 strate commitment to reform and poverty re- duction. Government officials and civil society 8 representatives generally supported the focus on better-performing states but recommended greater 6 transparency in the Bank's performance criteria and a more proactive stance on behalf of poor states that are weak performers (OED 2000f). 2 Allocation of Administrative Resources 0 The Bank's process for allocating administrative 1992-94 199597 1998-00 resources is not fully aligned with the drive for 27 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS more selective, performance-based lending. across country programs and to a tendency to Research for the forthcoming OED review of hoard budget resources and carry large "stocks" IDA identifies three interrelated factors that con- of lending work to ensure funding for special- tribute to misalignment: inertia, inflexibility, and ized staff. Currently, 50-80 percent over- fragmentation. programming is common 6-12 months before The Bank's process Inertia. Budget allo- the beginning of a fiscal year-compared with cations vary slowly in a norm of 25-30 percent. This is a rational re- for allocating response to changing coun- sponse by country managers faced by a budget administrative resources try conditions. Full and process separate from the CAS process with no is not fully aligned transparent costing of reliable mechanism for assuring consistency CASs and Sector Strategy between the two. Over time, the CAS and the with the drive for more Papers (SSPs) is still not budget can diverge significantly. Access to lim- selective, performance- the foundation of corpo- ited contingencies held at the institutional and based lending. rate budgeting."4 The cur- regional levels tends to be restricted to excep- rent system is governed tional situations (such as new, reactivating, or by country norms (in- post-conflict countries). As a result, the search volving such characteristics as population, per for incremental resources-for example, through capita income, and access to private capital) and trust funds-is pervasive and costly. also by normative input-output relationships related to lending plans and the size of the Are Bank Country-Level Partnerships portfolio at the time of budget construction. Strategic? CPIA ratings and portfolio performance influence In principle, partnership can be used to over- budgets only at the margin, through "overrides" come budget stress; being selective means leav- by Regional management. The Bank appears to ing some issues and sectors to other agencies. allocate more funds per capita for analytical But this assumes efficient specialization, coor- work to medium performers than to low or dination, and motivation (Picciotto 1998). Ab- high performers, which may reflect Regional sent such prerequisites, the cost of partnerships management's desire to deploy policy advice to- may add to overload and stress. Partnerships in ward potential "turnaround" countries. support of country programs flourish best when Inflexibility. High fixed costs limit country the Bank and its partners agree to a specific as- program directors' opportunities to be responsive sistance strategy, which the borrowing country to emerging development opportunities. After "owns." In Mozambique, Tanzania, and Uganda, providing for supervision, such mandated ana- for example, the Bank has established produc- lytical work as CASs, country office overhead, and tive partnerships with donors and nongovern- completion of work for loans already launched, mental organizations (NGOs). Policy frameworks little funding may be available for new analysis clearly specify each partner's role and ac- or loan preparation. Country work for demand- countability. The experience of the Strategic ing sectors and themes (such as forestry and Partnership for Africa (SPA) suggests potential gender) gets no preference in budget alloca- for more efficient partnerships and harmoniza- tions. Indeed, country directors tend to shy away tion of donor approaches. Partnerships work from risky operations likely to attract contro- well for the Bank when the Bank clearly lacks versy and burden country programs with high comparative advantage in an area. Local agen- transaction costs, even when these operations lie cies and well-selected NGOs are typically better at the core of the Bank's corporate priorities than the Bank at service delivery and inde- (such as gender, forestry, water resource devel- pendent verification of results, for example. opment, and so on). The Bank is still learning how to be a better Fragmentation. Decentralization of the partner with other agencies. Recurrent com- budget to relatively small country departments plaints about Bank staff "arrogance" are in part may be contributing to limited redeployment connected to the Bank's perceived reluctance to 28 ADAPTING TO THE COUNTRY CONTEXT Box 3.4 Moving to a New Level in Partnerships OEDWs recent study of aid coordination recommends thatthe Bank reach partnership agreements with donors that delineate nu- move closerto its enunciated policy to putthe country inthe dri- tual responsibilities and distinct accountabilities: (3) design ver's seatW (Eriksson 1999 To align Bank policy with the CDF prin- agreed coordination principles and procedures, and (4) present ciples of partnership, ownership, and results orientation, the a plan for carrying out joint monitoring and evaluation missions study suggests a new operational policy that would declare that with other donors to reduce barriers to country leadership. country leadership of the management and coordination of aid re- Succesful collaboration among donors is needed at the sources is Bank policy and that would emphasize partnership and global level. The Bank should work closely with the UNDP, collaboration among all governmental and nongovernmental ac- which shares with the Bank extensive experience in capacity tors at the country level. To bolster country capacity, each Bank building; Development Assistance Committee (DAC) membersm country team should work with government and other develop- including the EU; and regional development banks (ffDBs)to es- ment partners to fonnulate a strategy to (1 help government tablish a common understanding of, and conunitment to, build- strengthen its capacityfor aid management and coordination, (2) ing partnerships at the county level. adapt procedures to local needs (Eriksson 1999). The Bank's role is to catalyze support for se- There is still much to be done, especially in lected reforms and institutional development; adapting procurement, disbursement, and safe- facilitate cross-national learning; and promote guard policy compliance procedures toward re- global public goods (Hinds 2000). This makes sults. To be more successful at initiating and it important to establish a shared agenda with fostering partnerships, the Bank will need to the client, select the right mix of instruments, and change the way it does business and be willing establish appropriate partnerships, domestic to compensate for the weaknesses of some of and international. Additional priorities are likely its client-country partners by helping them build to include mitigating the effects of capital mar- capacity (box 3.4). Although the Bank and donors ket volatility-in partnership with the IMF- have taken steps toward harmonization of donor while seeking to minimize moral hazard procedures, progress has been slow, and some concerns associated with crisis lending. The donor governments have resisted change-some- Bank can also help catalyze private sector in- times for commercial or geopolitical reasons. vestment by supporting structural reforms to enhance the enabling environment for private Different Roles for the Bank? investment (Hinds 2000). The wide variation in This review suggests several broad conclusions income, capital market access, and institutional about matching Bank programs to borrower capacity suggest the need for differentiated ap- characteristics. proaches among middle-income countries. In re- sponse, Bank management has established a Middle-Income Countries Middle-Income Country Task Force to explore In middle-income countries, particularly those these issues in consultation with clients and with access to private capital, the Bank's fi- partners. nancial contribution is relatively small. Lending trends suggest that the Bank should reconsider Low-income Countries its role in countries with strong capacity and In many low-income countries, borrower capac- ready access to private capital and refine its ap- ity and local financial resources are binding con- proach to countries whose capital markets have straints. First, the Bank should strengthen capacity closed because of poor performance or exter- for aid coordination, help harmonize donor stan- nal shocks. Access to private capital varies by dards, and assist the borrowers in enhancing the country and sector and is not binary. coordination and coherence of donor activities 29 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS (Eriksson 1999). Second, even in "good perform- They need external finance but lack many of the ing" low-income countries, such as Ghana and prerequisites for effective assistance. Accord- Uganda, simplifying project and program design ingly, Bank diagnostic work should seek to and sequencing reforms is a priority. Third, as con- identify the underlying institutional, political, or firmed in CAEs for Burkina exogenous factors affecting weak performance. The Bank should Faso and Uganda, Bank But adjustment lending should be deployed efforts to strengthen coun- prudently since it is likely to be ineffective in strengthen capacityfor y offices in low-income the absence of commitment for reform aid coordination, help countries should continue (Cameroon, Ecuador, Kenya, Papua New harmonize donor since this helps to improve Guinea). In "turnaround" countries-following project supervision and en- a change in leadership, for example-adjustment hance country dialogue. can provide needed support for reform, but borrowers in enhancing Debt relief under the should be based on a sober assessment of the coordination and enhanced Highly In- prospects for genuine change. Equally, limited coherence of donor debted Poor Countries Ini- lending operations may help strengthen public tiative (HIPC) should help sector accountability systems or support tar- activities, release resources for geted and closely monitored poverty programs poverty reduction, im- (Evans 2000). prove the environment for public and private in- But because all lending is channeled through vestment, sharpen the poverty focus of country government, CAEs and Bank research suggest policies, and facilitate improved selectivity in fu- caution (Dollar and Pritchett 1999). Where pol- ture lending (Birdsall, Claessons, and Diwan icy reform and governance improvements have 2000). The impact is likely to be modest overall, yet to take hold, grants may be more effective however, both because of the limitations of con- instruments for building consensus and sup- ditionality (Killick 2000) and because debt is only porting civil society. Hence, economic and sec- one of many development constraints for these tor work, aid coordination, and policy dialogue countries (World Bank 2001; Morrison 2000). The with government and civil society organizations PRSI has potential for a substantial change in the can help build consensus for reform; effective way the Bank and international partners do busi- donor coordination is critical. These activities still ness in low-income countries-if the expecta- tend to be underused in poor-performing coun- tions for greater country ownership, increased tries (Cameroon, Kenya, Nepal, Tajikistan). In poverty focus, and significant harmonization of light of scarce administrative and grant resources, donor activities are fulfilled while meeting Bank the Bank faces difficult choices regarding ap- and IMF technical and procedural requirements. propriate levels of resources for poor-perform- ers, particularly given the demand for Poor Policy Environments complementary inputs to support lending in Countries with weak governance and poor poli- better performers. Guidelines are needed to en- cies have limited access to external capital. sure a coherent approach to these situations. 30 TOWARD GLOBAL RESULTS: SECTOR AND THEMATIC DIMENSIONS The Bank's shift toward the higher plane of country strategy has been paralleled by a shift toward improving sector and thematic perform- .1. ance at the country and global levels. The challenge has been to strengthen the strategic focus and policy content of Bank lending and non- lending services within particular sectors, and to integrate cross-cutting the- matic priorities-including poverty, participation, and gender-into a range of Bank activities. OED's analysis suggests that much remains to be done to deliver on the four "Ps" of successful sector strategies: a clear Policy frame- work; an action plan to improve on past Performance; a specific Program of development assistance at country and global levels; and a definition of the Bank's role, taking account of the ongoing and planned activities of Partners. The challenges of sector and thematic issues re- * What are the implications for the Bank's recently flect (but also go beyond) the dilemmas expe- established Sector Strategy Paper (SSP) process? rienced at the country level, especially where global objectives are sought in line with the What Affects Sector Performance? Bank's global mandate and reach. This chapter Achieving sustained poverty reduction requires identifies those challenges based on evaluation macroeconomic, sectoral, and local progress, information in response to four related questions: but the average performance of Bank projects * What explains the variations in project per- varies widely among sectors. What factors af- formance within and among sectors? fect these variations, and what are the impli- * What factors influence the Bank's relevance cations for Bank programs and strategies? and efficacy in promoting sector and thematic objectives at the country and global levels? Country Income and Performance * What lessons can be drawn about the design Bank projects generally perform better on aver- and implementation of Bank sector strategies age in middle-income countries than in low-in- and policies? come countries. Across 14 sectors, in only 2 31 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS cases do lower-middle-income countries per- Institutional Design form worse than low-income countries, and in Bank sectors differ in the extent to which sec- only 3 cases do upper-middle-income countries tor or project performance is influenced by, or perform worse than lower-middle-income ones insulated from, institutional or policy factors. For (figure 4.1). But the example, most social protection projects in low- sensitivity of sector per- income countries are social funds, which operate Bank projects generally formance to country in- outside line agencies to finance and implement perform better on average come varies considerably. projects, often through local communities. Yet in middle-income Social-protection project while "enclave" approaches can improve serv- countries than in low- outcomes show no sig- ice delivery to the poor, they may limit the cat- nificant variation by coun- alytic influence of projects and undermine income countries. try income group, but for government capacity if not properly designed projects in the energy and (Erickson 1999).2 In infrastructure sectors, such electric power; water supply and sanitation; and as telecommunications and transport, the Bank health, nutrition, and population sectors, the has shifted from supporting parastatal organi- portfolio performance gap between low-income zations toward encouraging competition, and upper-middle-income countries exceeds 30 "unbundling" of services, and privatization. percent.' When comparing sector outcomes by Bank-financed health and education projects, in country performance (as measured by CPIA), a contrast, generally rely on government min- similar pattern emerges, but with even greater dif- istries for implementation and seek to improve ferences in sector outcomes between strong and overall sector performance. The low pay and dis- weak performers. torted incentives that prevail in the civil services Percent 100 90 80 70 60 50 40 30 20 101s * Low income Eli Lower-middle income U Upper-middle income 32 TOWARD GLOBAL RESULTS: SECTOR AND THEMATIC DIMENSIONS of many low-income countries often have a relatively weak, concludes that in subsectors that negative effect on project implementation and implemented portfolio improvement plans in the sector performance and complicate efforts to mid-1990s, project performance improved and strengthen capacity (van de Walle and Johnston projects at risk declined. The quality of Bank 1996). Community-based approaches can im- supervision also influ- prove social service delivery, as demonstrated ences project perform- by the success of community-managed water ance, particularly in The quality of Bank provision or health services in many poor coun- countries with limited Supervision also tries (Parker and Skytta 2000; UNICEF 1999). But government capacity or influences project no consensus has emerged regarding the ap- weak institutions. Recent propriate balance of public and private provi- reviews by QAG suggest r p i a sion in the social sectors (Nelson 1999). that Bank sector man- in countries with limited Results-based approaches need to be devel- agers are taking more di- government capacity or oped given the high transaction costs and low rect responsibility for weak institutions. capacity utilization associated with traditional in- quality assurance, and vestment projects. that sector and thematic networks with proactive portfolio quality man- Quality of Analysis agement programs have seen greater improve- Project outcomes are influenced by the extent and ments in quality at entry and project supervision. quality of prior analytic work-both economic and sector studies, as well as analyses conducted Rating Standards during project preparation. Health project out- Comparing Bank performance across sectors is comes were significantly correlated with the qual- complicated by several factors. First, sectors have ity of institutional analysis during project different objectives and performance standards. preparation (Johnston and Stout 1999). Com- For example, water projects financed through a so- prehensive ESW completed prior to project de- cial fund may not be subject to the financial sus- sign is associated with better project outcomes tainability standards that govern water sector (Squire, Deninger, and Basu 1997), but most projects (Pitman forthcoming). Second, more than OED sector studies noted with concern the un- half of the interventions in some sectors are sub- even coverage and quality of ESW (Evans 2000; components of multisectoral projects and are not Lele 2000).3 The OED financial sector evaluation, separately rated. Telecommunications and forestry for example, found that only half of the sector's subcomponents of large, multisectoral projects re- projects were preceded by ESW in the previous ceive less attention and may perform less well five years. Financial sector outcomes at the coun- than stand-alone projects (Lele 2000). Third, proj- try level were strongly associated with prior ESW; ect portfolio performance does not capture the qual- only one-third of countries with no prior ESW had ity or contribution of nonlending services. In such satisfactory outcomes, while two-thirds of the sectors as energy and telecommunications, where countries with at least one ESW activity before lending is declining but the importance of non- project approval had a satisfactory outcome lending services has increased, the scarcity of (Mathieu 1998). quality measurements for nonlending services complicates assessments of Bank performance. Portfolio Management This is a particular concern given the rapid growth The quality of Bank portfolio management is in the Bank's knowledge management activities.4 critical to portfolio performance, particularly Finally, project performance does not effectively in weak institutional settings. Bank sector and capture the Bank's effectiveness in promoting the- thematic networks share responsibility with re- matic issues, such as gender or participation-al- gional departments for monitoring and strength- though OED analyses suggest that higher levels of ening project quality. OED's review of the water participation are associated with improved project sector, where portfolio performance has been outcomes (Evans 2000). 33 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Challenges in Achieving Sectoral and to provide guidance on country-specific priorities Thematic Objectives and implementation. Preparation of regional strate- OED supplements its project evaluations with pe- gies-in 1997, for the Bank's rural strategy, and later riodic reviews of Bank experience and per- for health and education-is increasingly common. formance in selected Regional sector strategies, which can leverage sec- sectors and thematic tor work resources across smaller, low-income The Bank has an areas. In the past three countries, are currently underfunded and tend to obligation to provide years, OED has com- be used to justify preordained lending programs. policy advice that is pleted studies of poverty Moreover, country-specific work is essential. The coherent across reduction, forestry, health, rural study, as well as the CAS retrospective, found finance, energy and the that the quality of the rural strategy in CASs is countries and Regions. environment, rural devel- strongly associated with the existence and quality opment, telecommunica- of rural sector work (Heath 2000) . Just as for coun- tions, and participation. try work, OED sector and thematic reviews express Reviews are also being done for gender, envi- concem about the decline in funding for economic ronment, water, cultural heritage, social funds, and and sector work. private sector development in energy. Collec- Institutional design. Adapting strategies to tively, these reviews identify tensions and chal- country institutions and capacity is fundamental lenges the Bank faces in promoting sectoral and for development effectiveness. Good institutional thematic objectives at the country and global design requires attention to public and private levels, and highlight the importance of balanc- roles, stakeholder incentives, and the nature of ing global strategies with local adaptation, bal- the goods provided-public, private, club, or ancing client and corporate priorities, establishing common pool (Picciotto and Weisner 1998). In- strategic partnerships, and promoting global pub- stitutions, both formal and informal, are country lic goods. and even community-specific. Forest sector strate- gies must match forest interventions to specific Adapting Global Strategies to Local Settings geographic, biophysical, demographic, socio- The Bank's ability to articulate a broad strategic cultural, institutional, and economic circum- vision is one of its greatest strengths. It is the trans- stances (Lele 2000). In communities with low lation of its vision into reality that is a central chal- levels of social capital, community water associ- lenge. The Bank has an obligation to provide ations require intensive support if project inter- policy advice that is coherent across countries and ventions are to succeed (Parker and Skytta 2000). Regions. But the Bank is sometimes viewed as The impact of World Bank support to agri- not adequately adapting its policies and projects cultural extension in Kenya was limited-despite to country settings (box 4.1). OED studies sug- extensive investments in capacity building-in gest that several factors influence adaptation of part because the training and visit system lacked sector and thematic strategies. incentives for responding to farmer needs (Gau- Policyguidance. Strategies for most of the sec- tam 2000). The Bank is beginning to develop tors and thematic areas reviewed recognized tools for assessing organizational capacity (Or- the importance of adapting policies and programs bach 1999) and for institutional analysis, both to country contexts, but few provided explicit at the country and sector levels, but these need guidance. The Bank's 1991 forestry strategy, for to be further mainstreamed. example, was too rigid and did not recognize the Tapping local knowledge. Involvement of need for different strategies in forest-rich and client country officials and technical specialists forest-poor countries (Lele 2000). In contrast, the in the development of Bank strategies, country- Bank's approach to participation encouraged based analytic work, and throughout the proj- adaptation to diverse community settings. ect cycle enhances adaptation and maximizes the Country and regional analysis. Country-specific synergies between global and local knowledge. strategies and economic and sector work are needed Similarly, involvement of beneficiaries in design 34 TOWARD GLOBAL RESULTS: SECTOR AND THEMATIC DIMENSIONS Results from five country focus groups (in Bangladesh, C6te nomic and social development and to be persuasive in its ar- d'lvoire, Morocco, Uganda, and Ukraine) conducted for the OED guments. poverty study reveal important perspectives on the strengths and * Sharing good practice: "fighting poverty is not just throwing weaknesses of the Bank's poverty strategy and its implemen- money at it." tation. Stakeholders praised the Bank's broad strategic per- spective on development issues but expressed concern that Weaknesses the Bank sometimes promoted a 'one-size-fits-all" approach to * Insufficient focus on national capacity building for poverty policy advice and poverty programs (Evans 2000). analysis, monitoring, and evaluation. * Too little local ownership of ideas and interventions. Strengths * The strategy istoo macrm; it does not connect well enough with * Analytical power high-quality studies, surveys, and techni- microeconomic issues and poverty. cal assistance. * Bank processes and procedures are too rigid to permit true * Ability to fold issues of poverty into broader discourses on eco- partnership and full participation. of community interventions increases the likeli- poverty in different countries (Evans 2000). Im- hood of local adaptation, as well as ownership proving monitoring and evaluation will require and sustainability. Thus it is promising that increased attention during project design and African scholars and policymakers contributed to supervision, as well as greater emphasis on eval- the Bank report, Can Africa Claim the 21st Cen- uation capacity building in central and sectoral tury? (World Bank 2000c). Participatory appraisal ministries, and among independent research or- and social assessments are increasingly used in ganizations (AfDB and OED 1999; MacKay 1998). project design. While the Bank is generally per- ceived to have become more responsive in re- Balancing Corporate and Client Priorities cent years, client surveys suggest that the Bank At the corporate level, a scorecard connecting in- could do a better job listening to local ideas and ternational development goals with Bank sector expertise. Visiting missions still spend most of strategies and country programs has yet to see their time in capital cities, with limited oppor- the light of day. This makes it difficult to track tunities for field visits during compressed mission progress of approved programs towards inter- schedules. On the other hand, decentralization national goals and corporate objectives and to en- to field offices has greatly increased opportuni- sure coherence between institutional priorities ties for local interaction. and individual Country Assistance Strategies. This Monitoring and evaluation. Effective moni- inhibits management of the tensions that in- toring and evaluation is essential for regularly up- evitably emerge between the priorities articulated dating strategies in response to country in sector or corporate strategies and those that experience. Nearly all OED sector and thematic result from "bottom-up" consultations with coun- studies point to continued weaknesses in mon- try clients and stakeholders. These tensions can itoring and evaluation, for both the Bank and bor- encompass development outcome targets-such rowers-both at the operational level and at the as reducing environmental protection or gender sector and country level. The poverty study, for disparities-and lending targets, such as for basic example, concludes that inadequate monitoring education. Divergence between the priorities of and evaluation has made it difficult to assess the the Bank and the borrower-or other stake- country-level impact of poverty reduction strate- holders-constrained implementation of several gies, so the Bank knows relatively little about the Bank strategies. For example, the overwhelming optimal mix of sectoral policies needed to reduce emphasis on conservation in the Bank's 1991 35 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS forestry strategy improved compliance with the long, drawn-out, expensive, and inconclusive Bank's Safeguard standards but failed to arrest de- policy consultation exercises (for example, on re- forestation in the very countries the policy was settlement and indigenous peoples) and a designed to affect. It inhibited experimentation plethora of poorly coordinated initiatives and with sustainable forest partnerships (Lele 2000). Divergence between the exploitation in coopera- tion with the private Are Bank Global Partnerships Strategic? sector (Brazil). By some estimates, Bank networks have entered the borrower constrained Participation. Strategy into more than 200 partnerships with interna- implementation of several development has tended tional agencies and NGOs. OED evaluations Bank strategies. to focus on participation suggest that strategic consensus facilitates part- from staff in Bank net- nership, and partnerships can facilitate consen- works. The rural strategy sus. Partnerships are increasingly important to the achieved strong "buy-in" from staff in the rural water sector, for example, where a broad con- sector but did not sufficiently involve Regional sensus has emerged, as documented recently in management and country teams (Heath 2000). the report of the World Water Commission at The Similarly, resistance by Bank Regional depart- Hague. This evolving consensus was supported ments to the global prescriptions of the "Fuel for by the relatively successful UNDP-World Bank Thought" strategy paper delayed its processing Water Supply and Sanitation partnership and and blurred its quantitative goals. By contrast, the guided by the 1992 Dublin Accord on interna- inclusive approach used to develop the Bank's tional water management. The Global Water participation guidelines helped to generate learn- Partnership and World Water Commission are ing and commitment within the Bank, which expected to contribute to the development com- partly compensated for the lack of a formal strat- munity's ability to meet major water challenges egy or policy. (Pitman forthcoming). Building consensus. In the water, gender, and While acknowledging the importance of se- forestry sectors, basic disagreements-both in- lectivity in global partnerships, Bank strategies side and outside the Bank-were "papered over" have not provided sufficient guidance regarding rather than explicitly resolved (Pitman forth- priorities or in defining the Bank's comparative coming; Gopal 2000; Lele 2000). This hindered advantage vis-a-vis partners. The rural strategy implementation of strategy and also affected the discussed types of partnership but not the issue rigorous implementation of Safeguard Policies. of comparative advantage. Under the Bank's If disagreements are acknowledged, appropriate current management structure, network part- strategies may be developed to address them. To nership agreements for staff do not ensure coun- address fundamental disagreements regarding try programs' commitment or resource allocation development policies and strategies, Bank net- to partnership goals. The networks need to clar- works could identify priorities for "knowledge ify the Bank's core competencies and compar- creation"-as well as knowledge management- ative advantage relative to its partners and to through research and analytic activities, as well establish more businesslike alliances to allow the as pilot projects and cross-national learning. The Bank to relinquish activities that partners are bet- Bank can use its convening role to help build ter equipped to handle. In policy matters how- consensus among stakeholders. Insufficient con- ever, "letting go" is not the full answer. For sultation contributes to a lack of broad-based sup- example, the Bank turned over the design of in- port for the Bank's strategies and policies. Thus, ternationally accepted standards for the design, international NGOs were consulted extensively construction, and operation of large dams to a during the formulation of forestry and water World Commission on Dams following a work- strategies, but borrowing countries and the pri- shop jointly sponsored by OED and the World vate sector were not. Lack of standards for pub- Conservation Union (IUCN 1997; WCD 2000). lic consultation and partnerships has also led to The final report of the conmiission is a major con- 36 TOWARD GLOBAL RESULTS: SECTOR AND THEMATIC DIMENSIONS tribution to the topic but it has not stilled the con- 2000k). But good examples of strategy and policy troversy over what constitutes adequate social review contributing to the development of au- and environmental standards for large dams. thoritative intemational standards and agreements are rare. In principle, this area of comparative ad- Contributing to Global Public Goods vantage deserves to be The benefits of global public goods are widely nurtured as a component The decline in funding for shared, but developed and developing countries of the knowledge Bank, ESWhighlight th alike are often reluctant to finance-or borrow- but only for activities for their production and delivery. International closely aligned with the importance of balancing institutions, such as the World Bank, can facili- Bank's core development knowledge management tate collective action-through their convening mission and in close col- with knowledge creation role, knowledge creation and dissemination, and laboration with other mul- provision of financing for piloting and imple- tilateral agencies. The mentation (World Bank 2001). Global initiatives Bank has made substantial investments in knowl- supported by the Bank encompass issues rang- edge management in recent years, but the decline ing from environmental protection and financial in funding for ESW highlights the importance of stability to disease control and research and de- balancing knowledge management with knowledge velopment. Because global public goods require creation. collective action, international partnerships are es- The Bank is being called on to take an increased sential. The Bank's status as convener and as cat- role in promoting global public goods, which has alyst for the financing of global public goods is highlighted issues of selectivity, the Bank's com- exemplified by its role in fostering the Consul- parative advantage, and the appropriate balance tative Group on International Agricultural Re- between country-based programs and global or search (Anderson and Dalrymple 1999). Grants regional initiatives, particularly in the allocation of funded by the Development Grant Facility and staff time and Bank resources. In response, Bank the Global Environment Facility have helped fill management-through both intemal discussion pa- the gap, but resources fall short of needs and have pers and external consultations-is seeking to been spread thin. For example, while the Bank's clarify its priorities and role. These discussions have 1991 forestry strategy diagnosed the problem of highlighted the importance of maintaining an ap- externalities, the Bank and international com- propriate balance between global or regional part- munity did not provide adequate financing to nerships and programs that promote "pure" public bridge the costs and benefits of conservation at goods, and country-based programs that have im- the global and local levels (Lele 2000). portant international spillover effects (such as ef- The Bank helps articulate global public policy fective disease control programs). The Bank also through global policy studies-such as the recent needs to ensure that developing countries are World Development Report: Attacking Poverty- able to make effective use of global knowledge and sector and regional studies or strategies. It has and public goods. OED recently launched an also begun to jointly develop policy studies with evaluation of the Bank's experience in global partners. For example, Bank staff were seconded public goods, which seeks to inform these debates to the World Health Organization to help with the (World Bank and UNDP 2000). 2000 World Health Report (WHO 2000). Financial stability is another global public good. As dis- Design and Implementation of Sector cussed in Chapter 3, while the IMF plays the lead Strategies role in financial crises (Fischer 1999), the Bank can OED's sector and thematic studies increasingly play an important supportive role (World Bank evaluate the Bank's experience implementing 2000b). At a regional level, the Bank contributed specific strategies and policies.5 To organize and to establishing standards and strengthening the build on evaluation findings, OED evaluators banking systems in the CFA countries of West assessed eight recently evaluated sector and Africa after the CFA devaluation of 1994 (OED thematic strategies along 13 dimensions.6 Most 37 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS of the strategies were developed in the early needs (Liebenthal 2000). The gender study con- to mid-1990s, and with the exception of the cludes that the Bank's strategy and policy are un- 1997 rural development strategy, predate the clear about whether the objective is to introduction of Sector Strategy Papers (SSPs) "mainstream" gender concerns into all Bank ac- (see Annex B). The qual- tivities or to selectively integrate them into prior- One weakness in many ity and relevance of the ity sectors or country programs (Gopal 2000). As sector strategies was the strategic framework re- the first official Sector Strategy Paper (SSP), the ceive the highest aver- 1997 rural strategy identified 15 countries and 4 paucity of guidance age rating, and the cross-country programs for special attention- to Bank staff or clients quality (and existence) although the selection criteria were not entirely about how to make of plans for implemen- transparent (Heath 2000). tation the lowest. Re- choce an rdvealing differences Implementation of Strategies the country level, among the sectors are Sector policies and strategies are not, as a rule, im- presented below. plemented direcdy, but rather through country programs-including lending and nonlending Strategy Design services-and related instruments such as the With notable exceptions, the strategies articulated PRSI. None of the strategies OED evaluated in- sound policy frameworks relevant to intemational cluded explicit plans for implementation-making and Bank objectives (figure 4.2). OED gave high this the weakest dimension of the strategies re- ratings to the policy frameworks underlying viewed. In some, additional resources were iden- Bank strategies for water, energy, participation, tified to upgrade staff skills (rural, poverty), or to and the financial sector. The OED water study assist country teams (poverty). But generally, lack concluded that the Bank's 1993 water strategy of planning and resources contributed to a gap be- remains highly relevant, despite challenges in im- tween strategic intentions and implementation. plementation that require greater selectivity and Resource problems are symptomatic of a mis- sensitivity to different country and subsectoral cir- match between the Bank's commitments- cumstances. The analytic frameworks for poverty, including increased requirements for Safeguards rural, health, and telecommunications strategies, and fiduciary accountability-and available staff however, were mixed or lagged behind evolv- and resources.7 Few of the strategies OED re- ing circumstances. The poverty strategy was rel- viewed included explicit indicators or monitoring evant, but a more comprehensive approach was arrangements to assess progress in implementa- needed to achieve broad-based, equitable tion or results. growth. Gender and forestry were flawed (Gopal 2000; Lele 2000), while the design of private sec- Operational Guidance tor strategies has lagged. The clarity and consistency of operational poli- cies and guidelines were rated modest on aver- Selectivity age. Policy guidance for the poverty strategy was One weakness in many sector strategies was the generally strong and accompanied by best prac- paucity of guidance to Bank staff or clients about tice guidelines and a poverty reduction hand- how to make choices and tradeoffs at the coun- book. The guidelines were regularly strengthened try level. Strategy papers did not provide adequate and updated during the 1990s.8 But the majority tools to help sort competing needs and oppor- of OED studies (participation, forestry, gender, en- tunities. Illustrating the problems that arise when vironment, telecommunications, water) concluded there is policy without strategy, OED's environ- that ambiguities or gaps in the policy framework ment study concludes that the Bank has tried to have made Bank staff, management, clients, and do many things without establishing clear prior- partners uncertain about expectations and re- ities and focusing its efforts on the areas most rel- quirements.9 Unclear guidance also has con- evant to its overarching mission and to country tributed to risk aversion, with many Bank staff and 38 TOWARD GLOBAL RESULTS: SECTOR AND THEMATIC DIMENSIONS country managers avoiding involvement with International Finance Corporation (IFC), and by controversial issues, such as forestry, large dams, the lack of a PSD strategy. The Bank recently es- or extractive industries projects. Finally, enforce- tablished the Private Sector Development Board, ment of Safeguard Policies has been inconsistent, however, and is developing a private sector strat- largely because of highly decentralized and dif- egy jointly with the IFC. fuse accountability combined with weak central Bank Management is Few of the strategies coordination and oversight (box 4.2). aware of concerns with included explicit matrix management, and indicators or monitoring An Unbalanced Matrix has taken some steps to- The Bank's current management structure makes ward "rebalancing the ma- arrangements to assess it difficult for sectors and networks to explicitly trix." Further changes are progress. plan for implementation, because country de- needed to fulfill the prom- partments have full authority over budgeting and ise of the Bank as a global institution. decisionmaking regarding country programs. This proved to be an obstacle to implementation of the Implications for Sector Strategy Papers rural strategy (Heath 2000). Accountability With the introduction of Sector Strategy Papers for implementation is a particular challenge for (SSPs) in 1996,1° the Bank began to address the cross-cutting thematic issues (gender, environ- weaknesses in sector strategy development. The ment). Some strategies (poverty, participation) Bank has produced 12 SSPs so far. SSP guide- addressed this challenge by providing good prac- lines have tried to address of the shortcomings tice guidance and support and training for in the earlier approach, including lack of selec- operational staff. Others, such as telecommuni- tivity, implementation guidance, and monitoring cations and gender, were less effective. The Bank's frameworks. efforts in telecommunications and private sector In a preliminary stock-taking, Bank manage- development (PSD) generally have been hindered ment found improvements over earlier strategies, by fragmentation of responsibility for PSD across but many problems remain. In general, SSPs various units within the Bank Group, including the have been stronger on conceptual and analyti- Implementation plan Mainstreaming in Bank Cross-sector links Participation Integrated with CASs Operational policies Selectivity Adaptable/flexible Strategic framework Negligible Modest Substantial High 39 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS OED reviews of Bank activities in forestry, environment, water, taken partial action to strengthen incentives and enhance and resettlement concluded that during the 1990s the Bank budgetsupport for compliance. Additional steps are needed to: made significant progress developing and implementing "do * Eliminate conflicts of interest for quality assurance. no harm" Safeguard Policies, which helped reduce the neg- * Further strengthen accountability for and enforcement of ative impact of Bank lending on the environment, indigenous Safeguards. peoples, and displaced communities. But further progress is * Conduct country level reviews and policy dialogues to ensure needed. Each of these studies found that inconsistencies per- consistency between Bank Safeguards standards and coun- sist in Safeguard compliance during project design and try regulatory frameworks. supervision, as well as in project completion reporting on * Help build domestic capacity for resettlement entironmen- resettlement performance and outcomes (OED 1998g; Lele tal protection, and independentverification upstream of prol- 2000; Liebenthal 2000). Reviews by the Bank's Quality Assur- ect processing. ance Group (GAG) confirm that, despite progress on compli- * Reduce transaction costs to the Bank and to borrowers. Ince, the Bank falls short of its target of 100 percent * Explorewaystoreduceriskaversionbyimprovingriskman- compliance. Safeguard Policy treatment was less than satis- agement in areas with necessarily strong Safeguardstandards factory in 9 percent of FY99 project design documents re- (such as large dams and forestry). viewed by QAG. A rapid supervision assessmentfound that only * Catalyze improvements in Safeguard standards for non-Bank- 84 percent of the projects involving the Bank's resettlement and financed projects in clientcountries (Lele2000; Liebenthal 2000). indigenous peoples safeguard policies had satisfactory Beforeinitiatingaproject,theBankshouldcarefullyexamine supervision. the borrower's legal and regulatory frameworks and, where In response to recent well-publicized incidents of non- appropriate, operate within them. An increased emphasis on up- compliance, Bank management affirmed its commitment to stream capacity building could help countries tackle issues be- the diligent execution of the Bank's Safeguard Policies, and has fore they become problems. cal issues and weaker on the implications for OED's analysis of Bank experience designing Bank Group business. They typically provide and implementing policies and sector strategies comprehensive coverage of the development suggests four factors-the four 'Ps'-that are rationale for Bank participation in the sector or essential to successful sector strategies, and thematic area and examples of good practice in could help to strengthen the Bank's sector and country approaches. But few thoroughly analyze thematic strategies: country strategies-particularly the tension be- * A clear policy framework, taking account of tween client objectives and Bank priorities-or global experience and best practice in the sec- the strengths and weaknesses of lending port- tor. Clarifying operational (including Safe- folios and nonlending services. Issues about the guard) policies is a priority. menu of products the Bank offers in the sector, * An assessment of past performance and an and implications for future products and services, action plan to improve on the past record to also deserve further attention. Recent SSPs have balance portfolio risk and rewards- paid more attention to such business matters as especially for controversial or institutionally internal organization, staffing, and performance difficult development challenges. While OED monitoring, but there is still room for improve- studies and QAG portfolio analyses are in- ment. Partnership issues have received more at- creasingly being used to develop SSPs and tention in recent years, but relatively few specify to improve the links between strategies and the Bank's and partners' comparative advan- evaluation and past experience, rigorous tages or establish priorities among potential self-evaluation remains the exception rather partnerships. than the rule. 40 TOWARD GLOBAL RESULTS: SECTOR AND THEMATIC DIMENSIONS * A specific program of assistance, specifying * An assessment of the comparative advantage monitorable global goals, focus countries, of the Bank and its partners, of the partners' proposed deployment of instruments, skills re- current and planned activities, and of the quirements, and budgets. This has improved need for new alliances. It should be a priority in some of the more recent SSPs, but the to establish more busi- continued imbalance in the matrix structure nesslike partnership Thefour'Ps'-policy, renders the planning and implementation arrangements, stream- performance, program, processes of sector strategies difficult to line the number of manage. Even within the constraints of the partnership arrange- current matrix, networks need to do a better ments, and back any successful sector strategies. job achieving "buy-in" from Regions and agreements with cor- country departments and keeping track of porate commitments, with full recognition of progress. their resource implications. 41 r CORPORATE AND POLICY IMPLICATIONS his year's ARDE reviews the Bank's development effectiveness in the context of an increasingly integrated but unequal and volatile world economy, a natural environment under stress, a fragmented political order, failed and failing states undermined by civil strife, and a global in- stitutional framework riddled with tensions among weakened public sectors, an ascending transnational private sector, and a network of vocal organi- zations from civil society. In the early 1990s, the Bank was slow to respond to this more complex operating environment. By the mid-1990s, under new leadership, the Bank had adopted a more comprehensive development agenda, committed itself to implementing more demanding social and en- vironmental policies, shifted its focus toward the higher plane of country and global programs, decentralized its operations, and launched new lend- ing and knowledge management initiatives to meet the diverse demands of its borrowers, shareholders, and other stakeholders. As a result, the Bank's financial and human re- daunting challenges, portfolio performance has sources have been spread thin and expectations improved in the past year. The Strategic Compact have risen farther than the Bank's capacity to de- target of 75 percent satisfactory performance is liver. Enormous stress resulted from organiza- likely to be met for FY00, even though project tional transformation, leading to widespread complexity, demandingness, and risk are high- calls for the Bank to be more selective, to man- and have increased the most for Africa and low- age resources more effectively, and to be more capacity countries. At the same time, progress disciplined in implementing corporate policies toward the International Development Goals has and in making new commitments. been slow. Over the past decade, the percentage OED's independent assessment of operational of the population in developing countries living performance trends shows that, despite these in poverty has declined, but the absolute num- 43 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS bers have increased, reflecting continued popu- of an integrated, volatile, and highly differenti- lation growth and declining per capita GDP in ated global economy. many transition economies and poor countries- especially those affected by civil conflict, politi- Reconciling Corporate Priorities and Client cal instability, and, Ownership Managing tensions when increasingly, HIV/AIDS. Managing tensions when the Bank, its partners, the Bank, its partners, and Partly for these reasons, and borrowing countries disagree about priori- project performance in ties ("what to do"), or specific policies and in- borrowing countries Africa remains disap- terventions ("how to do it") is a difficult balancing disagree about priorities or pointing, although QAG act. Strategies that are fully developed and owned specfic policies and data could signal future by client countries may contain different policies improvements. Adjust- or interventions than the Bank or its international interventions is a difficult ment lending perform- partners might propose. balancing act. ance has improved * Bank strategies should explicitly acknowl- steadily over the past five edge such differences in priorities. Fuller years, due to lessons from stakeholder analysis, which can help identify experience and a shift toward countries with bet- sources of disagreement and make strategies ter performance ratings. But findings from proj- more relevant, will be especially important as ect audits and country evaluations suggest that the the PRSI process moves forward. Moreover, achievement of immediate policy objectives that sector and country strategies can identify trigger a favorable outcome rating does not nec- ways to move toward greater consensus-for essarily translate into long-term impact, unless the example, through specifying priorities for country commitment to reform is sustained and knowledge creation and dissemination. exogenous factors do not undermine the impact * Bank instruments need to be modernized to of reforms. Investment lending performance has reduce transaction costs and focus on results. progressed at a more modest pace, while IBRD They also need to be mobilized more strate- lending volumes have declined, reflecting rising gically to build consensus at the country and resistance to the high cost of doing business with global levels. This requires an increased focus the Bank. on local involvement and dissemination of The Bank knows, through hard-won experi- findings from Bank analytic work, enhanced ence, much about what to do and how to do it use of pilots, strengthened monitoring and better: choose good projects, support the right evaluation, and greater use of the Bank's con- policies, be selective about which countries and vening role. Consensus building is resource- sectors to address through diverse instruments, intensive, so it should be focused on priority and continue to modernize business practices. issues-especially global public policy issues- But doing this is difficult, especially when re- where the Bank has a distinct comparative source allocation systems are not aligned with advantage. the Bank's thematic priorities, staff and senior * Conditions are most effective for reforms that managers continually face new initiatives, and in- can be easily specified, monitored, and im- ternal incentives are poorly adapted to the ef- plemented by a few decisionmakers. Condi- fective implementation of corporate policies and tions should be used judiciously and aligned strategies. with policy changes that command sufficient support to avoid policy reversals. Thus, ad- Addressing Tensions and Challenges justment lending could be made more effec- The Bank could improve its development ef- tive through continued progress toward more fectiveness by directly confronting the tensions consultative approaches, fewer conditions, highlighted in this review and adapting its or- better analysis of institutional constraints to re- ganizational structure, business practices, and op- form, and careful sequencing of nonlending erational instruments to the emerging demands and lending instruments. 44 CORPORATE AND POLICY IMPLICATIONS Adapting Global Prescriptions to Local reserving a capacity to intervene to mitigate Conditions the effects of volatility in international capital The Bank has usually been effective in provid- markets. ing strategic advice and in designing relevant country and sector strategies. But Bank policies Balancing the Bank's and programs could be better matched to bor- Pro-Poor Operational Bank policies and rowers' institutional, social, and political envi- Emphasis with Country ronments. PerformanceProrm * Improving the quality and coverage of country- Constraints matched to borrowers' based and regional analytic work is essential for Bank lending is more ef- institutional, social, and adapting Bank strategies and policies to local fective in countries with poUtical environments. contexts. Equally important, government officials, better policies, gover- technical experts, and stakeholder organiza- nance, and commitment tions need to be engaged to ensure both the rel- to poverty reduction, but many poor people live evance and ownership of analytic work and in countries that do not have these attributes. resulting recommendations. Engagement with . The rationale for performance-based lending local experts is increasingly common in middle- should be made transparent to borrowers and income countries and other large borrowers. In partners. Discussion of CPIA ratings with client small, low-income countries, economic and governments in FY01 is an important step. But sector work needs to be closely managed to en- greater transparency would encourage more sure a proper balance between strong local in- rigor in performance measurement and stim- volvement and high technical standards. ulate debate regarding reform within member * For social, environmental, and fiduciary poli- countries. Public disclosure could increase cies, the Bank should seek to harmonize its the costs of the exercise, however, and require Safeguard Policies with legal and regulatory changes in indicators to improve their credi- frameworks in client countries, and place bility and acceptance. greater emphasis on strengthening country * The Bank needs to clarify and strengthen its policies and enforcement rather than estab- approach to poor-performing countries. A se- lishing project enclaves. lective, targeted program of policy dialogue, * In low-income countries, priorities include: nonlending services, grants, and learning and (1) adapting project complexity to domestic innovation lending can help lay the ground- capacities and enhancing aid coordination; (2) work for improved performance. Targeted supporting governments in the development analysis and advice-and dissemination of of strategic frameworks for country and donor good practice lessons in concert with other programs, and encouraging selectivity among partners-deserve more emphasis. donors; (3) adopting comprehensive, long- * Adjustment lending can support "turnaround" term strategies to improve governance, pub- countries, but should follow concrete actions lic sector management, and the enabling toward reform. Expanding the Bank's adjust- environment for private investment. ment portfolio would be more effective in * To make lending and nonlending services in countries with moderate to good policy per- middle-income countries more selective and formance to support reforms on which there catalytic, the Bank should apply more stringent is mature consensus. Conditionality should be selection criteria so projects have "spin-off' ef- streamlined to reflect absorptive capacity fects on government programs and policies constraints. and on the institutional framework. The Bank should consider streamlining its presence in Achieving Selectivity Through Partnerships middle-income countries with weak per- Being selective means identifying the Bank's formance, limited commitment to poverty re- comparative advantage and "letting go" of ac- duction, or low demand for Bank services, and tivities that other agencies can handle. 45 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS * Networks need to clarify the Bank's and part- . Bank strategies, programs, and budgeting ners' core competencies and comparative ad- processes should become strongly linked to the vantages and establish businesslike alliances achievement of results consistent with the so that the Bank can relinquish activities and Bank's mission and comparative advantage. Pri- enhance the impact of its orities include: budget reform to align corporate Bank strategies, work. Trust fund activi- strategies with resource allocations; full cost- ties and global collabora- ing of approved country assistance strategies, programs, and tive programs should be sector strategies, and global partnerships; mod- budgeting processes closely aligned with cor- ernizing Bank instruments to shift staff incen- should become porate priorities. tives from a focus on inputs and processes * Administrative budget toward results and performance. processes should be . While knowledge, connectivity, and dissem- achievement of results. transparent and ensure ination are important, maintaining the Bank's consistency between the knowledge base is even more critical. Hence, costs of approved CASs and Regional budg- the Bank should strike an appropriate balance ets. Similarly, approved SSPs should be fully between knowledge management and knowl- funded and give greater attention to the pro- edge creation-at both the country and global grammatic, business, and partnership impli- levels-and establish quality monitoring and cations of strategy implementation. tracking processes, similar to those devel- The Bank should continue working with in- oped for projects, to strengthen the quality ternational partners to harmonize procedures and coherence of all Bank knowledge and make internal processes and requirements activities. more compatible with those of client countries. * Achieving selectivity will require the Bank to seriously and continuously assess its compar- Corporate Management Implications ative advantage-relative to international and The findings in this review have overarching im- client country partners-to improve the de- plications for corporate management as the velopment effectiveness of partnerships Bank seeks to reposition its strategic directions through businesslike processes and to "let go" to adjust to a complex and rapidly changing of activities which other development actors environment: can carry out. 46 ANNEXES ANNEX A RELEVANCE AND OUTCOME OF COUNTRY PROGRAMS OED evaluators rated the relevance of Bank standing with Bank staff. Third, the quality and strategy and the development effectiveness of its consistency of the Bank's analytic and advisory program for a dozen major issues and sectors, work varies among sectors. Recent reviews by based on 15 recently completed country assis- OED and Bank management of economic and tance evaluations. Because the sample is small sector work (ESW) have found that the quality and the rating methodology not yet mature, the and timeliness of macroeconomic ESW was gen- results are only indicative. Moreover, CAEs typ- erally good, but quality and coverage of analytic ically cover 5-10 year periods, so the ratings may work for poverty assessments and sectoral analy- not fully capture recent improvements in per- sis varies widely across countries and sectors. As formance. But they display revealing patterns and noted in Chapter 3, recent OED sector and the- supplement Chapter 3 findings (figure A. 1). matic studies have also raised concerns about the For every category, the relevance of the Bank's quality and coverage of ESW. Although the low strategy was rated higher than the programs' rating for poverty is disappointing, the OED development effectiveness. For macroeconomic poverty study noted an improvement in the stability, development effectiveness averaged quality of recent poverty assessments, and in the only slightly below fully satisfactory, and only coverage of poverty issues in CASs (Evans 2000). slightly lower than strategy ratings. Effective- The Bank is also piloting social and structural re- ness ratings for promoting growth, financial sec- views, which may help strengthen the relevance tor development, infrastructure, and human of Bank strategies in these areas. development averaged partially satisfactory, What explains the significant gap between rel- while poverty reduction, private sector devel- evance and development effectiveness across cat- opment, environmental sustainability, rural de- egories and the disappointing outcome ratings velopment, and public sector management were for poverty and other social and structural rated less than partially satisfactory, on average. objectives? First, while stabilization can be achieved For objectives that constitute the core social and by a limited number of actors, growth and struc- structural agenda in client countries, the gap tural reform require a broader number of players between the relevance ratings and outcome rat- confronting difficult and contentious institutional ing was significantly larger than for macroeco- issues. Second, macroeconomic stability is neces- nomic stabilization. sary but not sufficient to achieve sustained growth, Why are relevance ratings the highest for poverty reduction, and aid effectiveness. Achiev- macroeconomic stabilization, promoting growth, ing sustained economic growth depends on pri- and financial sectors? First, stabilization, growth, vate sector development, improved infrastructure, and financial sectors are areas in which the Bank rural and urban development, financial sector re- typically works in close partnership with the form, and the like. Similarly, translating growth into IMF. This symbiosis may help strengthen the poverty reduction requires action on additional Bank's impact. Second, the Bank's partners in the fronts-including access to social services, em- client country macroeconomic dialogue are usu- powerment, and security (World Bank 2000a). ally a small team of highly trained individuals Two critical areas-private sector development who share a common language and under- and public sector management-received low 47 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Rating * Relevance 4.0 n Effectiveness 3.5 3.0- 2.5- 2.0- 1.5 1.0 .D I 1r 10 1~0 1 0 01 i10 1 1 Note:The ratings are based on a four-point scale (4 = highly satisfactory, 3 = fully satisfactory, 2 = partially satisfactory, and 1 = unsatisfactnry). The relevance rating assesses the coherence and appropriateness of the Bank's strategy and programs (did the Bank 'do the right things?"). The effectiveness rating assesses the extent to which Bank advice and interventions contributed to development progress on the given issue or sector While the Bank has control over the relevance of its strategy, development effectiveness is influenced by a variety of factors outside the Bank's control. ratings, on average, for development outcomes. zation and public sector reform, as well as in- A number of CAEs-from low-income countries adequate attention to implementation and insti- in particular-expressed concern regarding the tutional issues in the Bank's approach. Improved lack of a coherent strategy for private sector de- project outcome ratings for public sector man- velopment and public sector management (in- agement are encouraging (see Chapter 2), but the cluding governance and corruption issues). But CAE findings warn against complacency in terms the "outcome gap" also may reflect the political of achieving sustained improvements in public and institutional difficulties inherent in privati- and private sector performance. 48 ANNEX B RECENTLY EVALUATED SECTOR AND THEMATIC STRATEGIES This annex summarizes eight sector and thematic *Participation. In the early 1990s, the Bank strategies recently evaluated by OED. Prior to established a Participatory Development Learn- 1996, sector and thematic strategies were articu- ing Group that engaged in a four-year process lated in the form of Policy Papers, which provided of consultation and learning, which resulted in a broad framework for Bank activities and policy the publication of the final report, including a Par- advice, but gave limited attention to implementa- ticipation Action Plan endorsed by the Board in tion issues. Sector Strategy Papers (SSPs) were 1994. The Bank has not issued formal Opera- launched in 1996, in part to give a more transparent tional Policies or Good Practice guidelines for accountability framework for Bank network ac- participation. tivities. Of the strategies below, only the rural "Vi- oPoverty. World Development Report 1990: sion to Action" strategy was a formal SSP. Poverty (World Bank 1990) provided the foun- *Environment. There is no comprehensive dation for the World Bank's strategy to achieve strategy that describes the Bank's environmen- poverty reduction. It encouraged broad-based, tal objectives. Instead, a set of policies, guide- labor-intensive growth; expanded provision of lines, and practices have emerged since the late basic social services; and improved targeting of 1980s to integrate environmental concerns into transfers and safety nets. Operational Directive Bank work. The Bank's ESSD Network is cur- 4.15 issued in 1991 provided further guidance to rently developing an environmental strategy. staff regarding implementation. The recent World *Forestry. In 1991, the World Bank published Development Report 2000/2001: Attacking Poverty Thbe Forest Sector: A World Bank Policy Paper, fol- (World Bank 2000a) confirms the importance of lowing an extensive consultation process with growth and broadens the 1990 strategy by em- nongovernmental organizations. The policy phasizing opportunity, empowerment, and called for a ban on World Bank financing for for- security. est production and advocated a multisectoral *RuralDevelopment. The 1997 "Rural Vision approach to conserve tropical moist forests and to Action" Sector Strategy Paper is organized expand forest cover. Parts of the strategy became around four goals and five strategic principles. the basis for Operational Policy 4.36 and Good The goals of the strategy are to promote poverty Practice 4.36, which were issued in 1993. alleviation, widely shared growth, food security, * Gender. The World Bank began to encour- and sustainable natural resource management. age the incorporation of women's issues into The five strategic principles proposed to achieve poverty strategies and lending operations in the these goals are a coherent policy framework, pri- mid-1980s. A 1994 strategy paper, "Enhancing vate sector development, restructuring govern- Women's Participation in Economic Develop- ment, participation, and decentralization. ment," signaled a shift toward a more compre- * Telecommunications. The Bank's policy is hensive gender approach. The Bank's gender embodied in OP 4.50 (1995). It advocates private policy and strategy are articulated in a number provision of telecommunications services, com- of documents, including the 1984 Operational petition and independent regulation. It makes Manual, the 1994 strategy paper, and the sub- lending to state enterprises contingent on sec- sequent OP 4.20. tor reform and encourages increased coopera- 49 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS tion among the Bank, IFC, and MIGA. No simi- Bank Policy Paper," issued in 1993, which ac- lar policy statement or operational strategy ex- companied the water resources management ists for the information technology or information policy statement (Operational Policy 4.7). The infrastructure sector. A World Bank Group SSP strategy calls for a comprehensive, multisectoral for the information and communication tech- approach to water resources that integrates water nology sector is scheduled to be issued in provision, sanitation, and irrigation. The strategy FY2001. also emphasizes the importance of charging eco- * Water. The Bank's water strategy is articu- nomic rates for water. A Sector Strategy Paper is lated in "Water Resources Management: A World under preparation for FY02. 50 Table C.i . . _ _ Exit FY90-94 Exit FY95-98 Exit FY99-00* Active Portfolio Sust % ID Impact Sust % ID Impact Sust% t3Impaect % Projects Share Out rikelyor %ksub-or Afg Projects Share Out likely or %sub.or Aggr. Projects Share Out likelyor % sub.or Aggr. Projects Share Projects I 1 % Sat better bener Index # % % Sat better better Index ;/ % Sat better bettwr Index # % at risk Sector Group Agriculture 323 29 60 35 29 6.1 223 23 69 43 37 6.3 57 15 58 44 31 6.1 251 17 17 H Education 83 7 80 62 38 6.9 85 9 73 39 25 6.4 42 11 79 52 36 6.8 169 11 10 Electric Power & Other Energy i111 10 64 57 30 6Q2 68 7 66 49 33 6.4 29 8 62 52 52 6.6 82 5 21 Environment … … … … … … … 12 1 58 50 25 6.6 16 4 75 50 50 6.9 96 6 17 t) Extractive Industries 49 4 71 51 57 6.9 31 3 77 68 52 7.1 15 4 73 67 40 6. 41 3 22 Finance 64 6 56 40 28 6.2 61 6 68 53 38 6.6 19 5 68 72 42 6.8 67 4 21 Industry 69 6 59 43 28 6.3 34 4 41 32 26 5.4 4 1 5 106 50 72 20 1 10 Multisector ! 6 7 70 5t 31 6.6 112 12 83 54 30 6.9 27 7 69 62 19 6.4 51 3 12 Z Health, Nutrition and Population 2$ a 69 45 17 63 52 5 65 56 25 6.3 23 6 65 48 43 6.4 150 10 17 Z PublicSectorManagement . 62 5 49 33 27 5.7 56 6 75 63 38 6.8 40 11 89 74 58 7A 126 8 15 Social Sector 4 0 75 5D 75 7.4 30 3 83 20 37 6.6 21 6 8 67 57 7.4 95 6 15 Telecommunications . 17 2 76 76 41 7.1 20 2 80 75 45 7.3 4 1 100 100 75 8. 14 1 0 Transportation 129 12 72 S7 Z 6.S 92 9 82 58 45 7.1 39 10 83 69 64 7.6 162 11 14 Urban Development . 53 5 75 39 24 6.3 52 5 67 41 24 6.2 20 5 80 45 35 6OS 82 5 15 WaterSupply&Sanitation 47 4 70 33 36 6E.4 43 4 49 37 23 5.7 22 6 59 32 27 6.9 87 6 15 Network Environmentally & Socially Sustainable Development 323 29 60 35 29 6.1 233 24 69 43 36 6.4 73 19 82 45 35 6.3 348 23 17 Finance, Private Sector & Infrastructure 542 49 67 49 32 6.4 403 41 68 51 36 6.5 160 42 73 58 47 63 608 41 16 Human Development 116 1t 77 57 34 6.3 167 17 72 41 27 6.4 87 23 77 55 43 6.8 415 28 14 Poverty Reduction & Economic Management IV f2 62 29 62 168 17 80 56 32 6.8 58 is 8471 42 7.8 122 8 11 Lending Type Adjustment 1:4 13 68 5Z 34 6.5 167 17 84 60 38 7.1 42 11 86 74 48 7.1 56 4 16 Investment 92 97 65 44 30 63 804 83 68 46 33 6.4 8 9 72 55 42 6.7 1437 96 15 Lending Source lBRD 58 6 3 69 E7 34 86. 474 49 74 56 37 6.7 178 47 77 64 48 7.1 705 47 16 IDA/blend 52 47 f 62 32 27 6.t 497 51 68 41 30 6.3 200 53 83 51 38 6.5 788 53 15 Regionj Africa 37 34 53 2i' 23 5.7 314 32 59 31 27 5.9 112 30 56 39 31 6.0 363 24 14 EastAsiaandPacific 204 IR O3 7Z 44 7.2 152 16 81 63 43 7.1 62 16 79 56 44 6.9 273 18 11 Europe and Central Asia 69 6 66 St 36 6.5 100 10 82 68 41 7.0 70 19 84 66 51 7.3 139 9 24 Latin America and Caribbean 198 18 65 53 33 6S5 193 20 80 57 42 7.0 75 20 80 71 54 7.2 291 19 18 Middle East and North Africa 118 't1 67 47 28 6.3 83 9 70 48 27 6.3 20 5 74 53 42 6.8 123 8 17 South Asia 151 14 72 41 31 .4 129 13 66 46 27 6.3 39 10 79 67 44 7.0 304 20 13 WDI 1999 Income Group Lower 5 46 59 30 25 5.9 472 49 63 35 28 6.1 172 46 67 47 35 6.3 662 44 16 Lowermiddle 369 33 72 55 34 6.6 324 33 74 55 36 6.7 123 33 78 58 50 7.1 597 40 15 Upper middle 179 16 69 63 37 6.8 155 16 86 71 48 7.2 73 19 79 72 49 7.2 227 15 13 High 29 3 86 86 50 7.6 20 2 80 65 32 7.2 10 3 80 100 50 7.6 7 0 0 Note: Percentages exclude projects not rated. Sectors represent Bank groupings as of the end of FYoO, with the exception of the oil & gas and misnig sectors which were combined into the extrative industries sector * The data for FYOO exits represents a pantial sample t 20 out of 269) and reflects the processing of all ICRs received through August 2000. The processing of the remainder of the FY00 sample is ongoing, expected to be completed by spring 200t Active Portfolio data reflects projects active as of July t, 2000 and it is taken from Annual Report on Portfolio Performance (AfPPI for the PYoo done by the Quality Assurance Group (0AG). 0,a1 n) C Exit FY90-94 Exit FY95-98 Exit FY99-00* Active Portfolio o C) Sust % ID Impact Sust % ID Impact Sust % ID Impact % Disburse Share Out likely or % subor Aggr. Disburse Share Out likely or % sub.or Aggr. Disburse Share Out likely or % sub.or Aggr. Disburse Share Disburse 2 $Millions % % Sat better better Index $ Millions % % Sat better better Index $ Millions % % Sat better better Index $ Millions % at risk c Sector Group > Agriculture 19,197 21 70 46 35 6.4 11,836 15 79 52 44 6.9 2,752 10 74 53 38 G.6 17,293 15 14 Education 3,533 4 90 77 61 7.5 4,574 6 73 40 24 6.5 3,194 10 79 70 46 7.4 11,221 10 11 M ElectricPower&OtherEnergy 13,144 15 64 71 34 6.3 7,448 9 73 55 36 6.6 3J148 12 64 59 65 7.0 11,635 10 19 Environment … … … … … …354 0 75 60 30 7.1 794 3 73 55 44 6.8 5,145 4 22 M Extractive Industries 3,341 4 72 86 45 7.3 3,030 4 84 87 34 7.1 1,601 6 43 45 19 5.8 3,535 3 36 Finance 6,446 7 56 49 37 6.4 13,158 16 79 75 41 7.1 1,541 4 89 69 52 7.1 6,247 5 26 0 Industry 8,561 10 72 65 30 6.7 4,686 6 51 44 38 6.1 350 1 98 100 98 8.2 1,386 1 17 11 Multisector 13,423 15 81 62 48 7.3 11,138 13 91 64 35 7.3 4,394 2 93 58 68 7.5 7,857 7 27 M Health, Nutrition and Population 854 1 76 56 30 7.0 2,613 3 84 72 31 6.8 1,328 5 71 57 57 6.9 9,646 8 20 Public Sector Management 2,322 3 67 52 60 6.9 2,964 4 95 63 56 7.5 2,520 10 95 87 67 7.6 5,468 5 8 M Social Sector 74 : 0 95 67 95 8.3 1,828 2 95 59 51 5.9 1,658 7 97 82 49 7.9 5,155 4 5 Telecommunications 807 1 73 75 35 5.9 986 1 91 94 80 8.7 149 1 100 100 70 9.0 845 1 0 '0 Transportation 10,407 12 79 57 30 5.8 8,489 11 90 61 46 7.2 2,448 8 91 75 71 7.8 18,975 16 12 Urban Development 4,383 5 76 52 30 6.7 4,083 5 83 56 18 6.6 1,157 4 91 55 54 7.6 6,223 5 14 M z Water Supply & Sanitation 2,722 3 71 29: 30 . 62 3,661 5 49 26 17 5.5 1,250 4 54 35 32 6.1 5,756 5 14 Network M Environmentally & Socially T Sustainable Development 19.197 21 70 45 35 6.4 12,151 15 79 53 44 6.9 3,527 13 74 54 39 6.7 22,454 20 15 M Finance, Private Sector & H: Infrastructure 50,053 55 70 61 33 6.5 45,145 56 76 62 37 6.8 10,788 41 73 61 57 7.0 56,605 31 17 Human Development 4,561 5 87 73 5G 7.4 8,956 11 81 53 32 6.7 5,808 22 83 74 47 7.4 26,566 21 15 M Poverty Reduction & Economic :: Management 15,692 17 78 60 50 7.2 13,696 17 92 64 39 7.3 6.259 24 94 74 75 7.6 10,762 28 17 on Lending type Adjustment 26,593 30 73 62 44 7,0 26,198 33 87 70 41 7.3 8,464 32 97 78 65 7.6 15,657 13 17 Investment 62,7Z1 70 72 57 35 6.8 53,750 67 76 54 36 6.7 17,918 68 72 61 49 7.0 100,731 87 16 Lending source IBRD 66,238 74 72 64 40 5.8 55,325 69 80 64 41 7.0 18,451 70 82 69 59 7.3 75,525 64 15 IDA/blend 23,175 26 71 41 32 5.4 24,622 31 77 49 31 6.6 7,901 30 73 60 44 6.9 40,863 36 18 Region Africa 14,597 16 60 27 25 5.9 11,799 15 70 34 28 6.4 3,430 13 59 43 27 6.0 13,341 11 12 East Asia and Pacific 18,807 21 90 83 51 7.5 20,388 25 86 74 41 7.2 8,408 32 88 70 74 7.5 31,189 27 6 Europe and Central Asia 7,778 9 61 50 38 6.5 10,132 13 89 77 45 7.3 4,355 17 74 52 47 6.9 17,636 15 27 Latin America and Caribbean 26,947 29 67 57 35 6.8 18,365 23 84 62 52 7.3 6,121 23 89 81 54 7.4 19,947 17 30 Middle East and North Africa 6,521 7 71 48 35 85. 6,132 8 68 42 22 6.1 943 4 84 77 66 7.5 7,150 6 19 South Asia 15,663 18 75 54 37 5.5 13,132 16 69 53 23 6.3 3,124 12 75 58 44 7.1 27,126 23 12 WDI 1999 Income Group Lower 30,828 34 B8 41 31 6.3 26,200 33 71 44 26 6.4 7,175 27 70 57 37 7.0 36,010 31 19 Lower middle 28,206 32 80 68 42 7.0 29,409 37 80 60 42 7.0 9,380 36 77 55 63 7.4 52,073 45 16 Upper middle 28,285 32 67 66 38 5 6.9 18,561 23 86 71 52 7.3 7,334 28 90 79 52 7.5 27,936 24 12 High 2,093 2 87 88 53 7.6 5,777 7 95 94 9 7.1 2,493 9 95 100 87 6.7 368 0 0 Note: Percentages exclude projects not rated. Sectors represent Bank groupings as of the end of FYOO, with the exception of the oil & gas and mining sectors which were combined into the extrative industries sector. 'The data for FYOO exits represents a panial sample 1t20 eut of 2691 and reflects the processing of all ICRs received through August 2000. The processing of the remainder of the FYOO sample is ongoing, expected to be completed by spring 2001. Active Portfolio data reflects projects active as of July 1, 2000 and it is taken from Annual Repont on Portfolio Performance IARPPI for the FYOO done by the Quality Assurance Group IQAG). ANNEX D MANAGING DEVELOPMENT EFFECTIVENESS: AN OVERVIEW FROM THE CODE CHAIRPERSON The Committee is pleased to report on its delib- allocation, reconciling client ownership and erations on January 29, 2001, on the Annual corporate goals, adapting global knowledge to Review of Development Effectiveness: From local conditions, and selectivity through part- Strategy to Results prepared by the Operations nerships. The Committee also supported the Evaluation Department (OED). This year's Annual recommendations made by OED that the Bank: Review of Development Effectiveness (ARDE) use its instruments more strategically to gener- assessed the development effectiveness of the ate consensus at the global and country levels implementation of the Bank's operational strate- and to build client ownership and effective part- gies in three areas: projects (based on assessments nerships; maintain its flexibility in adapting poli- of 269 completed projects); country assistance pro- cies and strategies to diverse country institutional, grams (based on CAEs of 36 borrowing countries); social, and political environments; strengthen the and sector and thematic activities (through a link between programmatic lending instruments review of 12 sector and thematic areas). and country strategies; and tailor its approaches The report brings welcome news. It confirms to poor and to middle income countries, while that significant progress has been made in the qual- broadening its quality measures to track the ity of the Bank's portfolio in key aspects and in poverty orientation of the Bank's portfolio. almost all regions and networks. The active AFR As the ARDE points out-the gains are still frag- portfolio has improved significantly. The risk of the ile. The Bank must continue to improve the portfolio has been cut in half. The portfolio development effectiveness of its activities in an performance has exceeded the Strategic Compact increasingly complex, risky, and demanding target of 75 percent satisfactory outcomes. environment. The tradeoffs will not be easy for Sustainabiity and institutional development ratings Management or for staff. CODE will refer to the are up. Management has made progress in evaluation reports throughout the year to ensure addressing implementation issues related to matrix that these outstanding issues continue to be management. Decentralization appears to be ben- addressed. Despite the overall improvements in efiting Bank-client relations. quality ratings, a number of areas remain prob- Management and Staff are to be commended lematic. I would like to highlight five issues that for these achievements and for the overall con- emerged out of our Committee discussion that we tinued trend in improvements in the quality of can build on in today's discussion. the Bank's portfolio. These gains are all the more impressive having been achieved under Adjustment/Programmatic Lending increasingly complex external circumstances and The Committee welcomed the improvements in within tighter budget constraints. adjustment lending, noting that the quality ratings The Committee was in broad agreement with for adjustment operations have increased in the conclusions and recommendations of the recent years to quite high levels. The Commit- report-including the four areas of tension men- tee recognized the methodological difficulties in tioned in the ARDE that the Bank needs to assessing the impact of adjustment lending and address in order to increase its effectiveness- noted the need to improve instruments and balancing performance and poverty in resource methods of evaluation so as to enhance the 53 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS poverty focus in the evaluation of both adjust- behind other regions, and asked that a serious ment and investment lending. Some Directors effort be made to bring the size of the portfolio called for a closer look at the underlying drivers back up to the pre-cancellation restructuring level. of adjustment lending, including how lending The Committee could not judge whether the sec- volumes are determined and the links between toral shirts were intended, the result of the dif- lending volumes and past and ongoing country ferential costs of doing business or of other donors dialogue. The Committee looks forward to stepping into these sectors. The Committee noted discussing the Adjustment Lending Retrospective that the two Regions and three sectors were crit- being prepared by Management and to hearing ical to the Bank's realizing its mission of poverty from OED on improvements in assessment meth- alleviation and that these downward trends could ods for programmatic lending. raise questions about the effectiveness of the Bank's Poverty Reduction Strategy. In this respect Quality Trends and Prospects the Committee also noted that it would be criti- The Committee expressed several viewpoints in cal to reverse the trend of decreasing resources regard to the budget constraints and the ability of devoted to ESW. The Committee noted the the Bank to sustain the broad recovery in portfo- persistent lack of progress in developing an lio quality while maintaining and expanding its instrument to gauge Bank performance in assist- knowledge base. Some members felt that the ing client countries in reducing poverty and asked rigidity of the budget had a negative impact on the Management for renewed efforts in this area, Bank's work (for example, the decline in the including progress on the Corporate Scorecard. quality of ESW and increase in staff stress) and they Finally, the Committee noted the progress that suggested revisiting budget issues. Other members had been made with regard to rebalancing the believed that quality gains and the knowledge base matrix towards the networks but noted that much could be maintained and improved through the more still remains to be done to achieve a balance more efficient utilization of current resources and among corporate priorities, sector strategies, and through applying greater selectivity. While country programming. The Committee highlighted endorsing 100 percent quality targets for appraisal the key role that the Country Director could play and supervision, the Committee noted that 100 per- in assisting the Bank and countries in aligning cent success rate for project outcomes could not sector priorities with country priorities and asked be realistically expected. It was in broad agreement Management how it planned to further improve with the recommendation to create headroom for the matrix. The Committee also asked Manage- calculated risk-taking and innovation. The Com- ment how it planned to operationalize the results mittee noted the reputational risks associated with of the report to improve performance in the setting targets at too low a level and also urged a poorer performing regions and sectors, as well as cautious approach to setting different targets for with regard to the matrix and to continued weak- different regions. nesses in areas such as sustainability and institutional development ratings. Realigning Resources to Strategic Priorities and to Quality Improvements Country Performance The Committee agreed with OED that strategic The Committee supported the emphasis placed on selectivity and how to realign resources to strate- helping the poor performers but underlined that gic priorities would be the key challenges in it would be important to differentiate between poor moving forward. The Committee was concerned performance as a result of weak capacity versus about the significant shifts in the portfolio away a lack of commitment and poor policy environ- from the AFR and SAR regions and from agricul- ment. The Committee also underlined the impor- ture/rural development, infrastructure, and edu- tance of the quality of dialogue and capacity cation sectors. The Committee was especially building in these countries, and the importance of concerned that, despite the improvements in the country ownership and the quality of Bank-country Africa region portfolio, the region still lagged dialogue for reconciling country and corporate 54 ANNEX D MANAGING DEVELOPMENT EFFECTIVENESS: AN OVERVIEW FROM THE CODE CHAIRPERSON priorities but cautioned that country ownership and at the Strategic Forum demonstrate that Man- harmonization should not result in sub-optimum agement is committed to making the Bank more standards. The Committee supported the wider use selective in both its country and global work; of stakeholder analysis to conduct broader-based in setting corporate priorities and aligning its dialogue within countries in order to build con- resources with those priorities. The Strategic sensus, especially for difficult reforms. Framework provides the principles to guide the Banks towards greater selectivity and prioriti- Partnerships zation. The Committee urges Management to The Committee emphasized that partnerships address the severe pressures faced by staff, should decrease and not increase the Bank's without whom the gains in quality are unsus- cost of doing business and that the Bank had yet tainable. CODE will continue to promote the to reap the benefits of partnerships. The Com- adoption of lessons learned and best practice mittee noted that sector strategies and networks through evaluation and looks forward to work- had a key role to play in helping the Bank be ing with Management to further improve the more selective in partnerships and in develop- development effectiveness of the Bank Group ing more businesslike approaches. policies, programs, instruments, and corporate Accountability for results remains a chal- processes. lenge. Our discussions of the Strategic Frame- work paper and our deliberations most recently Pieter Stek, Chairmnan 55 ENDNOTES Chapter 1 While initially quite distinct, the two have become in- 1. In the past three years, OED has completed stud- creasingly alike given the need for specificity in econ- ies of energy and the environment (Mason 1998), fi- omywide interventions and the recognized synergies nance (Mathieu 1998), forestry (Lele 2000), health across sectors. There is surprisingly little difference in (Johnston and Stout 1999), participation, poverty re- either the number or subject area of loan conditions duction (Evans 2000), resettlement (OED 1998g), rural between SALs and SECALs. The two other types of ad- development (Heath 2000), telecommunications, and justment lending-debt reduction loans and rehabil- water (Pitman forthcoming). Mid-term reports have itation import loans-make up less than 10 percent been issued for gender (Gopal 2000) and environment of disbursements of FY95-00 adjustment exits. (Liebenthal 2000). Studies are under way for cultural 6. Bank and borrower performance are measured heritage, social funds, and private sector develop- at key stages in the project cycle. The exogenous en- ment in energy. ARDE 2000 also draws on OED work- vironment includes measures of both the macroeco- ing papers in rural water supply (Parker and Skytta nomic policy outcomes (such as inflation, fiscal balance, 2000) and an impact evaluation of agricultural ex- debt servicing, and trade openness) and the quality of tension in Kenya (Gautam 2000). the country's institutional framework. See Battaile and Mini (forthcoming) for a more detailed discussion of Chapter 2 the model, estimation method, and results. 1. Demandingness refers to the extent to which 7. Growth is measured by real GDP per capita, the project is expected to strain the economic, in- when available for at least one-third of the project's stitutional, and human resources of the govern- implementation period. Institutional quality is meas- ment/implementing agency. Complexity refers to ured by the average score on the public sector man- factors such as the range of policy and institutional agement and institutions section of the 1998-00 Country improvements contemplated, the number of insti- Policy and Institutional Assessment exercise. The data tutions involved, the number of project compo- for 1998-00 are used to proxy for the broader imple- nents and their geographic dispersion, and the mentation period of recently exiting projects, given that number of cofinancers. Riskiness refers to the like- institutional change occurs slowly over time. lihood that the project, as designed, might fail to 8. The adjustment operation model was devel- meet relevant objectives efficiently. In determining oped in last year's ARDE, and the findings showed sim- project riskiness, evaluators consider the extent to ilar results after controlling for political economy which the project could reasonably have been ex- considerations. pected-at the time of preparation and appraisal- to face known risk factors, such as lack of borrower Chapter 3 commitment, inadequate counterpart funding, and 1. In the past 4 years, OED has completed 38 war or civil disorder. CAEs, covering a wide range of Bank client countries. 2. The figures in the chapter show portfolio per- Various criteria were used to select CAE countries (the formance by project rather than by disbursement un- Bank's CAS schedule, development learning potential, less otherwise noted. FY00 data represent a partial regional balance, and the like). The CAEs therefore sample (120 out of 269) and reflect the processing of do not represent a random sample. Hence, country all ICRs received through August 2000. outcome ratings cannot be used to assess the evolu- 3. For information regarding APPI methodology, tion of Bank performance. please refer to Annex 1, 1998 ARDE (Buckley 1999). 2. CAEs for Cameroon and Tanzania gave sepa- 4. In Kenya OED outcomes for FY96-00 exits rate ratings for different periods covered by the were 19 percent satisfactory, for Senegal 69 percent, evaluation. Performance of the Cameroon country and for Uganda 58 percent. QAG data reports 64 per- program improved following devaluation of the cent for Kenya, 100 percent for Senegal, and 92 per- CFA franc in 1994, although corruption remains en- cent for Uganda of projects not at risk in the active demic. Bank projects and programs performed portfolio. poorly in Tanzania until the government began im- 5. There are two dominant adjustment lending plementing macroeconomic and sector reform in the instruments-structural and sectoral adjustment loans. mid-1990s. 56 END NOTES 3. The Costa Rica CAE concluded that the Bank's to increased incomes and employment for the poor 1993 Country Assistance Strategy was based on a cor- and non-poor alike. In reality, the non-poor are rect diagnosis and included relevant objectives. The likely to benefit earlier from growth than the poor. outcome, however, was unsatisfactory because the A mix of policies must be identified that places the strategy included poorly designed structural adjustment highest priority on enhancing the poor's prospects lending instruments, overlooked lessons of experience, for benefiting from growth and protecting those and ignored political conditions determining owner- who are likely to lose out most in the short run" ship of the program. The Bank made some limited but (Evans 2000). useful contributions through nonlending services. Re- 9. According to data from the Development Assis- cently, the strategy has regained relevance, and dia- tance Committee (DAC), during 1994-98 an average logue has improved. of 12 donor agencies were active in the agriculture, 4. For this Review, OED applied a standard ques- education, and health sectors in low-income countries, tionnaire to the 15 Country Assistance Evaluations with a dozen low-income countries having more than listed in table 3.1. Based on the CAEs, country strate- 20 donors in education and agriculture. During the gies and programs were rated according to the qual- same period the Bank provided support to education, ity of diagnosis, the relevance of the country program, health, and agriculture for two-thirds of all low-income the effectiveness of program implementation (in- countries. cluding for lending and nonlending services), as well 10. Lending instruments include investment, ad- as Bank, borrower, and exogenous factors influenc- justment, and new instruments, including Learning and ing program outcomes. To improve consistency, the Innovation Loans, Adaptable Program Loans (APLs), questionnaires were completed by an experienced and Emergency Recovery Loans. Nonlending services evaluator and the ratings cross-checked with the CAE include economic and sector work, advisory services, task managers. donor coordination, and consultative activities. 5. For example, macroeconomic issues usually are 11. The Country Policy and Institutional Assessment well covered in CASs, but only 20 percent of those (CPIA) rating system currently consists of 20 criteria sampled-from the mid- to late-1990s-explicitly ad- grouped into four categories: economic management, dressed equity issues or microeconomic constraints to structural policies, policies for social inclusion, and pub- poverty reduction (Evans 2000). More recent CASs lic sector management. The CPIA reflects policies and show signs of improvement. institutional arrangements that are subject to govern- 6. The CAE for Burkina Faso concluded that ment control-as opposed to such performance out- progress in governance has been mixed, reflecting come indicators as GDP growth or poverty reduction. "the absence of a unifying vision of the improvements The ratings are updated annually. required to realize good governance in a restructured 12. The Burkina Faso CAE found that in the early economy." 1990s relatively open public discussion of proposed 7. In Kazakhstan, "the IBRD (along with other policy reforms resulted in somewhat higher accept- donors) was overly optimistic in its expectation that ance of structural adjustment than in other African the transition from a planned to a market economy countries undertaking reform. in the former Soviet Union countries could be shed 13. The Tanzania CAE notes that the Bank's strat- in a short time and at low social costs." (OED 2000c) egy for agriculture, and for private sector develop- 8. The OED poverty study's CAS review found ment in general, "appears to be based on the that more than 90 percent of CASs recommend a assumption that once prices were liberalized and paras- strategy of macrostability, liberalization, and trade tatals dismantled, the private sector would step in and and tariff reform to support broad-based growth. automatically fill the vacuum. This has not occurred." "The linkages or transmission mechanisms between The CAE further notes that parallel reviews by OED in this policy agenda and expected changes in the other countries are arriving at similar conclusions, and conditions facing the poor are largely tnclear, how- the Bank needs to grapple with this problem. ever. The implicit assumption seems to be that 14. CASs are now required to be costed, but this growth resulting from macroeconomic stability and budget information is not included in the CAS docu- improved resource allocation will automatically lead ment that is presented to the Bank's Board. 57 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS Chapter 4 soundness of the analytic or conceptual framework, the 1. These same patterns do not necessarily apply adequacy and clarity of operational policies, guidance when examining projects at risk in the ongoing port- on selectivity, adaptability of the strategy to differing folio. For example, only 15 percent of HNP projects country settings, attention to cross-sectoral issues, and in low-income countries are at risk, compared with degree of participation in strategy development. Crite- the Bankwide average of 17 percent. And for water ria for rating implementation included the existence and supply and sanitation, projects at risk are close to Bank quality of an implementation plan, the degree of main- averages for low- and lower-middle-income coun- streaming of the strategy within the Bank, and progress tries, but slightly above average for upper-middle- in incorporating the strategy into country strategies and income countries (18 percent at risk compared with programs. Task managers completed the questionnaire 13 percent Bankwide). for the following studies: environment, forestry, gender, 2. OED is currently conducting an evaluation of so- participation, poverty, rural development, telecommu- cial funds, which will be completed later in 2001. nications, and water. The Bank's 1997 health SSP was 3. The forestry study concluded that "strengthening not assessed because it was prepared concurrendy with the Bank's analytic base on forests requires much more the OED health study. explicit integration of forest sector issues into the Bank's 7. The current SSP guidelines call for an explicit ESW and more resources need to be devoted to ana- discussion of the business implications of the strategy. lytically sound forest sector work" (Lele 2000). 8. For example, in 1997 the Poverty Sector Board 4. Although the Bank's Quality Assurance Group issued a note that called for a shift away from describing (QAG) has begun rating the quality of economic and poverty and counting the number of poverty-focus pro- sector work, conclusions at the sectoral and country grams in Country Assistance Strategies and poverty as- levels are limited by sample size constraints. Other sessments, toward formulating explicit strategies for knowledge management activities have largely eluded poverty reduction and assessing the impact on the poor. systematic and objective evaluations, despite their 9. These gaps took several forms. Some OPs are too rapid expansion. brief-only one page in the case of gender, telecom- 5. Prior to the introduction of Sector Strategy Papers munications, and water-and discuss only general prin- (SSPs) in 1996, policy papers articulated broad strategies ciples, with no accompanying good practice guidelines. for sectors and themes, but were not expected to ad- For gender, this generality has meant that staff were un- dress implementation issues. Today, SSPs provide strate- clear whether and how to take action (Gopal 2000). In gic guidance to staff and management, while Operational telecommunications, it meant that Bank operations and Policies (OPs) provide formal guidance to staff regard- policy advice lacked consistency and strategic coherence. ing official Bank policies. In the text, "strategy paper" 10. SSP are expected to ground the strategy the refers both to SSPs and the earlier policy papers. Bank's corporate priorities, current development 6. The questions were based on the current one-page knowledge, and Bank experience; actively involve Re- outline for Sector Strategy Papers, and were grouped gional departments and other relevant stakeholders; under three broad headings: relevance, implementation, develop an explicit plan for implementation (includ- and outcome. The strategies were assessed against cur- ing implications for resources and staffing); and es- rent SSP standards, although only the 1997 rural strat- tablish indicators to monitor strategy implementation egy was an SSP. The criteria for relevance included the and results. 58 BIBLIOGRAPHY This report has drawn on a wide range of Wo rld Evans, A., and William Battaile. 1998. 1997 An- Bank documents and outside sources. World nual Review of Development Effectiveness. Bank sources include OED's major studies, Coun- OED Study Series. Washington, D.C.: World try Assistance Evaluations, project audits, and on- Bank. going research. Feinstein, Osvaldo, and Robert Picciotto. 2000. Evaluation and Poverty Reduction: Proceedings AfDB (African Development Bank) and OED from a World Bank Conference. Washington, (Operations Evaluation Department). 1999. D.C.: World Bank, Operations Evaluation De- Evaluation Capacity Development in Africa: partment. Selected Proceedingsfrom a Seminar inAbid- Fischer, Stanley. 1999. "On the Need for an In- jan. Washington, D.C.: World Bank, Opera- ternational Lender of Last Resort." Journal of tions Evaluation Department. Economic Perspectives 13 (4): 85-104. Anderson, Jock, and Dana Dalrymple. 1999. The Gautam, Madhur. 2000. Agricultural Extension, World Bank, the Grant Program, and the The Kenya Experience: An Impact Evalua- CGL4R: A Retrospective Review. OED Working tion. OED Study Series. Washington, D.C.: Paper No. 1. Washington, D.C.: World Bank, World Bank. Operations Evaluation Department. Girishankar, Navin. 1999. Civil Service Reform: A Battaile, William, and Federico Mini. Forthcoming. Review of World Bank Assistance. World Bank DeterminantsofInvestmentProjects'Success:An Report No. 19599. Washington, D.C.: World Empirical Analysis. Washington, D.C.: World Bank, Operations Evaluation Department. Bank, Operations Evaluation Department. Gopal, Gita. 2000. Integrating Gender in World Birdsall, Nancy, Stijn Claessons, and Ishac Diwan. Bank Assistance. Washington, D.C.: World 2000. Debt Reduction in Africa: Understand- Bank, Operations Evaluation Department. ing the Creditors. Washington, D.C. Hassan, Fareed. 1999. HigherImpactAdjustment Branson, William, and Nagy Hanna. 2000. Own- Lending (HI4L). Initial Evaluation. World Bank ership and Conditionality. OED Working Report No. 19797. Washington, D.C.: World Paper No. 8. Washington, D.C.: World Bank, Bank, Operations Evaluation Department. Operations Evaluation Department. Heath, John. 2000. Rural Development-From Buckley, R. 1999. 1998 Annual Review of De- Vision to Action? (Phase II). Washington, D.C.: velopment Effectiveness. OED Study Series. World Bank, Operations Evaluation Depart- Washington, D.C.: World Bank. ment. Devarajan, S., D. Dollar, and T. Hohmgren. 1999. Hinds, Manuel. 2000. "The Bank's Role in Mid- Aid and Reform in Africa. Washington, D.C: dle-Income Countries." ARDE 2000 back- World Bank. ground paper. World Bank, Operations Dollar, David, and Lant Pritchett. 1999. Assess- Evaluation Department, Washington, D.C. ing Aid: What Works, What Doesn't, and Hirschman, A.O. 1958. The Strategy ofEconomic Why. Washington, D.C.: Oxford University Development. New Haven, CT: Yale Univer- Press. sity Press. Dollar, David, and Jakob Svensson. 1998. What IMF (International Monetary Fund). 1998. Exter- Explains the Success or Failure of Structural nal Evaluation of the ESAF: Report by a Group Adjustment Programs? World Bank Policy Re- of Independent Experts. Washington, D.C. search Working Paper. Washington, D.C. IUCN. 1997. Large Dams-Learning from the Eriksson, John. 1999. The Drive to Partnership: Past, Looking at the Future. Workshop pro- Aid Coordination and the World Bank. OED ceedings. Gland, Switzerland: IUCN and the Study Series. Washington, D.C.: World Bank. World Bank Group. Evans, Alison. 2000. Poverty Reduction in the Jayarajah, C., and W. Branson. 1995. Social Di- 1990s: An Evaluation of Strategy and Per- mensions of Adjustment.. World Bank Expe- formance. OED Study Series. Washington, rience, 1980-93. Washington, D.C.: World D.C.: World Bank. Bank, Operations Evaluation Department. 59 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS . 1994. Structural and Sectoral Adjust- MacKay, Keith. 1998. Evaluation Capacity De- ment: World Bank Experience 1980-99. velopment: Lessonsfrom National Experience. Washington, D.C.: World Bank, Operations OED ECD Working Paper No. 1. Washington, Evaluation Department. D.C.: World Bank. Johnson, John, and Sulaiman Wasty. 1993. Bor- Mason, Melody. 1998. "The World Bank Envi- rower Ownership ofAdjustment Programs and ronmental Strategy for the Energy Sector: An the Political Economy of Reform. World Bank OED Perspective." Report No. 17359, World Discussion Paper No. 199. Washington, D.C. Bank, Operations Evaluation Department, Johnston, Timothy, and Susan Stout. 1999. In- Washington, D.C. vesting in Health: Development Effectiveness Mathieu, N. 1998. Financial Sector Reform: A Re- in the Health, Nutrition and Population Sec- view of World Bank Assistance. OED Study Se- tor. OED Study Series. Washington, D.C.: ries. Washington, D.C.: World Bank. World Bank. Morrison, Kevin. 2000. "Debt and Development Jones-Carroll, Charlotte. 2000. "IDA Budget Effectiveness." ARDE 2000 background paper. Processes and Their Implications." IDA Review World Bank, Operations Evaluation Depart- background paper. World Bank, Operations ment, Washington D.C. Evaluation Department, Washington, D.C. Nelson, Joan M. 1999. Reforming Health and Ed- Jones, Stephen, and Andrew Lawson. 1999. "Scal- ucation: The World Bank, The IDB, and Com- ing Up: Issues in Moving from Projects to plex Institutional Change. ODC Policy Essay Programmatic Aid." Oxford Policy Manage- No. 26. Washington, D.C.: Overseas Devel- ment. opment Council. Killick, Tony. 2000. "HIPC II and Conditionality: Nelson, Joan M., and Stephanie Eglinton. 1993. Business as Before or a New Beginning?" Global Goals, Contentious Means: Issues of Paper commissioned by Commonwealth Sec- Multiple Aid Conditionally. ODC Policy Essay retariat for Policy Workshop on Debt, HIPC, No. 10. Washington, D.C.: Overseas Devel- and Poverty Reduction. opment Council. Killick, T., R. Gunatilaka, and A. Marr. 1998. Aid OED (Operations Evaluation Department). 2000a. and the Political Economy of Policy Change. "Cameroon-Country Assistance Evaluation." London: Overseas Development Institute. Washington, D.C. Kreimer, A., J. Eriksson, R. Muscat, M. Arnold, . 2000b. "Costa Rica-Country Assistance and C. Scott. 1998. The World Bank's Experi- Evaluation." Washington, D.C. ence with Post-Conflict Reconstruction. OED . 2000c. "Republic of Kazakhstan-Coun- Study Series. Washington, D.C.: World Bank. try Assistance Evaluation." Washington, D.C. Krueger, Anne. 1997. Whither the World Bank . 2000d. "Egypt-Country Assistance Eval- and the IMF? NBER Working Paper Series No. uation." Washington, D.C. 6327. Cambridge, MA: National Bureau for . 2000e. "Ghana-Country Assistance Economic Research. Evaluation." Washington, D.C. Leem, K.S., and R. Gilbert. 1999. Developing . 2000f. "Papua New Guinea-Country As- Towns and Cities: Lessonsfrom Brazil and the sistance Evaluation." Washington, D.C. Philippines. OED Study Series. Washington, . 2000g. "Paraguay-Country Assistance D.C.: World Bank. Evaluation." Washington, D.C. Lele, Uma. 2000. The World Bank Forest Strat- . 2000h. "Tanzania-Country Assistance egy: Striking theRight Balance. OED Study Se- Evaluation." Washington, D.C. ries. Washington, D.C.: World Bank. . 2000i. "Uganda-Country Assistance Liebenthal, Andres. 2000. "Phase I Interim Report. Evaluation." Washington, D.C. Promoting Environmental Sustainability in De- . 2000j. "Uruguay-Country Assistance velopment-An Evaluation of the World Bank's Evaluation." Washington, D.C. Performance." World Bank, Operations Eval- . 2000k. World Bank Assistance to CFA uation Department, Washington, D.C. Countries: An Evaluation of Selected Social, 60 BIBLIOGRAPHY EconomicandRegionalAspectsoftheBank's Pitman, Keith. Forthcoming. Bridging Troubled Performance. Washington D.C. Waters: Assessing the WaterResources Strategy . 1999a. Poverty Assessments: A Follow-up Since 1993. Washington, D.C.: World Bank, Review. Washington D.C. Operations Evaluation Department. . 1999b. "Burkina Faso-Country Assis- QAG (Quality Assurance Group). 2000. "Qual- tance Note." Washington, D.C. ity at Entry in CY99: A QAG Assessment." . 1999c. "Indonesia-Country Assistance World Bank Quality Assurance Group, Wash- Note." Washington, D.C. ington, D. C. . 1998a. "Albania-Country Assistance Squire, Lyn, Klaus Deininger, and Swati Basu. Note." Washington, D.C. 1997. "Does Economic Analysis Improve Qual- . 1998b. "Bolivia-Country Assistance ity of Foreign Assistance?" World Bank Eco- Note." Washington, D.C. nomic Review 12: 385-418. . 1998c. "Jamaica-Country Assistance UNICEF (United Nations Children's Fund). 1999. Note." Washington, D.C. Implementing Health SectorReforms in Africa: . 1998d. "Kenya-Country Assistance A Review of Eight Country Experiences. New Note." Washington, D.C. York. . 1998e. Evaluation Capacity Development van de Walle, Nicolas. 1999. Aid's Crisis of Legit- in Africa: A Selected Proceedingsfrom a Sem- imacy: CurrentProposalsandFutureProspects. inar in Abidjan. Washington, D.C.: African De- Ann Arbor: Michigan State University. velopment Bank, Abidjan, and the World Bank. van de Walle, Nicolas, and Timothy Johnston. 1998f. The Special Program of Assis- 1996. "Improving Aid to Africa." ODC Policy tanceforAfrica (SPA): An Independent Eval- Essay No. 21. Washington, D.C.: Overseas uation. Washington, D.C. Development Council. -. -1998g. Recent Experience with Invol- WCD (World Commission on Dams). 2000. Dams untary Resettlement. Overview. World Bank Re- and Development-A New Framework for port No. 17538. Washington, D.C. Decision-Making. London and Sterling, VA: . 1997. "Mozambique-Country Assis- Earthscan. tance Note." Washington, D.C. White, Howard, Tony Killick, Stephen Kayizzi- . 1996. "Argentina-Country Assistance Mugerwa, and Marie-Angelique Savane. 2000. Note." Washington, D.C. Africa Poverty at the Millennium: Causes, Orbach, Eliezer. 1999. "Organizational Capacity Complexities and Challenges. Report prepared and Capacity Building: An Operational Defi- for the Strategic Partnerships for Africa (SPA). nition and Assessment Methodology, Capac- World Bank, Washington, D.C. ity Building Unit." World Bank, Africa Region, WHO (World Health Organization). 2000. Health Washington, D.C. Systems: Improving Performance. Geneva. Parker, R., and T. Skytta. 2000. Rural WaterPro- Williamson, John. 1999. Country Ownership, Pub- jects: Lessons from OED Evaluations. OED lic Participation, and the CDF Washington, Working Paper No. 3. Washington, D.C.: World D.C.: Institute for International Economics. Bank, Operations Evaluation Department. Wolfensohn, J.D. 1999. "A Proposal for a Com- Picciotto, R. 1998. "Logic of Partnership." World prehensive Development Framework." World Bank, Operations Evaluation Department, Bank, Washington, D.C. Washington, D.C. World Bank. 2001. Global Development Finance. . 1995. Putting Institutional Economics to Washington, D.C. Work: FromParticipation to Governance. World . 2000a. World Development Report Bank Discussion Paper 304. Washington, D.C. 2000/2001: Attacking Poverty. Washington, Picciotto, R., and Eduardo Weisner. 1998. Eval- D.C.: Oxford University Press for the World uation and Development: The Institutional Bank. Dimension. New Brunswick, NJ, and Lon- . 2000b. East Asia Recovery and Beyond. don: Transaction for the World Bank. Washington, D.C. 61 2000 ANNUAL REVIEW OF DEVELOPMENT EFFECTIVENESS . 2000c. Can Africa Claim the 21st Cen- . 1996. World Bank Participation Source- tury? Washington, D.C. book. Environmentally and Socially Sustainable . 2000d. Fuel for Thought: An Environ- Development Work in Progress, Report 14838. mental Strategy for the Energy Sector. Wash- Washington, D.C. ington, D.C. . 1990. World Development Report 1990: . 1999. "Assessing Borrower Ownership Poverty. New York: Oxford University Press Using Reform Readiness Analysis." PREMnote for the World Bank. 25. Washington, D.C. World Bank and UNDP (United Nations Devel- . 1997a. World Development Report 1997: opment Programme). 2000. "Global Public The State in a Changing World. Washington, Policies and Programs: Implications for Fi- D.C.: Oxford University Press for the World nancing and Evaluation." Proceedings of a Bank. workshop held in Washington, D.C., July . 1997b. Sector Strategy. Health, Nutrition, 11-12, 2000. World Bank Operations Evalua- and Population. Washington, D.C. tion Department, Washington, D.C. 62 OPERATIONS EVALUATION DEPARTMENT PUBLICATIONS The Operations Evaluation Department (OED), an Ordering World Bank Publications independent evaluation unit reporting to the World Customers in the United States and in territories not Bank's Executive Directors, rates the development served by any of the Bank's publication distributors may impact and performance of all the Bank's completed send publication orders to: lending operations. Results and recommendations are reported to the Executive Directors and fed back into The World Bank the design and implementation of new policies and P.O. Box 960 projects. In addition to the individual operations and Herndon, VA 20172-0960 country assistance programs, OED evaluates the Bank's Fax: (703) 661-1501 policies and processes. Telephone: (703) 661-1580 Summaries of studies and the full text of the Precis The address for the World Bank publication database and Lessons & Practices can be read on the Internet at on the Internet is: http://www.worldbank.org (select http://www.worldbank.org/html/oed publications/project info). E-mail: pic@worldbank.org How To Order OED Publications Fax number: (202) 522-1500 Operations evaluation studies, World Bank discussion Telephone number: (202) 458-5454 papers, and all other documents are available from the World Bank InfoShop. The World Bank InfoShop serves walk-in customers Documents listed with a stock number and price code only. The InfoShop is located at: may be obtained through the World Bank's mail order service or from its InfoShop in downtown Washington, 701 18th Street, NW D.C. For information on all other documents, contact Washington, DC 20433, USA the World Bank InfoShop. For more information about this study or OED's All other customers must place their orders through their other evaluation work, please contact Elizabeth local distributors. Campbell-Page or the OED Help Desk. Ordering by e-mail Operations Evaluation Department If you have an established account with the World Bank, Partnerships & Knowledge Programs (OEDPK) you may transmit your order by electronic mail on the E-mail: ecampbellpage@worldbank.org Internet to: books@worldbank.org. Please include your E-mail: eline@worldbank.org account number, billing and shipping addresses, the tide Telephone: (202) 458-4497 and order number, quantity, and unit price for each Facsimile: (202) 522-3200 item. 63 OED PUBLICATIONS Study Series 1999 Annual Review of Development Effectiveness Agricultural Extension: The Kenya Experience Agricultural Extension and Research: Achievements and Problems in National Systems Bangladesh: Progress Through Partnership Developing Towns and Cities: Lessons from Brazil and the Philippines Financial Sector Reform: A Review of World Bank Assistance Financing the Global Benefits of Forests: The Bank's GEF Portfolio and the 1991 Forest Strategy and Its Implementation Fiscal Management in Adjustment Lending India: The Dairy Revolution Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Mainstreaming Gender in World Bank Lending: An Update Nongovernmental Organizations in World Bank-Supported Projects: A Review Paddy Irrigation and Water Management in Southeast Asia Poland Country Assistance Review: Partnership in a Transition Economy Poverty Reduction in the 1990s: An Evaluation of Strategy and Performance Reforming Agriculture: The World Bank Goes to Market The World Bank's Experience with Post-Conflict Reconstruction The World Bank's Forest Strategy: Striking the Right Balance Zambia Country Assistance Review: Turning an Economy Around Evaluation Country Case Series Bosnia and Herzegovina: Post-Conflict Reconstruction Brazil: Forests in the Balance: Challenges of Conservation with Development Cameroon: Forest Sector Development in a Difficult Political Economy China: From Afforestation for Poverty Alleviation and Natural Forest Management Costa Rica: Forest Strategy and the Evolution of Land Use El Salvador: Post-Conflict Reconstruction India: Alleviating Poverty through Forest Development Indonesia: The Challenges of World Bank Involvement in Forests Uganda: Post-Conflict Reconstruction Proceedings Lessons of Fiscal Adjustment Lesson from Urban Transport Evaluating the Gender Impact of World Bank Assistance Evaluation and Development: The Institutional Dimension (Transaction Publishers) Evaluation and Poverty Reduction Monitoring & Evaluation Capacity Development in Africa Public Sector Performance-The Critical Role of Evaluation Multilingual Editions Assessing Development Effectiveness: Evaluation in the World Bank and the International Finance Corporation Appreciation de l'efficacite du developpement: L'evaluation a la Banque mondiale et a la Societefinanciere internationale Determinar la eficacia de las actividades de desarrollo: La evaluaci6n en el Banco Mundial y la Corporaci6n Financiera Internacional C6te d'Ivoire: Revue de l'aide de la Banque mondiale aupays Philippines: From Crisis to Opportunity Filipinas: Crisis y oportunidades Rebuilding the Mozambique Economy: Assessment of a Development Partnership Reconstruir a Economia de Mo,cambique www.worldbank.org/html/oed TH E WO R LD BAN K 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. Telephone: (202) 477-1234 Facsimile: (202) 477-6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org IN 11VF - II1 ISB 0.82134929-